Media display method and device, first electronic equipment and second electronic equipment

Abstract

The embodiment of the invention provides a media display method and device, first electronic equipment and second electronic equipment. The method comprises the following steps: acquiring first media collected by a camera of first electronic equipment and motion data of the first electronic equipment; generating motion track information according to the first media and the motion data; and transmitting the first media and the motion trail information to the second electronic equipment, so that the second electronic equipment generates and displays the second media according to the first media and the motion trail information, and picture jitter generated by large-amplitude low-frequency jitter of the first electronic equipment can be reduced. And picture jitter generated when the first electronic equipment performs relatively large visual angle conversion is reduced, so that the stability of a video picture is improved, and the visual experience is better.

Description

[Technical field]

[0001] The present application relates to the field of electronic technology, and in particular to a media display method, device, first electronic device and second electronic device. [Background technology]

[0002] Media is the medium for transmitting information. Media may include video. Video is usually in various codec formats such as H263 / 4 / 5, all of which are derived from the compression of the original video source without processing the original video source.

[0003] When a user shoots a video with an electronic device, the user's hands cannot hold the electronic device steadily for shooting. If the electronic device has large-scale low-frequency jitter and / or large viewing angle changes, the picture shot by the electronic device will shake violently, causing great discomfort to the user. The electronic device used for playback currently does not support the ability to stabilize video, and the stability of the video picture is low. [Summary of the invention]

[0004] In view of this, embodiments of the present application provide a media display method, device, first electronic device and second electronic device to improve the stability of video images.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a media display method, which is applied to a first electronic device, including:

[0007] Acquire motion data of the first media and the first electronic device collected by a camera of the first electronic device;

[0008] generating motion trajectory information according to the first media and the motion data;

[0009] The first media and the motion track information are transmitted to the second electronic device, so that the second electronic device generates and displays the second media according to the first media and the motion track information.

[0010] According to the media display method provided by the present application, the first electronic device synchronously transmits the motion trajectory information and the first media to the second electronic device, and the second electronic device can perform motion compensation on the first media based on the motion trajectory information and the first media obtained from the first electronic device. However, in the related art, de-jitter processing is not performed based on the motion trajectory information but only based on the de-jitter algorithm, which easily causes the video picture to be blurred and the de-jitter quality to be low. In the technical solution provided by one embodiment of the present application, the picture can be motion compensated according to the jitter process of the first electronic device when shooting the first media to reduce the jitter amplitude of the displayed picture, and the video picture is clear and the de-jitter quality is high while improving the stability of the video picture.

[0011] In an implementation of the first aspect, the motion trajectory information includes a video frame identifier, so that when the second electronic device generates the second media based on the first media and the motion trajectory information, the second electronic device generates the second media after performing motion trajectory compensation processing on each video frame based on the video frame identifier of each video frame and the motion trajectory information corresponding to each video frame identifier.

[0012] Furthermore, the motion trajectory information includes a video frame identifier, and the motion trajectory information corresponding to each video frame identifier can be determined according to the video frame identifier of each video frame, thereby ensuring the accuracy of motion trajectory compensation for each video frame.

[0013] In an implementation of the first aspect, the method further includes:

[0014] Acquire real-time motion data of the first media and the first electronic device collected in real time by a camera of the first electronic device;

[0015] generating motion trajectory information in real time according to the first media and the motion data;

[0016] The first media and the motion track information are transmitted to the second electronic device in real time, so that the second electronic device can generate and display the second media in real time according to the first media and the motion track information.

[0017] Furthermore, the camera of the first electronic device collects the first media and the real-time motion data of the first electronic device in real time, and the first electronic device generates motion trajectory information in real time based on the first media and the motion data, so that the second electronic device can generate and display the second media in real time based on the first media and the motion trajectory information, that is, while the user is recording the first frame of the first media on the first electronic device, the second electronic device can display the first frame of the second media. In the technical solution provided in one embodiment of the present application, real-time and stable video transmission can be achieved by synchronously transmitting the motion trajectory information and the first media.

[0018] In an implementation of the first aspect, the method further includes:

[0019] A first medium captured by a camera of a first electronic device is displayed.

[0020] In an implementation manner of the first aspect, a frame whose frame identifier is the first frame identifier in a first media displayed by the first electronic device is different from a frame whose frame identifier is also the first frame identifier in a second media displayed by the second electronic device.

[0021] In an implementation manner of the first aspect, before acquiring the first media and the motion data, the method further includes:

[0022] Establishing a first communication connection with at least one second electronic device and receiving a viewing request sent by the second electronic device;

[0023] In response to the viewing request, the step of acquiring the first media and the motion data is continued.

[0024] In an implementation manner of the first aspect, the first communication connection includes a wifi connection or a Bluetooth connection.

[0025] In an implementation of the first aspect, generating the motion trajectory information according to the first medium and the motion data specifically includes:

[0026] Determining first track information of a first medium;

[0027] Determining second trajectory information of the motion data;

[0028] generating third trajectory information according to the first trajectory information and the second trajectory information;

[0029] Perform trajectory smoothing processing on the third trajectory information to generate motion trajectory information.

[0030] In the technical solution provided by an embodiment of the present invention, performing trajectory smoothing processing on the third trajectory information can reduce trajectory noise and improve the accuracy of generating motion trajectory information.

[0031] In an implementation manner of the first aspect, the motion data includes acceleration data and angular velocity data, and determining the second trajectory information of the motion data specifically includes: generating the second trajectory information according to the acceleration data and the angular velocity data.

[0032] In an implementation manner of the first aspect, the motion trajectory information includes horizontal trajectory information, vertical trajectory information, rotation angle information, and scale factor information.

[0033] In the technical solution provided by an embodiment of the present invention, the motion trajectory information can reflect the shaking process of the camera of the first electronic device when the user uses the first electronic device to record the first media, which can improve the accuracy of motion trajectory compensation.

[0034] In an implementation of the first aspect, determining first track information of the first media specifically includes:

[0035] The first medium is calculated by an image processing algorithm to generate first trajectory information.

[0036] In an implementation manner of the first aspect, the image processing algorithm includes an image registration algorithm or an optical flow method.

[0037] In an implementation manner of the first aspect, when the image processing algorithm includes an image registration algorithm, calculating the first medium through the image processing algorithm to generate the first trajectory information specifically includes:

[0038] Acquire first feature point information of a first video frame of a first media and second feature point information of a second video frame;

[0039] Determine a cross-power spectrum function between the first video frame and the second video frame according to the first feature point information and the second feature point information;

[0040] Performing an inverse Fourier transform on the cross power spectrum function to generate an impulse function of the second video frame;

[0041] generating a displacement parameter between the first video frame and the second video frame according to the impulse function;

[0042] Generate a rotation parameter between the first video frame and the second video frame by performing discrete Fourier transform on the first feature point information and the second feature point information;

[0043] First trajectory information is generated according to the displacement parameter and the rotation parameter.

[0044] In an implementation manner of the first aspect, generating third trajectory information according to the first trajectory information and the second trajectory information specifically includes:

[0045] Performing time synchronization processing on the first trajectory information and the second trajectory information;

[0046] The first trajectory information and the second trajectory information after synchronous processing are calculated by a Kalman filter model to generate third trajectory information.

[0047] In an implementation of the first aspect, performing time synchronization processing on the first trajectory information and the second trajectory information specifically includes:

[0048] Time synchronization processing is performed on the first trajectory information and the second trajectory information according to a first timestamp corresponding to the first trajectory information and a second timestamp corresponding to the second trajectory information.

[0049] In the technical solution provided by an embodiment of the present invention, by performing time synchronization processing on the first trajectory information and the second trajectory information, the consistency between the first trajectory information and the second trajectory information can be improved, and the accuracy of motion trajectory compensation can be further improved.

[0050] In an implementation of the first aspect, the first trajectory information and the second trajectory information are processed by a Kalman filter model to generate third trajectory information, specifically including:

[0051] generating observation data according to the displacement parameter and the rotation parameter of the first trajectory information;

[0052] using the motion data of the second trajectory information after time synchronization processing as a motion parameter;

[0053] The motion parameters are input into the motion model to generate prediction data;

[0054] Perform Kalman fusion on the observed data and the predicted data to generate fused data;

[0055] The fused data is corrected to generate third trajectory information.

[0056] In the technical solution provided by an embodiment of the present invention, the first trajectory information and the second trajectory information are processed based on a Kalman filter model. The Kalman filter model can fuse the observed data and the predicted data to reduce the error of the third trajectory information.

[0057] In an implementation manner of the first aspect, the first media includes streaming media, omnidirectional video, augmented reality (AR) video, or virtual reality (VR) video.

[0058] In a second aspect, an embodiment of the present application provides a media display method, which is applied to a second electronic device, including:

[0059] Acquire motion track information and first media captured by a camera of the first electronic device from the first electronic device;

[0060] generating a second media according to the first media and the motion trajectory information;

[0061] Display the second media.

[0062] In an implementation manner of the second aspect, before acquiring the first media and the motion trajectory information from the first electronic device, the method further includes:

[0063] A first communication connection is established with a first electronic device, and in response to a first operation input by a user, a viewing request is sent to the first electronic device, so that the first electronic device responds to the viewing request, obtains first media and motion data, and generates motion trajectory information according to the first media and the motion data.

[0064] In an implementation manner of the second aspect, the motion trajectory information includes a video frame identifier, and generating the second media according to the first media and the motion trajectory information specifically includes:

[0065] Parsing a plurality of video frames of a first media;

[0066] A motion trajectory compensation process is performed on each video frame according to the video frame identifier and the motion trajectory information of each video frame to generate a second media.

[0067] In an implementation of the second aspect, performing motion trajectory compensation processing on each video frame according to the video frame identifier and motion trajectory information of each video frame to generate the second media specifically includes:

[0068] According to each video frame identifier, each video frame is reversely processed according to the motion trajectory information to perform motion trajectory compensation on each video frame to generate a second media.

[0069] In the technical solution provided by one embodiment of the present invention, the second media generated by performing motion trajectory compensation on each video frame can be de-jittered based on the motion trajectory information, that is, the image can be motion compensated according to the shaking process of the first electronic device when shooting the first media, so as to reduce the shaking amplitude of the image displayed by the second media and improve the stability of the image of the second media.

[0070] In a third aspect, an embodiment of the present application provides a media display system, including: a first electronic device and a second electronic device;

[0071] The first electronic device is configured to obtain motion data of a first medium and the first electronic device collected by a camera of the first electronic device; generate motion track information according to the first medium and the motion data; and transmit the first medium and the motion track information to a second electronic device;

[0072] The second electronic device is used to generate and display the second media according to the first media and the motion track information.

[0073] In an implementation of the third aspect, the first electronic device includes a camera and a gesture sensor;

[0074] A camera, for recording the first media;

[0075] The gesture sensor is used to obtain motion data of the first electronic device.

[0076] In an implementation manner of the third aspect, the attitude sensor includes an inertial measurement unit IMU.

[0077] In an implementation of the third aspect, the IMU includes an accelerometer ACC and a gyroscope GYRO;

[0078] Motion data includes acceleration data and angular velocity data;

[0079] ACC is used to obtain acceleration data, GYRO is used to obtain angular velocity data; or,

[0080] Motion data includes vehicle speed, yaw angle, and steering wheel angle;

[0081] ACC is used to obtain vehicle speed, and GYRO is used to obtain yaw angle and steering wheel angle.

[0082] In a fourth aspect, an embodiment of the present application provides a media display device, including:

[0083] A first acquisition module, used to acquire motion data of a first medium and a first electronic device collected by a camera of the first electronic device;

[0084] A first generating module, used to generate motion trajectory information according to the first media and motion data;

[0085] The sending module is used to transmit the first media and the motion track information to the second electronic device, so that the second electronic device generates and displays the second media according to the first media and the motion track information.

[0086] In a fifth aspect, an embodiment of the present application provides a media display device, including:

[0087] A second acquisition module, configured to acquire motion track information and a first medium captured by a camera of the first electronic device from the first electronic device;

[0088] A second generating module, used for generating a second media according to the first media and the motion trajectory information;

[0089] The display module is used to display the second media.

[0090] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the media display method as described in the first aspect; or, the media display method as described in the second aspect.

[0091] In the seventh aspect, an embodiment of the present application provides a first electronic device, wherein the first electronic device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the steps of the media display method as described in the first aspect.

[0092] In an eighth aspect, an embodiment of the present application provides a second electronic device, wherein the second electronic device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the second electronic device is triggered to execute the steps of the media display method described in the second aspect.

Brief Description of the Drawings

[0093] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0094] Figure 1 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application;

[0095] Figure 2a A schematic diagram of a communication scenario of an electronic device provided according to an embodiment of the present application;

[0096] Figure 2b A schematic diagram of a communication scenario of another electronic device provided according to an embodiment of the present application;

[0097] Figure 2c A schematic diagram of a scenario of a media display method provided according to an embodiment of the present application;

[0098] Figure 3 A schematic diagram of obtaining first media and motion data according to an embodiment of the present application;

[0099] Figure 4 A schematic diagram of determining third trajectory information according to an embodiment of the present application;

[0100] Figure 5a A schematic diagram of motion trajectory information provided according to an embodiment of the present application;

[0101] Figure 5b A schematic diagram of another type of motion trajectory information provided according to an embodiment of the present application;

[0102] Figure 5c A schematic diagram of another type of motion trajectory information provided according to an embodiment of the present application;

[0103] Figure 6 A schematic diagram of a first media screen and a second media screen provided according to an embodiment of the present application;

[0104] Figure 7 A flowchart of a media display method provided according to an embodiment of the present application;

[0105] Figure 8 A flowchart of generating motion trajectory information according to an embodiment of the present application;

[0106] Fig. 9 A schematic diagram of first feature point information provided according to an embodiment of the present application;

[0107] Fig.10 A flowchart of generating a second media according to an embodiment of the present application;

[0108] Fig.11 A schematic diagram of the structure of a media display device provided according to an embodiment of the present application;

[0109] Fig.12 A schematic diagram of the structure of a first generation module provided according to an embodiment of the present application;

[0110] Fig.13 The figure is a schematic diagram of the structure of another media display device provided according to an embodiment of the present application. [Specific implementation method]

[0111] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0112] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0113] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0114] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0115] When a user uses an electronic device to track a high-speed moving object to shoot a video, the user's hand cannot hold the electronic device stably for shooting, and the electronic device is prone to large-scale low-frequency jitter and / or large perspective changes. The picture taken by the electronic device will produce low-frequency large-scale jitter, which will cause great discomfort to the user. The electronic device used for playback currently does not have this processing capability or does not support the ability to stabilize the video. This problem is particularly important for distributed camera tasks such as on-site multi-camera recording. The video de-jitter technology in the related art mostly uses a gyroscope to eliminate high-frequency jitter, which is very weak for low-frequency large-scale jitter. The video de-jitter technology in the related art mostly de-jitters the video that has been shot to generate a de-jittered video, and the electronic device then plays the de-jittered video, so that the user cannot watch the de-jittered video in time, and the de-jitter efficiency is low.

[0116] In a related technology, a Harris corner detection algorithm, a greedy algorithm, and feature point sub-pixelization are used to obtain a reasonable number of feature points that are evenly distributed and accurately positioned. The feature points are tracked by the pyramid LK optical flow method to achieve accurate matching of feature point pairs. The background feature points are screened out by the RANSAC algorithm and the motion parameters of the background feature points are optimized by the simplex method to improve the image stabilization effect. However, the above technical solution solves the technical problem of image video screen jitter caused by camera tremor. The above camera shake removal technology faces small-amplitude high-frequency jitter, which is generally caused by small vibrations caused by the environment (such as wind, rain, frost, snow) or mechanical structure vibrations. The vibration amplitude is mostly less than 50 mm, and the change angle is less than 10 degrees. The above technical solution has a low ability to improve the stability of the video. For video jitters with a video sequence offset amplitude greater than ±6%, it cannot be effectively eliminated.

[0117] In another related technology, during the video recording process, a frame of image is acquired through a camera, and then the image is segmented to obtain at least one object image and a background image, and then the object image is anti-shake processed to obtain an anti-shake image. After that, the background image is anti-shake processed to obtain a background anti-shake image. Finally, the target anti-shake image and the background anti-shake image are fused to obtain a fused image, and the target video is obtained based on the fused image. However, the above technical solution relies heavily on image segmentation technology, which can easily cause blurred boundaries and ghosting of the video. In addition, the above technical solution can only perform anti-shake processing on one of the object images or background images with the same depth of field in the video image, resulting in the problem that the jitter in the video image cannot be completely eliminated and the picture is distorted.

[0118] In order to solve the technical problems in the related art, an embodiment of the present application provides an electronic device.

[0119] In some embodiments, electronic devices include but are not limited to Or devices with other operating systems.

[0120] In one embodiment of the present application, the electronic device includes a first electronic device or a second electronic device. The first electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a wearable device, a car computer, a camera, etc. The second electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a wearable device, a driving recorder, etc.

[0121] In some embodiments, the first electronic device and the second electronic device may communicate with each other via wireless communication. For example, the wireless communication may include Bluetooth (BT), wifi, or near field communication (NFC).

[0122] Figure 1 FIG. 1 is a schematic diagram of a structure of an electronic device provided according to an embodiment of the present application. Figure 1 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 125, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor (GYRO for short) 180B, an air pressure sensor 180C, a magnetic sensor 180D, an accelerometer (ACC for short) 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0123] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0124] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0125] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0126] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0127] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0128] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0129] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0130] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 125, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0131] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0132] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0133] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0134] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0135] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0136] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0137] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0138] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0139] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0140] The internal memory 125 can be used to store computer executable program codes, which include instructions. The internal memory 125 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 125 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 125 and / or instructions stored in a memory provided in the processor.

[0141] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0142] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0143] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0144] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0145] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0146] In order to solve the technical problem of low stability of the video image with large-scale low-frequency jitter or large perspective conversion, based on the above-mentioned electronic device, the embodiment of the present application can combine motion data and the first media to solve the problem of large-scale jitter of the media image caused by video perspective transfer and / or camera movement, thereby greatly improving the stability of the video.

[0147] Below is a combination of Figure 2 to Figure 6 c. A detailed description of the media display method provided in the embodiment of the present application is given.

[0148] The first electronic device establishes a first communication connection with at least one second electronic device, and the second electronic device sends a viewing request to the first electronic device in response to a first operation input by a user. In one embodiment of the present application, the first communication connection may include a wifi connection or a bluetooth connection.

[0149] Figure 2a A communication scenario diagram of an electronic device provided according to an embodiment of the present application is shown in FIG. Figure 2aAs shown, when the number of second electronic devices is 2, user A holds the first electronic device 1, user B holds the second electronic device 2a, and user C holds the second electronic device 2b. At this time, when user A wants to shoot a video through the first electronic device 1 and share it with user B and user C, the first electronic device 1 establishes a first communication connection with the second electronic device 2a and the second electronic device 2b, and sends media broadcast information to the second electronic device 2a and the second electronic device 2b to indicate that user A wants to share the media with user B and user C. At this time, the second electronic device 2a of user B and the second electronic device 2b of user C receive a pop-up window, which may include a prompt, a confirmation button and a rejection button. The prompt may be "Do you want to receive the media shared by user A?" If user B wants to watch the media, the second electronic device 2a responds to the first operation input by the user, that is, user B sends a viewing request to the first electronic device 1 by clicking the "confirmation button" displayed on the second electronic device 2a. If user C does not want to watch the media, he clicks the "reject button" displayed on the second electronic device 2b, and the second electronic device 2b does not send a viewing request to the first electronic device 1.

[0150] Figure 2b A schematic diagram of a communication scenario of another electronic device provided according to an embodiment of the present application is shown in FIG. Figure 2b As shown, when the number of second electronic devices is 2, user A holds the first electronic device 1, user B holds the second electronic device 2a, and user C holds the second electronic device 2b. User A discusses the time for media sharing with user B and user C in advance, user A provides the media sharing address through the first electronic device 1, the first electronic device 1 establishes a first communication connection with the second electronic device 2a and the second electronic device 2b, the second electronic device 2a responds to the first operation input by user B, that is, user B clicks the media sharing address through the second electronic device 2a to send a viewing request to the first electronic device 1 of user A, at this time, the first electronic device 1 of user A receives a pop-up window, the pop-up window may include a prompt, a confirmation button and a rejection button, and the prompt may be "Do you agree to user B viewing the media?" The second electronic device 2b responds to the first operation input by user C, that is, user C clicks the media sharing address through the second electronic device 2b to send a viewing request to the first electronic device 1 of user A, at this time, the first electronic device 1 of user A receives a pop-up window, the pop-up window may include a prompt, a confirmation button and a rejection button, and the prompt may be "Do you agree to user C viewing the media?" In response to the viewing request, the first electronic device obtains the first media collected by the camera of the first electronic device and the motion data of the first electronic device. For example, user A clicks the "confirm button" on the first electronic device to open the camera application (Application, APP for short) to start obtaining the first media and motion data.

[0151] When the first electronic device 1 receives the viewing request sent by the second electronic device 2a, the first electronic device 1 can shoot the first media. For example, user A tracks a high-speed car with the first electronic device 1 to shoot the first media. The hand of user A cannot hold the first electronic device 1 stably for shooting, which makes the first electronic device 1 easily skewed. Figure 2c A schematic diagram of a scenario of a media display method provided according to an embodiment of the present application is shown in FIG. Figure 2c As shown, when user A tracks a high-speed car through the first electronic device 1 he holds, the first electronic device 1 tilts to the left, causing the front of the car displayed in the first electronic device 1 to tilt downward when the first electronic device 1 is placed vertically, causing the picture taken by the first electronic device 1 to produce low-frequency large-scale jitter.

[0152] The first electronic device may include a camera and a gesture sensor, the camera is used to record a first media, the first media may include streaming media, omnidirectional video, augmented reality AR video or virtual reality VR video. The gesture sensor is used to obtain motion data of the first electronic device. For example, the gesture sensor may be an inertial measurement unit (IMU), a compass, a micro-electro-mechanical system (MEMS) accelerometer or a MEMS gyroscope. Figure 1 As shown, the posture sensor may include Figure 1 In one embodiment of the present application, the IMU may obtain motion data of the first electronic device, wherein the motion data may include acceleration data obtained by the acceleration sensor 180E and angular velocity data obtained by the gyro sensor 180B. When the first electronic device is a vehicle computer, the motion data may include vehicle motion data. For example, the vehicle motion data may include: yaw rate, vehicle speed and / or steering wheel angle, the acceleration sensor 180E is used to obtain the vehicle speed, and the gyro sensor 180B is used to obtain the yaw angle and the steering wheel angle. The compass, MEMS accelerometer and MEMS gyroscope in the attitude sensor can all obtain the motion data of the first electronic device in the above manner, which will not be repeated here.

[0153] Figure 3 FIG. 1 is a schematic diagram of obtaining first media and motion data according to an embodiment of the present application. Figure 3 As shown, each frame of the first media generates motion data due to the camera's viewing angle shift and / or camera movement. Exemplarily, the motion data may include acceleration data acquired by ACC and angular velocity data acquired by GYRO.

[0154] The first electronic device generates motion trajectory information according to the first media and motion data. Specifically, the first electronic device determines the first trajectory information of the first media. In a possible implementation, the first electronic device calculates the first media through an image processing algorithm to generate the first trajectory information. The image processing algorithm may include an image registration algorithm or an optical flow method, and the image registration algorithm may include an image registration algorithm based on image features and an image registration algorithm based on image transform domain. When the image processing algorithm is an image registration algorithm, the first electronic device obtains the first feature point information of the first video frame of the first media and the second feature point information of the second video frame; the first electronic device determines the mutual power spectrum function of the first video frame and the second video frame according to the first feature point information and the second feature point information; the first electronic device performs an inverse Fourier transform on the mutual power spectrum function to generate an impulse function of the second video frame; the first electronic device generates a displacement parameter between the first video frame and the second video frame according to the impulse function; the first electronic device generates a rotation parameter between the first video frame and the second video frame by performing a discrete Fourier transform on the first feature point information and the second feature point information; the first electronic device generates the first trajectory information according to the displacement parameter and the rotation parameter. The first electronic device determines the second trajectory information of the motion data. In a possible implementation, the motion data may include acceleration data and angular velocity data, and the first electronic device generates the second trajectory information according to the acceleration data and the angular velocity data. The first electronic device generates the third trajectory information according to the first trajectory information and the second trajectory information. In a possible implementation, the first electronic device performs time synchronization processing on the first trajectory information and the second trajectory information; the first electronic device calculates the first trajectory information and the second trajectory information after synchronization processing through a Kalman filter model to generate the third trajectory information. Specifically, the first electronic device performs time synchronization processing on the first trajectory information and the second trajectory information according to the first timestamp corresponding to the first trajectory information and the second timestamp corresponding to the second trajectory information.

[0155] Figure 4 Schematic diagram of determining third trajectory information according to an embodiment of the present application. Figure 4 As shown, the first electronic device generates observation data according to the displacement parameters and rotation parameters of the first trajectory information; the first electronic device uses the motion data of the second trajectory information after time synchronization processing as the motion parameter; the first electronic device inputs the motion parameter into the motion model to generate prediction data; the first electronic device performs Kalman fusion on the observation data and the prediction data to generate fusion data; the first electronic device corrects the fusion data to generate third trajectory information. The first electronic device performs trajectory smoothing processing on the third trajectory information to generate motion trajectory information. In one embodiment of the present application, the motion trajectory information may include horizontal trajectory information, vertical trajectory information, rotation angle information, and scale factor information.

[0156] Figure 5a is a schematic diagram of motion trajectory information provided according to an embodiment of the present application, Figure 5b is a schematic diagram of another type of motion trajectory information provided according to an embodiment of the present application, Figure 5c FIG. 1 is a schematic diagram of another type of motion trajectory information provided according to an embodiment of the present application. Figure 5a As shown in , the horizontal trajectory information is represented by a line graph, the horizontal axis of the line graph is time, and the vertical axis is horizontal displacement. Figure 5b As shown in , the longitudinal trajectory information is represented by a line graph, the horizontal axis of the line graph is time, and the vertical axis is the longitudinal displacement. Figure 5c As shown, the rotation angle information is represented by a line graph, the horizontal axis of the line graph is time, and the vertical axis is the rotation angle. Among them, the solid line part is the third trajectory information, and the dotted line part is the motion trajectory information, that is, the motion trajectory information generated after the third trajectory information is smoothed. As an optional solution, when the scale factor information is 1.0, the line graph of the scale factor information is a positive proportional function image.

[0157] The first electronic device transmits the first media and the motion track information to the second electronic device.

[0158] The second electronic device generates and displays the second media based on the first media and the motion trajectory information. Specifically, the motion trajectory information may include a video frame identifier, and the second electronic device parses multiple video frames of the first media; the second electronic device performs motion trajectory compensation processing on each video frame according to the video frame identifier and motion trajectory information of each video frame to generate the second media. In one possible implementation, the second electronic device performs reverse processing on each video frame according to the motion trajectory information according to each video frame identifier to perform motion trajectory compensation on each video frame to generate the second media.

[0159] In one embodiment of the present application, a first electronic device can obtain real-time motion data of a first media and the first electronic device collected in real time by a camera of the first electronic device; the first electronic device generates motion trajectory information in real time based on the first media and the motion data; the first electronic device transmits the first media and the motion trajectory information to a second electronic device in real time, so that the second electronic device can generate and display the second media in real time based on the first media and the motion trajectory information.

[0160] In one embodiment of the present application, the first electronic device can display the first media collected by the camera of the first electronic device, and the first electronic device transmits the first media and motion track information to the second electronic device. The second electronic device generates and displays the second media according to the first media and motion track information. Figure 6 is a schematic diagram of a first media screen and a second media screen provided according to an embodiment of the present application, Figure 6include Figure 6 a to Figure 6 f, such as Figure 6 As shown, Figure 6 The image on the left is the image of the first media, that is, Figure 6 a. Figure 6 b and Figure 6 c, the image on the right is the second media screen, i.e. Figure 6 d. Figure 6 e and Figure 6 f.

[0161] In one embodiment of the present application, a frame whose frame identifier is a first frame identifier in a first media displayed by a first electronic device is different from a frame whose frame identifier is also a first frame identifier in a second media displayed by a second electronic device.

[0162] The first media is video streaming, Figure 6 The image on the left is three consecutive frames of the first media. Figure 6 a is the first frame of the first media, Figure 6 b is the second frame of the first media, Figure 6 c is the third frame of the first media. When the user shoots the first media through the first electronic device, in order to track the object in the video, the perspective shift and / or camera movement of the first electronic device when the user holds the first electronic device causes the first media image to shake significantly. The ground plane on the first media image is horizontal in real life. The bridge pillars and the trunks of the trees are perpendicular to the ground. The car is driving on the ground plane. Figure 6 As shown in Figure a, the first frame of the first media shows images of five large trees, a bridge pillar and a car. The user uses the first electronic device to track and shoot the car in the video. Due to the high speed of the car, the user cannot track and shoot the car steadily while holding the first electronic device. Due to the tilt of the first electronic device, the ground plane shown in the first frame of the first media shot by the first electronic device tilts significantly to the lower right. Figure 6 As shown in b, the second frame of the first media shows that the car has reached the bridge pillar, and the ground plane is tilted to the lower right less. Figure 6 As shown in c, in the third frame of the first media, due to the shift of the camera's viewing angle, after the car passes the bridge pillar, the bridge pillar can no longer be seen in the third frame of the first media, and at this time, the ground plane tilts significantly to the upper right.

[0163] In one embodiment of the present application, the second media is a video stream. Figure 6 The image on the right side of the middle is three consecutive frames of the second media. Figure 6 d is the first frame of the second media, Figure 6 e is the second frame of the second media, Figure 6f is the third frame of the second media. Figure 6 As shown in FIG. 4 , the first frame of the second media shows five large trees, a bridge pillar, and a car. Figure 6 Compared with a, Figure 6 In the first frame of the second media in d, the ground plane shown in the image is tilted less to the lower right. Figure 6 As shown in FIG. 4 , the second frame of the second media shows that the car has reached the bridge pillar. Figure 6 Compared with the second frame of the first media in b, the ground plane is less inclined to the lower right. Figure 6 As shown in FIG. 5 , due to the camera's viewing angle shift, after the car passes the bridge pillar, the bridge pillar can no longer be seen in the third frame of the second media. Figure 6 c, the ground plane is less inclined to the upper right, so that the jitter of the second media image received by the second electronic device is reduced, greatly improving the stability of the video. In a possible implementation, the first electronic device displays Figure 6 When the first frame of the first media in a is displayed, the second electronic device synchronously displays Figure 6 d, the first electronic device displays the first frame of the second media Figure 6 When the second frame of the first media in b is displayed, the second electronic device synchronously displays Figure 6 e, the first electronic device displays a second frame of the second media Figure 6 When the third frame of the first media in c is displayed, the second electronic device synchronously displays Figure 6 The third frame of the second media in f. From the above content, it can be seen that the jitter amplitude of the first media image is greater than the jitter amplitude of the second media image. The technical solution provided by this application can reduce the jitter of the first media image and improve the stability of the video.

[0164] Figure 6 The schematic diagram of the first media screen and the second media screen shown are exemplary displays of an embodiment of the present application. The schematic diagram of the first media screen and the schematic diagram of the second media screen may also be in other styles, which is not limited in the embodiment of the present application.

[0165] To implement the above technical solution, the following is a specific embodiment to illustrate the workflow of the media display method. An embodiment of the present application provides a media display method, Figure 7 A flow chart of a media display method provided according to an embodiment of the present application is shown in FIG. Figure 7 As shown, the method includes:

[0166] Step 102: The first electronic device establishes a first communication connection with at least one second electronic device, and receives a viewing request sent by the second electronic device.

[0167] In one possible implementation, a first electronic device establishes a first communication connection with at least one second electronic device, the first electronic device sends media broadcast information to each second electronic device, and each second electronic device can send a viewing request to the first electronic device through a first operation input by a user in response to the media broadcast information.

[0168] In one possible implementation, a first electronic device establishes a first communication connection with at least one second electronic device, and the first electronic device sends a media sharing address to at least one second electronic device. Each second electronic device can send a viewing request to the first electronic device based on the media sharing address through a first operation input by a user.

[0169] Step 104: The first electronic device obtains the first media and the motion data of the first electronic device collected by the camera of the first electronic device in response to the viewing request.

[0170] In one embodiment of the present application, the motion data may include acceleration data and angular velocity data. A camera and a gesture sensor are installed in the first electronic device, the camera acquires the first media, and the gesture sensor acquires the acceleration data and angular velocity data.

[0171] In a possible implementation manner, the first electronic device may display the first media captured by a camera of the first electronic device.

[0172] Step 106: The first electronic device generates motion track information according to the first media and the motion data.

[0173] Specifically, Figure 8 A flowchart of generating motion trajectory information according to an embodiment of the present application is provided, such as Figure 8 As shown, step 106 includes:

[0174] Step 1062: The first electronic device obtains first feature point information of a first video frame and second feature point information of a second video frame of the first media.

[0175] In one embodiment of the present application, the first feature point information may include a window corner point of a group of pictures (Group of Pictures, GOP for short).

[0176] Fig. 9 is a schematic diagram of first feature point information provided according to an embodiment of the present application, such as Fig. 9 As shown, when the first video frame of the first media is Figure 5aWhen the first feature point information is a plurality of feature points in a picture of a first video frame of the first media.

[0177] Step 1064: The first electronic device determines a cross-power spectrum function between the first video frame and the second video frame according to the first feature point information and the second feature point information.

[0178] Specifically, if the first video frame is f[x, y] and the second video frame after displacement is g[x, y], then g[x, y] = f[x+x0, y+y0], that is, assuming that the second video frame is displaced by x0 in the horizontal direction and y0 in the vertical direction. According to the displacement invariance of Fourier transform, the frequency domain function of g[x, y] is G[u, v] = F[u, v]e j2π(ux0+vy0) Then the cross power spectrum function between the first video frame f[x, y] and the second video frame g[x, y] is:

[0179] Step 1066: The first electronic device performs an inverse Fourier transform on the cross-power spectrum function to generate an impulse function of the second video frame.

[0180] Specifically, performing an inverse Fourier transform on the cross power spectrum function can generate an impulse function m[x, y] of the second video frame (x+x0, y+y0), where m[x, y]=δ[x+x0, y+y0].

[0181] Step 1068: The first electronic device generates a displacement parameter between the first video frame and the second video frame according to the impulse function.

[0182] In one embodiment of the present application, the first electronic device may generate a displacement parameter (x0, y0) between the first video frame and the second video frame according to the impulse function δ[x+x0, y+y0].

[0183] Step 1070: The first electronic device generates a rotation parameter between the first video frame and the second video frame by performing a discrete Fourier transform on the first feature point information and the second feature point information.

[0184] Specifically, assume that the second video frame g[x, y] is the video frame after the first video frame f[x, y] is rotated by θ0, that is, g[x, y]=f[xcosθ0+ysinθ0, xcosθ0-ysinθ0]. Then in polar coordinates, the relationship between the second video frame g[x, y] and the first video frame f[x, y] is g[x, y]=g[ρ, θ]=f[ρ, θ+θ0]. Then, the discrete Fourier transform of the second video frame g[x, y] in polar coordinates is G[ω, φ]=F[ω, φ+θ0], and it can be concluded that the rotation parameter between the first video frame and the second video frame is θ0.

[0185] Step 1072: The first electronic device generates first trajectory information according to the displacement parameter and the rotation parameter.

[0186] Specifically, the first electronic device generates first trajectory information according to the displacement parameter (x0, y0) and the rotation parameter θ0.

[0187] Step 1074: The first electronic device generates second trajectory information according to the acceleration data and the angular velocity data.

[0188] Specifically, the first electronic device generates the second trajectory information according to the acceleration data and the angular velocity data acquired in step 104 .

[0189] Step 1076: The first electronic device performs time synchronization processing on the first trajectory information and the second trajectory information according to the first timestamp corresponding to the first trajectory information and the second timestamp corresponding to the second trajectory information.

[0190] In one embodiment of the present application, it is necessary to time-align the first media and the motion data, and synchronize the first media and the motion data corresponding to the same time. Since the time interval of each video frame is relatively long, it is not sensitive to the time difference between the motion data and the first media, so the system timestamp can be recorded while acquiring the first media, and the system timestamp can also be recorded when acquiring the motion data. Then, the first electronic device can perform time synchronization processing on the first trajectory information and the second trajectory information according to the first timestamp corresponding to the first trajectory information and the second timestamp corresponding to the second trajectory information.

[0191] Step 1078: The first electronic device generates observation data according to the displacement parameters and rotation parameters of the first trajectory information.

[0192] In one embodiment of the present application, steps 1078 to 1086 are specific steps of processing the first trajectory information and the second trajectory information through a Kalman filter model to generate the third trajectory information.

[0193] Specifically, the first electronic device performs filtering processing on the displacement parameter and the rotation parameter of the first trajectory information to generate observation data.

[0194] Step 1080: The first electronic device uses the motion data of the second trajectory information after time synchronization processing as a motion parameter.

[0195] Step 1082: The first electronic device inputs the motion parameters into the motion model to generate prediction data.

[0196] In one embodiment of the present application, the prediction data is a predicted value of the motion state of the current camera predicted by the motion state of the camera in the previous video frame.

[0197] In one embodiment of the present application, the motion parameters may include position data, speed data, direction data, and angle data of the current video frame of each video frame in the first media.

[0198] In one embodiment of the present application, the motion model may include a position calculation formula, a speed calculation formula, and a direction calculation formula. The position calculation formula is P k =P k-1 +Δtv k-1 +(Δt 2 / 2)(C ns f k-1 -g), P k is the position data of the predicted video frame, P k-1 is the position data of the current video frame, Δt is the time difference between the predicted video frame and the current video frame, g is the gravitational acceleration, and f k-1 Characterizes the specific force of k-1 steps, that is, the acceleration exerted by external forces other than gravity, C ns is the rotation change matrix relative to the previous step. Based on the rotation change matrix, the posture change of the first electronic device at the current (kth step) can be inferred. The speed calculation formula is V k =V k-1 +Δt(C ns f k-1 -g), V k is the speed data of the predicted video frame, V k-1 is the speed data of the current video frame. The direction calculation formula is q k =Ω(q(ω k-1 Δt))q k-1 ,q k is the direction data of the predicted video frame, q k-1 is the direction data of the current video frame, ω k-1 is the angle data of the current video frame, and Ω is used to characterize the quaternion change calculation process.

[0199] Step 1084: The first electronic device performs Kalman fusion on the observed data and the predicted data to generate fused data.

[0200] Step 1086: The first electronic device corrects the fused data to generate third trajectory information.

[0201] In one possible implementation, the first electronic device corrects the fused data based on the Kalman coefficient and the residual weight, updates the estimate of the state variable according to the predicted data obtained by using the physical model at the previous moment and the observed data obtained from the image in the first media at the current moment, and calculates the estimated value at the current moment to generate third trajectory information.

[0202] Step 1088: The first electronic device performs trajectory smoothing processing on the third trajectory information to generate motion trajectory information.

[0203] Specifically, the first electronic device performs trajectory smoothing processing on the third trajectory information by using a trajectory smoothing algorithm to generate motion trajectory information.

[0204] Step 108: The first electronic device transmits the first media and the motion track information to the second electronic device.

[0205] In one embodiment of the present application, when the first electronic device is a vehicle computer, the motion data may include vehicle motion data. For example, the vehicle motion data may include: yaw rate, vehicle speed and / or steering wheel angle, which can estimate the vehicle's forward direction and forward speed, thereby sending the vehicle's actual motion trajectory information to the second electronic device to achieve a stable video image effect.

[0206] In the related art, the first electronic device performs de-jitter processing on the captured video to generate a de-jitter video. At this time, the first electronic device can watch the de-jitter video, so that the user cannot watch the de-jitter video in time, and the de-jitter efficiency is low.

[0207] In one possible implementation, a first electronic device generates motion trajectory information in real time based on first media and motion data; the first electronic device transmits the first media and motion trajectory information to a second electronic device in real time, so that the second electronic device can generate and display second media in real time based on the first media and motion trajectory information. This allows the first electronic device to synchronously display the first media after de-jittering, i.e., the second media, while shooting the first media, thereby improving de-jittering efficiency.

[0208] In the technical solution provided in one embodiment of the present application, by synchronously transmitting the motion trajectory information and the first media, real-time and stable media transmission can be achieved, and through the following algorithm, large-scale jitter in the picture of the first media can be automatically eliminated.

[0209] Step 110: The second electronic device generates and displays the second media according to the first media and the motion trajectory information. Specifically, Fig.10 A flowchart of generating a second media according to an embodiment of the present application is provided, such as Fig.10 As shown, step 110 includes:

[0210] Step 1102: The second electronic device parses multiple video frames of the first media.

[0211] Step 1104: The second electronic device performs reverse processing on each video frame according to the motion trajectory information according to each video frame identifier, so as to perform motion trajectory compensation on each video frame and generate a second media.

[0212] In one embodiment of the present application, a frame whose frame identifier is a first frame identifier in a first media displayed by a first electronic device is different from a frame whose frame identifier is also a first frame identifier in a second media displayed by a second electronic device.

[0213] In one embodiment of the present application, the motion trajectory information may include a video frame identifier, and the video frame identifier may include horizontal trajectory information, vertical trajectory information, rotation angle information and scale factor information. In a possible implementation, the video frame identifier is shown in Table 1 below.

[0214] Table 1

[0215]

[0216] The displacement change and rotation angle change between two frames of the first medium are recorded in a data packet, and the values ​​in the data packet can be represented by decimal, wherein the displacement change can be represented by horizontal trajectory information and longitudinal trajectory information, and the rotation angle change can be represented by rotation angle information. As shown in Table 1 above, the motion trajectory information identity document (Id for short) is 1, the frame ID is 5, the horizontal trajectory information (13 bits) is 0 mm, the longitudinal trajectory information (13 bits) is -30 mm, the rotation angle information (48 bits) is 0.010 degrees, i.e. 10 / 1000 degrees, the scale factor information (12 bits) is 1.0, and the reserved bits are 10 bits.

[0217] For example, if the motion trajectory information is a horizontal movement of 10 pixels to the right, a vertical movement of 2 pixels downward, and a rotation of 2° to the right, then the reverse processing is a horizontal movement of 10 pixels to the left, a vertical movement of 2 pixels upward, and a rotation of 2° to the left. In the related art, the second electronic device parses multiple video frames of the first media, the second electronic device decodes each video frame, and the second electronic device plays the decoded multiple video frames in sequence according to each video frame identifier. At this time, the video played by the second electronic device is the first media, and the jitter amplitude of the first media picture is relatively large.

[0218] In one embodiment of the present application, the second electronic device parses multiple video frames of the first media, and the second electronic device decodes each video frame. The second electronic device in step 1104 performs reverse processing on each decoded video frame according to the motion trajectory information according to each video frame identifier, that is, the process of performing motion trajectory compensation on each video frame can be set between video decoding and video playback, and each decoded video frame is reversely processed according to the motion trajectory information to generate the second media. At this time, the video played by the second electronic device is the second media, and the jitter amplitude of the second media picture is smaller than the jitter amplitude of the first media picture. Since the first electronic device generates the motion trajectory information based on the motion data of the first electronic device and the first media, the second electronic device performs motion compensation on the first media according to the motion trajectory information obtained from the first electronic device and the first media, that is, the picture can be motion compensated according to the jitter process of the first electronic device when shooting the first media, so as to reduce the jitter amplitude of the picture displayed by the second media and improve the stability of the picture of the second media.

[0219] In the technical solution provided in one embodiment of the present application, a stable second media can be obtained in real time, ensuring a smooth and stable video viewing experience. The video transmission and video encoding and decoding adopt industry standard methods, and the motion trajectory compensation process only occurs between the video decoding and video playback. Different from the motion trajectory compensation in the encoding and decoding method, the motion trajectory compensation in the present application focuses on the displacement and rotation caused by large-scale jitter, wherein the motion trajectory compensation method is to perform reverse translation changes on each video frame based on the displacement on the X-axis and Y-axis (height and width directions), and rotate the Z-axis (vertical to the plane) according to the rotation angle, and upsample or downsample the image according to whether the aperture is enlarged or reduced, wherein no resampling is required if the aperture does not change, and the video frame compensated by the motion trajectory is provided to the second electronic device as a playback video frame (second media), and the second electronic device displays the second media.

[0220] In the technical solution provided in the embodiment of the present application, the motion data of the first media and the first electronic device collected by the camera of the first electronic device are obtained; motion trajectory information is generated according to the first media and the motion data; the first media and the motion trajectory information are transmitted to the second electronic device, so that the second electronic device can generate and display the second media according to the first media and the motion trajectory information. In the technical solution provided in the embodiment of the present application, the first electronic device synchronously transmits the motion trajectory information and the first media to the second electronic device, and the second electronic device generates and displays the second media according to the first media and the motion trajectory information, which can reduce the picture jitter caused by the large-scale low-frequency jitter of the first electronic device, and reduce the picture jitter caused by the large viewing angle conversion of the first electronic device, thereby improving the stability of the video picture and providing a better visual experience.

[0221] An embodiment of the present application provides a media display system, the system comprising: a first electronic device and a second electronic device. The first electronic device is connected to the second electronic device.

[0222] The first electronic device is used to obtain the first media and motion data of the first electronic device collected by the camera of the first electronic device; generate motion track information according to the first media and the motion data; and transmit the first media and the motion track information to the second electronic device.

[0223] The second electronic device is used to generate and display the second media according to the first media and the motion track information.

[0224] In one embodiment of the present application, the first electronic device includes a camera and a gesture sensor. The camera is used to record the first media, and the gesture sensor is used to obtain motion data of the first electronic device.

[0225] In one embodiment of the present application, the attitude sensor includes an inertial measurement unit IMU.

[0226] In one embodiment of the present application, the IMU includes an accelerometer ACC and a gyroscope GYRO; the motion data includes acceleration data and angular velocity data; ACC is used to obtain acceleration data, and GYRO is used to obtain angular velocity data; or, the motion data includes vehicle speed, yaw angle and steering wheel angle; ACC is used to obtain vehicle speed, and GYRO is used to obtain yaw angle and steering wheel angle.

[0227] In the technical solution provided in the embodiment of the present application, the motion data of the first media and the first electronic device collected by the camera of the first electronic device are obtained; motion trajectory information is generated according to the first media and the motion data; the first media and the motion trajectory information are transmitted to the second electronic device, so that the second electronic device can generate and display the second media according to the first media and the motion trajectory information. In the technical solution provided in the embodiment of the present application, the first electronic device synchronously transmits the motion trajectory information and the first media to the second electronic device, and the second electronic device generates and displays the second media according to the first media and the motion trajectory information, which can reduce the picture jitter caused by the large-scale low-frequency jitter of the first electronic device, and reduce the picture jitter caused by the large viewing angle conversion of the first electronic device, thereby improving the stability of the video picture and providing a better visual experience.

[0228] An embodiment of the present application provides a media display device, Fig.11 FIG. 1 is a schematic diagram of a structure of a media display device provided according to an embodiment of the present application. Fig.11 As shown, the device includes: a first acquisition module 11, a first generation module 12 and a sending module 13.

[0229] The first acquisition module 11 is used to acquire the first media and the motion data of the first electronic device collected by the camera of the first electronic device.

[0230] The first generating module 12 is used to generate motion track information according to the first medium and the motion data.

[0231] The sending module 13 is used to transmit the first media and the motion track information to the second electronic device, so that the second electronic device can generate and display the second media according to the first media and the motion track information.

[0232] In one embodiment of the present application, the motion trajectory information includes a video frame identifier, so that when the second electronic device generates the second media according to the first media and the motion trajectory information, the second electronic device generates the second media after performing motion trajectory compensation processing on each video frame according to the video frame identifier of each video frame and the motion trajectory information corresponding to each video frame identifier.

[0233] In one embodiment of the present application, the first acquisition module 11 may be used to acquire real-time motion data of the first media and the first electronic device collected in real time by a camera of the first electronic device.

[0234] The first generating module 12 may be configured to generate motion track information in real time according to the first media and motion data.

[0235] The sending module 13 may be used to transmit the first media and the motion track information to the second electronic device in real time, so that the second electronic device can generate and display the second media in real time according to the first media and the motion track information.

[0236] In one embodiment of the present application, the device further includes: a display module 14 .

[0237] The display module 14 is used to display the first media collected by the camera of the first electronic device.

[0238] In one embodiment of the present application, a frame whose frame identifier is a first frame identifier in a first media displayed by a first electronic device is different from a frame whose frame identifier is also a first frame identifier in a second media displayed by a second electronic device.

[0239] In one embodiment of the present application, the device further includes: a receiving module 15.

[0240] The receiving module 15 is used to establish a first communication connection with at least one second electronic device, receive a viewing request sent by the second electronic device, and in response to the viewing request, trigger the first obtaining module 11 to continue to execute the step of obtaining the first media and motion data.

[0241] In one embodiment of the present application, the first communication connection includes a wifi connection or a Bluetooth connection.

[0242] In one embodiment of the present application, Fig.12 This is a schematic diagram of the structure of a first generation module provided according to an embodiment of the present application. The first generation module 12 includes: a first determination submodule 121, a second determination submodule 122, a first generation submodule 123 and a second generation submodule 124.

[0243] The first determining submodule 121 is used to determine first track information of a first medium.

[0244] The second determining submodule 122 is used to determine second trajectory information of the motion data.

[0245] The first generating submodule 123 is used to generate third trajectory information according to the first trajectory information and the second trajectory information.

[0246] The second generating submodule 124 is used to perform trajectory smoothing processing on the third trajectory information to generate motion trajectory information.

[0247] In an embodiment of the present application, the motion data includes acceleration data and angular velocity data, and the second determination submodule 122 is specifically configured to generate the second trajectory information according to the acceleration data and the angular velocity data.

[0248] In one embodiment of the present application, the motion trajectory information includes horizontal trajectory information, vertical trajectory information, rotation angle information and scale factor information.

[0249] In an embodiment of the present application, the first determining submodule 121 is specifically configured to calculate the first media through an image processing algorithm to generate first trajectory information.

[0250] In one embodiment of the present application, the image processing algorithm includes an image registration algorithm or an optical flow method.

[0251] In one embodiment of the present application, when the image processing algorithm includes an image registration algorithm, the first determination submodule 121 is specifically used to obtain first feature point information of a first video frame and second feature point information of a second video frame of a first media; determine the mutual power spectrum function of the first video frame and the second video frame based on the first feature point information and the second feature point information; perform an inverse Fourier transform on the mutual power spectrum function to generate an impulse function of the second video frame; generate a displacement parameter between the first video frame and the second video frame based on the impulse function; generate a rotation parameter between the first video frame and the second video frame by performing a discrete Fourier transform on the first feature point information and the second feature point information; and generate first trajectory information based on the displacement parameter and the rotation parameter.

[0252] In one embodiment of the present application, the first generating submodule 123 is specifically used to perform time synchronization processing on the first trajectory information and the second trajectory information; calculate the first trajectory information and the second trajectory information after the synchronization processing through the Kalman filter model to generate the third trajectory information.

[0253] In one embodiment of the present application, the first generating submodule 123 is specifically configured to perform time synchronization processing on the first trajectory information and the second trajectory information according to a first timestamp corresponding to the first trajectory information and a second timestamp corresponding to the second trajectory information.

[0254] In one embodiment of the present application, the first generation submodule 123 is specifically used to generate observation data based on the displacement parameters and rotation parameters of the first trajectory information; use the motion data of the second trajectory information after time synchronization as motion parameters; input the motion parameters into the motion model to generate predicted data; perform Kalman fusion on the observation data and the predicted data to generate fused data; and correct the fused data to generate third trajectory information.

[0255] In one embodiment of the present application, the first media includes streaming media, omnidirectional video, augmented reality AR video or virtual reality VR video.

[0256] In the technical solution provided in the embodiment of the present application, the motion data of the first media and the first electronic device collected by the camera of the first electronic device are obtained; motion trajectory information is generated according to the first media and the motion data; the first media and the motion trajectory information are transmitted to the second electronic device, so that the second electronic device can generate and display the second media according to the first media and the motion trajectory information. In the technical solution provided in the embodiment of the present application, the first electronic device synchronously transmits the motion trajectory information and the first media to the second electronic device, and the second electronic device generates and displays the second media according to the first media and the motion trajectory information, which can reduce the picture jitter caused by the large-scale low-frequency jitter of the first electronic device, and reduce the picture jitter caused by the large viewing angle conversion of the first electronic device, thereby improving the stability of the video picture and providing a better visual experience.

[0257] An embodiment of the present application provides another media display device, Fig.13 FIG. 1 is a schematic diagram of the structure of another media display device provided according to an embodiment of the present application. Fig.13 As shown, the device includes: a second acquisition module 21, a second generation module 22 and a display module 23.

[0258] The second acquisition module 21 is used to acquire motion track information and a first medium captured by a camera of the first electronic device from the first electronic device.

[0259] The second generating module 22 is used to generate the second media according to the first media and the motion track information.

[0260] The display module 23 is used to display the second media.

[0261] In one embodiment of the present application, the device further includes a communication module 24 .

[0262] The communication module 24 is used to establish a first communication connection with the first electronic device, and send a viewing request to the first electronic device in response to a first operation input by a user.

[0263] In one embodiment of the present application, the motion trajectory information includes a video frame identifier, and the second generation module 22 is specifically used to parse multiple video frames of the first media; perform motion trajectory compensation processing on each video frame according to the video frame identifier and motion trajectory information of each video frame to generate the second media.

[0264] In one embodiment of the present application, the second generating module 22 is specifically configured to perform reverse processing on each video frame according to the motion trajectory information according to each video frame identifier, so as to perform motion trajectory compensation on each video frame and generate the second media.

[0265] In the technical solution provided in the embodiment of the present application, the motion data of the first media and the first electronic device collected by the camera of the first electronic device are obtained; motion trajectory information is generated according to the first media and the motion data; the first media and the motion trajectory information are transmitted to the second electronic device, so that the second electronic device can generate and display the second media according to the first media and the motion trajectory information. In the technical solution provided in the embodiment of the present application, the first electronic device synchronously transmits the motion trajectory information and the first media to the second electronic device, and the second electronic device generates and displays the second media according to the first media and the motion trajectory information, which can reduce the picture jitter caused by the large-scale low-frequency jitter of the first electronic device, and reduce the picture jitter caused by the large viewing angle conversion of the first electronic device, thereby improving the stability of the video picture and providing a better visual experience.

[0266] The embodiment of the present application provides a first electronic device, which can be a terminal device or a circuit device built into the terminal device. The first electronic device includes: a display screen; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the device can be used to perform each step in the above-mentioned media display method embodiment.

[0267] The embodiment of the present application provides a second electronic device, which can be a terminal device or a circuit device built into the terminal device. The second electronic device includes: a display screen; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the device can be used to perform each step in the above-mentioned media display method embodiment.

[0268] An embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the steps in the above-mentioned media display method embodiment.

[0269] The embodiment of the present application provides a computer program product including instructions. When the computer program product is executed on a computer or any at least one processor, the computer is enabled to execute each step in the above-mentioned media display method embodiment.

[0270] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0271] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0272] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0273] In several embodiments provided in the present application, any function can be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0274] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A media display method, characterized in that: The method is applied to a first electronic device, comprising: Acquire motion data of a first medium and the first electronic device collected by a camera of the first electronic device; generating motion trajectory information according to the first medium and the motion data; The first media and the motion track information are transmitted to a second electronic device, so that the second electronic device generates and displays a second media according to the first media and the motion track information.

2. The method according to claim 1, characterized in that: The motion trajectory information includes a video frame identifier, so that when the second electronic device generates the second media based on the first media and the motion trajectory information, the second electronic device can generate the second media after performing motion trajectory compensation processing on each video frame based on the video frame identifier of each video frame and the motion trajectory information corresponding to each video frame identifier.

3. The method according to claim 1, characterized in that Also includes: Acquire real-time motion data of the first media and the first electronic device collected in real time by the camera of the first electronic device; generating motion trajectory information in real time according to the first medium and the motion data; The first media and the motion track information are transmitted to a second electronic device in real time, so that the second electronic device can generate and display a second media in real time according to the first media and the motion track information.

4. The method according to claim 1, characterized in that: Also includes: The first media captured by the camera of the first electronic device is displayed.

5. The method according to claim 4, characterized in that The frame whose frame identifier is the first frame identifier in the first media displayed by the first electronic device is different from the frame whose frame identifier is also the first frame identifier in the second media displayed by the second electronic device.

6. The method according to claim 1, characterized in that Before acquiring the first media and the motion data, the method further includes: Establishing a first communication connection with at least one of the second electronic devices, and receiving a viewing request sent by the second electronic device; In response to the viewing request, the step of obtaining the first media and the motion data is continued.

7. The method according to claim 6, characterized in that The first communication connection includes a Wi-Fi connection or a Bluetooth connection.

8. The method according to claim 1, characterized in that The generating of the motion trajectory information according to the first medium and the motion data specifically includes: Determining first trajectory information of the first media; Determining second trajectory information of the motion data; generating third trajectory information according to the first trajectory information and the second trajectory information; Perform trajectory smoothing processing on the third trajectory information to generate motion trajectory information.

9. The method according to claim 8, characterized in that The motion data includes acceleration data and angular velocity data, and determining the second trajectory information of the motion data specifically includes: generating the second trajectory information according to the acceleration data and the angular velocity data.

10. The method according to claim 1, 2, 3 or 8, characterized in that: The motion track information includes horizontal track information, vertical track information, rotation angle information and scale factor information.

11. The method according to claim 9, characterized in that The determining the first track information of the first media specifically includes: The first medium is calculated by an image processing algorithm to generate first trajectory information.

12. The method according to claim 11, characterized in that The image processing algorithm includes an image registration algorithm or an optical flow method.

13. The method according to claim 12, characterized in that When the image processing algorithm includes an image registration algorithm, the step of calculating the first medium using the image processing algorithm to generate the first trajectory information specifically includes: Acquire first feature point information of a first video frame and second feature point information of a second video frame of the first media; Determine a cross-power spectrum function between the first video frame and the second video frame according to the first feature point information and the second feature point information; Performing an inverse Fourier transform on the cross power spectrum function to generate an impulse function of the second video frame; generating a displacement parameter between the first video frame and the second video frame according to the impulse function; Generate a rotation parameter between the first video frame and the second video frame by performing discrete Fourier transform on the first feature point information and the second feature point information; First trajectory information is generated according to the displacement parameter and the rotation parameter.

14. The method according to claim 13, characterized in that The generating the third trajectory information according to the first trajectory information and the second trajectory information specifically includes: Performing time synchronization processing on the first trajectory information and the second trajectory information; The first trajectory information and the second trajectory information after synchronization are calculated through a Kalman filter model to generate third trajectory information.

15. The method according to claim 14, characterized in that The performing time synchronization processing on the first trajectory information and the second trajectory information specifically includes: Time synchronization processing is performed on the first trajectory information and the second trajectory information according to a first timestamp corresponding to the first trajectory information and a second timestamp corresponding to the second trajectory information.

16. The method according to claim 15, characterized in that The processing of the first trajectory information and the second trajectory information by using a Kalman filter model to generate third trajectory information specifically includes: generating observation data according to the displacement parameter and the rotation parameter of the first trajectory information; using the motion data of the second trajectory information after time synchronization processing as motion parameters; Inputting the motion parameters into a motion model to generate prediction data; Performing Kalman fusion on the observed data and the predicted data to generate fused data; The fused data is corrected to generate third trajectory information.

17. The method according to claim 1, characterized in that The first media includes streaming media, omnidirectional video, augmented reality (AR) video or virtual reality (VR) video.

18. A media display method, characterized in that: The method is applied to a second electronic device, comprising: Acquire motion trajectory information and first media captured by a camera of the first electronic device from a first electronic device; generating a second medium according to the first medium and the motion trajectory information; The second media is displayed.

19. The method according to claim 18, characterized in that Before acquiring the first media and the motion track information from the first electronic device, the method further includes: Establishing a first communication connection with the first electronic device; In response to a first operation input by a user, a viewing request is sent to the first electronic device, so that the first electronic device responds to the viewing request, obtains first media and motion data, and generates motion trajectory information according to the first media and the motion data.

20. The method according to claim 18, characterized in that The motion trajectory information includes a video frame identifier, and generating the second media according to the first media and the motion trajectory information specifically includes: parsing a plurality of video frames of the first media; A motion trajectory compensation process is performed on each video frame according to the video frame identifier of each video frame and the motion trajectory information to generate a second media.

21. The method according to claim 20, characterized in that The performing motion trajectory compensation processing on each video frame according to the video frame identifier of each video frame and the motion trajectory information to generate the second media specifically includes: According to each video frame identifier, each video frame is reversely processed according to the motion trajectory information to perform motion trajectory compensation on each video frame to generate a second media.

22. A media display system, characterized in that: include: a first electronic device and a second electronic device; The first electronic device is used to obtain motion data of a first medium and the first electronic device collected by a camera of the first electronic device; generate motion track information according to the first medium and the motion data; and transmit the first medium and the motion track information to the second electronic device; The second electronic device is used to generate and display a second medium according to the first medium and the motion trajectory information.

23. The system according to claim 22, characterized in that The first electronic device includes a camera and a gesture sensor; The camera is used to record the first media; The posture sensor is used to obtain the motion data of the first electronic device.

24. The system according to claim 23, characterized in that The attitude sensor includes an inertial measurement unit IMU.

25. The system according to claim 24, characterized in that The IMU includes an accelerometer ACC and a gyroscope GYRO; The motion data includes acceleration data and angular velocity data; The ACC is used to obtain the acceleration data, and the GYRO is used to obtain the angular velocity data; or, The motion data includes vehicle speed, yaw angle and steering wheel angle; The ACC is used to obtain the vehicle speed, and the GYRO is used to obtain the yaw angle and the steering wheel angle.

26. A media display device, characterized in that: include: A first acquisition module, configured to acquire motion data of a first medium and the first electronic device collected by a camera of the first electronic device; A first generating module, configured to generate motion trajectory information according to the first media and the motion data; The sending module is used to transmit the first media and the motion track information to a second electronic device, so that the second electronic device can generate and display a second media according to the first media and the motion track information.

27. A media display device, characterized in that: include: A second acquisition module, configured to acquire motion track information and a first medium captured by a camera of the first electronic device from the first electronic device; A second generating module, configured to generate a second medium according to the first medium and the motion trajectory information; The display module is used to display the second medium.

28. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the media display method according to any one of claims 1 to 17; or the media display method according to any one of claims 18 to 21.

29. A first electronic device, characterized in that: The first electronic device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the steps of the media display method as described in any one of claims 1 to 17.

30. A second electronic device, characterized in that: The second electronic device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the second electronic device is triggered to execute the steps of the media display method as described in any one of claims 18 to 21.

Images