Video playback method, device, electronic device and storage medium

Abstract

The present disclosure relates to a video playback method, device, electronic device and storage medium, the method comprising: displaying a first-perspective picture of the current video on a preset page; displaying a second-perspective picture of the current video on a preset page in response to a perspective adjustment instruction triggered by a posture adjustment of a mobile device; the second-perspective picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video. The embodiment of the present application can adjust the viewing angle of the current video by adjusting the posture of the mobile device without performing gesture operations on the mobile phone screen, which can not only achieve conflict-free coexistence with the general gestures of some existing players, but also can achieve switching without screen occlusion when switching perspectives because there is no need to perform operations on the screen, thereby improving the quality of video playback.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a video playback method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the current mobile Internet, the dissemination of information based on mobile terminals has become more and more mature. Usually, information can be disseminated through various browsers or applications on the terminal.

[0003] Video is a medium for exchanging and interacting with information and other resources. In order to provide viewers with a good viewing experience, video-related settings are becoming more and more intelligent. However, videos are usually captured by a camera and are streamed. Then, users can only watch a video stream from one perspective, resulting in low playback quality of videos produced with a lot of resources, which cannot meet the needs of users with different viewing needs. Summary of the invention

[0004] The present disclosure provides a video playback method, device, electronic device and storage medium. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a video playback method is provided, including:

[0006] Display the first-person perspective of the current video on the preset page;

[0007] In response to a view adjustment instruction triggered by a posture adjustment of a mobile device, a second view picture of a current video is displayed on a preset page; the second view picture is generated based on a correspondence between a device posture and a playback view; the correspondence between the device posture and the playback view represents a correspondence between the posture information of the mobile device and the playback view of the current video;

[0008] The multiple perspective pictures included in the current video include a first perspective picture and a second perspective picture.

[0009] In some possible embodiments, in response to a viewing angle adjustment instruction triggered by a posture adjustment of a mobile device, displaying a second viewing angle picture of a current video on a preset page includes:

[0010] If it is detected that the posture of the mobile device is adjusted, a viewing angle adjustment instruction is generated;

[0011] Determine the adjusted position information of the mobile device based on the perspective adjustment instruction;

[0012] Based on the correspondence between the device posture and the playback perspective, determine the second perspective corresponding to the adjusted posture information;

[0013] The second perspective picture of the current video corresponding to the second perspective is displayed on the preset page.

[0014] In some possible embodiments, before the preset page displays the first-person perspective picture of the current video, the method further includes:

[0015] In response to a play instruction of the current video, obtaining angular velocity information and acceleration information of the mobile device;

[0016] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0017] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0018] The corresponding relationship between the device posture and the playback viewing angle is determined based on the initial posture information in the preset coordinate system and the preset initial viewing angle.

[0019] In some possible embodiments, the method further includes:

[0020] Obtain video rendering data corresponding to a preset initial viewing angle;

[0021] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0022] In some possible embodiments, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the clamping device that clamps the mobile device;

[0023] Alternatively, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the biological part that clamps the mobile device.

[0024] In some possible embodiments, when the posture adjustment of the mobile device is based on the conduction implementation of the posture adjustment of the biological part that clamps the mobile device, after displaying the second perspective picture of the current video on the preset page, it also includes:

[0025] If the duration of displaying the second perspective picture of the current video on the preset page exceeds the preset duration, a corresponding relationship between the posture and perspective is generated based on the second perspective corresponding to the second perspective picture and the initial posture information in the preset coordinate system;

[0026] The correspondence between the device posture and the playback perspective is replaced by the correspondence between the posture and perspective.

[0027] In some possible embodiments, after replacing the correspondence between the device posture and the playback perspective with the correspondence between the posture and the perspective, the method further includes:

[0028] Displaying posture adjustment guidance information on a preset page; the posture adjustment guidance information indicates that the initial posture information in the preset coordinate system corresponds to the second viewing angle;

[0029] If a posture adjustment corresponding to the posture adjustment guidance information is detected, during the adjustment process, the second perspective screen of the current video is maintained on the preset page; the posture adjustment representation corresponding to the posture adjustment guidance information is adjusted from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

[0030] In some possible embodiments, the method further includes:

[0031] Displaying a playback function page; the playback function page includes multiple playback components with different playback capabilities;

[0032] If a first display instruction triggered by a multi-angle playback component among playback components based on multiple different playback capabilities is detected, displaying a video display page;

[0033] At least one video to be played is displayed on the video display page; the at least one video to be played includes the current video.

[0034] In some possible embodiments, the method further includes:

[0035] In response to a play instruction of the current video, the video display page is switched to a preset page;

[0036] Play the first preset viewing angle of the current video on the preset page; the preset page includes prompt information for rotating the viewing angle of the picture;

[0037] If a perspective rotation instruction triggered by a preset operation executed based on the prompt information is detected, a second preset perspective picture of the current video corresponding to the preset operation is played on the preset page, wherein the preset operation is a contact operation on the screen of the mobile device.

[0038] In some possible embodiments, the preset page includes a posture adjustment sensing component, and the method further includes:

[0039] In response to a posture adjustment sensing instruction triggered by the posture adjustment sensing component, obtaining angular velocity information and acceleration information of the mobile device;

[0040] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0041] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0042] Determine the correspondence between the device posture and the playback viewing angle based on the initial posture information in the preset coordinate system and the preset initial viewing angle;

[0043] Obtain video rendering data corresponding to a preset initial viewing angle;

[0044] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0045] According to a second aspect of an embodiment of the present disclosure, a video playback device is provided, including:

[0046] A first display module is configured to display a first-viewing angle picture of a current video on a preset page;

[0047] A second display module is configured to execute a view angle adjustment instruction triggered in response to a posture adjustment of the mobile device, and display a second view angle picture of the current video on a preset page; the second view angle picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video;

[0048] The multiple perspective pictures included in the current video include a first perspective picture and a second perspective picture.

[0049] In some possible embodiments, the second display module is configured to execute:

[0050] If it is detected that the posture of the mobile device is adjusted, a viewing angle adjustment instruction is generated;

[0051] Determine the adjusted position information of the mobile device based on the perspective adjustment instruction;

[0052] Based on the correspondence between the device posture and the playback perspective, determine the second perspective corresponding to the adjusted posture information;

[0053] The second perspective picture of the current video corresponding to the second perspective is displayed on the preset page.

[0054] In some possible embodiments, the apparatus further includes a corresponding relationship determination module configured to execute:

[0055] In response to a play instruction of the current video, obtaining angular velocity information and acceleration information of the mobile device;

[0056] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0057] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0058] The corresponding relationship between the device posture and the playback viewing angle is determined based on the initial posture information in the preset coordinate system and the preset initial viewing angle.

[0059] In some possible embodiments, the device further includes a rendering module configured to execute:

[0060] Obtain video rendering data corresponding to a preset initial viewing angle;

[0061] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0062] In some possible embodiments, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the clamping device that clamps the mobile device;

[0063] Alternatively, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the biological part that clamps the mobile device.

[0064] In some possible embodiments, when the posture adjustment of the mobile device is based on the conduction implementation of the posture adjustment of the biological part that clamps the mobile device, the apparatus further includes a correspondence adjustment module configured to execute:

[0065] If the duration of displaying the second perspective picture of the current video on the preset page exceeds the preset duration, a corresponding relationship between the posture and perspective is generated based on the second perspective corresponding to the second perspective picture and the initial posture information in the preset coordinate system;

[0066] The correspondence between the device posture and the playback perspective is replaced by the correspondence between the posture and perspective.

[0067] In some possible embodiments, the device further includes a third display module configured to execute:

[0068] Displaying posture adjustment guidance information on a preset page; the posture adjustment guidance information indicates that the initial posture information in the preset coordinate system corresponds to the second viewing angle;

[0069] If a posture adjustment corresponding to the posture adjustment guidance information is detected, during the adjustment process, the second perspective screen of the current video is maintained on the preset page; the posture adjustment representation corresponding to the posture adjustment guidance information is adjusted from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

[0070] In some possible embodiments, the device further includes a fourth display module configured to execute:

[0071] Displaying a playback function page; the playback function page includes multiple playback components with different playback capabilities;

[0072] If a first display instruction triggered by a multi-angle playback component among playback components based on multiple different playback capabilities is detected, displaying a video display page;

[0073] At least one video to be played is displayed on the video display page; the at least one video to be played includes the current video.

[0074] In some possible embodiments, the device further includes a fifth display module configured to execute:

[0075] In response to a play instruction of the current video, the video display page is switched to a preset page;

[0076] Play the first preset viewing angle of the current video on the preset page; the preset page includes prompt information for rotating the viewing angle of the picture;

[0077] If a perspective rotation instruction triggered by a preset operation executed based on the prompt information is detected, a second preset perspective picture of the current video corresponding to the preset operation is played on the preset page, wherein the preset operation is a contact operation on the screen of the mobile device.

[0078] In some possible embodiments, the preset page includes a posture adjustment sensing component, and the device also includes a rendering processing module configured to execute:

[0079] In response to a posture adjustment sensing instruction triggered by the posture adjustment sensing component, obtaining angular velocity information and acceleration information of the mobile device;

[0080] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0081] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0082] Determine the correspondence between the device posture and the playback viewing angle based on the initial posture information in the preset coordinate system and the preset initial viewing angle;

[0083] Obtain video rendering data corresponding to a preset initial viewing angle;

[0084] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0085] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement a method as described in any one of the first aspects above.

[0086] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods in the first aspect of the embodiment of the present disclosure.

[0087] According to the fifth aspect of an embodiment of the present disclosure, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a readable storage medium, at least one processor of a computer device reading and executing the computer program from the readable storage medium, so that the computer device performs any one of the methods in the first aspect of the embodiment of the present disclosure.

[0088] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0089] The first perspective picture of the current video is displayed on a preset page; in response to a perspective adjustment instruction triggered by the posture adjustment of the mobile device, the second perspective picture of the current video is displayed on the preset page; the second perspective picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video; wherein the multiple perspective pictures contained in the current video include the first perspective picture and the second perspective picture. The embodiment of the present application can adjust the viewing angle of the current video by adjusting the posture of the mobile device without performing gesture operations on the mobile phone screen. It can not only achieve conflict-free coexistence with the general gestures of some existing players, but also can achieve switching without screen occlusion when switching perspectives because there is no need to perform operations on the screen, thereby improving the video playback quality.

[0090] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0092] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0093] Figure 2 is a schematic diagram of an application environment according to an exemplary embodiment;

[0094] Figure 3 is a flow chart of a video playing method according to an exemplary embodiment;

[0095] Figure 4 is a schematic diagram showing six degrees of freedom according to an exemplary embodiment;

[0096] Figure 5 is a flow chart of a method for determining a correspondence between a device posture and a playback viewing angle according to an exemplary embodiment;

[0097] Figure 6 is a schematic diagram of screen direction information of a mobile device according to an exemplary embodiment;

[0098] Figure 7 is a flow chart showing a method for triggering a change in a viewing angle image by adjusting a posture of a mobile device according to an exemplary embodiment;

[0099] Figure 8 is a schematic diagram showing a first-perspective picture of a current video according to an exemplary embodiment;

[0100] Fig. 9 is a schematic diagram of a second viewing angle picture of a current video according to an exemplary embodiment Figure 1 ;

[0101] Fig.10 is a schematic diagram of a second viewing angle picture of a current video according to an exemplary embodiment Figure 2 ;

[0102] Fig.11 is a schematic diagram of a second viewing angle picture of a current video according to an exemplary embodiment Figure 3 ;

[0103] Fig.12 is a schematic diagram showing a switching of a play function page to a video display page according to an exemplary embodiment;

[0104] Fig.13 is a schematic diagram of a preset page according to an exemplary embodiment;

[0105] Fig.14 is a flow chart of a method for playing a video according to an exemplary embodiment;

[0106] Fig.15 is a block diagram of a video playback device according to an exemplary embodiment;

[0107] Fig.16 The invention is a block diagram of an electronic device for video playback according to an exemplary embodiment. DETAILED DESCRIPTION

[0108] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0109] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar first objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0111] See also Figure 1 , Figure 1 is a schematic diagram of an application environment of a video playback method according to an exemplary embodiment. Figure 1 As shown, the application environment may include a server 011 and a client 012 .

[0112] In some possible embodiments, the server 011 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The operating system running on the server may include but is not limited to Android, IOS, Linux, Windows, Unix, etc.

[0113] In some possible embodiments, the client 012 may include but is not limited to a smartphone, a desktop computer, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, a smart wearable device, etc. It may also be software running on the client, such as an application, a small program, etc. Optionally, the operating system running on the client may include but is not limited to Android system, IOS system, Linux, Windows, Unix, etc.

[0114] See also Figure 2 , Figure 2 is a schematic diagram of an application environment of a video playback method according to an exemplary embodiment. Figure 2 As shown, the application environment may include a camera array 021 , a server 022 , and a client 023 .

[0115] In some possible embodiments, the camera array 021 is composed of several or dozens of cameras arranged around the target scene, the server 022 may refer to the above-mentioned server 011 , and the client 023 may refer to the above-mentioned client 012 .

[0116] In some possible embodiments, client 012 or client 023 displays the first perspective picture of the current video on a preset page; in response to the perspective adjustment instruction triggered by the posture adjustment of the mobile device, the second perspective picture of the current video is displayed on the preset page; the second perspective picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video; wherein the multiple perspective pictures contained in the current video include the first perspective picture and the second perspective picture. The embodiment of the present application can adjust the viewing angle of the current video by adjusting the posture of the mobile device without performing gesture operations on the mobile phone screen. It can not only achieve conflict-free coexistence with the general gestures of some existing players, but also can achieve switching without screen occlusion when switching perspectives because there is no need to operate on the screen, thereby improving the video playback quality.

[0117] In an exemplary embodiment, the databases corresponding to the client and the server can both be node devices in the blockchain system, and can share the acquired and generated information with other node devices in the blockchain system, thereby realizing information sharing among multiple node devices. Multiple node devices in the blockchain system can be configured with the same blockchain, which is composed of multiple blocks, and the adjacent blocks have an association relationship, so that when the data in any block is tampered, it can be detected by the next block, thereby preventing the data in the blockchain from being tampered with, and ensuring the security and reliability of the data in the blockchain.

[0118] Figure 3 is a flowchart of a video playing method according to an exemplary embodiment. Figure 3 As shown, the video playing method can be applied to a client, such as a mobile device, including a smart phone or an IPAD, or a desktop computer and the like. The flow chart includes at least the following steps S301-S303.

[0119] In step S301, the first-person perspective picture of the current video is displayed on a preset page.

[0120] In a possible embodiment, the mobile device may display the first-perspective picture of the current video on a preset page. Optionally, the current video may be a video being played vertically or horizontally on the mobile device.

[0121] In the embodiment of the present application, the current video is a free viewpoint video (Free Viewpoint Video, FVV). Free viewpoint video refers to a new type of video form in which the user can freely choose the viewing angle during the viewing process. Optionally, a free viewpoint video with full freedom is equivalent to a six-degree-of-freedom video. Among them, the first perspective corresponding to the first perspective picture is the first perspective in the freely selectable viewing perspective.

[0122] Optionally, Six Degree of Freedom (6DOF) means that an object has six degrees of freedom in space. Figure 3 is a schematic diagram showing six degrees of freedom according to an exemplary embodiment, such as Figure 4 As shown in the figure, there are three degrees of freedom of movement along the three rectangular coordinate axes of x, y, and z (front and back, up and down, and left and right) and the degrees of freedom of rotation around these three coordinate axes (pitch, yaw, and roll). Six-degree-of-freedom video means that the viewing angle of the video has multiple degrees of freedom, that is, within a certain range, the user can freely choose the viewing angle to watch the scene in the video.

[0123] In an embodiment of the present application, the mobile device needs to obtain the current video before displaying the first-view picture of the current video on the preset page. In the production stage of the current video, several or dozens of cameras can be set around the target scene, and the several or dozens of cameras can be referred to as a camera array. In an optional embodiment, the cameras in the camera array can be arranged around the target scene in sequence at a certain interval, and the target scene is photographed at the same time. In another optional embodiment, the cameras in the camera array can be arranged around the target scene according to the importance of the scenes involved in all directions of the target scene, and the cameras in the camera array can be arranged around the target scene according to actual needs, and the target scene is photographed at the same time. So that the cameras in the camera array can record the scene information from different positions and angles, and then generate the three-dimensional structure information of the entire target scene through a three-dimensional reconstruction algorithm, and compress this information into the video stream (shot by the cameras in the camera array) using video encoding technology to form a free-view video.

[0124] In an optional embodiment, the first perspective picture may be the initial perspective picture after the current video is started to play. Therefore, the mobile device may respond to the play instruction of the current video, calculate the video rendering data corresponding to the preset initial perspective from the video rendering data corresponding to different perspectives contained in the current video, and render the video rendering data corresponding to the preset initial perspective on a preset page to generate the initial perspective picture of the current video.

[0125] Optionally, while the mobile device is generating the initial viewing angle picture of the current video, or before the mobile device generates the initial viewing angle picture of the current video, the mobile device can also determine the correspondence between the device posture and the playback viewing angle, so that when the posture information of the mobile device changes subsequently, the mobile device can determine the new playback viewing angle based on the changed new posture information and the correspondence between the device posture and the playback viewing angle, and then present the picture of the current video corresponding to the new playback viewing angle on the preset page.

[0126] Figure 5 is a flow chart showing a method for determining a correspondence between a device posture and a playback viewing angle according to an exemplary embodiment. Figure 5 As shown, at least S501-S507 are included.

[0127] In step S501, in response to a play instruction of the current video, angular velocity information and acceleration information of the mobile device are obtained.

[0128] In the embodiment of the present application, in response to a play instruction of the current video, the mobile device may obtain angular velocity information and acceleration information of the mobile device.

[0129] Specifically, when the user clicks on the current video in the video list, the mobile device can detect that the current video is triggered to play. Optionally, the mobile device can perform initialization operations through the player, such as obtaining the address of the current video from the address server, and obtaining the resources of the current video according to the address of the current video. In addition, the player can notify the device posture tracking module to track the posture of the mobile device. Optionally, the player can be a player built into the mobile device. Alternatively, the player can be a player group built into the application, wherein the application can be a video application that can play the current video.

[0130] Correspondingly, after receiving the notification from the player, the device posture tracking module on the mobile device can turn on the tracking function, register the device sensors (such as gyroscope sensors and accelerometers) to monitor events, monitor the gyroscope sensors and accelerometers, and collect the angular velocity information corresponding to the gyroscope sensors and the acceleration information corresponding to the accelerometers.

[0131] In the embodiment of the present application, the gyroscope sensor can be called an angular velocity sensor, which is different from an accelerometer (G-sensor). Its measured physical quantity is the angular velocity of rotation during deflection and tilt. The gyroscope sensor is a device used to sense and maintain direction, designed based on the theory of conservation of angular momentum. The gyroscope sensor is mainly composed of a rotatable rotor located at the axis. Once the gyroscope sensor starts to rotate, it tends to resist changes in direction due to the angular momentum of the rotor. In the embodiment of the present application, the acceleration sensor is a device in a mobile device that can measure acceleration force. Acceleration force is the force acting on an object when the object is accelerating.

[0132] In step S503, initial posture information in the sensor coordinate system is determined based on the angular velocity information and the acceleration information.

[0133] In the embodiment of the present application, the pose is a description of the position and posture of an object in a specified coordinate system. The position is the location in space, which can be represented by a 3*1 matrix, and the posture is the orientation in space, which can be represented by a 3*3 matrix. The initial pose information may include position information and posture information, or the initial pose information may include posture information.

[0134] In an optional implementation, the mobile device may fuse the angular velocity information and the acceleration information and filter the fused data to remove noise in the data to obtain the initial pose information in the sensor coordinate system. The initial pose information in the sensor coordinate system may be expressed as a device pose rotation matrix in the sensor coordinate system.

[0135] Optionally, when the initial posture information includes attitude information, the scheme for data fusion of the angular velocity information and the acceleration information is: after the mobile device obtains the angular velocity information and the acceleration information, it can determine the rotation angle by integrating the angular velocity information, and then correct the rotation angle by integrating and proportioning the acceleration information to obtain the final fused data, which can be called attitude information.

[0136] Optionally, when the initial posture information includes position information and attitude information, the solution for data fusion of angular velocity information and acceleration information is: after the mobile device obtains the attitude information according to the quaternion attitude solution algorithm, it can also determine the three-dimensional space positioning information in the specified coordinate system through the positioning system in the mobile device, and then combine the attitude information and the three-dimensional space positioning information into an initial posture information (4*4 matrix).

[0137] In step S505, the initial position and posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial position and posture information in the preset coordinate system.

[0138] In an optional embodiment, the screen information of the mobile device may include screen direction information of the mobile device. Figure 6 is a schematic diagram of screen direction information of a mobile device according to an exemplary embodiment. Figure 6 As shown, taking the mobile device as a smartphone and the XYZ three-dimensional coordinate system in the figure as an example, assuming Figure 6 The smartphones in the four situations shown in the figure are all on the plane where the X-axis and Y-axis are located. When the screen orientation information of the smartphone is 0 degrees, the top edge of the smartphone screen corresponds to the positive direction of the Y-axis; when the screen orientation information of the smartphone is 90 degrees, the right side of the smartphone screen corresponds to the positive direction of the Y-axis; when the screen orientation information of the smartphone is 180 degrees, the bottom edge of the smartphone screen corresponds to the positive direction of the Y-axis; when the screen orientation information of the smartphone is 270 degrees, the left side of the smartphone screen corresponds to the positive direction of the Y-axis.

[0139] Optionally, the preset coordinate system may include a world coordinate system. The pose information of the mobile device in the world coordinate system refers to the position of the mobile device in three-dimensional space and the pose it exhibits. The position of the mobile device is generally described by the coordinate points of the mobile device in three-dimensional space. The pose of the mobile device can generally be described by a rotation matrix or Euler angles, which can be converted to each other. Euler angles consist of three angles, namely:

[0140] Pitch Angle refers to the angle of rotation of an object around the X-axis, which is used to describe the pitch degree of the object;

[0141] Yaw Angle refers to the angle at which an object rotates around the Y axis and is used to describe the degree of yaw of an object.

[0142] Roll Angle refers to the angle at which an object rotates around the Z axis and is used to describe the degree of rotation of an object.

[0143] Optionally, the posture information of the mobile device in the world coordinate system may also be referred to as the device posture rotation matrix in the world coordinate system.

[0144] In step S507, the correspondence between the device posture and the playback viewing angle is determined based on the initial posture information in the preset coordinate system and the preset initial viewing angle.

[0145] In an embodiment of the present application, if the initial setting information of the posture information does not exist in the mobile device, the mobile device can regard the initial posture information in the world coordinate system as the initial setting information of the posture information, and store it, and set the flag position indicating whether the initial setting information of the posture information exists to "exists".

[0146] Subsequently, when the player detects that the position of the flag is set to "exist", the initial posture information can be obtained from the storage area, and the initial posture information can be matched with the preset initial viewing angle to obtain the corresponding relationship between the device posture and the playback viewing angle. The corresponding relationship between the device posture and the playback viewing angle represents the corresponding relationship between the posture information of the mobile device and the playback viewing angle of the current video.

[0147] In the embodiments of the present application, for ease of understanding, the preset initial viewing angle can be imagined as a viewing angle determined by a virtual camera based on its position and posture. A virtual camera refers to a camera that does not physically exist and is placed in a three-dimensional space. The camera has a position and posture like a real camera, and the image corresponding to the virtual camera can be generated by an algorithm. When the current video starts playing, the generated image is the image corresponding to the preset initial viewing angle that the user sees through interaction.

[0148] That is, after the mobile device determines the correspondence between the device posture and the playback perspective, the mobile device can calculate and determine the video rendering data corresponding to the preset initial perspective from the video rendering data corresponding to the different perspectives contained in the current video, render the video rendering data corresponding to the preset initial perspective, and generate the initial perspective screen of the current video on the preset page.

[0149] In an optional embodiment, since the current video is a free-viewpoint video, and the free-viewpoint video is obtained by compressing video streams shot by cameras at different positions and angles in the camera array. Therefore, each video stream can carry viewpoint identification information determined based on different positions and angles. Optionally, when the mobile device obtains a preset initial viewpoint, the preset initial viewpoint and different viewpoint identification information can be matched. The viewpoint identification information that best matches the preset initial viewpoint is obtained from the different viewpoint identification information and is determined as the target viewpoint identification information, and the video stream corresponding to the target viewpoint identification information is determined as the video rendering data corresponding to the preset initial viewpoint.

[0150] In another optional embodiment, the first perspective image is not the initial perspective image after the current video is started to play. Therefore, after detecting that the posture of the mobile device is adjusted, the mobile device can trigger the generation of a perspective adjustment instruction and display the first perspective image of the current video after the posture adjustment on a preset page.

[0151] In step S203, in response to the perspective adjustment instruction triggered by the posture adjustment of the mobile device, the second perspective picture of the current video is displayed on the preset page. The second perspective picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video; wherein the multiple perspective pictures contained in the current video include the first perspective picture and the second perspective picture.

[0152] In an optional embodiment, the posture adjustment of the mobile device can be caused by changing the posture of the mobile device, for example, the user shakes the mobile device, rotates the mobile device, and tilts the mobile device, that is, the posture adjustment of the mobile device can be achieved based on the transmission of the posture adjustment of the biological part that clamps the mobile device. In another optional embodiment, the posture adjustment of the mobile device is achieved based on the conduction of the posture adjustment of the clamping device that clamps the mobile device. For example, the clamping device can be a clamping member that clamps the mobile device (such as a selfie stick), and the user can adjust the posture of the clamped mobile device by adjusting the posture of the clamping device.

[0153] Figure 7 is a flow chart showing a method for triggering a change in a viewing angle image by adjusting a posture of a mobile device according to an exemplary embodiment. Figure 7 As shown, at least S701-S707 are included.

[0154] In step S701, if it is detected that the posture of the mobile device is adjusted, a viewing angle adjustment instruction is generated.

[0155] In an optional implementation, when the mobile device detects that the posture of the mobile device has been adjusted, a viewing angle adjustment instruction may be generated.

[0156] In step S703, the adjusted posture information of the mobile device is determined based on the viewing angle adjustment instruction.

[0157] Optionally, the mobile device can determine the adjusted posture information of the mobile device based on the viewing angle adjustment instruction. Specifically, when the mobile device detects that the posture of the mobile device has been adjusted, the device posture tracking module tracks the posture of the mobile device and obtains the angular velocity information corresponding to the gyroscope sensor and the acceleration information corresponding to the acceleration sensor at the current moment. Subsequently, the mobile device can perform data fusion on the angular velocity information and acceleration information at the current moment, and filter the fused data to remove noise in the data to obtain the posture information in the sensor coordinate system at the current moment, and according to the screen direction information at this time, convert the posture information in the sensor coordinate system at the current moment into the posture information in the preset coordinate system, that is, the posture information of the mobile device after adjustment.

[0158] Optionally, when the adjusted posture information of the mobile device includes posture information, the scheme for data fusion of the angular velocity information and acceleration information at the current moment is: after the mobile device obtains the angular velocity information and acceleration information at the current moment, it can determine the rotation angle at the current moment by integrating the angular velocity information at the current moment, and then correct the rotation angle at the current moment by integrating and proportioning the acceleration information at the current moment to obtain the final fused data at the current moment. The fused data can be called the adjusted posture information of the mobile device.

[0159] Optionally, when the adjusted posture information of the mobile device includes position information and posture information, the solution for data fusion of the angular velocity information and acceleration information at the current moment is: after the mobile device obtains the posture information at the current moment according to the technical solution in the previous paragraph, it can also determine the three-dimensional space positioning information in the specified coordinate system through the positioning system in the mobile device, and then combine the posture information at the current moment and the three-dimensional space positioning information into an adjusted posture information of the mobile device (a 4*4 matrix).

[0160] In step S705, based on the correspondence between the device posture and the playback perspective, a second perspective corresponding to the adjusted posture information is determined.

[0161] Optionally, the mobile device may obtain a correspondence between the device posture and the playback angle of view from the storage area, where the correspondence between the device posture and the playback angle of view represents a correspondence between the posture information of the mobile device and the playback angle of view of the current video.

[0162] Optionally, the mobile device may determine a second viewing angle from the correspondence between the device's posture and the playback viewing angle based on the adjusted posture information, where the second viewing angle represents the posture of the virtual camera at the current moment and the viewing angle corresponding to the posture.

[0163] In step S707, the second perspective picture of the current video corresponding to the second perspective is displayed on the preset page.

[0164] In an embodiment of the present application, the mobile device can calculate the video rendering data corresponding to the second perspective from the video rendering data corresponding to different perspectives contained in the current video, render the video rendering data corresponding to the second perspective, and generate a second perspective screen of the current video on a preset page.

[0165] Figure 8 is a schematic diagram showing a first-viewing angle picture of a current video according to an exemplary embodiment. Figure 8 As shown, it includes a first-person perspective picture 801 of the current video displayed on a preset page. Figure 9-11 FIG. 1 is a schematic diagram showing a second viewing angle picture of a current video according to an exemplary embodiment. Fig. 9 As shown, the second viewing angle picture 802 of the current video is displayed on the preset page; Fig.10 As shown, the second viewing angle picture 803 of the current video is displayed on the preset page; Fig.11 As shown, it includes a second perspective picture 804 of the current video displayed on the preset page.

[0166] Combine the following Figure 8-11 The perspective adjustment instruction triggered in response to the posture adjustment of the mobile device is explained by switching from displaying the first perspective picture of the current video on the preset page to the second perspective picture of the current video. Optionally, when the mobile device detects that the mobile device is deflected to the left by 30 degrees, the mobile device can switch from displaying the first perspective picture 801 of the current video on the preset page to the second perspective picture 802 of the current video. Optionally, when the mobile device detects that the mobile device is deflected to the left by 50 degrees, the mobile device can switch from displaying the first perspective picture 801 of the current video on the preset page to the second perspective picture 803 of the current video. Optionally, when the mobile device detects that the mobile device is deflected to the right by 60 degrees, the mobile device can switch from displaying the first perspective picture 801 of the current video on the preset page to the second perspective picture 804 of the current video.

[0167] The above-mentioned 30 degree deflection to the left, 50 degree deflection to the left and 60 degree deflection to the right are only examples. Optionally, the position adjustment of the mobile device may also include pitch adjustment and roll adjustment.

[0168] In an optional embodiment, when the mobile device detects that the mobile device is deflected to the right by 60 degrees, it can sequentially display on a preset page the first perspective picture 801 of the current video switching to the first second perspective picture 802 of the current video, then switching from the first second perspective picture 802 of the current video to the second second perspective picture 803 of the current video, and then switching from the second second perspective picture 803 of the current video to the third second perspective picture 804 of the current video.

[0169] In the embodiment of the present application, the user can adjust the angle of viewing the current video by simply adjusting the posture of the hand holding the mobile device or the clamping device holding the mobile device, without having to perform gesture operations on the mobile phone screen. Not only can it coexist without conflict with the common gestures of some existing players (such as fast forward), but it can also switch without screen obstruction when switching viewing angles, providing a strong sense of immersion.

[0170] In some possible embodiments, it is assumed that the first viewing angle picture is the initial viewing angle picture, and the initial posture information in the preset coordinate system is Figure 6 The posture information displayed by the mobile device in the upper left corner of the middle image can be displayed on the preset page when the mobile device responds to the viewing angle adjustment instruction triggered by the posture adjustment of the mobile device (for example, the mobile device deflects 30 degrees to the left). Fig. 9 The second perspective picture 802 of the current video is shown. However, if the user's hand is currently holding the mobile device, then if the user does not adjust the body posture (or head posture) or it is not easy to adjust the body posture (or head posture), always holding the mobile device with a 30-degree angle to the left will inevitably cause discomfort to the user when watching the current video. Based on this, the embodiment of the present application can adjust the correspondence between the device posture and the playback perspective under the preset conditions, so that the second perspective corresponding to the second perspective picture corresponds to the initial posture information, that is, keep the second perspective picture of the current video playing when the user is in a more comfortable posture of holding the mobile device.

[0171] In an optional embodiment, satisfying the preset condition may mean that if the mobile device determines that the duration of displaying the second perspective picture of the current video on the preset page exceeds the preset duration (for example, 2 minutes), the mobile device may generate a correspondence between the posture and perspective based on the second perspective corresponding to the second perspective picture and the initial posture information in the preset coordinate system, and use the correspondence between the posture and perspective to replace the correspondence between the device posture and the playback perspective.

[0172] In another optional embodiment, satisfying the preset condition may mean that if the mobile device detects a correspondence adjustment instruction, the mobile device may generate a correspondence between the posture and perspective based on the second perspective corresponding to the second perspective image and the initial posture information in the preset coordinate system, and use the correspondence between the posture and perspective to replace the correspondence between the device posture and the playback perspective.

[0173] Optionally, the correspondence adjustment instruction may be generated based on a trigger of a virtual component set on a preset page.

[0174] That is to say, the mobile device can generate a new correspondence between the device posture and the playback perspective based on the second perspective corresponding to the second perspective image and the initial posture information in the preset coordinate system, and use the new correspondence between the device posture and the playback perspective to replace the previous correspondence between the device posture and the playback perspective.

[0175] This embodiment takes into account the user's usage habits. At the initial moment of watching the current video, the user generally holds the mobile device in the most comfortable posture. When it is necessary to adjust the playback angle of the current video on the mobile device by adjusting the posture information of the mobile device, it will usually cause inconvenience to the user's viewing if the second perspective is maintained. Therefore, the mobile device can reset the corresponding relationship between the new device posture and the playback angle to prepare for the subsequent adjustment of the mobile device back to the original initial posture information.

[0176] In an embodiment of the present application, after replacing the correspondence between the device posture and the playback perspective with the correspondence between the posture and perspective, the mobile device can display posture adjustment guide information on a preset page, and the posture adjustment guide information indicates that the initial posture information in the preset coordinate system corresponds to the second perspective, so as to prompt the user to adjust the mobile device from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

[0177] Optionally, if a posture adjustment corresponding to the posture adjustment guidance information is detected, the mobile device can maintain displaying the second perspective image of the current video on the preset page during the adjustment process, that is, when the user converts the mobile device from the adjusted posture information corresponding to the second perspective back to the initial posture information in the preset coordinate system, the second perspective image is kept unchanged on the preset page.

[0178] Among them, the posture adjustment representation corresponding to the posture adjustment guidance information is adjusted from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

[0179] In an embodiment of the present application, replacing the previous correspondence between the device posture and the playback perspective with the correspondence between the new device posture and the playback perspective is an optional implementation. In the actual application of the mobile device, the user can choose to turn this function on or off. In the case of choosing to turn on this function, the mobile device can adjust the correspondence between the device posture and the playback perspective under the preset conditions, so that the second perspective corresponding to the second perspective picture corresponds to the initial posture information in the preset coordinate system.

[0180] In an optional embodiment, each video in the application where the current video is located can be a free perspective video. Optionally, when the user clicks on the current video in the video list, the initial perspective picture of the current video corresponding to the preset initial perspective can be displayed on the preset page. Optionally, when the mobile device displays the initial perspective picture of the current video corresponding to the preset initial perspective on the preset page, the posture adjustment sensing component can be displayed on the preset page. When the mobile device detects that the posture sensing adjustment component is clicked, the current video can enter the free perspective adjustment state, that is, enter the perspective picture rendered on the preset page by adjusting the posture of the mobile device. Optionally, after the current video enters the free perspective adjustment state, the posture adjustment sensing component can be hidden or displayed in a preset area of ​​the preset page. When the mobile device detects that the posture sensing adjustment component is clicked again, the current video can exit the free perspective adjustment state.

[0181] In another optional embodiment, since the production process of free viewpoint videos consumes a lot of resources and not all videos are suitable for free viewpoint, the application can put all free viewpoint videos on the same page for users to choose to watch.

[0182] Fig.12 is a schematic diagram showing a switching of a play function page to a video display page according to an exemplary embodiment. Fig.12 As shown, after the mobile device starts the application, it can enter the playback function page 1001. The playback function page includes multiple playback components with different playback capabilities, such as a "panoramic sound effect" component, a "bullet screen protection" component, a "multi-angle playback" component, a "quality audio and video" component, and a "quality clear stream" component. If a first display instruction triggered by a multi-angle playback component 1003 among playback components based on multiple different playback capabilities is detected, the playback function page can be switched to display a video display page 1002. Optionally, the mobile device can display at least one video to be played on the video display page 1002, wherein at least one video to be played includes the current video, for example, the first video may be the current video played above.

[0183] In the embodiment of the present application, the mobile device can switch from the video display page to the preset page in response to the play instruction of the current video. Fig.13 The present invention is a schematic diagram of a preset page according to an exemplary embodiment, wherein a mobile device can play a first preset viewing angle picture of a current video on the preset page. Optionally, the first preset viewing angle picture can be a picture corresponding to a preset initial viewing angle.

[0184] In an optional embodiment, if Fig.13 As shown, the preset page includes prompt information 1100 for rotating the screen angle of view. If a rotation instruction of the angle of view triggered by a preset operation executed based on the prompt information is detected, the second preset angle of view screen of the current video corresponding to the preset operation is played on the preset page. The preset operation is a contact operation on the screen of the mobile device. For example, the mobile terminal detects a leftward sliding track or a rightward sliding track on the screen, which can trigger the generation of a corresponding rotation instruction of the angle of view.

[0185] Optionally, when a 5-degree leftward sliding on the screen is detected based on the prompt information, the mobile device can trigger the generation of a 5-degree leftward viewing angle rotation instruction. Subsequently, the mobile device can play the second preset viewing angle picture of the current video corresponding to the 5-degree leftward rotation on the preset page.

[0186] Optionally, when a 5-degree rightward sliding on the screen is detected based on the prompt information, the mobile device can trigger the generation of a 5-degree rightward viewing angle rotation instruction. Subsequently, the mobile device can play the second preset viewing angle picture of the current video corresponding to the 5-degree rightward rotation on the preset page.

[0187] Optionally, when the mobile device detects a 5-degree leftward slide on the screen, the "0 degree" in the prompt message may become "-5 degrees". When the mobile device detects a 5-degree rightward slide on the screen, the "0 degree" in the prompt message may become "5 degrees".

[0188] Optionally, within a period of time after the mobile device detects that the user has completed a preset operation, the prompt message for rotating the screen viewing angle may disappear, and may be displayed only when the user performs the preset operation again.

[0189] The above-mentioned viewing angle picture adjustment is adjusted by gestures on the screen, and the present application also provides a viewing angle picture adjustment method that is not implemented by gestures. Optionally, the preset page includes a posture adjustment sensing component.

[0190] Fig.14 is a flowchart of a method for playing a video according to an exemplary embodiment. Fig.14 As shown, it includes at least S1401-S1411.

[0191] In step S1401, in response to a posture adjustment sensing instruction triggered by a posture adjustment sensing component, angular velocity information and acceleration information of the mobile device are obtained.

[0192] In the embodiment of the present application, the mobile device can respond to the posture adjustment sensing instruction triggered by the posture adjustment sensing component, and notify the device posture tracking module to track the posture of the mobile device. Correspondingly, after receiving the notification from the player, the device posture tracking module on the mobile device can turn on the tracking function, register the device sensor (such as gyroscope sensor, acceleration sensor) to monitor events, monitor the gyroscope sensor and acceleration sensor, and collect the angular velocity information corresponding to the gyroscope sensor and the acceleration information corresponding to the acceleration sensor.

[0193] In step S1403, initial posture information in the sensor coordinate system is determined based on the angular velocity information and the acceleration information.

[0194] In an optional implementation, the mobile device may perform data fusion on the angular velocity information and the acceleration information, and filter the fused data to remove noise in the data, thereby obtaining initial posture information in the sensor coordinate system.

[0195] In step S1405, the initial position and posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial position and posture information in the preset coordinate system.

[0196] Optionally, the mobile device may convert the pose information in the sensor coordinate system according to the screen information of the mobile device to obtain the initial pose information in the world coordinate system. Figure 6 As shown, the screen information of the mobile device may include screen direction information of the mobile device.

[0197] In step S1407, the correspondence between the device posture and the playback viewing angle is determined based on the posture information in the preset coordinate system and the preset initial viewing angle.

[0198] In an embodiment of the present application, if the initial setting information of the posture information does not exist in the mobile device, the mobile device can regard the initial posture information in the world coordinate system as the initial setting information of the posture information, and store it, and set the flag position indicating whether the initial setting information of the posture information exists to "exists".

[0199] Subsequently, when the player detects that the position of the flag is set to "exist", the initial posture information can be obtained from the storage area, and the initial posture information can be matched with the preset initial viewing angle to obtain the corresponding relationship between the device posture and the playback viewing angle. The corresponding relationship between the device posture and the playback viewing angle represents the corresponding relationship between the posture information of the mobile device and the playback viewing angle of the current video.

[0200] In an optional embodiment, the mobile device may correspond the initial posture information to the preset initial viewing angle, and obtain the relationship between the posture information adjustment data and the viewing angle adjustment data. The correspondence between the initial posture information and the preset initial viewing angle, and the relationship between the posture information adjustment data and the viewing angle adjustment data are determined as the correspondence between the device posture and the playback viewing angle.

[0201] For example, based on the correspondence between the initial posture information and the preset initial viewing angle, the viewing angle is adjusted by one unit of data for each adjustment of the posture information. Taking the adjustment of the deflection data as an example, optionally, for each 1 degree deflection in the direction A, the viewing angle is also adjusted by 1 degree toward A. Alternatively, for each 1 degree deflection in the direction A, the viewing angle is adjusted by 5 degrees toward A.

[0202] In step S1409, video rendering data corresponding to a preset initial viewing angle is obtained.

[0203] In step S1411, the video rendering data is rendered to generate an initial viewing angle picture of the current video on a preset page.

[0204] In an embodiment of the present application, the mobile device can calculate the video rendering data corresponding to the preset initial perspective from the video rendering data corresponding to the different perspectives contained in the current video, render the video rendering data corresponding to the preset initial perspective, and generate an initial perspective screen of the current video on a preset page.

[0205] In this way, the current video enters the free perspective adjustment state, that is, the perspective screen rendered on the preset page is adjusted by adjusting the posture of the mobile device. Optionally, when the current video enters the free perspective adjustment state, the posture adjustment sensing component can be hidden or displayed in a preset area of ​​the preset page. When the mobile device detects that the posture sensing adjustment component is clicked again, the current video can exit the free perspective adjustment state.

[0206] Fig.15 is a block diagram of a video playback device according to an exemplary embodiment. The device has the function of implementing the data processing method in the above method embodiment, and the function can be implemented by hardware or by hardware executing corresponding software. Fig.15 , the device includes a first display module 1501 and a second display module 1502:

[0207] The first display module 1501 is configured to display a first-viewing angle of view of the current video on a preset page;

[0208] The second display module 1502 is configured to execute the view angle adjustment instruction triggered in response to the posture adjustment of the mobile device, and display the second view angle picture of the current video on a preset page; the second view angle picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video;

[0209] The multiple perspective pictures included in the current video include a first perspective picture and a second perspective picture.

[0210] In some possible embodiments, the second display module is configured to execute:

[0211] If it is detected that the posture of the mobile device is adjusted, a viewing angle adjustment instruction is generated;

[0212] Determine the adjusted position information of the mobile device based on the perspective adjustment instruction;

[0213] Based on the correspondence between the device posture and the playback perspective, determine the second perspective corresponding to the adjusted posture information;

[0214] The second perspective picture of the current video corresponding to the second perspective is displayed on the preset page.

[0215] In some possible embodiments, the apparatus further includes a corresponding relationship determination module configured to execute:

[0216] In response to a play instruction of the current video, obtaining angular velocity information and acceleration information of the mobile device;

[0217] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0218] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0219] The corresponding relationship between the device posture and the playback viewing angle is determined based on the initial posture information in the preset coordinate system and the preset initial viewing angle.

[0220] In some possible embodiments, the device further includes a rendering module configured to execute:

[0221] Obtain video rendering data corresponding to a preset initial viewing angle;

[0222] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0223] In some possible embodiments, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the clamping device that clamps the mobile device;

[0224] Alternatively, the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the biological part that clamps the mobile device.

[0225] In some possible embodiments, when the posture adjustment of the mobile device is based on the conduction implementation of the posture adjustment of the biological part that clamps the mobile device, the apparatus further includes a correspondence adjustment module configured to execute:

[0226] If the duration of displaying the second perspective picture of the current video on the preset page exceeds the preset duration, a corresponding relationship between the posture and perspective is generated based on the second perspective corresponding to the second perspective picture and the initial posture information in the preset coordinate system;

[0227] The correspondence between the device posture and the playback perspective is replaced by the correspondence between the posture and perspective.

[0228] In some possible embodiments, the device further includes a third display module configured to execute:

[0229] Displaying posture adjustment guidance information on a preset page; the posture adjustment guidance information indicates that the initial posture information in the preset coordinate system corresponds to the second viewing angle;

[0230] If a posture adjustment corresponding to the posture adjustment guidance information is detected, during the adjustment process, the second perspective screen of the current video is maintained on the preset page; the posture adjustment representation corresponding to the posture adjustment guidance information is adjusted from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

[0231] In some possible embodiments, the device further includes a fourth display module configured to execute:

[0232] Displaying a playback function page; the playback function page includes multiple playback components with different playback capabilities;

[0233] If a first display instruction triggered by a multi-angle playback component among playback components based on multiple different playback capabilities is detected, displaying a video display page;

[0234] At least one video to be played is displayed on the video display page; the at least one video to be played includes the current video.

[0235] In some possible embodiments, the device further includes a fifth display module configured to execute:

[0236] In response to a play instruction of the current video, the video display page is switched to a preset page;

[0237] Play the first preset viewing angle of the current video on the preset page; the preset page includes prompt information for rotating the viewing angle of the picture;

[0238] If a perspective rotation instruction triggered by a preset operation executed based on the prompt information is detected, a second preset perspective picture of the current video corresponding to the preset operation is played on the preset page, wherein the preset operation is a contact operation on the screen of the mobile device.

[0239] In some possible embodiments, the preset page includes a posture adjustment sensing component, and the device also includes a rendering processing module configured to execute:

[0240] In response to a posture adjustment sensing instruction triggered by the posture adjustment sensing component, obtaining angular velocity information and acceleration information of the mobile device;

[0241] Determine the initial position information in the sensor coordinate system based on the angular velocity information and the acceleration information;

[0242] The initial posture information in the sensor coordinate system is converted according to the screen information of the mobile device to obtain the initial posture information in the preset coordinate system;

[0243] Determine the correspondence between the device posture and the playback viewing angle based on the initial posture information in the preset coordinate system and the preset initial viewing angle;

[0244] Obtain video rendering data corresponding to a preset initial viewing angle;

[0245] The video rendering data is rendered and processed to generate the initial viewing angle picture of the current video on a preset page.

[0246] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0247] Fig.16 3000 is a block diagram of a device for video playback according to an exemplary embodiment. For example, the device 3000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0248] Reference Fig.16 , the device 3000 may include one or more of the following components: a processing component 3002 , a memory 3004 , a power component 3006 , a multimedia component 3008 , an audio component 3010 , an input / output (I / O) interface 3012 , a sensor component 3014 , and a communication component 3016 .

[0249] The processing component 3002 generally controls the overall operation of the device 3000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 3002 may include one or more processors 3020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 3002 may include one or more modules to facilitate the interaction between the processing component 3002 and other components. For example, the processing component 3002 may include a multimedia module to facilitate the interaction between the multimedia component 3008 and the processing component 3002.

[0250] The memory 3004 is configured to store various types of data to support operations on the device 3000. Examples of such data include instructions for any application or method operating on the device 3000, contact data, phone book data, messages, pictures, videos, etc. The memory 3004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0251] The power supply component 3006 provides power to the various components of the device 3000. The power supply component 3006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 3000.

[0252] The multimedia component 3008 includes a screen that provides an output interface between the device 3000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 3008 includes a front camera and / or a rear camera. When the device 3000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0253] The audio component 3010 is configured to output and / or input audio signals. For example, the audio component 3010 includes a microphone (MIC), and when the device 3000 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 3004 or sent via the communication component 3016. In some embodiments, the audio component 3010 also includes a speaker for outputting audio signals.

[0254] I / O interface 3012 provides an interface between processing component 3002 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0255] The sensor assembly 3014 includes one or more sensors for providing various aspects of status assessment for the device 3000. For example, the sensor assembly 3014 can detect the open / closed state of the device 3000, the relative positioning of components, such as the display and keypad of the device 3000, and the sensor assembly 3014 can also detect changes in the position of the device 3000 or a component of the device 3000, the presence or absence of user contact with the device 3000, the orientation or acceleration / deceleration of the device 3000, and the temperature change of the device 3000. The sensor assembly 3014 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 3014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 3014 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0256] The communication component 3016 is configured to facilitate wired or wireless communication between the device 3000 and other devices. The device 3000 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 3016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 3016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0257] In an exemplary embodiment, the apparatus 3000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0258] An embodiment of the present invention also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a video playback method. The at least one instruction or the at least one program is loaded and executed by the processor to implement the video playback method provided by the above method embodiment.

[0259] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 3004 including instructions, and the instructions can be executed by the processor 3020 of the device 3000 to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0260] An embodiment of the present invention further provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any method in the first aspect of the embodiment of the present disclosure.

[0261] An embodiment of the present invention further provides a computer program product, which includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the computer device performs any method in the first aspect of the embodiments of the present disclosure.

[0262] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0263] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0264] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0265] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video playback method, characterized in that: Applicable to mobile devices, including: Display the first-person perspective of the current video on the preset page; In response to a perspective adjustment instruction triggered by the posture adjustment of the mobile device, a second perspective picture of the current video is displayed on the preset page; the second perspective picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video; wherein the multiple perspective pictures contained in the current video include the first perspective picture and the second perspective picture; In the case where the posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the biological part clamping the mobile device, if the duration of displaying the second viewing angle picture of the current video on the preset page exceeds the preset duration, a corresponding relationship between the posture and viewing angle is generated based on the second viewing angle corresponding to the second viewing angle picture and the initial posture information in the preset coordinate system; Replacing the correspondence between the device posture and the playback perspective with the correspondence between the posture and perspective; Displaying posture adjustment guidance information on the preset page; the posture adjustment guidance information indicates that the initial posture information in the preset coordinate system corresponds to the second viewing angle; If a posture adjustment corresponding to the posture adjustment guide information is detected, during the adjustment process, the second perspective screen of the current video is maintained on the preset page; the posture adjustment representation corresponding to the posture adjustment guide information is adjusted from the adjusted posture information corresponding to the second perspective to the initial posture information in the preset coordinate system.

2. The video playback method according to claim 1, characterized in that: The displaying of the second viewing angle picture of the current video on the preset page in response to the viewing angle adjustment instruction triggered by the posture adjustment of the mobile device comprises: If it is detected that the posture of the mobile device is adjusted, generating the viewing angle adjustment instruction; Determining the adjusted position information of the mobile device based on the viewing angle adjustment instruction; Based on the correspondence between the device posture and the playback perspective, determining a second perspective corresponding to the adjusted posture information; The second perspective picture of the current video corresponding to the second perspective is displayed on the preset page.

3. The video playback method according to claim 2, characterized in that: Before displaying the first-person perspective picture of the current video on the preset page, the method further includes: In response to a play instruction of the current video, acquiring angular velocity information and acceleration information of the mobile device; Determine initial position information in a sensor coordinate system based on the angular velocity information and the acceleration information; Converting the initial position and posture information in the sensor coordinate system according to the screen information of the mobile device to obtain the initial position and posture information in a preset coordinate system; The corresponding relationship between the device posture and the playback viewing angle is determined based on the initial posture information in the preset coordinate system and the preset initial viewing angle.

4. The video playback method according to claim 3, characterized in that: The method further comprises: Obtaining video rendering data corresponding to the preset initial viewing angle; The video rendering data is rendered to generate an initial viewing angle picture of the current video on the preset page.

5. The video playback method according to any one of claims 1 to 4, characterized in that: The posture adjustment of the mobile device is implemented based on the conduction of the posture adjustment of the clamping device that clamps the mobile device.

6. The video playback method according to claim 1 or 2, characterized in that: The method further comprises: Displaying a playback function page; the playback function page includes multiple playback components with different playback capabilities; If a first display instruction triggered by a multi-angle playback component among the playback components with different playback capabilities is detected, displaying a video display page; At least one video to be played is displayed on the video display page; the at least one video to be played includes the current video.

7. The video playback method according to claim 6, characterized in that: The method further comprises: In response to a play instruction of the current video, switching from the video display page to the preset page; Playing a first preset viewing angle picture of the current video on the preset page; the preset page includes prompt information for rotating the viewing angle of the picture; If a perspective rotation instruction triggered by a preset operation performed based on the prompt information is detected, a second preset perspective picture of the current video corresponding to the preset operation is played on the preset page, wherein the preset operation is a contact operation on the screen of the mobile device.

8. The video playback method according to claim 7, characterized in that: The preset page includes a posture adjustment sensing component, and the method further includes: In response to a posture adjustment sensing instruction triggered by the posture adjustment sensing component, acquiring angular velocity information and acceleration information of the mobile device; Determine initial position information in a sensor coordinate system based on the angular velocity information and the acceleration information; Converting the initial position and posture information in the sensor coordinate system according to the screen information of the mobile device to obtain the initial position and posture information in a preset coordinate system; Determining the correspondence between the device posture and the playback viewing angle based on the initial posture information in the preset coordinate system and the preset initial viewing angle; Obtaining video rendering data corresponding to the preset initial viewing angle; The video rendering data is rendered to generate an initial viewing angle picture of the current video on the preset page.

9. A video playback device, characterized in that: include: A first display module is configured to display a first-viewing angle picture of a current video on a preset page; A second display module is configured to execute a viewing angle adjustment instruction triggered in response to a posture adjustment of the mobile device, and display a second viewing angle picture of the current video on the preset page; The second viewing angle picture is generated based on the correspondence between the device posture and the playback perspective; the correspondence between the device posture and the playback perspective represents the correspondence between the posture information of the mobile device and the playback perspective of the current video; wherein the multiple viewing angle pictures included in the current video include the first viewing angle picture and the second viewing angle picture; The corresponding relationship adjustment module is configured to execute, in the case where the posture adjustment of the mobile device is based on the conduction implementation of the posture adjustment of the biological part clamping the mobile device, if the duration of displaying the second perspective picture of the current video on the preset page exceeds the preset duration, generate a corresponding relationship between the posture and perspective based on the second perspective corresponding to the second perspective picture and the initial posture information in the preset coordinate system; and use the corresponding relationship between the posture and perspective to replace the corresponding relationship between the device posture and the playback perspective; The third display module is configured to display posture adjustment guide information on the preset page; the posture adjustment guide information indicates that the initial posture information in the preset coordinate system corresponds to the second viewing angle; if a posture adjustment based on the posture adjustment guide information is detected, during the adjustment process, the second viewing angle screen of the current video is maintained on the preset page; the posture adjustment representation corresponding to the posture adjustment guide information is adjusted from the adjusted posture information corresponding to the second viewing angle to the initial posture information in the preset coordinate system.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video playback method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video playback method as claimed in any one of claims 1 to 8.

Images