View angle control method and device, electronic equipment and storage medium

Abstract

The invention relates to a visual angle control method and device, electronic equipment and a storage medium. The visual angle control method comprises the steps that under a first display visual angle of a target video, first control information and second control information are acquired, the first control information represents sliding information on a screen, and the second control information represents rotation information of the electronic equipment; determining a model matrix based on the first control information; determining a view matrix based on the second control information; and adjusting the display view angle of the target video from the first display view angle to a second display view angle based on the model matrix and the view matrix. According to the method, the second display view angle is consistent with the view angle control result expected by the user, the accuracy of target video playing view angle control is ensured, and the watching experience of the user on the target video is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of panoramic video, and in particular, to a viewing angle control method, apparatus, electronic device, and storage medium. Background Art

[0002] With the continuous development of network technology, the types of videos are becoming more and more diverse. Compared with ordinary single-view videos, panoramic videos can control the playback viewing angle by operating the video playback device, enabling users to view the video from multiple different viewing angles and bringing a more immersive video viewing experience to users. However, in related technologies, there are certain limitations in the control of the panoramic video playback viewing angle, and the viewing angle control accuracy is relatively poor, affecting the user's viewing experience of panoramic videos. Summary of the Invention

[0003] To overcome the problems existing in related technologies, the present disclosure provides a viewing angle control method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a viewing angle control method is provided, and the method includes:

[0005] Obtain first control information and second control information at a first display viewing angle of a target video, where the first control information represents swiping information on a screen, and the second control information represents rotation information of an electronic device;

[0006] Determine a model matrix based on the first control information;

[0007] Determine a view matrix based on the second control information;

[0008] Adjust the display viewing angle of the target video from the first display viewing angle to a second display viewing angle based on the model matrix and the view matrix.

[0009] In an exemplary embodiment, the determining the model matrix based on the first control information includes:

[0010] Determine a first rotation angle at which the target video rotates around the coordinate axes in a world coordinate system based on the first control information, where the first rotation angle includes a first horizontal axis rotation angle and a first vertical axis rotation angle;

[0011] Determine a first rotation matrix based on the first rotation angle, where the first rotation matrix includes a first horizontal axis rotation matrix and a first vertical axis rotation matrix;

[0012] Determine the model matrix based on the first rotation matrix.

[0013] In an exemplary embodiment, the first control information is the first swiping distance in the model coordinate system, and the first swiping distance includes the swiping distance of the horizontal axis and the swiping distance of the vertical axis in the model coordinate system;

[0014] Determining, based on the first control information, a first rotation angle of rotation about a coordinate axis in the world coordinate system includes:

[0015] Determining a second swiping distance in the world coordinate system based on the first swiping distance;

[0016] Determining the first rotation angle based on the second swiping distance.

[0017] In an exemplary embodiment, determining the first rotation angle based on the second swiping distance includes:

[0018] Determining the first rotation angle based on the second swiping distance and a historical swiping distance in the world coordinate system.

[0019] In an exemplary embodiment, determining a view matrix based on the second control information includes:

[0020] Determining an origin, a viewpoint, and an up vector of the camera coordinate system based on the second control information;

[0021] Determining the view matrix based on the origin, the viewpoint, and the up vector.

[0022] In an exemplary embodiment, the second control information is a second rotation angle of rotation about a coordinate axis of the electronic device in the world coordinate system, and the second rotation angle includes a second horizontal axis rotation angle, a second vertical axis rotation angle, and a second vertical axis rotation angle;

[0023] Determining an origin, a viewpoint, and an up vector of the camera coordinate system based on the second control information includes:

[0024] Taking the origin of the world coordinate system as the origin of the camera coordinate system;

[0025] Rotating a preset point in the world coordinate system by the second horizontal axis rotation angle about the horizontal axis and by the second vertical axis rotation angle about the vertical axis to obtain the coordinates of the viewpoint in the world coordinate system;

[0026] Determining the up vector according to the origin and the viewpoint.

[0027] In an exemplary embodiment, determining the up vector according to the origin and the viewpoint includes:

[0028] Determining a first vector according to the origin and the viewpoint;

[0029] Rotate the preset vector in the world coordinate system by the second horizontal axis rotation angle around the horizontal axis, by the second vertical axis rotation angle around the vertical axis, and by the second vertical axis rotation angle around the first vector to obtain the upward vector.

[0030] In an exemplary embodiment, adjusting the display perspective of the target video from the first display perspective to the second display perspective based on the model matrix and the view matrix includes:

[0031] In the first display perspective, obtain the initial positions of all key points on the target video in the world coordinate system;

[0032] Based on the model matrix and the view matrix, adjust the initial positions of the key points to obtain the target positions of the key points;

[0033] Based on the target positions of the key points, display the second display perspective of the target video.

[0034] In an exemplary embodiment, the target video includes a panoramic video.

[0035] According to the second aspect of the embodiments of the present disclosure, there is provided a perspective control device, the device includes:

[0036] An acquisition module, configured to acquire first control information and second control information in a first display perspective of a target video, where the first control information represents swipe information on a screen, and the second control information represents rotation information of an electronic device;

[0037] A calculation module, configured to determine a model matrix based on the first control information; and determine a view matrix based on the second control information;

[0038] A rendering module, configured to adjust the display perspective of the target video from the first display perspective to the second display perspective based on the model matrix and the view matrix.

[0039] According to the third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0040] A processor;

[0041] A memory for storing processor-executable instructions;

[0042] Wherein, the processor is configured to execute the method described in the first aspect of the embodiments of the present disclosure.

[0043] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the first aspect of the embodiments of the present disclosure.

[0044] By adopting the above method of the present disclosure, the following beneficial effects are achieved: It can make the second display viewing angle consistent with the viewing angle control result expected by the user, ensure the accuracy of the target video playback viewing angle control, and improve the user's viewing experience of the target video.

[0045] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0047] Figure 1 is a schematic diagram of panoramic video playback shown according to an exemplary embodiment;

[0048] Figure 2 is a flowchart of a viewing angle control method shown according to an exemplary embodiment;

[0049] Figure 3 is a schematic diagram of viewing angle control shown according to an exemplary embodiment;

[0050] Figure 4 is a flowchart of a viewing angle control method shown according to an exemplary embodiment;

[0051] Figure 5A is a schematic diagram of a first swiping distance shown according to an exemplary embodiment;

[0052] Figure 5B is a schematic diagram of a second swiping distance shown according to an exemplary embodiment;

[0053] Figure 6 is a block diagram of a viewing angle control device shown according to an exemplary embodiment;

[0054] Figure 7 is a block diagram of an electronic device shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0056] Figure 1 is a schematic diagram of panoramic video playback shown according to an exemplary embodiment, as Figure 1 shown, the panoramic video obtained after decoding is a distorted picture and cannot be directly rendered and drawn on the screen of the playback device. When playing a panoramic video on the playback device, first, the decoded video picture is rendered onto the surface of a sphere through OpenGL or other 3D rendering tools, and then by setting a coordinate transformation matrix, a part of the picture on the sphere surface is rendered and drawn on the screen of the playback device.

[0057] In the related art, the control of the playback perspective of a panoramic video is achieved by the user swiping the screen of the playback device or the user rotating the playback device. When the playback device receives the swiping information or the rotation information, the swiping information or the rotation information is converted into the rotation angles of the sphere where the panoramic video is located around the X, Y, and Z axes of the world coordinate system, and then the sphere is rotated according to the corresponding rotation angles, so that the pictures of other parts of the sphere are rendered and drawn on the screen of the playback device to complete the switching of the playback perspective, thus changing the user's viewing perspective. However, when rotating the sphere around the three axes in the world coordinate system, the rotation result is related to the rotation order, and the rotation order includes six types: X - Y - Z, X - Z - Y, Y - X - Z, Y - Z - X, Z - X - Y, and Z - Y - X. Therefore, the playback perspective control result corresponding to the same control information is not unique, and it is very easy to have a deviation between the perspective control result and the perspective control result expected by the user. Moreover, after rotating multiple times, it is often very difficult to switch back to the initial playback perspective, which affects the user's viewing experience of the panoramic video.

[0058] In an exemplary embodiment of the present disclosure, to overcome the problem of inaccurate perspective control in panoramic video playback in the related art, a perspective control method is provided, including: obtaining first control information and second control information from a first display perspective of a target video, where the first control information represents swiping information on a screen, and the second control information represents rotation information of an electronic device; determining a model matrix based on the first control information; determining a view matrix based on the second control information; and adjusting the display perspective of the target video from the first display perspective to a second display perspective based on the model matrix and the view matrix. This method can make the second display perspective consistent with the perspective control result expected by the user, ensure the accuracy of the target video playback perspective control, and improve the user's viewing experience of the target video.

[0059] In an exemplary embodiment of the present disclosure, a perspective control method is provided. Figure 2 FIG. is a flowchart of a perspective control method shown according to an exemplary embodiment, as Figure 2 shown, and includes the following steps:

[0060] Step S201, obtaining first control information and second control information from a first display perspective of a target video, where the first control information represents swiping information on a screen, and the second control information represents rotation information of an electronic device;

[0061] Step S202, determining a model matrix based on the first control information;

[0062] Step S203, determining a view matrix based on the second control information;

[0063] Step S204, adjusting the display perspective of the target video from the first display perspective to a second display perspective based on the model matrix and the view matrix.

[0064] The perspective control method in an exemplary embodiment of the present disclosure is applied to an electronic device, including electronic devices with panoramic video playback functions such as smartphones, tablets, smart screens, smart wearable devices, smart home devices, and intelligent vehicle systems. As the perspective control method for playing panoramic videos by default in the built-in application of the electronic device operating system, for third-party application programs with other perspective control methods, the perspective control method in the present disclosure can be used by default to replace the original perspective control method, or relevant settings for the user to select the perspective control method can be provided, and the user can set whether to use the perspective control method in the present disclosure.

[0065] In step S201, the target video is a video with multiple playback perspectives, such as a panoramic video, such as a cityscape video, a tour video, an interior design video, a virtual reality video, etc., which are videos for playing panoramas. The target video in this disclosure takes the panoramic video as an example. The first display perspective represents the current display perspective, which can be the default initial display perspective of the panoramic video or the display perspective after adjusting the perspective based on the initial display perspective. In the first display perspective, a partial picture content of the panoramic video is displayed. For example, Figure 1 The picture rendered on the screen as shown is the first display perspective of the target video.

[0066] The first control information represents the swiping information on the screen. When the user swipes the screen of the electronic device, the electronic device receives the swiping information, which includes the swiping position and the swiping distance. For example, when the user swipes from point A to point B on the screen, the swiping position is from point A to point B, and the swiping distance is the length from point A to point B. The first control information is obtained by reading the swiping information received by the electronic device. The second control information represents the rotation information of the electronic device. When the user shakes the electronic device, the angle by which the electronic device rotates around the coordinate axes in the world coordinate system is the second control information, and the second control information is obtained by sensors such as gyroscopes, accelerometers, and magnetometers.

[0067] In step S202, the model coordinate system represents the electronic device coordinate system, with the electronic device as the reference object of the coordinate system. For example, with the center of the electronic device as the origin, the width of the electronic device as the horizontal axis direction, and the length of the electronic device as the vertical axis direction, the established coordinate system is the model coordinate system. The world coordinate system represents the horizontal direction as the horizontal axis direction and the vertical direction as the vertical axis direction. Since the position of the electronic device in the world coordinate system is not fixed, in order to accurately obtain the swiping information, the data included in the swiping information are all data in the model coordinate system. For example, the position from point A to point B on the screen is in the model coordinate system.

[0068] However, when the user swipes the screen, the expected swiping operation or the swiping operation in the mind is relative to the world coordinate system. For example, no matter how the electronic device is placed, when the user swipes the screen vertically upward, the swiping operation in the mind is vertically upward, and vertically upward is a concept in the world coordinate system. Therefore, when controlling the perspective according to the first control information, it is necessary to rotate the sphere in the world coordinate system to make the rotation effect correspond to the swiping operation in the user's mind and ensure the accuracy of perspective control.

[0069] The model matrix is used to control the rotation of the sphere where the panoramic video is located in the world coordinate system. According to the first control information, the swiping operation in the user's awareness and the expected perspective transformation result can be obtained. For example, when the first control information is the swiping distance from point A to point B on the screen, it indicates that the user expects the sphere to rotate in the world coordinate system in the vector direction. Therefore, by determining the model matrix according to the first control information, the sphere can be rotated in the world coordinate system based on the swiping information to adjust to the user's expected viewing perspective.

[0070] In step S203, the camera coordinate system is also called the observer coordinate system. Taking the observer as the reference object of the coordinate system, for example, taking the human eye as the origin of the coordinate system, the direction directly in front of the human eye as one axis direction, the direction directly above the observer as one axis direction, and the left or right side of the observer as one axis direction, the established coordinate system is the camera coordinate system. In the camera coordinate system, the effect of rotating the sphere is achieved by rotating the human eye. Therefore, the picture in the target video does not rotate relative to the human eye.

[0071] The second control information represents the rotation information of the electronic device, that is, the rotation angle of the electronic device in the world coordinate system. The view matrix can convert the points in the world coordinate system into the points in the camera coordinate system. Therefore, by determining the view matrix according to the second control information, the rotation information of the electronic device in the world coordinate system can be converted to the camera coordinate system. And since the video picture in the camera coordinate system does not rotate relative to the human eye, it can be ensured that the rotation information of the electronic device in the world coordinate system will not affect the perspective control. That is, no matter how the electronic device is placed, the target video can be switched in perspective with the user's expected swiping operation.

[0072] In step S204, the second display perspective represents the display perspective after perspective control. Another part of the video picture is displayed in the panoramic video under the second display perspective. Through the model matrix, the sphere where the panoramic video is located can be rotated to the target position in the world coordinate system according to the user's expected rotation direction. Since the swiping information only has the information of two coordinate axes, and when rotating the sphere around two coordinate axes in the world coordinate system, the rotation result is independent of the rotation order. That is, whether rotating around the horizontal axis first and then the vertical axis, or rotating around the vertical axis first and then the horizontal axis, the obtained result is the same. Therefore, there will be no inaccurate problem. Then, through the view matrix, the rotation information of the electronic device can be converted to the camera coordinate system to ensure that the video picture does not rotate relative to the human eye, avoiding rotating the sphere around three coordinate axes in the world coordinate system. Therefore, it can be ensured that the second display perspective is accurate in any case.

[0073] In one example, Figure 3 is a schematic diagram of perspective control shown according to an exemplary embodiment, as Figure 3As shown in the figure, the first row in the figure represents the rendering effect of the first display perspective, and the second row represents the rendering effect of the second display perspective. On the left is the rendering effect in the embodiment of the present disclosure, and on the right is the rendering effect in the related art. When the electronic device is placed at a certain inclination angle in the vertical direction, the coordinate system in the figure represents the direction in the world coordinate system. The user swipes the screen to the left in the horizontal direction. At this time, the user expects to rotate the sphere to the left in the horizontal direction and view the video image directly to the right of the current video image. Since the electronic device has a rotation angle around the z-axis in the world coordinate system, in the related art, it is necessary to rotate around three coordinate axes in the world coordinate system, and the obtained second display perspective is the video image of the sky in the upper right of the current video image, which is different from the video image that the user expects to view and is inaccurate. In the embodiment of the present disclosure, the sphere is rotated according to the effect expected by the user through the model matrix, and the view matrix is used to ensure that the video image does not rotate relative to the human eye. The obtained second display perspective is the video image directly to the right of the current video image, which is the same as the video image that the user expects to view and is accurate.

[0074] It should be noted that when the user does not swipe the screen, the relevant value in the first control information can be set to 0. When the electronic device does not rotate, the relevant value in the second control information can be set to 0.

[0075] In the exemplary embodiment of the present disclosure, in the first display perspective of the target video, based on the swiping information of the user on the screen, the model matrix is determined, and based on the rotation information of the electronic device, the view matrix is determined. Based on the model matrix and the view matrix, the display perspective of the target video is adjusted from the first display perspective to the second display perspective, which can make the second display perspective consistent with the perspective control result expected by the user, ensure the accuracy of the target video playback perspective control, and improve the user's viewing experience of the target video.

[0076] In an exemplary embodiment of the present disclosure, a perspective control method is provided. Figure 4 It is a flowchart of a perspective control method shown according to an exemplary embodiment, as Figure 4 shown, and includes the following steps:

[0077] Step S401, in the first display perspective of the target video, obtain the first control information and the second control information. The first control information represents the swiping information on the screen, and the second control information represents the rotation information of the electronic device;

[0078] Step S402, based on the first control information, determine the first rotation angle of the target video rotating around the coordinate axes in the world coordinate system. The first rotation angle includes the first horizontal axis rotation angle and the first vertical axis rotation angle;

[0079] Step S403: Determine a first rotation matrix based on the first rotation angle. The first rotation matrix includes a first horizontal axis rotation matrix and a first vertical axis rotation matrix.

[0080] Step S404: Determine a model matrix based on the first rotation matrix.

[0081] Step S405: Determine the origin, the viewpoint, and the up vector of the camera coordinate system based on the second control information.

[0082] Step S406: Determine a view matrix based on the origin, the viewpoint, and the up vector.

[0083] Step S407: Obtain the initial positions of all key points on the target video in the world coordinate system under the first display perspective.

[0084] Step S408: Adjust the initial positions of the key points based on the model matrix and the view matrix to obtain the target positions of the key points.

[0085] Step S409: Display the second display perspective of the target video based on the target positions of the key points.

[0086] Specifically, for the implementation manner of step S401, refer to step S201, which will not be elaborated here.

[0087] In step S402, the first rotation angle of the target video rotating around the coordinate axes in the world coordinate system is determined by the swiping distance. When the swiping distance is long, it indicates that the user expects a large amplitude of view rotation. When the swiping distance is short, it indicates that the user expects a small amplitude of view rotation. Since the first control information is the swiping information in the model coordinate system, and view control requires the target video to rotate in the world coordinate system, it is necessary to convert the first control information into the swiping information in the world coordinate system, and then determine the first rotation angle according to the swiping information in the world coordinate system. The first rotation angle includes a first horizontal axis rotation angle and a first vertical axis rotation angle, that is, the rotation angle around the horizontal axis and the rotation angle around the vertical axis.

[0088] In some possible implementation manners, the first control information includes a first swiping distance in the model coordinate system. The first swiping distance includes the swiping distance of the horizontal axis in the model coordinate system and the swiping distance of the vertical axis in the model coordinate system.

[0089] When the user swipes the screen, the electronic device obtains the first control information, which includes the swiping distance and the swiping direction. For the convenience of the user's calculation, the swiping distance will be mapped to the coordinate axes of the model coordinate system. The first swiping distance is the swiping distance on the coordinate axes, including the swiping distance mapped to the horizontal axis and the swiping distance mapped to the vertical axis. In one example, Figure 5A is a schematic diagram of the first swiping distance shown according to an exemplary embodiment, asFigure 5A As shown, the electronic device is placed at an inclination angle of γ in the world coordinate system. Taking the center of the electronic device as the origin, the length of the electronic device as the Y'-axis, and the width of the electronic device as the X'-axis, a model coordinate system X'Y' is established. The user swipes upward in the vertical direction on the screen, and the swiping distance is represented by the vector That is, the user swipes from point O to point P, maps the swiping distance to the coordinate axes of the model coordinate system X'Y', and obtains the swiping distance vector on the Y'-axis and the swiping distance vector on the X'-axis, and satisfies the relationship: When reading the first swiping distance from the electronic device, that is, reading and

[0090] In some possible implementation manners, based on the first control information, determining a first rotation angle of rotation around the coordinate axis in the world coordinate system includes the following steps:

[0091] S42-1, based on the first swiping distance, determining a second swiping distance in the world coordinate system.

[0092] Converting the first swiping distance into a second swiping distance in the world coordinate system. In one example, Figure 5B is a schematic diagram of the second swiping distance shown according to an exemplary embodiment. As Figure 5B shown, on the basis of Figure 5A , taking the horizontal direction as the X-axis and the vertical direction as the Y-axis, a world coordinate system XY is established. The coordinate representation of the actual swiping distance of the user in the world coordinate system is (X, Y), that is, the coordinate representation of the second swiping distance is (X, Y). Determining the coordinate representation of the first swiping distance in the world coordinate system, then the coordinate representation of the swiping distance vector on the Y'-axis in the world coordinate system is (x1, y1), and the coordinate representation of the swiping distance vector on the X'-axis in the world coordinate system is (x2, y2). Then the coordinate representation of the actual swiping distance of the user in the world coordinate system is (x1 + x2, y1 + y2). Therefore, the horizontal axis coordinate of the second swiping distance is X = x1 + x2, and the vertical axis coordinate is Y = y1 + y2.

[0093] S42-2, based on the second swiping distance, determining the first rotation angle.

[0094] After obtaining the second swiping distance, according to the mapping relationship between the second swiping distance and the first rotation angle, the first rotation angle is determined. The mapping relationship between the second swiping distance and the first rotation angle is a preset relationship. For example, the second swiping distance and the first rotation angle satisfy a preset proportional relationship. When the relationship between the second swiping distance and the first rotation angle is 1:1, if the second swiping distance is 10 pixel lengths, the first rotation angle is 10 degrees; when the relationship between the second swiping distance and the first rotation angle is 1:3, if the second swiping distance is 10 pixel lengths, the first rotation angle is 30 degrees. Map the second swiping distance onto the coordinate axes of the world coordinate system. The swiping distance on the horizontal axis corresponds to the first horizontal axis rotation angle, and the swiping distance on the vertical axis corresponds to the first vertical axis rotation angle. Figure 5A For example, if the swiping distance on the horizontal axis of the second swiping distance is 0, the first horizontal axis rotation angle is 0 degrees. If the swiping distance on the vertical axis is assumed to be 10 pixel lengths, the first vertical axis rotation angle is the rotation angle corresponding to 10 pixel lengths.

[0095] In some possible implementation manners, when determining the first rotation angle based on the second swiping distance, the first rotation angle is determined based on the second swiping distance and the historical swiping distance in the world coordinates.

[0096] If the first display perspective is the initial default display perspective of the target video, the historical swiping distance is 0; if the first display perspective is not the initial default display perspective of the target video, the historical swiping distance is the total swiping distance from the initial default display perspective to the first display perspective. Take the sum of the second swiping distance and the historical swiping distance as the target swiping distance, that is, the target swiping distance is the cumulative sum of the swiping distances, and determine the first rotation angle according to the target swiping distance. Figure 5B In the example of, the horizontal axis coordinate of the target swiping distance is represented as X+ = x1 + x2, and the vertical axis coordinate is represented as Y+ = y1 + y2. Among them, for the specific implementation manner of determining the first rotation angle according to the target swiping distance, refer to the implementation manner of determining the first rotation angle according to the second swiping distance.

[0097] In step S403, denote the first horizontal axis rotation angle as α 1 , denote the first vertical axis rotation angle as β 1 , denote the first horizontal axis rotation matrix as R x (α 1 ), denote the first vertical axis rotation matrix as R y (β 1 ), then the first rotation matrix is expressed as:

[0098]

[0099]

[0100] In step S404, the model matrix is denoted as R model , then the model matrix is expressed as:

[0101] R model = R x (α 1 ) × R y (β 1 )

[0102] In step S405, the origin, view point, and up vector of the camera coordinate system are the three elements that determine the camera coordinate system. The origin is the position where the observer is located, the view point is in the direction directly in front of the observer's line of sight, and the up vector is on the immediate left or right of the observer's line of sight, perpendicular to the view point direction. The up vector is also called the up vector. According to the second control information, the coordinate representations of the above three parameters in the world coordinate system are obtained respectively, and then the camera coordinate system can be established.

[0103] In some possible implementation manners, the second control information is the second rotation angle of the electronic device rotating around the coordinate axes in the world coordinate system. The second rotation angle includes the second horizontal axis rotation angle, the second vertical axis rotation angle, and the second vertical axis rotation angle.

[0104] The second rotation angle is obtained through sensors such as a gyroscope and an accelerometer. Figure 5A Taking... as an example, the second rotation angle obtained at this time is: the second horizontal axis rotation angle is 0 degrees, the second vertical axis rotation angle is 0 degrees, and the second vertical axis rotation angle is γ degrees.

[0105] In some possible implementation manners, based on the second control information, determining the origin, view point, and up vector of the camera coordinate system includes the following steps:

[0106] S45-1, taking the origin of the world coordinate system as the origin of the camera coordinate system.

[0107] The origin of the camera coordinate system is the origin coordinate (0, 0, 0) of the world coordinate system.

[0108] S45-2, rotating the preset point in the world coordinate system by the second horizontal axis rotation angle around the horizontal axis and by the second vertical axis rotation angle around the vertical axis to obtain the coordinates of the view point in the world coordinate system.

[0109] The preset point in the world coordinate system is a point on the vertical axis of the world coordinate system. For example, a point with coordinates (0, 0, 1). Denote the second horizontal axis rotation angle as α 2 , and the second vertical axis rotation angle as β 2 . Rotate the preset point by α 2 degrees around the horizontal axis of the world coordinate system and by β 2Degree, and the coordinates of the obtained point are the coordinates of the view point in the world coordinate system.

[0110] S45-3. Determine the upward vector according to the origin and the view point.

[0111] According to the origin and the view point, determine the first vector. Denote the origin as point O and the view point as point E. Then the first vector is Rotate the preset vector in the world coordinate system by the second horizontal axis rotation angle around the horizontal axis, by the second vertical axis rotation angle around the vertical axis, and by the second vertical axis rotation angle around the first vector to obtain the upward vector. The preset vector in the world coordinate system is a vector in the direction of the vertical axis of the world coordinate system. For example, a vector with coordinates (0, 1, 0). Denote the second horizontal axis rotation angle as α 2 , and denote the second vertical axis rotation angle as β 2 , and denote the second vertical axis rotation angle as γ degrees. Rotate the preset vector by α 2 degrees around the horizontal axis of the world coordinate system, by β 2 degrees around the vertical axis of the world coordinate system, and by γ degrees around the first vector. The coordinates of the obtained vector are denoted as the coordinates of the upward vector in the world coordinate system.

[0112] In step S406, after obtaining the origin, view point, and upward vector of the camera coordinate system, the camera coordinate system is obtained. The view matrix can be determined by matrix derivation. Translate and rotate the camera coordinate system through the translation matrix and rotation matrix to make it coincide with the world coordinate system. The product of the translation matrix and the rotation matrix is the view matrix; alternatively, a function for calculating the view matrix in the 3D rendering tool can be called. For example, call the LookUp function in openGL, and input the origin, view point, and upward vector of the camera coordinate system to output the view matrix.

[0113] In step S407, the key points on the target video represent the key points in the sphere where the target video is located. For example, the sphere can be divided into multiple small triangles, and the vertices of each small triangle are the key points. The positions of all key points in the world coordinate system are the coordinates of the vertices of each small triangle in the world coordinate system. In the first display perspective, the positions of all key points in the world coordinate system are denoted as the initial positions. Obtain the initial positions of all key points through the rendering tool of the panoramic video (such as openGL).

[0114] In step S408, the target positions of the key points represent the positions of the key points after perspective control, that is, in the second display perspective, the positions of the key points in the world coordinate system. The target positions of each key point can be obtained through the following matrix calculation formula:

[0115] P 2 =R projection ×Rview ×R model ×P 1

[0116] Among them, P 2 represents the target position coordinates of each key point, and R projection represents the perspective projection matrix, and R view represents the view matrix, and R madel represents the model matrix, and P 1 represents the initial position coordinates of each key point. Among them, the perspective projection matrix is used to map the 3D space to the 2D space to map and display the panoramic video on the screen of the electronic device.

[0117] The target position of each key point can also be obtained through a panoramic video rendering tool. By inputting the initial position, model matrix, and view matrix of each key point into the rendering tool of the panoramic video, such as openGL, the target position of each key point can be output.

[0118] In step S409, after obtaining the target positions of all key points, through the panoramic video rendering tool, the video frame corresponding to the second display perspective of the target video can be rendered and drawn on the screen of the electronic device, such as using the vertex shader in openGL.

[0119] In an exemplary embodiment of the present disclosure, a viewing angle control device is provided. Figure 6 It is a block diagram of a viewing angle control device shown according to an exemplary embodiment, as Figure 6 shown, including:

[0120] An acquisition module 601, configured to acquire first control information and second control information in the first display perspective of the target video. The first control information represents the swiping information on the screen, and the second control information represents the rotation information of the electronic device;

[0121] A calculation module 602, configured to determine the model matrix based on the first control information; and determine the view matrix based on the second control information;

[0122] A rendering module 603, configured to adjust the display perspective of the target video from the first display perspective to the second display perspective based on the model matrix and the view matrix.

[0123] In an exemplary embodiment, the calculation module 602 is further configured to:

[0124] Based on the first control information, determine the first rotation angle of the target video rotating around the coordinate axis in the world coordinate system. The first rotation angle includes the first horizontal axis rotation angle and the first vertical axis rotation angle;

[0125] Based on the first rotation angle, determine a first rotation matrix, where the first rotation matrix includes a first horizontal axis rotation matrix and a first vertical axis rotation matrix;

[0126] Based on the first rotation matrix, determine a model matrix.

[0127] In an exemplary embodiment, the first control information is a first sliding distance in the model coordinate system, where the first sliding distance includes a sliding distance of the horizontal axis and a sliding distance of the vertical axis in the model coordinate system;

[0128] The calculation module 602 is further configured to:

[0129] Based on the first sliding distance, determine a second sliding distance in the world coordinate system;

[0130] Based on the second sliding distance, determine the first rotation angle.

[0131] In an exemplary embodiment, the calculation module 602 is further configured to:

[0132] Based on the second sliding distance and a historical sliding distance in the world coordinate system, determine the first rotation angle.

[0133] In an exemplary embodiment, the calculation module 602 is further configured to:

[0134] Based on the second control information, determine the origin, the view point, and the up vector of the camera coordinate system;

[0135] Based on the origin, the view point, and the up vector, determine a view matrix.

[0136] In an exemplary embodiment, the second control information is a second rotation angle by which the electronic device rotates around the coordinate axes in the world coordinate system, where the second rotation angle includes a second horizontal axis rotation angle, a second vertical axis rotation angle, and a second vertical axis rotation angle;

[0137] The calculation module 602 is further configured to:

[0138] Use the origin of the world coordinate system as the origin of the camera coordinate system;

[0139] Rotate a preset point in the world coordinate system by the second horizontal axis rotation angle around the horizontal axis and by the second vertical axis rotation angle around the vertical axis to obtain the coordinates of the view point in the world coordinate system;

[0140] Determine the up vector according to the origin and the view point.

[0141] In an exemplary embodiment, the calculation module 602 is further configured to:

[0142] Determine a first vector according to the origin and the view point;

[0143] Rotate a preset vector in the world coordinate system by a second horizontal axis rotation angle about the horizontal axis, by a second vertical axis rotation angle about the vertical axis, and by a second vertical axis rotation angle about the first vector to obtain an upward vector.

[0144] In an exemplary embodiment, the rendering module 603 is further configured to:

[0145] In a first display perspective, obtain the initial positions of all key points on the target video in the world coordinate system;

[0146] Based on the model matrix and the view matrix, adjust the initial positions of the key points to obtain the target positions of the key points;

[0147] Based on the target positions of the key points, display a second display perspective of the target video.

[0148] In an exemplary embodiment, the target video includes a panoramic video.

[0149] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0150] Figure 7 It is a block diagram of an electronic device 700 shown according to an exemplary embodiment.

[0151] Refer to Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0152] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0153] The memory 704 is configured to store various types of data to support the operation of the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, and the like. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0154] The power supply component 706 provides power to various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 700.

[0155] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0156] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 further includes a speaker for outputting audio signals.

[0157] The I / O interface 712 provides an interface between the processing component 702 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power-on button, and a lock button.

[0158] The sensor assembly 714 includes one or more sensors for providing an assessment of various aspects of the status of the electronic device 700. For example, the sensor assembly 714 can detect the on / off state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700. The sensor assembly 714 can also detect a change in the position of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0159] The communication component 716 is configured to facilitate communication between the electronic device 700 and other devices in a wired or wireless manner. The electronic device 700 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0160] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described method.

[0161] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 704 including instructions, is also provided. The above instructions can be executed by the processor 720 of the electronic device 700 to complete the above-described method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0162] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute a perspective control method, and the method includes any of the above methods.

[0163] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only illustrative, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0164] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A perspective control method, characterized in that, the method includes: Obtaining first control information and second control information under a first display perspective of a target video, where the first control information represents swiping information on a screen, and the second control information represents rotation information of an electronic device; Determining a model matrix based on the first control information; Determining a view matrix based on the second control information; Adjusting the display perspective of the target video from the first display perspective to a second display perspective based on the model matrix and the view matrix.

2. The method according to claim 1, characterized in that, the determining the model matrix based on the first control information includes: Determining a first rotation angle of the target video rotating around coordinate axes in a world coordinate system based on the first control information, where the first rotation angle includes a first horizontal axis rotation angle and a first vertical axis rotation angle; Determining a first rotation matrix based on the first rotation angle, where the first rotation matrix includes a first horizontal axis rotation matrix and a first vertical axis rotation matrix; Determining the model matrix based on the first rotation matrix.

3. The method according to claim 2, characterized in that, the first control information is a first swiping distance in a model coordinate system, and the first swiping distance includes a swiping distance of the horizontal axis and a swiping distance of the vertical axis in the model coordinate system; the determining the first rotation angle of rotating around coordinate axes in the world coordinate system based on the first control information includes: Determining a second swiping distance in the world coordinate system based on the first swiping distance; Determining the first rotation angle based on the second swiping distance.

4. The method according to claim 3, characterized in that, the determining the first rotation angle based on the second swiping distance includes: Determining the first rotation angle based on the second swiping distance and a historical swiping distance in the world coordinate system.

5. The method according to claim 1, characterized in that, the determining the view matrix based on the second control information includes: Determining the origin, the view point, and the up vector of a camera coordinate system based on the second control information; Determining the view matrix based on the origin, the view point, and the up vector.

6. The method according to claim 5, characterized in that, the second control information is a second rotation angle of the electronic device rotating around coordinate axes in the world coordinate system, and the second rotation angle includes a second horizontal axis rotation angle, a second vertical axis rotation angle, and a second vertical axis rotation angle; the determining the origin, the view point, and the up vector of the camera coordinate system based on the second control information includes: Taking the origin of the world coordinate system as the origin of the camera coordinate system; Rotating a preset point in the world coordinate system by the second horizontal axis rotation angle around the horizontal axis and by the second vertical axis rotation angle around the vertical axis to obtain the coordinates of the view point in the world coordinate system; Determining the up vector according to the origin and the view point.

7. The method according to claim 6, characterized in that, Determining the upward vector according to the origin and the viewpoint includes: Determining a first vector according to the origin and the viewpoint; Rotating a preset vector in the world coordinate system by a second horizontal axis rotation angle around the horizontal axis, by a second vertical axis rotation angle around the vertical axis, and by a second vertical axis rotation angle around the first vector to obtain the upward vector.

8. The method according to claim 1, wherein, Adjusting the display perspective of the target video from the first display perspective to a second display perspective based on the model matrix and the view matrix includes: Obtaining the initial positions of all key points on the target video in the world coordinate system under the first display perspective; Adjusting the initial positions of the key points based on the model matrix and the view matrix to obtain the target positions of the key points; Displaying the second display perspective of the target video based on the target positions of the key points.

9. The method according to claim 1, wherein, The target video includes a panoramic video.

10. A perspective control device, wherein, The device includes: An acquisition module configured to acquire first control information and second control information under a first display perspective of a target video, where the first control information represents swiping information on a screen, and the second control information represents rotation information of an electronic device; A calculation module configured to determine a model matrix based on the first control information; and determine a view matrix based on the second control information; A rendering module configured to adjust the display perspective of the target video from the first display perspective to a second display perspective based on the model matrix and the view matrix.

11. An electronic device, wherein, It includes: A processor; A memory for storing processor-executable instructions; wherein, the processor is configured to execute the method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium, wherein, When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1-9.

Images