Electronic device and display method thereof, storage medium

Abstract

The application relates to an electronic device and a display method thereof and a storage medium. An embodiment of the application provides a display method applied to an electronic device, which comprises the following steps: the electronic device plays a first video based on a first frame rate and a first pulse frequency; when the playing state is a first state, the electronic device is adjusted based on a second frame rate and a second pulse frequency corresponding to the change of the displacement size of the display picture of the first video relative to the screen of the electronic device. Through the above display method, in the process that a user wears the electronic device to watch a video, when the displacement size of the display picture of the video relative to the screen is small, a lower preset frame rate and a higher preset pulse frequency are set for the electronic device, and when the displacement size is large, a higher preset frame rate and a lower preset pulse frequency are set for the electronic device, so that the display picture of the video cannot be seriously trailed due to the movement of the electronic device, and the watching effect of the user on the video is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of extended reality. In particular, it relates to an electronic device, a display method thereof, and a storage medium. BACKGROUND

[0002] With the development of extended reality (XR) technologies such as virtual reality (VR) and augmented reality (AR), electronic products based on virtual reality technology are becoming more and more popular. For example, more and more users use VR glasses to watch videos, which can bring users a more independent and immersive viewing experience. At the same time, the sound and picture of the video do not affect people nearby, and the user experience is good.

[0003] However, in the process of a user wearing VR glasses to watch a video, in a dynamic scene, the screen of the VR glasses moves with the movement of the VR glasses (such as the movement of the user's head), and the display picture of the video (i.e., the video region) also moves in the screen with the movement of the screen. If the head movement (especially the head rotation) is fast, the user's eyes will feel that the video has serious trailing, which affects the viewing effect and user experience. Therefore, a display scheme is needed that can improve the viewing effect of the video. SUMMARY

[0004] The present application provides an electronic device, a display method thereof, and a storage medium.

[0005] In a first aspect, an embodiment of the present application provides a display method applied to an electronic device, the method comprising:

[0006] In the process of playing a first video, a playing state of the electronic device is determined, wherein the electronic device plays the first video based on a first frame rate and a first pulse frequency;

[0007] When the playing state is a first state, a displacement size of a display picture of the first video relative to a screen of the electronic device is obtained;

[0008] The electronic device is adjusted based on a second frame rate and a second pulse frequency corresponding to a change in the displacement size, wherein the second frame rate and the second pulse frequency are the same as or different from the first frame rate and the first pulse frequency.

[0009] In the present application, the electronic device herein can include various electronic devices supporting extended reality technology (virtual reality, augmented reality, and mixed reality), for example, the applicable electronic device can include an XR device, an AR device, and a VR device, etc. The playing state can be a viewing scene of the electronic device, the playing state can be determined by an application running on the electronic device for playing the first video, and the first state can be a dynamic scene. The first frame rate herein represents the number of display pictures of the first video that the screen of the electronic device can display per second, and the first pulse frequency represents the number of times that the screen of the electronic device displays one display picture and lights up the display picture within a single frame time. The change of the displacement size represents the displacement of the display picture corresponding to the first video in the screen of the electronic device relative to the coordinate system of the screen of the electronic device, i.e., the number of pixel movements.

[0010] It can be seen that, by the display method of the first aspect, in the process of the user wearing the electronic device to watch the video, the displacement size of the display picture of the video displayed in the screen of the electronic device relative to the screen can be determined according to the motion of the electronic device. In the case that the displacement size is small, a lower preset frame rate and a higher preset pulse frequency are set for the electronic device to ensure that the user does not easily suffer from eye fatigue in the process of long-term watching of the video. In the case that the displacement size is relatively large, a higher preset frame rate and a lower preset pulse frequency are set for the electronic device to ensure that the display picture of the video does not appear serious trailing due to the motion of the electronic device, and the viewing effect of the user on the video is ensured.

[0011] In a possible implementation of the above first aspect, the displacement size of the display picture of the first video relative to the screen of the electronic device is obtained by:

[0012] The inertia data of the electronic device is obtained, and the posture and position of the electronic device are predicted according to the inertia data;

[0013] The number of pixel movements of the display picture of the first video relative to the coordinate system of the screen of the electronic device is determined according to the posture and position of the electronic device, wherein the coordinate system includes a plane coordinate system established based on the screen of the electronic device, and wherein the inertia data includes at least one of three-axis acceleration and three-axis angular velocity of the electronic device.

[0014] In the present application, the plane coordinate system herein includes a plane coordinate system established with any one of the corner points of the screen of the electronic device as the origin, such as the left lower corner point, the right lower corner point, etc., or a plane coordinate system established with the midpoint of the screen as the origin.

[0015] In a possible implementation of the above first aspect, the method further comprises:

[0016] In the second state, the electronic device keeps playing the first video based on the first frame rate and the first pulse frequency.

[0017] In a possible implementation of the first aspect, the first state is a dynamic scene in a process in which the electronic device plays the first video, and the second state is a static scene of the electronic device.

[0018] In a possible implementation of the first aspect, adjusting the electronic device based on the change in the displacement size corresponds to a second frame rate and a second pulse frequency, and the adjusting includes:

[0019] Corresponding to the displacement size being less than the first displacement threshold, the second frame rate and the second pulse frequency are equal to the first frame rate and the first pulse frequency.

[0020] In the present application, in the case of a small displacement size, the frame rate and the pulse frequency of the electronic device can be kept unchanged.

[0021] In a possible implementation of the first aspect, adjusting the electronic device based on the change in the displacement size corresponds to a second frame rate and a second pulse frequency, and the adjusting further includes:

[0022] Corresponding to the displacement size being greater than the first displacement threshold and less than a second displacement threshold, the second frame rate is higher than the first frame rate and the second pulse frequency is lower than the first pulse frequency.

[0023] In a possible implementation of the first aspect, adjusting the electronic device based on the change in the displacement size corresponds to a second frame rate and a second pulse frequency, and the adjusting further includes:

[0024] Corresponding to the displacement size being greater than the second displacement threshold, the second frame rate is higher than the first frame rate and the second pulse frequency is lower than the first pulse frequency.

[0025] In the present application, in the case of a large displacement size, a higher frame rate and a lower pulse frequency can be set for the electronic device.

[0026] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0027] a memory configured to store instructions executed by one or more processors of the electronic device, and

[0028] a processor, which is one of the processors of the electronic device, configured to execute the display method of the first aspect.

[0029] In a third aspect, an embodiment of the present application provides a storage medium, and the storage medium stores instructions, which, when executed on an electronic device, cause the electronic device to execute the display method of the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising: a non-volatile computer readable storage medium, the non-volatile computer readable storage medium containing computer program codes for executing the display method of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figures 1(a) to 1(b) A scene diagram of the electronic device provided by the embodiment of the present application presenting a display picture of a video on a screen;

[0032] Figure 2 A flowchart of the display method of the electronic device provided by the embodiment of the present application;

[0033] Figures 3(a) to 3(b) A scene diagram of the electronic device provided by the embodiment of the present application presenting a display picture of a video on a screen;

[0034] Figure 4 A module function diagram of the electronic device provided by the embodiment of the present application;

[0035] Figure 5 A structure diagram of the electronic device provided by the embodiment of the present application;

[0036] Figure 6 A software structure block diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings.

[0038] It can be understood that the technical solutions of the present application are applicable to various electronic devices capable of supporting extended reality technology (virtual reality, augmented reality and mixed reality), for example, the applicable electronic devices can include XR devices, AR devices and VR devices, etc., taking the VR device as an example, it can include VR glasses, VR headsets and VR all-in-one machines, etc.

[0039] Reference Figures 1(a) to 1(b) Taking the VR device in the XR device as the VR glasses as an example, Figures 1(a) to 1(b)An exemplary scene diagram of a display picture of a screen rendering video of the VR glasses 100 is shown. As shown in FIG. 1(a), a user wears the VR glasses 100 to watch a video, and the screen 101 of the VR glasses 100 displays a display picture 102 of the video. The display picture 102 can be located at a fixed position in the screen 101, for example, the center of the display picture 102 is aligned with the center of the screen 101. The method of displaying the video by the VR glasses 100 can include that the screen 101 of the VR glasses 100 displays the display picture of the video at a preset frame rate (for example, M frames per second) and a preset pulse frequency (N pulses per frame). For example, the preset frame rate can be 72 frames per second (FPS), and the preset pulse frequency can be 1 pulse per frame. Wherein, 72 frames per second means that the screen 101 of the VR glasses 100 can display 72 display pictures of the video per second, that is, the single frame time between the Nth frame and the N+1th frame is 1s / 72≈13.9ms, and 1 pulse per frame means that the number of light-on pulses in the single frame time (i.e., 13.9ms) is 4, that is, the screen 101 of the VR glasses 100 displays one display picture every 13.9ms and lights up the display picture four times. At this time, the viewing scene can be a static scene, that is, when the user wears the VR device to watch the video, the display picture is static relative to the screen, and only zero to a few pixels of the content in the display picture move relative to the screen.

[0040] In some embodiments, the preset frame rate and the preset pulse frequency correspond to fixed values. In a dynamic scene, if the user constantly moves / turns the head during the process of wearing the VR glasses 100 to watch the video, the VR glasses 100 generates motion, in order to maintain the sense of immersion, the display picture 102 of the video needs to be kept at a fixed position in the environment, and the video area moves relative to the screen. Referring to FIG. 1(b), the screen 101 of the VR glasses 100 displays a smear 103 corresponding to the display picture 102 of the video. In particular, when the preset frame rate of the VR glasses is low and the preset pulse frequency is high, for example, the preset frame rate is 30 frames per second, and the preset pulse frequency is 4 pulses per frame, that is, one display picture needs to be displayed and lighted up four times in each frame time, and the screen 101 of the VR glasses 100 displays the display picture 102 of the video with more obvious smear.

[0041] In other embodiments, the VR glasses 100 may be set to a higher preset frame rate and a lower preset pulse frequency to avoid severe motion blur in the video display caused by the movement of the VR glasses 100. For example, the preset frame rate is 90 frames per second, that is, the time between the Nth frame and the N+1th frame is 1 second / 90 ≈ 11 milliseconds, and the preset pulse frequency is 1 pulse / frame. However, a sustained high frame rate will cause the VR glasses 100 to generate high power consumption, reducing the usage time of the VR glasses 100, and the preset pulse frequency of 1 pulse / frame will cause eye fatigue after the user watches the video for a long time.

[0042] To address the aforementioned issues, this application proposes a display method for VR devices. When a user is playing a video while wearing a VR device, if the viewing scene corresponding to the VR device is detected to be dynamic, the number of pixels that the video's display image on the VR device's screen has moved relative to the VR device's screen is calculated. This pixel movement count can also represent the displacement of one or more display images relative to the VR device's screen coordinate system within a unit of time (one or more single-frame times, or a preset fixed time period, such as 10 seconds). In other words, it represents the displacement of the video's display image on the VR device's screen after movement relative to the VR device's screen before movement. The preset frame rate and preset pulse frequency of the VR device are adjusted based on the pixel movement count. For example, when the pixel movement count is small, the preset frame rate of the VR device can be adjusted to a lower value and the preset pulse frequency to a higher value; when the pixel movement count is large, the preset frame rate of the VR device can be adjusted to a higher value and the preset pulse frequency to a lower value.

[0043] The above display method may include the following steps: during the playback of the first video, determining the playback state of the electronic device, wherein the electronic device plays the first video based on a first frame rate and a first pulse frequency; when the playback state is a first state (dynamic scene), obtaining the displacement of the display screen of the first video relative to the screen of the electronic device; adjusting the electronic device based on the second frame rate and a second pulse frequency corresponding to the change in displacement, wherein the second frame rate and the second pulse frequency are the same as or different from the first frame rate and the first pulse frequency.

[0044] The coordinate system of the VR device screen can be a planar coordinate system established based on the VR device screen itself. For example, it can be a planar coordinate system established with any one of the corner points of the screen as the origin, such as the bottom left corner, the bottom right corner, etc., or it can be a planar coordinate system established with the center point of the screen as the origin. The displacement of the displayed image relative to the coordinate system of the VR device screen can include the magnitude of the displacement of the center point of the displayed image of the video being played (i.e., the coordinates of the center of the video area) relative to the coordinate system of the VR device screen.

[0045] In some embodiments of the present application, the manner of obtaining the number of pixel movements of the display picture corresponding to the video in the screen of the VR device can include: collecting inertial data of an inertial data sensor (IMU) configured by the VR device during playing of the video, including three-axis acceleration, three-axis angular velocity, etc. of the VR device, and calculating a motion trajectory of the VR device per unit time based on the inertial data. Further, the number of pixel movements of the display picture corresponding to the video in the screen of the VR device is determined according to the motion trajectory of the VR device.

[0046] In some embodiments, if the user wearing the VR device rotates the head in the horizontal direction, the display picture corresponding to the video in the screen of the VR device will also present horizontal movement. For example, the user wearing the VR device faces the front, and then the user rotates the head to the left in parallel, and converts the rotation into a corresponding number of pixel movements. For example, the displacement of the VR device rotating to the left in parallel is 1°, and continuing to refer to FIG. 1(b), it can be determined that the display picture corresponding to the video in the screen of the VR device moves to the right in parallel by 40 pixels (the number of pixel movements), and the display picture 102 is the display picture after movement. The coordinates of the display picture corresponding to the video in the screen of the VR device (i.e. the coordinates of the center of the video area) are displaced by 40 pixels (the number of pixel movements) in the direction of the x-axis in the coordinate system of the screen of the VR device, and the ghosting 103 is close to the display picture before movement.

[0047] For another example, if the user wearing the VR device faces the front and then rotates the head up and down, the number of pixel movements of the coordinates of the display picture corresponding to the video in the screen of the VR device in the direction of the y-axis can be determined through the inertial data.

[0048] It can be understood that the above viewing scene (playing state) can include a dynamic scene (3DoF / 6DoF, Three / Six degrees of freedom tracking, three / six degrees of freedom tracking) and a static scene (0DoF). The dynamic scene can refer to that when the user wearing the VR device watches the video, the video area is fixed relative to the environment and moves on the screen to maintain the sense of immersion. In contrast, the static scene can refer to that when the user wearing the VR device watches the video, the video area is stationary relative to the screen. The application program (APP) for playing the video running on the VR device can determine whether the viewing scene belongs to the dynamic scene or the static scene.

[0049] The 3DoF / 6DoF here refers to that a user wearing a VR device can freely view a video at any position and in any direction in a physical space, and movement of the user wearing the VR device can be captured by sensors of the VR device to determine displacement of the user in the space and changes in the posture of the head. The 3DoF refers to that the VR device can detect rotation of the head of the user in different directions in a three-dimensional space (for example, X-axis rotation, i.e., pitching, Y-axis rotation, i.e., rolling, and Y-axis rotation, i.e., yawing), but cannot detect spatial displacement of the head in up, down, forward, backward, left, and right directions. In comparison, the 6DoF refers to that the VR device can detect changes in spatial displacement of the head in up, down, forward, backward, left, and right directions due to body movement in addition to changes in the field of view angle due to rotation of the head.

[0050] It can be seen that, by using the display method of the VR device, in the process of the user wearing the VR device to view a video, in a dynamic scene, the displacement size of a display picture of the video displayed in a screen of the VR device relative to the screen can be determined according to movement of the VR device. In the case of a small displacement size, a lower preset frame rate and a higher preset pulse frequency are set for the VR device to ensure that the user does not easily experience eye fatigue in a long-term process of viewing the video. In comparison, in the case of a large displacement size, a higher preset frame rate and a lower preset pulse frequency are set for the VR device to ensure that the display picture of the video does not appear to be severely blurred due to movement of the VR device, and the viewing effect of the user on the video is ensured.

[0051] The display method provided in the embodiments of the present application will be described in detail below based on the method flowchart shown in FIG. 2. Figure 2 The method shown in FIG. 2 can be implemented by a processor of the VR device 100 (i.e., the VR glasses 100) shown in FIG. 1(a) executing related instructions. Figure 2 The method shown in FIG. 2 can be implemented by a processor of the VR device 100 (i.e., the VR glasses 100) shown in FIG. 1(a) executing related instructions.

[0052] Referring to FIG. 2, the display method can include the following steps. Figure 2 The display method can include the following steps.

[0053] S201: It is detected that a user uses the VR glasses 100 to view a video, and a viewing scene is determined.

[0054] Exemplarily, after the user wears the VR glasses 100 and turns on the power supply of the VR glasses 100, if the user opens a video playing application of the VR glasses 100, the VR glasses 100 can determine that the user uses the VR glasses 100 to view a video.

[0055] The viewing scene herein can include a dynamic scene and a static scene. The dynamic scene means that during the user's viewing of the video using the VR glasses 100, the display picture (video area) of the video moves on the screen of the VR glasses 100 relative to the environment as the user's head moves. In contrast, the static scene means that during the user's viewing of the video using the VR glasses 100, the video area is static relative to the screen of the VR glasses 100.

[0056] S202: Determine whether the viewing scene is a static scene.

[0057] It can be understood that the basis for determining whether the viewing scene belongs to a dynamic scene or a static scene comes from the application program running on the VR glasses 100 / terminal device (such as a mobile phone or a tablet computer, etc.) in communication connection with the VR glasses 100 playing the video.

[0058] Exemplarily, if the viewing scene is a static scene, step S203 is performed; otherwise, it is indicated that the viewing scene is a dynamic scene, and step S204 is performed to further acquire the inertia data corresponding to the VR glasses 100, i.e., the inertia data corresponding to the motion generated by the VR glasses 100.

[0059] S203: Configure the VR glasses 100 with a first frame rate and a first pulse frequency corresponding to the static scene.

[0060] Exemplarily, the first frame rate herein can be the frame rate of the video source (such as 24fps or 30fps) viewed by the user, and the first pulse frequency herein can be 4 pulses per frame. In some embodiments, if the video source frame rate is not used, a commonly used value can also be used, such as setting the video source frame rate to 30fps.

[0061] S204: Acquire the inertia data corresponding to the VR glasses 100.

[0062] Exemplarily, the inertia data herein can be the inertia data collected by the inertia data sensor configured by the VR glasses 100, and the inertia data can include three-axis acceleration, three-axis angular velocity, etc. of the VR glasses 100.

[0063] S205: Calculate the pixel movement number of the video area center coordinate of the display picture relative to the screen of the VR glasses 100 based on the inertia data.

[0064] Exemplarily, the motion trajectory of the VR glasses 100 is calculated based on the inertial data. Further, the pixel movement number of the display picture corresponding to the video in the screen of the VR glasses 100 is determined according to the motion trajectory of the VR glasses 100, i.e., the pixel movement number of the display picture relative to the video area center coordinate of the screen of the VR glasses 100. Here, the pixel movement number can also represent the displacement size of one or more display pictures relative to the coordinate system of the screen of the VR glasses 100 within a unit time (within one single frame time), i.e., the displacement size of the display picture corresponding to the video in the screen of the VR glasses 100 after the motion relative to the coordinate system of the screen of the VR glasses 100 before the motion.

[0065] In some embodiments, for example: the user wears the VR glasses 100 facing the front, then the user turns the head to the left in parallel, the posture and position of the next frame of the VR glasses 100 can be calculated by the previous counting data corresponding to the three-axis acceleration and three-axis angular velocity of the previous N frames of IMU, and then it can be determined that the display picture corresponding to the video in the screen of the next frame (i.e., N+1 frame) of the VR glasses 100 moves to the right in parallel by 50 pixels (pixel movement number), i.e., in the coordinate system of the screen of the VR glasses 100, the display picture moves in the x-axis direction by 50 pixels (pixel movement number) between two frames.

[0066] In other embodiments, the user can also turn the head up and down in addition to turning the head left and right, at this time, the pixel movement number of the display picture corresponding to the video in the screen of the VR glasses 100 determined according to the motion trajectory of the VR glasses 100 can be the pixel movement number of the display picture moving in the y-axis direction in the coordinate system of the screen of the VR glasses 100.

[0067] It can be understood that the user can also turn the head in multiple directions, at this time, the pixel movement number of the display picture corresponding to the video in the screen of the VR glasses 100 determined according to the motion trajectory of the VR glasses 100 can be the pixel movement number of the display picture moving in the projection direction of the x-axis or y-axis in the coordinate system of the screen of the VR glasses 100.

[0068] S206: Determine whether the pixel movement number is less than a first pixel movement number threshold.

[0069] Exemplarily, if the pixel movement number is less than the first pixel movement number threshold, step S207 is performed, and the VR glasses 100 is configured based on a second frame rate corresponding to the first pixel movement number threshold and a second pulse frequency. If the pixel movement number is greater than the first pixel movement number threshold, step S208 is performed, and the pixel movement number is further compared with a second pixel movement number threshold.

[0070] S207: configuring the VR glasses 100 with the second frame rate and the second pulse frequency corresponding to the first pixel movement threshold.

[0071] For example, as shown in FIG. 3(a), if the first pixel movement threshold is 10, and the pixel movement determined according to step S205 is 8 pixels, the VR glasses 100 is configured with the second frame rate and the second pulse frequency corresponding to the first pixel movement threshold. Here, the second frame rate can also be the frame rate of the video source (such as 24 fps or 30 fps) watched by the user, and the second pulse frequency can also be 4 pulses per frame.

[0072] As can be seen, when the pulse frequency is 4 pulses per frame, the pixel movement is 8, and the frame rate is 30 fps, in a single frame time, the first pulse appears visual persistence, and there is a deviation between the video area center coordinates corresponding to the last three pulses and the first pulse. The ghosting 103 in the screen 101 of the VR glasses 100 is not obvious compared with the display picture 102.

[0073] As can be seen, the second frame rate here is the same as the first frame rate in step S203, and the second pulse frequency is also the same as the first pulse frequency in step S203. In other embodiments, different values can also be set respectively, and the embodiments of the present application are not limited in this regard.

[0074] S208: determining whether the pixel movement is less than a second pixel movement threshold.

[0075] For example, if the pixel movement is greater than the first pixel movement threshold and less than the second pixel movement threshold, step S209 is performed, and the VR glasses 100 is configured with a third frame rate and a third pulse frequency corresponding to the second pixel movement threshold. If the pixel movement is greater than the second pixel movement threshold, step S210 is performed, and the VR glasses 100 is configured with a default frame rate and a default pulse frequency.

[0076] S209: configuring the VR glasses 100 with the third frame rate and the third pulse frequency corresponding to the second pixel movement threshold.

[0077] For example, if the first pixel movement threshold is 10 and the second pixel movement threshold is 20, and the pixel movement determined according to step S205 is 18, the VR glasses 100 is configured with the third frame rate and the third pulse frequency corresponding to the second pixel movement threshold. Here, the third frame rate can be a multiple (such as 2 times) of the frame rate of the video source (such as 24 fps or 30 fps) watched by the user, and the third pulse frequency can be 2 pulses per frame.

[0078] S210: configuring the VR glasses 100 with a default frame rate and a default pulse frequency.

[0079] For example, as shown in FIG. 3(b), if the pixel movement number determined in step S205 is 50 pixels, the VR glasses 100 are configured with a default frame rate and a default pulse frequency. The default frame rate can be a multiple of the frame rate of the video source (e.g., 24 fps or 30 fps) viewed by the user, such as 3 times or 4 times, and the default pulse frequency can be 1 pulse per frame.

[0080] In some embodiments, when the pulse frequency is 1 pulse per frame, the pixel movement number is 50 pixels, and the frame rate is 90 fps, in a single frame time, by setting a higher preset frame rate and a lower preset pulse frequency, it is ensured that the display image of the video does not appear to be trailing due to the movement of the VR glasses 100, so that the trailing 103 in the screen 101 of the VR glasses 100 is not obvious compared with the display image 102.

[0081] It can be understood that after steps S203, S207, S209, and S210 are executed, they can all return to step S201 to continuously detect whether the VR glasses 100 are in a static scene or a dynamic scene.

[0082] It can be seen that the above values are all exemplary, the pulse frequency is not limited to 1, 2, 4 pulses per frame, the frame rate is not limited to 72 fps, 90 fps, 120 fps, 24 fps, 30 fps, 48 fps, 60 fps, and the pixel movement threshold is not limited to 10, 20, 40, 50, 200, etc.

[0083] Through the above Figure 2 The display method described above can determine the displacement size of the display image of the video displayed on the screen of the VR device relative to the screen according to the movement of the VR device during the process of the user wearing the VR device to watch the video. In the case of small displacement size, a lower preset frame rate and a higher preset pulse frequency are set for the VR device to ensure that the user does not easily experience eye fatigue during a long process of watching the video. In the case of large displacement size, a higher preset frame rate and a lower preset pulse frequency are set for the VR device to ensure that the display image of the video does not appear to be severely trailing due to the movement of the VR device, and the viewing effect of the user on the video is ensured.

[0084] The process of implementing the display method of the embodiments of the present application will be described below through the hardware and software structure of the VR device 100, such as Figure 4As shown, the software side of the VR device 100 can include a video application 401, and the hardware side of the VR device 100 can include a system chip and a display screen component, wherein the system chip (System on Chip, SOC) includes a data source 402, a graphics processing unit (Graphics Processing Unit, GPU) 403, a display processing module (Display Processing Unit, DPU) 404, and an inertial data sensor 405; and the display screen component includes a display driver IC (Display Driver IC, DDIC) 406, a column driving circuit 407, a row driving circuit 408, and a pixel matrix 409.

[0085] The video application 401 is configured to play a video file and display a display picture of the video on a screen of the VR device 100, and the video application 401 is in communication connection with the data source 402 and configured to transmit video data corresponding to the video file to the data source 402.

[0086] The data source 402 is configured to store the received video data and transmit the video data to the graphics processing unit 403.

[0087] The graphics processing unit 403 is configured to receive the video data from the data source 402 and receive inertial data from the inertial data sensor 405, and transmit the video data and the inertial data to the data processor 404.

[0088] The display processing module 404 is configured to transmit the video data to the display driver IC 406 for display through a MIPI interface, i.e., a Mobile Industry Processor Interface (Mobile Industry Processor Interface, MIPI), and the display processing module 404 can also be configured to determine whether the preset frame rate and the preset pulse frequency of the VR device 100 need to be adjusted based on the inertial data, i.e., adjust the preset frame rate and the preset pulse frequency of the display screen component for displaying the video data, and if so, send an adjustment instruction to the display driver IC 406.

[0089] The inertial data sensor 405 is configured to acquire inertial data of the VR device 100, such as three-axis acceleration and three-axis angular velocity.

[0090] The display driver IC 406 is configured to transmit the video data to the column driving circuit 407 and the row driving circuit 408 based on the received video data, and adjust the preset frame rate and the preset pulse frequency of the display screen component for displaying the video data based on the received adjustment instruction.

[0091] The column driving circuit 407 and the row driving circuit 408 are configured to display the video data on the pixel matrix 409 through a line-by-line scanning display mode, respectively.

[0092] Figure 5 is a structural schematic diagram of an example of a VR device provided in this embodiment. As shown in Figure 5 The VR device can include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a loudspeaker 570A, a receiver 570B, a microphone 570C, a headset jack 570D, a sensor module 580, a key 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, and the like.

[0093] The sensor module 580 can include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and the like.

[0094] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the VR device. In other embodiments, the VR device can include more or fewer components than those illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0095] The processor 510 can include one or more processing units, for example: the processor 510 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Different processing units can be independent devices, or can be integrated into one or more processors.

[0096] The processor 510 can also include a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. The memory can hold instructions or data that the processor 510 has just used or is using repeatedly. If the processor 510 needs to use the instructions or data again, it can call them directly from the memory. This avoids repeated access and reduces the waiting time of the processor 510, thus improving the efficiency of the system.

[0097] In some embodiments, the processor 510 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0098] It can be understood that the interface connection relationship between the modules shown in the embodiments is only illustrative and does not constitute a structural limitation of the VR device. In other embodiments, the VR device can also use different interface connection methods or combinations of multiple interface connection methods.

[0099] The VR device realizes the display function through the GPU, the display screen 594, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 can include one or more GPUs that execute program instructions to generate or change display information.

[0100] The display screen 594 is used to display images, videos, etc. The display screen 594 includes a display panel.

[0101] The VR device can implement the photographing function through an ISP, a camera 593, a video codec, a GPU, a display screen 594, and an application processor, and the like. The ISP is configured to process data fed back by the camera 593. The camera 593 is configured to capture a still image or a video. In some embodiments, the VR device can include one or N cameras 593, where N is a positive integer greater than 1.

[0102] The external memory interface 520 can be configured to connect an external memory card, for example, a Micro SD card, to expand the storage capacity of the VR device. The external memory card communicates with the processor 510 through the external memory interface 520 to implement a data storage function. For example, files such as music and videos are stored in the external memory card.

[0103] The internal memory 521 can be configured to store computer-executable program codes including instructions. The processor 510 implements various function applications and data processing of the VR device by running the instructions stored in the internal memory 521. For example, in the embodiments of the present application, the processor 510 can run the instructions stored in the internal memory 521 to implement various function applications and data processing of the VR device. The internal memory 521 can include a program storage area and a data storage area.

[0104] The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, and the like), and the like. The data storage area can store data (such as audio data, a phonebook, and the like) created during use of the VR device, and the like. In addition, the internal memory 521 can include a high-speed random access memory, and can further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0105] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the VR device. In other embodiments of the present application, the VR device (such as a mobile phone, a vehicle machine, and the like) can include more or fewer components than those shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0106] Figure 6 is a software structure block diagram of the VR device of the embodiments of the present application.

[0107] The layered architecture divides the software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.

[0108] like Figure 6 As shown, the application layer can include applications such as lifestyle, video, reading, shopping, gallery, calendar, calling, navigation, and music. It can be understood that these applications can be those already installed on the VR device 100, or those not installed on the VR device 100 as retrieved through the Quick Service Center.

[0109] The application framework layer can include layout services, window management services, event input services, display adjustment services, and so on. The layout service determines the position of each display element on the VR device's screen. The event input service responds to user actions on the VR device, changing the content displayed on the VR device's screen. The window management service retrieves the attributes of the application's corresponding windows. The display adjustment service determines the displacement of the video displayed on the VR device's screen relative to the screen based on the VR device's movement; when the displacement is small, it sets a lower preset frame rate and a higher preset pulse frequency for the VR device.

[0110] The hardware abstraction layer can include a rendering service module (SurfaceFlinger), a call module, and so on. The rendering service module (SurfaceFlinger) is used to render and composite one or more layers from one or more application windows to obtain frame data. The call module can be used to transmit received incoming call messages to the call processing module, and to process and transmit received hang-up messages, etc.

[0111] The kernel layer includes display drivers, event drivers, and sensor drivers, among others.

[0112] It should be understood that although the terms "first," "second," etc., may be used herein to describe various features, these features should not be limited by these terms. The use of these terms is merely for distinction and should not be construed as indicating or implying relative importance. For example, without departing from the scope of the exemplary embodiments, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature.

[0113] Furthermore, the various operations will be described as multiple separate operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order of description, and many of these operations may be performed in parallel, concurrently, or simultaneously. Moreover, the order of the operations may also be rearranged. The process may be terminated when the described operations are completed, but additional operations not included in the figures may also be present. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0114] Reference throughout this specification to "one embodiment", "an embodiment", "certain embodiments", "certain implementations" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrases in various places in the specification are not necessarily all referring to the same embodiment.

[0115] The terms "including", "has", "having" and "comprises" and variations thereof as used herein are meant to be synonymous with the term "comprising". The phrase "A / B" means "A or B". The phrase "A and / or B" means "(A), (B) or (A and B)".

[0116] As used herein, the term "module" can refer to, be part of, include an memory (shared, dedicated or group) that stores one or more software or firmware programs, an application specific integrated circuit (ASIC), an electronic circuit and / or a processor (shared, dedicated or group) that execute the software or firmware programs to perform the functions described herein, a combination of hardware and software, and / or other suitable components that provide the described functionality.

[0117] In the drawings, some of the structural or methodological acts can be shown in a particular arrangement and / or order. However, it should be understood that such is merely an example, and that the acts can be performed in a different manner and / or order than as discussed herein. Additionally, some of the acts can not be required in some embodiments, or can be combined with other acts, or can be performed in conjunction with other acts.

[0118] The embodiments of the present application described above with reference to the drawings are merely exemplary and the technical solutions of the present application are not limited to the above examples. Various changes can be made to the above examples without departing from the spirit of the present application. Various modifications and changes can be made to the embodiments of the present application without departing from the spirit and scope of the present application, which can be implemented by those skilled in the art on the basis of the embodiments of the present application disclosed above.

Claims

1. A display method, characterized in that, Applied to electronic devices, the method includes: During the playback of the first video, the playback status of the electronic device is determined. The electronic device plays the first video based on a first frame rate and a first pulse frequency. The first pulse frequency represents the number of times the screen of the electronic device displays one display image and lights up the display image within a single frame time. When the playback state is in the first state, the displacement of the display screen of the first video relative to the screen of the electronic device is obtained. The first state is when the electronic device is in a dynamic scene while playing the first video. The frame rate and pulse frequency of the electronic device are adjusted based on the second frame rate and the second pulse frequency corresponding to the change in the displacement magnitude, wherein the second frame rate and the second pulse frequency are the same as or different from the first frame rate and the first pulse frequency.

2. The method according to claim 1, characterized in that, The step of obtaining the displacement of the displayed image of the first video relative to the screen of the electronic device includes: Acquire the inertial data of the electronic device, and predict the attitude and position of the electronic device based on the inertial data; The number of pixels the first video display screen moves relative to the coordinate system of the screen of the electronic device is determined based on the attitude and position of the electronic device, wherein the coordinate system includes a planar coordinate system established based on the screen of the electronic device, and wherein the inertial data includes at least one of the three-axis acceleration and three-axis angular velocity of the electronic device.

3. The method according to claim 1, characterized in that, Also includes: When the playback state is the second state, the electronic device continues to play the first video based on the first frame rate and the first pulse frequency. The second state is when the electronic device is in a static scene.

4. The method according to claim 3, characterized in that, The adjustment of the frame rate and pulse frequency of the electronic device based on the second frame rate and second pulse frequency corresponding to the change in displacement magnitude includes: When the displacement magnitude is less than the first displacement threshold, the second frame rate and the second pulse frequency are equal to the first frame rate and the first pulse frequency.

5. The method according to claim 3, characterized in that, The method of adjusting the frame rate and pulse frequency of the electronic device based on the second frame rate and second pulse frequency corresponding to the change in displacement magnitude further includes: The displacement magnitude is greater than the first displacement threshold and less than the second displacement threshold, the second frame rate is higher than the first frame rate, and the second pulse frequency is lower than the first pulse frequency.

6. The method according to claim 3, characterized in that, The method of adjusting the frame rate and pulse frequency of the electronic device based on the second frame rate and second pulse frequency corresponding to the change in displacement magnitude further includes: The second frame rate is the default frame rate, and the second pulse frequency is the default pulse frequency, corresponding to the displacement magnitude being greater than the second displacement threshold, the second frame rate being higher than the first frame rate, and the second pulse frequency being lower than the first pulse frequency.

7. An electronic device, characterized in that, include: Memory, used to store instructions executed by one or more processors of an electronic device, and A processor is one of the processors in an electronic device, used to execute the display method according to any one of claims 1-6.

8. A storage medium, characterized in that, The storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the display method according to any one of claims 1-6.

9. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, implements the display method according to any one of claims 1-6.

Images