A video display method and device

Abstract

The present application relates to the field of display technologies, and provides a video display method and device. It detects whether a target video sent by an external device is received. When the target video is not received, it performs time warping on its own screen and displays it. When the initial video frame of the target video is received, it performs three-dimensional full-screen display on the received initial video frame and prohibits time warping on the video frames of the target video, preventing picture jitter caused by incorrect compensation. Moreover, when the video frame is displayed full screen, the user's viewing point always faces the center of the video frame, which can prevent seeing empty pixels during the process of head rotation. During the display of the target video, according to the comparison result between the decoding frame rate of the target video and a preset stuttering threshold, it reasonably controls whether to perform time warping and display on the video frame, thereby reducing the video picture jitter caused during the video transmission process, and further reducing the sense of head dizziness.

Description

Technical Field

[0001] This application relates to the field of display technologies, and particularly to a video display method and device. Background Art

[0002] Generally, when the wearer of an XR device watches a video from the XR device application itself, as the wearer's head rotates, the displayed video image can continuously change within a 360-degree space. Thus, when the rendering frame rate of the XR device is lower than the human eye refresh rate, or there is a visual jump caused by head rotation, it will cause a sense of head dizziness.

[0003] Currently, the time-warp method is mostly used for frame compensation to reduce the sense of dizziness. Time-warp predicts frame compensation through the previous video frame and the current pose information (including position and orientation) of the head-mounted XR device, so as to make the video image smoothly transition and reduce the sense of dizziness.

[0004] When the wearer of an XR device watches a video transmitted from an external device, since the pose information of the head-mounted XR device is transmitted to the external device through the network, and the video image of the external device is also transmitted to the head-mounted XR device through the network. However, during the data transmission process, the head will rotate. If the time-warp is directly used to predict the compensation frame, then the pose information corresponding to the compensation frame is the data before the current pose information, which will have a certain delay. When the head-mounted XR device receives the transmitted video frame, the compensation frame is immediately replaced, causing a jump in the image, and when it is fed back to the human eye, it will cause image jitter and result in a sense of head dizziness. Summary of the Invention

[0005] Embodiments of this application provide a video display method and device to reduce image jitter and thus reduce the sense of head dizziness.

[0006] In a first aspect, embodiments of this application provide a video display method, including:

[0007] Detect whether a target video sent by an external device is received;

[0008] When the initial video frame of the target video sent by the external device is not received, maintain time-warping and display of its own image, where the time-warping is used for three-dimensional image frame compensation;

[0009] When the initial video frame of the target video sent by the external device is received, perform three-dimensional full-screen display of the initial video frame and prohibit time-warping of the video frames of the target video.

[0010] In a second aspect, an XR device provided by an embodiment of the present application includes a processor, a memory, a display, and at least one external communication interface. The at least one external communication interface, the display, and the memory are connected to the processor via a bus;

[0011] The memory stores computer program instructions, and the processor performs the following operations according to the computer program instructions:

[0012] Detect whether a target video sent by an external device is received through the at least one external communication interface;

[0013] When an initial video frame of the target video sent by the external device is not received, maintain time warping of its own picture and display it through the display, where the time warping is used for three-dimensional image frame compensation;

[0014] When an initial video frame of the target video sent by the external device is received, perform three-dimensional full-screen display of the initial video frame through the display and prohibit time warping of the video frames of the target video.

[0015] Optionally, after the processor prohibits time warping of the video frames of the target video, it also performs the following operations:

[0016] According to the comparison result between the decoding frame rate of the target video and a preset stuttering threshold, control whether to perform time warping on the video frames of the target video and perform three-dimensional full-screen display of the video frames. During the display process, the user's viewpoint always faces the center of the video frame.

[0017] Optionally, the processor controls whether to perform time warping on the video frames in the target video according to the comparison result between the decoding frame rate of the target video and a preset stuttering threshold, and performs three-dimensional full-screen display of the video frames. The specific operation is as follows:

[0018] If the decoding frame rate is greater than the preset stuttering threshold, prohibit time warping of the video frames of the target video and directly perform three-dimensional full-screen display of the decoded video frames through the display;

[0019] If the decoded frame rate is less than or equal to the preset stuttering threshold, time warping is performed on the current video frame according to the predicted pose information to obtain a compensated frame, and three-dimensional full-screen display is performed through the display. When the video frame corresponding to the predicted pose information is received, the compensated frame is replaced, and three-dimensional full-screen display is performed on the replaced video frame through the display, where the predicted pose information is determined according to the pose information corresponding to the current video frame and the head movement state, and the predicted pose information is located between the pose information corresponding to the current video frame and the current pose information after head movement.

[0020] Optionally, after performing three-dimensional full-screen display on the video frames of the target video, when the target video is interrupted or ended, the processor further performs the following operations:

[0021] Perform time warping and display on the popped-up prompt screen.

[0022] Optionally, when time warping of the video frames of the target video is prohibited, before exiting the display, the processor further performs the following operations:

[0023] Resume time warping and display on its own screen.

[0024] Optionally, when receiving the video frames in the target video sent by the external device, the processor further performs the following operations:

[0025] Send the pose information to the external device to control the corresponding video frames.

[0026] Optionally, the XR device includes a virtual reality device VR, an augmented reality device AR, or a mixed reality device MR.

[0027] In the above embodiments of the present application, for whether the target video sent by the external device is received, different display methods are adopted. When the initial video frame of the target video is not received, time warping and display on its own screen are maintained, so that the video picture is smoothly transitioned and the dizziness is reduced; when the initial video frame of the target video is received, three-dimensional full-screen display is performed on the received initial video frame, and time warping of the video frames of the target video is prohibited to prevent picture jitter caused by incorrect compensation. By detecting whether the target video sent by the external device is received, whether to perform time warping on the display screen is reasonably controlled, so as to reduce the video picture jitter caused by the video transmission process, and further reduce the head dizziness.

[0028] In a third aspect, an embodiment of the present application provides a video display method, including:

[0029] Detect whether a target video sent by an external device is received;

[0030] When the initial video frame of the target video sent by the external device is not received, keep performing time warping on its own screen and displaying it, where the time warping is used for three-dimensional image frame compensation;

[0031] When the initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is greater than the preset stuttering threshold, prohibit time warping of the video frames of the target video and directly perform three-dimensional full-screen display on the decoded video frames;

[0032] When the initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is less than or equal to the preset stuttering threshold, start performing time warping on the video frames of the target video and perform three-dimensional full-screen display on the decoded video frames;

[0033] Wherein, during the display of the target video, the user's viewing point always faces the center of the video frame.

[0034] Optionally, starting to perform time warping on the video frames of the target video and performing three-dimensional full-screen display on the decoded video frames includes:

[0035] Perform time warping on the current video frame according to the predicted pose information to obtain a compensated frame and perform three-dimensional full-screen display;

[0036] When the video frame corresponding to the predicted pose information is received, replace the compensated frame and perform three-dimensional full-screen display on the replaced video frame, where the predicted pose information is determined according to the pose information corresponding to the current video frame and the head movement state, and the predicted pose information is located between the pose information corresponding to the current video frame and the current pose information after head movement.

[0037] Optionally, after performing three-dimensional full-screen display on the video frames of the target video, when the target video is interrupted or ended, the method further includes:

[0038] Perform time warping on the popped-up prompt screen and display it.

[0039] Optionally, when time warping of the video frames of the target video is prohibited, before exiting the display, the method further includes:

[0040] Resume performing time warping on its own screen and displaying it.

[0041] Optionally, when receiving the video frames in the target video sent by the external device, the method further includes:

[0042] Send the pose information to the external device to control the corresponding video frames.

[0043] Optionally, the method is applied to an XR device, which includes a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device.

[0044] In a fourth aspect, an embodiment of the present application provides an XR device, including a processor, a memory, a display, and at least one external communication interface. The at least one external communication interface, the display, and the memory are connected to the processor through a bus;

[0045] The memory stores computer program instructions, and the processor performs the following operations according to the computer program instructions:

[0046] Detect whether a target video sent by an external device is received through the at least one external communication interface;

[0047] When an initial video frame of the target video sent by the external device is not received, maintain time warping of its own screen and display it through the display, where the time warping is used for three-dimensional image frame compensation;

[0048] When an initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is greater than a preset stuttering threshold, prohibit time warping of the video frames of the target video, and directly perform three-dimensional full-screen display of the decoded video frames through the display;

[0049] When an initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is less than or equal to the preset stuttering threshold, start time warping of the video frames of the target video, and perform three-dimensional full-screen display of the decoded video frames through the display;

[0050] During the display of the target video, the user's viewing point always faces the center of the video frame.

[0051] Optionally, when the processor starts time warping of the video frames of the target video and performs three-dimensional full-screen display of the decoded video frames, the specific operations are as follows:

[0052] Perform time warping on the current video frame according to the predicted pose information to obtain a compensated frame, and perform three-dimensional full-screen display through the display;

[0053] When receiving the video frame corresponding to the predicted pose information, replace the compensation frame, and perform three-dimensional full-screen display of the replaced video frame through the display, where the predicted pose information is determined according to the pose information corresponding to the current video frame and the head movement state, and the predicted pose information is located between the pose information corresponding to the current video frame and the current pose information after head movement.

[0054] Optionally, after performing three-dimensional full-screen display on the video frames of the target video, when the target video is interrupted or ended, the processor further performs the following operations:

[0055] Perform time warping on the popped-up prompt screen and display it.

[0056] Optionally, when time warping of the video frames of the target video is prohibited, before exiting the display, the processor further performs the following operations:

[0057] Resume time warping and display of its own screen.

[0058] Optionally, when receiving the video frames in the target video sent by the external device, the processor further performs the following operations:

[0059] Send the pose information to the external device to control the corresponding video frames.

[0060] Optionally, the XR device includes a virtual reality device VR, an augmented reality device AR, or a mixed reality device MR.

[0061] In the above embodiments of the present application, when the target video sent by the external device is not detected, time warping and display of its own screen are performed, so that the video screen smoothly transitions and the dizziness is reduced; when the initial video frame of the target video sent by the external device is detected, the decoding frame rate of the target video is compared with the preset frame drop threshold. If the decoding frame rate is greater than the preset frame drop threshold, the initial video frame is directly subjected to three-dimensional full-screen display, and time warping of the video frames of the target video is prohibited to prevent screen jitter caused by incorrect compensation. Otherwise, time warping of the video frames of the target video is started, and the decoded video frames are subjected to three-dimensional full-screen display. By detecting whether the target video sent by the external device is received and the size relationship between the decoding frame rate of the target video and the preset frame drop threshold, the time warping of the display screen is reasonably controlled, thereby reducing the video screen jitter caused during the video transmission process, and further reducing the head dizziness. During the process of full-screen display of the video frame, the user's viewing point always faces the center of the video frame, which can prevent seeing empty pixels during the process of head rotation.

[0062] Fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing an XR device to execute the video display method provided by the embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for description in the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0064] Figure 1 Exemplarily shows a schematic diagram of an application scenario provided by an embodiment of the present application;

[0065] Figure 2 Exemplarily shows a schematic diagram of a target application displaying its own screen provided by an embodiment of the present application;

[0066] Figure 3 Exemplarily shows a schematic diagram of a target application displaying an external video provided by an embodiment of the present application;

[0067] Figure 4 Exemplarily shows a flowchart of a method for displaying a streaming video provided by an embodiment of the present application;

[0068] Figure 5 Exemplarily shows a flowchart of a control method for a time warping function provided by an embodiment of the present application;

[0069] Figure 6 Exemplarily shows a complete flowchart of a method for displaying a streaming video provided by an embodiment of the present application;

[0070] Figure 7 Exemplarily shows a structural diagram of an XR device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] To make the objectives, implementation manners, and advantages of the present application clearer, the following will clearly and completely describe the exemplary implementation manners of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0072] Based on the exemplary embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope protected by the appended claims of the present application. In addition, although the disclosure in the present application is introduced according to one or several exemplary instances, it should be understood that each aspect of these disclosures can also constitute a complete implementation manner alone.

[0073] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, rather than intending to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0074] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to those clearly listed components, but may include other components not clearly listed or inherent to these products or devices.

[0075] The term "module" used in the present application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to the element.

[0076] The following gives explanations of the terms in the present application.

[0077] XR: The general term for Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), etc.

[0078] The following describes the embodiments of the present application in detail with reference to the drawings.

[0079] Figure 1 An exemplary application scenario diagram provided by the embodiments of the present application is shown; as Figure 1As shown, the XR device 100 interacts with the external device 300 via the network 200. The XR device 100 transmits head posture information to the external device 300 via the network 200 to control the video frame. The external device 300 transmits the two-dimensional video to the XR device 100 via the network 200 for display. The XR device 100 predicts the compensation frame based on its own posture information and the previous video frame received, and replaces the compensation frame when receiving the corresponding video frame transmitted by the external device 300. The external device 300 includes but is not limited to a desktop computer 301, a laptop computer 302, and a smart TV 303. The XR device 100 may be installed with a first application and / or a second application. The first application is used to play local videos, and the second application is used to play videos transmitted by the external device 300.

[0080] Figure 2 The XR device provided in the embodiment of the present application displays the effect of local video; Figure 2 As shown in the figure, when the wearer of the XR device rotates freely in 360° space, the displayed video image changes continuously as the line of sight changes.

[0081] Figure 3 The XR device provided in the embodiment of the present application displays a video effect diagram transmitted by an external device; Figure 3 As shown in the figure, when the wearer of an XR device turns their head, the human eye's visual range changes. However, because the video transmitted by the external device is a two-dimensional image, when the XR device displays the received video to the wearer, it needs to display the video in full screen and fix the video image in front of the wearer's eyes to prevent the viewing of empty pixels during head rotation.

[0082] When the wearer of an XR device watches a video transmitted from an external device, since the posture information of the head-mounted XR device is transmitted to the external device through the network, the video image of the external device is also transmitted to the head-mounted XR device through the network. Compared with the head-mounted XR device displaying local video, the transmission delay is increased. The wearer of the XR device can move during the data transmission process. In this way, the jump between the compensated frame predicted by the Time-warp method and the real image transmitted is large, which can easily cause dizziness.

[0083] In view of this, an embodiment of the present application provides a video display method and device, which dynamically detects the displayed video to determine whether it is the application's own video (i.e., non-streaming video) or a video received from an external device (i.e., streaming video). After determining the source of the video, the XR device is dynamically controlled to run Time-warp to solve the jitter of the video image and reduce head dizziness.

[0084] Figure 4 The video display method provided in the embodiment of the present application is as follows: Figure 4As shown, this process is executed by an XR device and mainly includes the following steps:

[0085] S401: Detect whether the target video sent by the external device is received. If not, execute S402; otherwise, execute S403.

[0086] In the embodiments of the present application, the XR device can communicate with the external device and receive the two-dimensional video sent by the external device. By detecting whether the target video sent by the external device is received, the display can be performed in different cases.

[0087] S402: Keep performing time warping on its own screen and display it.

[0088] In the embodiments of the present application, the own screen can be the screen generated by the XR device according to its own data, or the screen in the application installed on the XR device for playing videos.

[0089] Taking the own screen as the screen of the target application in the XR device as an example, the target application is an application for playing the video received from the external device, also known as a streaming application. Generally, in addition to displaying the video received from the external device, when the video is not being played, the target application will also display some skyboxes or operation prompt interfaces (such as: selecting a virtual scene). The skybox or operation prompt interface can be regarded as the screen originating from the target application itself. Whether it is the external video or the own screen, the content displayed by the target application is three-dimensional to generate a sense of immersion, and the user's head can rotate during the display process. Therefore, the applications in the XR device are usually set to default enable time warping for three-dimensional image frame compensation, thereby reducing the jitter of the screen.

[0090] In the embodiments of the present application, when the initial video frame of the target video sent by the external device is not received, the own screen of the XR device or the application can be displayed (such as: skybox or operation guidance interface). For the own screen, when executing S402, the Time-warp algorithm is used to perform time warping on the own screen, that is, according to the current pose information of the XR device and the previous moment's screen, the predicted compensation frame is calculated and displayed.

[0091] S403: Perform three-dimensional full-screen display on the initial video frame and prohibit time warping on the video frames of the target video.

[0092] In S403, when the XR device receives the initial video frame of the target video sent by the external device, it converts the two-dimensional initial video frame into three-dimensional and renders it on the display screen. By adjusting the distance between the display screen and the user's viewpoint, the converted three-dimensional initial video frame is full-screen displayed, making the initial video frame fill the fields of view of both the left and right eyes, and completely covering other areas outside the video frame to prevent the user from seeing empty pixels when turning the head. Since the XR device has started receiving the target video from the external device and there is no need to predict compensation frames, it is possible to prohibit time warping of the video frames of the target video by calling the system interface to prevent image jitter caused by incorrect compensation.

[0093] In an embodiment of the present application, when the XR device receives the initial video frame sent by the external device, it indicates that the transmission channel has been established. At this time, the XR device can transmit the pose information back to the external device in real time through the transmission channel to control the video picture displayed by the external device, without using a remote control device with an external handle.

[0094] In some embodiments of the present application, after the transmission channel between the external device and the XR device is established, the XR device can continuously receive video frames. By dynamically calculating the decoding frame rate of the XR device for the target video, it is possible to select whether to perform time warping on the target video or prohibit time warping of the target video.

[0095] Generally, the decoding frame rate of the XR device is affected by multiple factors, such as the resolution and format of the target video, the GPU and CPU of the XR device, etc. When executing S403, first calculate the decoding rate of the XR device for the target video, compare the decoding rate with a preset jitter threshold, and according to the comparison result, reasonably control whether to perform time warping on the video frames of the target video, and perform three-dimensional full-screen display on the decoded video frames.

[0096] Specifically, if the decoding frame rate is greater than the preset jitter threshold, it indicates that the decoding frame rate of the target video meets the visual requirements of the human eye and will not cause visual discomfort, then prohibit time warping of the video frames of the target video and directly perform three-dimensional full-screen display on the decoded video frames; if the decoding frame rate is less than or equal to the preset jitter threshold, it indicates that the decoding frame rate of the target video will cause video jitter, then determine the predicted pose information according to the pose information corresponding to the current video frame and the head movement state (such as: rotation speed, rotation direction, etc.), perform time warping on the current video frame according to the predicted pose information to obtain a compensation frame and perform three-dimensional full-screen display, and when receiving the video frame corresponding to the predicted pose information, replace the compensation frame and perform three-dimensional full-screen display on the replaced video frame, where the predicted pose information is between the pose information corresponding to the current video frame and the current pose information after head movement.

[0097] In the embodiments of the present application, there are no restrictive requirements for the setting method of the preset stuttering threshold. For example, it can be set according to the screen refresh rate of the hardware, or it can also be set according to empirical values.

[0098] It should be noted that during the process of full-screen display of the entire target video, the user's viewpoint (i.e., the camera) is located at the origin of the virtual three-dimensional space and always faces the center of the video frame to prevent the user from seeing empty pixel points when turning the head.

[0099] In some embodiments, due to reasons such as network anomalies, the transmission of the target video is interrupted, or when the target video playback ends, the target application pops up a prompt screen to inform the user of the playback status of the target video. At this time, the prompt screen belongs to the screen of the target application itself, and time warping should be performed on the prompt screen and displayed.

[0100] In other embodiments, when time warping of the video frames of the target video is prohibited, since there are also some screens of the XR device and the target application itself, to ensure the normal display of its own screen, before exiting the display of the target video, time warping of its own screen is restored and displayed.

[0101] In the above embodiments of the present application, on the one hand, after the XR device receives the target video transmitted by the external device, during the entire display process, the user's viewpoint located at the origin of the virtual three-dimensional space always faces the center of the video frame to fix the video screen in front of the person's eyes and perform three-dimensional full-screen display on the video frame to make the video frame fill the fields of view of both eyes, thereby effectively preventing the human eye from seeing empty pixels during the head rotation process. On the other hand, when rendering the received initial video frame to the display screen, the time warping function is stopped in a timely manner to prevent the jitter of the video screen caused by incorrect compensation. During the display process, by comparing the decoding frame rate of the target video with the preset stuttering threshold, the opening and closing of the time warping function are controlled to reasonably control the compensation for the target video and reduce the jitter of the screen, thereby reducing head dizziness. On the other hand, after the target video ends, the time warping function is restored in a timely manner to prevent the jitter of other screens.

[0102] The embodiments of the present application also provide another control of the time warping function during the video display process. For details, please refer to Figure 5 :

[0103] S501: Detect whether the target video sent by the external device is received. If not, execute S502; otherwise, execute S503.

[0104] For the specific description of this step, please refer to S401 and will not be repeated here.

[0105] S502: Keep time warping and displaying the own screen.

[0106] In this step, time warping is used for three-dimensional image frame compensation to smooth the displayed video image and reduce image jitter. For a detailed description, see S402, which will not be repeated here.

[0107] S503: Calculate the decoding frame rate of the target video.

[0108] Generally, the decoding frame rate of XR devices is affected by multiple factors, such as the resolution and format of the target video, the GPU and CPU of the XR device, etc. By calculating the decoding frame rate of the target video, it can be determined whether the target video meets the rendering requirements.

[0109] S504: Determine whether the decoding frame rate is greater than the preset stuttering threshold. If so, execute S505; otherwise, execute S506.

[0110] In S504, if the decoding frame rate is greater than the preset stuttering threshold, it indicates that the decoding frame rate of the target video meets the visual requirements of the human eye, and stopping the time warping function will not cause visual discomfort; if the decoding frame rate is less than or equal to the preset stuttering threshold, it indicates that the decoding frame rate of the target video will cause video stuttering, and stopping the time warping function will cause visual discomfort.

[0111] S505: Prohibit time warping of the video frames of the target video and directly perform three-dimensional full-screen display on the decoded video frames.

[0112] In S505, since the decoding frame rate is greater than the preset stuttering threshold, there is no need for frame compensation through time warping. After converting the decoded video frames into three dimensions, the three-dimensional video frames can be directly displayed full screen.

[0113] S506: Start time warping of the video frames of the target video and perform three-dimensional full-screen display on the decoded video frames.

[0114] In S506, since the decoding frame rate is less than or equal to the preset stuttering threshold, frame compensation needs to be performed through time warping. After receiving the corresponding video frames, the compensation frames are replaced and three-dimensional full-screen display is performed.

[0115] Specifically, when implemented, the XR device determines the predicted pose information based on the pose information and head movement state (such as rotation speed, rotation direction, etc.) corresponding to the current video frame, performs time warping on the current video frame according to the predicted pose information to obtain compensation frames and perform three-dimensional full-screen display. When receiving the video frames corresponding to the predicted pose information, the compensation frames are replaced, and three-dimensional full-screen display is performed on the replaced video frames, where the predicted pose information is between the pose information corresponding to the current video frame and the current pose information after head movement.

[0116] It should be noted that in Figure 5During the process of full - screen display of the target video shown, the user's viewing point always faces the center of the video frame to prevent seeing empty pixels due to head rotation.

[0117] Taking the XR device as a VR device as an example, Figure 6 An exemplary complete display process of the target video provided by the embodiments of the present application is shown, mainly including the following steps:

[0118] S601: The VR device detects whether it has received the target video sent by an external device. If not, it executes S602; otherwise, it executes S603.

[0119] In the embodiments of the present application, the VR device can communicate with an external device and receive the two - dimensional video sent by the external device. In S601, by detecting whether the target video sent by the external device is received, the on - off of the time - warping function can be reasonably controlled to reduce screen jitter and thus relieve head dizziness.

[0120] S602: The VR device maintains the time - warping function of the target application to compensate its own screen and display it.

[0121] In this step, the target application is used to play the video received from the external device. After the target application is started, the transmission channel of the video has not been established yet, and the screen displayed at this time is the own screen of the target application. To ensure the normal display of the own screen of the target application, in S602, the time - warping function of the target application is kept on. For a detailed description, see S402 and will not be repeated here.

[0122] S603: The VR device turns off the time - warping function of the target application.

[0123] In this step, after the VR device receives the initial video frame of the target video, it indicates that the transmission channel has been established. First, the time - warping function of the target application is turned off to prevent screen jitter caused by incorrect compensation, and it can be selected to be turned on or remain off according to actual needs later.

[0124] S604: The VR device dynamically calculates the decoding frame rate of the target video.

[0125] In S604, the target video can be decoded in a hardware decoding method, a software decoding method, or a combination of software and hardware decoding methods.

[0126] S605: The VR device determines whether the decoding frame rate is greater than the preset jitter threshold. If so, it executes S606; otherwise, it executes S607.

[0127] S606: The VR device keeps the time - warping function of the target application turned off and performs three - dimensional full - screen display on the received video frames.

[0128] In S606, when the decoded frame rate is greater than the preset stuttering threshold, it indicates that visual discomfort will not be caused even without performing temporal warping on the target video. Therefore, the temporal warping function of the target application can be kept turned off to prevent picture jitter caused by incorrect compensation.

[0129] S607: The VR device enables the temporal warping function of the target application and performs three-dimensional full-screen display on the received video frames.

[0130] In S607, when the decoded frame rate is less than or equal to the preset stuttering threshold, it indicates that temporal warping needs to be performed on the target video. Since the head rotates during the data transmission process, at this time, when enabling the temporal warping function of the target application, the predicted compensation frame should use the pose information between the pose information corresponding to the current video frame and the current pose information after the head rotation (i.e., the predicted pose information). That is, the VR device determines the predicted pose information between the pose information corresponding to the current video frame and the current pose information based on the pose information corresponding to the current video frame and the head movement state, and performs temporal warping on the current video frame according to the predicted pose information to obtain a compensation frame and display it. When receiving the corresponding adaptation frame, replace the compensation frame and perform full-screen display.

[0131] S608: When the VR device determines that the target video is interrupted or ended, restore the temporal warping function of the target application.

[0132] In S608, after the target video terminates or ends, restore the temporal warping function of the target application to prevent jitter of other pictures.

[0133] Based on the same technical concept, the embodiment of the present application provides a video display terminal. This terminal can execute the video display method flow provided by the embodiment of the present application and achieve the same technical effect, which will not be repeated here.

[0134] See Figure 7 , this terminal includes a processor 701, a memory 702, a display 703, and at least one external communication interface 704. The at least one external communication interface 704, the display 703, and the memory 702 are connected to the processor 701 through a bus 705. The terminal communicates and performs video transmission with external devices through the at least one external communication interface 704. The display 703 is used to display the target video and its own picture. The memory 702 stores computer program instructions, and the processor 701 executes the video display method in the above embodiment according to the computer program instructions.

[0135] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which can complete the method in the foregoing embodiment when executed.

[0136] The embodiments of the present application further provide a computer program product for storing a computer program, which is used to execute the method of the foregoing embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0138] For the sake of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A video display method, characterized in that: Including: Detecting whether a target video sent by an external device is received; When the initial video frame of the target video sent by the external device is not received, maintaining time warping and display of its own screen, where the time warping is used for three-dimensional image frame compensation; When the initial video frame of the target video sent by the external device is received, performing three-dimensional full-screen display of the initial video frame, first prohibiting time warping of the video frames of the target video, then controlling whether to perform time warping on the video frames of the target video according to the comparison result between the decoding frame rate of the target video and a preset stuttering threshold, and performing three-dimensional full-screen display of the video frames. During the display process, the user's viewing point always faces the center of the video frame.

2. The method according to claim 1, characterized in that The controlling whether to perform time warping on the video frames in the target video according to the comparison result between the decoding frame rate of the target video and a preset stuttering threshold, and performing three-dimensional full-screen display of the video frames includes: If the decoding frame rate is greater than the preset stuttering threshold, prohibiting time warping of the video frames of the target video and directly performing three-dimensional full-screen display on the decoded video frames; If the decoding frame rate is less than or equal to the preset stuttering threshold, performing time warping on the current video frame according to predicted pose information to obtain a compensated frame and performing three-dimensional full-screen display. When the video frame corresponding to the predicted pose information is received, replacing the compensated frame and performing three-dimensional full-screen display, where the predicted pose information is determined according to the pose information corresponding to the current video frame and the head movement state, and the predicted pose information is between the pose information corresponding to the current video frame and the current pose information after head movement.

3. The method according to claim 1, wherein After performing three-dimensional full-screen display on the video frames of the target video, when the target video is interrupted or ended, the method further includes: Performing time warping and display on the popped-up prompt screen.

4. The method according to claim 1, wherein When time warping of the video frames of the target video is prohibited, before exiting the display of the target video, the method further includes: Restoring time warping and display of its own screen.

5. The method according to any one of claims 1-4, characterized in that, When receiving the video frames in the target video sent by the external device, the method further includes: Sending pose information to the external device to control the corresponding video frames.

6. A video display method, characterized in that: Applied to an extended reality (XR) device, including: Detecting whether a target video sent by an external device is received; When the initial video frame of the target video sent by the external device is not received, maintaining time warping and display of its own screen, where the time warping is used for three-dimensional image frame compensation; When the initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is greater than the preset stuttering threshold, prohibiting time warping of the video frames of the target video and directly performing three-dimensional full-screen display on the decoded video frames; When the initial video frame of the target video sent by the external device is received and the decoding frame rate of the target video is less than or equal to a preset stuttering threshold, start performing time warping on the video frames of the target video and perform three-dimensional full-screen display on the time-warped video frames; Among them, during the display of the target video, the user's viewpoint always faces the center of the video frame.

7. The method according to claim 6, characterized in that The step of starting to perform time warping on the video frames of the target video and performing three-dimensional full-screen display on the time-warped video frames includes: Performing time warping on the current video frame according to the predicted pose information to obtain a compensated frame and perform three-dimensional full-screen display; When the video frame corresponding to the predicted pose information is received, replace the compensated frame and perform three-dimensional full-screen display on the replaced video frame, where the predicted pose information is determined according to the pose information corresponding to the current video frame and the head movement state, and the predicted pose information is located between the pose information corresponding to the current video frame and the current pose information after head movement.

8. The method according to claim 6 or 7, characterized in that, The extended reality (XR) device includes a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device.

9. An extended reality (XR) device, characterized in that: It includes a processor, a memory, a display, and at least one external communication interface, and the at least one external communication interface, the display, and the memory are connected to the processor through a bus; The at least one external communication interface is configured to communicate with an external device and transmit video, the display is configured to display three-dimensional images, the memory stores computer program instructions, and the processor executes the method according to any one of claims 1-8 according to the computer program instructions.

Images