A method and device for drawing a virtual reality scene and a virtual reality device
By using vertical synchronization signals to control the rendering thread and drawing thread in virtual reality scene drawing, and estimating the rendering time, awakening the rendering thread in advance to complete the rendering before the vertical synchronization signal arrives, the delay problem caused by the rendering thread waiting is solved, and the User experience.
Patent Information
- Application Number
- CN201611267022.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2016-12-31
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2036-12-31
AI Technical Summary
In the existing virtual reality scene drawing process, the rendering thread is in a waiting state while waiting for the drawing thread to wake up, resulting in delayed image display after rendering, causing user stunning and other bad experiences.
By setting a vertical synchronization signal of a specific frequency, controlling the drawing thread and rendering thread, and estimating the time it takes for the rendering thread to complete a frame of virtual reality scene rendering, awakening the rendering thread in advance to complete the rendering before the vertical synchronization signal arrives, reducing the invalid waiting of the rendering thread time.
It effectively reduces the time interval between rendering thread rendering completion and image sending to the screen, reduces latency and improves user experience.
Smart Images

Figure CN106843859B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of virtual reality technology, and in particular to a method and device for drawing a virtual reality scene and a virtual reality device. Background Art
[0002] At present, the mainstream virtual reality scene rendering process adopts the Timewarp asynchronous refresh method. Figure 1 As shown, there are generally two threads, a rendering thread (render) and a drawing thread (timewarp). The result of the rendering thread is obtained by the drawing thread and sent to the screen, and the drawing thread controls the operation of the two threads according to the vertical synchronization signal (VSYNC).
[0003] After the rendering thread completes a drawing, it does not directly proceed to the next drawing, but waits for the drawing thread to wake up. Each time the drawing thread receives a vertical synchronization signal, it draws and wakes up the rendering thread to render the next frame of the image. Taking a 60hz screen as an example, the drawing cycle of each of the two is 16.67ms.
[0004] The disadvantage of this mode is that the rendering thread has to wait for the drawing thread to wake up each time it finishes drawing. If the drawing task of the rendering thread is relatively light, it will be completed in a short time. Then the rendered image will need to wait longer before being displayed on the screen. The longer the time interval from rendering to displaying a frame of image on the screen, the greater the delay felt by the user, which will cause dizziness and other adverse experiences. Summary of the invention
[0005] The present invention provides a method and device for drawing a virtual reality scene and a virtual reality device, so as to reduce the invalid waiting time of a rendering thread during the drawing process, reduce delay and improve user experience.
[0006] According to one aspect of the present invention, the present invention provides a method for drawing a virtual reality scene, comprising:
[0007] Set a specific frequency of vertical synchronization signal to control the drawing thread and rendering thread;
[0008] Estimate the time it takes for the rendering thread to complete rendering of a frame of the VR scene;
[0009] Setting a wake-up time, waking up the rendering thread at the wake-up time to start rendering a frame of the virtual reality scene, so that the rendering thread just completes the rendering of the frame of the virtual reality scene before the vertical synchronization signal arrives; after the rendering is completed, the rendering thread enters a waiting state to be awakened;
[0010] When the vertical synchronization signal arrives, the drawing thread sends the virtual reality scene that has been rendered by the rendering thread to the screen.
[0011] According to another aspect of the present invention, the present invention provides a drawing device for a virtual reality scene, comprising a vertical synchronization unit, a rendering unit and a drawing unit;
[0012] The vertical synchronization unit is used to send a vertical synchronization signal of a specific frequency to the rendering unit and the drawing unit;
[0013] The rendering unit is used to estimate the time taken by the rendering thread to complete rendering of one frame of the virtual reality scene, set a wake-up time, wake up the rendering thread at the wake-up time to start rendering of one frame of the virtual reality scene, so that the rendering thread just completes the rendering of one frame of the virtual reality scene before the vertical synchronization signal arrives; and control the rendering thread to enter a waiting state to be awakened after the rendering is completed;
[0014] The drawing unit is used to send the virtual reality scene that has been rendered by the rendering thread to the screen when the vertical synchronization signal arrives.
[0015] According to yet another aspect of the present invention, the present invention provides a virtual reality device, comprising the above-mentioned virtual reality scene drawing device.
[0016] The beneficial effect of the present invention is as follows: in the process of rendering a virtual reality scene, the embodiment of the present invention controls the drawing thread and the rendering thread according to the vertical synchronization signal, estimates the time taken by the rendering thread to complete the rendering of one frame of the virtual reality scene, sets the rendering time according to the time, wakes up the rendering thread at the time to start rendering one frame of the virtual reality scene, enables the rendering thread to just complete the rendering of one frame of the virtual reality scene before the vertical synchronization signal arrives, and sends the virtual reality scene that the rendering thread has rendered to the screen when the vertical synchronization signal arrives, thereby reducing the interval between the moment when the rendering thread completes rendering one frame of the virtual reality scene and the moment when the drawing thread sends the scene to the screen, effectively reducing the ineffective waiting time of the rendering thread, reducing the delay, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a schematic diagram of the principle of the existing mainstream virtual reality scene drawing scheme;
[0018] Figure 2 is a flow chart of a method for drawing a virtual reality scene provided by an embodiment of the present invention;
[0019] Figure 3 is a schematic diagram of a method for drawing a virtual reality scene provided by an embodiment of the present invention;
[0020] Figure 4 It is a functional block diagram of a virtual reality scene rendering device provided by one embodiment of the present invention;
[0021] Figure 5 It is a functional block diagram of a virtual reality device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The design concept of the present invention is: in the existing virtual reality scene drawing process, the rendering thread is in a waiting state after each drawing, waiting for the awakening of the drawing thread. If the drawing task of the rendering thread is relatively light, it will be drawn in a short time, then the rendered image needs to wait for a longer time to be displayed on the screen. The longer the time interval from rendering to displaying a frame of image on the screen, the greater the delay felt by the user, which will bring adverse experiences such as dizziness. In view of this situation, in the process of rendering the virtual reality scene, the present invention controls the drawing thread and the rendering thread according to the vertical synchronization signal, estimates the time taken by the rendering thread to complete the rendering of a frame of virtual reality scene, and sets the rendering time according to the time, wakes up the rendering thread at this time to start rendering a frame of virtual reality scene, so that the rendering thread just completes the rendering of a frame of virtual reality scene before the vertical synchronization signal arrives, and when the vertical synchronization signal arrives, the drawing thread sends the virtual reality scene that the rendering thread has rendered to the screen, thereby reducing the interval between the moment when the rendering thread completes the rendering of a frame of virtual reality scene and the moment when the drawing thread sends the scene to the screen, effectively reducing the invalid waiting time of the rendering thread, reducing the delay, and improving the user experience.
[0023] Embodiment 1
[0024] Figure 2 is a flow chart of a method for drawing a virtual reality scene provided by an embodiment of the present invention, such as Figure 2 As shown, the method for drawing a virtual reality scene provided in this embodiment includes:
[0025] Step S210: Setting a vertical synchronization signal (VSYNC) of a specific frequency to control the drawing thread (timewarp) and the rendering thread (render).
[0026] Step S220: estimating the time taken by the rendering thread to complete rendering of one frame of the virtual reality scene.
[0027] The time it takes to render a frame of a VR scene is related to the hardware configuration of the device. For example, a high-end graphics card takes less time to render the same scene. On the other hand, the time it takes to render a frame of a VR scene is also related to the scene being rendered. When the scene is more complex, it takes longer to render.
[0028] In order to estimate the rendering time more accurately, the preferred embodiment dynamically obtains and updates the estimated value of the rendering time during program execution: first, insert a fence synchronization signal into the rendering instruction of the rendering thread, and use the fence synchronization signal to query the time taken by the rendering thread to render each frame of the virtual reality scene. Then, at regular intervals, the longest time taken by the rendering thread to render a frame of the virtual reality scene within the time period is counted as the estimated time taken by the rendering thread to complete the rendering of a frame of the virtual reality scene. For example, the longest time T taken by the rendering thread to render a frame of the virtual reality scene within the past one second is counted once every one second to update the estimated value of the rendering time, so as to ensure that the rendering thread can complete the rendering of a frame of the virtual reality scene within this time. Since in this embodiment, the time taken by the rendering thread to complete the rendering of a frame of the virtual reality scene is dynamically adjusted, even if the complexity of the scene changes and the rendering task volume increases or decreases, the estimated value can also be dynamically adjusted according to the recent working conditions of the rendering thread, thereby obtaining a more accurate estimate of the rendering time.
[0029] Step S230: setting a wake-up time, waking up the rendering thread at this time to start rendering a frame of the virtual reality scene, so that the rendering thread just completes the rendering of a frame of the virtual reality scene before the vertical synchronization signal arrives; after the rendering is completed, the rendering thread enters a state of waiting to be awakened.
[0030] Preferably, the time point at which the vertical synchronization signal arrives can be used as a reference to estimate in advance the time T that the rendering thread takes to complete rendering of a frame of the virtual reality scene, and the obtained time can be used as the wake-up time. Figure 3 As shown, according to the time point when the next vertical synchronization signal arrives and the currently estimated rendering time T, a time point t can be calculated, and the time point t is used as the time when the rendering thread is awakened, because waking up the rendering thread at this time can ensure that the rendering of a frame of virtual reality scene is completed before the next vertical synchronization signal arrives, and when the next vertical synchronization signal arrives, the drawing thread will send the frame scene to the screen. The interval between the moment when the scene rendering is completed and the moment when the scene is sent to the screen is the shortest, which effectively reduces the ineffective waiting time of the rendering thread and reduces the resulting delay to a certain extent.
[0031] Step S240: When the vertical synchronization signal arrives, the drawing thread sends the virtual reality scene that has been rendered by the rendering thread to the screen, completing the display of one frame of virtual reality image.
[0032] and Figure 1 Compared with the existing drawing scheme in the embodiment, after the drawing thread of this embodiment sends the rendered virtual reality scene to the screen when the vertical synchronization signal arrives, it does not immediately wake up the rendering thread to start rendering the next frame of the scene, but waits for a period of time until the rendering thread of the virtual reality scene is sent to the screen. Figure 3The rendering thread is awakened at time t to start rendering the next frame of the scene. If the rendering thread has indeed been awakened at time t1 before the awakening time t, if the rendering of a frame of the scene is immediately started at time t1, the rendering task will be completed some time in advance before the next vertical synchronization signal arrives, thereby generating an extra delay. Therefore, in this case, the choice is to continue waiting for a while until rendering at time t1. If the rendering thread is awakened at time t2 after the awakening time t, it will directly render without waiting.
[0033] In this embodiment, by controlling the time when the rendering thread starts rendering in the drawing process of the virtual reality scene, the ineffective waiting time of the rendering thread is effectively reduced, the delay is reduced, and the user experience is improved.
[0034] Embodiment 2
[0035] Figure 4 is a functional block diagram of a virtual reality scene rendering device provided by an embodiment of the present invention, such as Figure 4 As shown, the virtual reality scene drawing device provided in this embodiment includes a vertical synchronization unit 410, a rendering unit 420 and a drawing unit 430.
[0036] The vertical synchronization unit 410 sends a vertical synchronization signal of a specific frequency to the rendering unit 420 and the drawing unit 430. The rendering unit 420 estimates the time taken by the rendering thread to complete the rendering of a frame of the virtual reality scene, and sets a wake-up time, at which the rendering thread is woken up to start rendering a frame of the virtual reality scene, so that the rendering thread just completes the rendering of a frame of the virtual reality scene before the vertical synchronization signal arrives; and controls the rendering thread to enter a waiting state after the rendering is completed. When the vertical synchronization signal arrives, the drawing unit 430 sends the virtual reality scene that has been rendered by the rendering thread to the screen to complete the drawing of a frame of the virtual reality scene.
[0037] In a preferred embodiment, the rendering unit 420 includes a rendering time estimation module 421. The rendering time estimation module 421 inserts a fence synchronization signal into the rendering instruction of the rendering thread, uses the fence synchronization signal to query the time taken by the rendering thread to render each frame of the virtual reality scene, and counts the longest time taken by the rendering thread to render one frame of the virtual reality scene within the time period at preset intervals, for example, the longest time T taken by the rendering thread to render one frame of the virtual reality scene within the past one second is counted once every one second as the estimated time taken by the rendering thread to complete rendering one frame of the virtual reality scene.
[0038] In another preferred embodiment, the rendering unit 420 further includes a wake-up time setting module 422. The wake-up time setting module 422 uses the time point at which the vertical synchronization signal arrives as a reference, and uses the time T estimated by the rendering time estimation module 421 in advance to complete the rendering of one frame of the virtual reality scene. The obtained time is used as the wake-up time. Figure 3 The time t is shown.
[0039] If the rendering thread has been awakened before the awakening time t, for example, at time t1, then the rendering unit 420 controls the rendering thread to wait until the awakening time t before starting the rendering of a frame of the virtual reality scene, so as to prevent the rendering task from being completed a period of time in advance before the next vertical synchronization signal arrives, thereby generating an unnecessary delay. If the rendering thread is awakened after the awakening time t, for example, at time t2, the rendering is performed directly without waiting. The virtual reality scene drawing device provided in this embodiment can effectively reduce the invalid waiting time of the rendering thread, reduce delays, and improve user experience.
[0040] Embodiment 3
[0041] Figure 5 is a functional block diagram of a virtual reality device provided by an embodiment of the present invention, such as Figure 5 As shown, the virtual reality device 500 provided in this embodiment includes the virtual reality scene rendering device 510 in the above-mentioned embodiment 2.
[0042] During the process of rendering the virtual reality scene by the virtual reality device 500, the drawing device 510 of the virtual reality scene controls the drawing thread and the rendering thread according to the vertical synchronization signal, estimates the time taken by the rendering thread to complete the rendering of one frame of the virtual reality scene, sets the rendering time according to the time, wakes up the rendering thread at the time to start rendering one frame of the virtual reality scene, and enables the rendering thread to complete the rendering of one frame of the virtual reality scene just before the arrival of the vertical synchronization signal. When the vertical synchronization signal arrives, the drawing thread sends the virtual reality scene that has been rendered by the rendering thread to the screen, thereby reducing the interval between the moment when the rendering thread completes the rendering of one frame of the virtual reality scene and the moment when the drawing thread sends the scene to the screen, effectively reducing the ineffective waiting time of the rendering thread, reducing the delay of the virtual reality device 500, and improving the user experience of the product.
[0043] The above is only a specific embodiment of the present invention. Under the above teachings of the present invention, those skilled in the art can make other improvements or modifications on the basis of the above embodiments. Those skilled in the art should understand that the above specific description is only to better explain the purpose of the present invention, and the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for drawing a virtual reality scene, characterized in that: include: Set a specific frequency of vertical synchronization signal to control the drawing thread and rendering thread; Estimate the time it takes for the rendering thread to complete rendering of a frame of the VR scene; Setting a wake-up time, waking up the rendering thread at the wake-up time to start rendering a frame of the virtual reality scene, so that the rendering thread just completes the rendering of the frame of the virtual reality scene before the vertical synchronization signal arrives; after the rendering is completed, the rendering thread enters a waiting state to be awakened; Calculate the time point t according to the arrival time point of the next vertical synchronization signal and the currently estimated rendering time, and use the time point t as the wake-up time; When the vertical synchronization signal arrives, the drawing thread sends the virtual reality scene that has been rendered by the rendering thread to the screen; wherein estimating the time taken by the rendering thread to complete rendering of one frame of the virtual reality scene specifically includes: Insert fence synchronization signal into the rendering instruction of the rendering thread; Using the fence synchronization signal to query the time taken by the rendering thread to render each frame of the virtual reality scene; At every preset time period, the longest time taken by the rendering thread to render one frame of the virtual reality scene within the time period is counted as the estimated time taken by the rendering thread to complete rendering one frame of the virtual reality scene.
2. The method according to claim 1, characterized in that The method further comprises: If the rendering thread has been awakened before the awakening moment, the rendering thread is controlled to wait until the awakening moment before starting to render a frame of the virtual reality scene.
3. A virtual reality scene rendering device, characterized in that: It includes a vertical synchronization unit, a rendering unit and a drawing unit; The vertical synchronization unit is used to send a vertical synchronization signal of a specific frequency to the rendering unit and the drawing unit; The rendering unit is used to estimate the time taken by the rendering thread to complete rendering of one frame of the virtual reality scene, set a wake-up time, wake up the rendering thread at the wake-up time to start rendering of one frame of the virtual reality scene, so that the rendering thread just completes the rendering of one frame of the virtual reality scene before the vertical synchronization signal arrives; and control the rendering thread to enter a waiting state to be awakened after the rendering is completed; The rendering unit includes a wake-up time setting module, which is used to: calculate the time point t according to the arrival time point of the next vertical synchronization signal and the current estimated rendering time, and use the time point t as the wake-up time; The drawing unit is used to send the virtual reality scene that has been rendered by the rendering thread to the screen when the vertical synchronization signal arrives; wherein the rendering unit includes a rendering time estimation module, which is used to: insert a fence synchronization signal into the rendering instruction of the rendering thread; and use the fence synchronization signal to query the time used by the rendering thread to render each frame of the virtual reality scene; At every preset time period, the longest time taken by the rendering thread to render one frame of the virtual reality scene within the time period is counted as the estimated time taken by the rendering thread to complete rendering one frame of the virtual reality scene.
4. The device according to claim 3, characterized in that The rendering unit is further configured to, if the rendering thread has been awakened before the awakening moment, control the rendering thread to wait until the awakening moment before starting rendering of a frame of the virtual reality scene.
5. A virtual reality device, characterized in that: A device for drawing a virtual reality scene comprising the device described in any one of claims 3-4.
Citation Information
Patent Citations
Image updating method and device
CN102004620A
Image display updating method and device
CN105307021A
Method and apparatus for optimizing motion image in virtual reality scene
CN105635805A
System optimization method for virtual reality application delay and system optimization device thereof
CN105913371A
Processing method and device for displaying stereo images
CN105979243A
