3D Scene Rendering Method, Device, Wearable Device and Storage Medium

Abstract

The present application relates to the technical field of three-dimensional visualization processing, and discloses a three-dimensional scene rendering method, device, wearable device, and storage medium. The method is applied to a wearable device and includes: starting and running the wearable device, calculating initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and storing the initial three-dimensional scene rendering data in a memory; obtaining the current pose of the wearable device at the current moment; determining whether three-dimensional scene rendering data corresponding to the current pose is stored in the memory; if the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, calculating the current three-dimensional scene rendering data corresponding to the current pose, and using the current three-dimensional scene rendering data to update the three-dimensional scene rendering data stored in the memory; and rendering a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory. By the above method, the present application can reduce the requirements for the computing power of the wearable device.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of three-dimensional visualization processing, and particularly to a three-dimensional scene rendering method, apparatus, wearable device, and storage medium. Background Art

[0002] Extended Reality (XR) refers to the integration of virtual content and real scenes through a computer to create a virtual environment for human-computer interaction. XR technology includes various technologies such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR).

[0003] When displaying an interactive screen in an XR scene, an XR device (such as an XR glasses) can first calculate three-dimensional scene rendering data and then use the three-dimensional scene rendering data to render a three-dimensional scene for the user to view. If the three-dimensional scene rendering data is calculated in real time, the computational load is large, and there are high requirements for the computing power of the XR device. Currently, there is a lack of a three-dimensional scene rendering method applicable to XR devices with limited computing power. Summary of the Invention

[0004] In view of the above problems, the embodiments of the present application provide a three-dimensional scene rendering method to solve the problem of high requirements for the computing power of XR devices in the prior art.

[0005] According to one aspect of the embodiments of the present application, a three-dimensional scene rendering method is provided, which is applied to a wearable device. The method includes: starting and running the wearable device, calculating initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and storing the initial three-dimensional scene rendering data in a memory; obtaining the current pose of the wearable device at the current moment; determining whether three-dimensional scene rendering data corresponding to the current pose is stored in the memory; if the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, calculating the current three-dimensional scene rendering data corresponding to the current pose, and using the current three-dimensional scene rendering data to update the three-dimensional scene rendering data stored in the memory; rendering a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory.

[0006] In an alternative manner, calculating the current 3D scene rendering data corresponding to the current pose and updating the 3D scene rendering data stored in the memory using the current 3D scene rendering data includes: determining, by a pose perception module in the wearable device, whether the wearable device has moved; if the wearable device has not moved, calculating the current 3D scene rendering data corresponding to the current pose and storing the current 3D scene rendering data in the memory.

[0007] In an alternative manner, after determining, by the pose perception module in the wearable device, whether the wearable device has moved, the method further includes: if the wearable device has moved, clearing the 3D scene rendering data stored in the memory; calculating the current 3D scene rendering data corresponding to the current pose and storing the current 3D scene rendering data in the memory.

[0008] In an alternative manner, the method further includes: if the 3D scene corresponding to the current pose is a dynamic scene, obtaining the clearing time when the 3D scene rendering data stored in the memory was last cleared, where if the 3D scene rendering data stored in the memory has not been cleared, the clearing time is the startup and running time of the wearable device; determining whether the interval duration between the current time and the clearing time reaches a preset duration; if the interval duration reaches the preset duration, clearing the 3D scene rendering data stored in the memory.

[0009] In an alternative manner, the 3D scene rendering data stored in the memory is also created according to the resolution of the screen of the wearable device; the method further includes: obtaining the current resolution of the screen of the wearable device at the current time; determining whether the current resolution is consistent with the resolution corresponding to the 3D scene rendering data stored in the memory; if the current resolution is not consistent with the resolution corresponding to the 3D scene rendering data stored in the memory, clearing the 3D scene rendering data stored in the memory.

[0010] In an alternative manner, storing the initial 3D scene rendering data in the memory includes: creating a target texture map and storing the target texture map in the memory, where the target texture map is a cube texture map, a 2D texture map, or a 3D texture map; filling the initial 3D scene rendering data into the target texture map; storing the current 3D scene rendering data in the memory includes: filling the current 3D scene rendering data into the target texture map.

[0011] In an alternative manner, the filling of the initial three-dimensional scene rendering data into the target texture map includes: dividing the target texture map into multiple regions; filling the initial three-dimensional scene rendering data into the region corresponding to the initial pose among the multiple regions; marking the regions among the multiple regions that have been filled with the three-dimensional scene rendering data as filled; the filling of the current three-dimensional scene rendering data into the target texture map includes: determining unfilled regions among the multiple regions that are not marked as filled and correspond to the current three-dimensional scene rendering data; filling the three-dimensional scene rendering data corresponding to the unfilled regions in the current three-dimensional scene rendering data into the unfilled regions.

[0012] According to another aspect of the embodiments of the present application, there is provided a three-dimensional scene drawing device, which is applied to a wearable device. The device includes: a startup and operation module, configured to start and operate the wearable device, calculate initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in a memory; an acquisition module, configured to acquire the current pose of the wearable device at the current moment; a judgment module, configured to judge whether there is three-dimensional scene rendering data corresponding to the current pose stored in the memory; an update module, configured to, if there is no three-dimensional scene rendering data corresponding to the current pose stored in the memory, calculate the current three-dimensional scene rendering data corresponding to the current pose, and update the three-dimensional scene rendering data stored in the memory with the current three-dimensional scene rendering data; a drawing module, configured to draw a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory.

[0013] According to another aspect of the embodiments of the present application, there is provided a wearable device, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions cause the processor to perform the operations of the three-dimensional scene drawing method as described above.

[0014] According to yet another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which executable instructions are stored, and the executable instructions perform the operations of the three-dimensional scene drawing method as described above when running.

[0015] In the embodiments of the present application, according to the usage characteristics and habits of users for wearable devices, first, the initial three-dimensional scene rendering data corresponding to the initial pose is stored in the memory, and then it is determined whether the three-dimensional scene rendering data corresponding to the current pose of the wearable device is stored in the memory. If so, the corresponding three-dimensional scene rendering data is directly used to draw the three-dimensional scene, avoiding recalculating the three-dimensional scene rendering data corresponding to the current pose, thereby reducing the amount of calculation. If not, after calculating the current three-dimensional scene rendering data corresponding to the current pose, the current three-dimensional scene rendering data is used to draw the three-dimensional scene, and the current three-dimensional scene rendering data is stored in the memory for subsequent use, avoiding repeated calculation of the three-dimensional scene rendering data when the pose of the wearable device is the same as the current pose in the future, thereby reducing the amount of calculation.

[0016] Moreover, usually when a user views a three-dimensional scene using a wearable device, after starting the wearable device, the user usually looks around and scans the three-dimensional scene first, and then rotates the head to carefully observe the images of the three-dimensional scene corresponding to different perspectives. Therefore, in the embodiments of the present application, according to the usage characteristics and habits of users for wearable devices, during the process that the user looks around and scans the three-dimensional scene once, the three-dimensional scene rendering data corresponding to different poses can be stored in the memory. Then, when the user rotates the head (i.e., makes the wearable device in different poses) to carefully observe the images of the three-dimensional scene corresponding to different perspectives, the three-dimensional scene rendering data corresponding to different poses can be directly obtained from the memory to draw the three-dimensional scene, avoiding repeated calculation of the three-dimensional scene rendering data, greatly reducing the amount of data calculation, reducing resource consumption, reducing the heat generated by the wearable device, increasing the battery life of the wearable device, and enhancing the user experience.

[0017] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to be able to understand the technical means of the embodiments of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the embodiments of the present application more obvious and understandable, the following specifically illustrates the embodiments of the present application. Brief Description of the Drawings

[0018] The drawings are only used to illustrate the embodiments and are not considered as a limitation to the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0019] Figure 1 A flowchart showing the method for drawing a three-dimensional scene provided by an embodiment of the present application is shown;

[0020] Figure 2 A flowchart showing the method for drawing a three-dimensional scene provided by another embodiment of the present application is shown;

[0021] Figure 3Shows a schematic diagram of the visible area after the user wears the XR glasses and the cube texture map provided by the embodiment of the present application;

[0022] Figure 4 Shows a schematic diagram of the cube texture map provided by another embodiment of the present application;

[0023] Figure 5 Shows a schematic diagram of the structure of the three-dimensional scene rendering device provided by the embodiment of the present application;

[0024] Figure 6 Shows a schematic diagram of the structure of the wearable device provided by the embodiment of the present application. Detailed implementation manners

[0025] Hereinafter, the exemplary embodiments of the present application will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0026] The XR device can render the pictures in different regions of the three-dimensional scene respectively to display the three-dimensional scene. For a dynamic three-dimensional scene, in order to enable the user to view a relatively smooth three-dimensional scene through the XR device, the XR device usually needs to reach 90 - 120 FPS (Frames per Second, the number of frames of the picture rendered per second), that is, display 90 - 120 frames of images per second. However, in order to display 90 - 120 frames of images per second, if the XR device needs to calculate 90 - 120 frames of three-dimensional scene rendering data per second and use the three-dimensional scene rendering data to draw the three-dimensional scene, the computing power requirements for the XR device are relatively high.

[0027] The XR device generally refers to wearable devices such as AR, VR, and MR. In order to facilitate the user to wear, it is usually necessary to control its volume within a certain range. Therefore, there are certain limitations in various aspects such as the Central Processing Unit (CPU), Graphics Processing Unit (GPU), memory, motherboard bandwidth, and battery life in the XR device, resulting in a large gap in its computing power compared with desktop computers and laptop computers. And when performing three-dimensional scene rendering, this performance difference will be more obvious. Specifically, the frame rate of the three-dimensional scene displayed by the XR device is relatively low, and when the three-dimensional scene is relatively large and the rendering pressure is relatively heavy, there will also be problems such as stuttering and non-smooth display of the three-dimensional scene.

[0028] To reduce the computational load during the three-dimensional scene rendering of XR devices, the foveated rendering method can be adopted. All the computing resources of the XR device are used to render the display area being gazed at by the human eye. The three-dimensional scene content is rendered at a high resolution in this area, while the non-gazed area is rendered at a low resolution, thereby reducing the resources required for rendering the three-dimensional scene, improving the user experience, and reducing energy consumption. However, to utilize the foveated rendering method, the XR device needs to be equipped with a highly sensitive eye movement tracking sensor module to determine the gaze point of the user's eyes. Additionally, it is necessary to combine a camera, a reflector, and specific algorithms to accurately capture the gaze point of the user's eyes in real time. This approach has high hardware requirements for the XR device and is costly due to the need for additional eye movement tracking accessories. Moreover, in this method, regardless of whether the gaze point changes, the rendering data for each frame of the three-dimensional scene needs to be calculated, resulting in a still relatively large computational load.

[0029] When a user wears an XR device to view a three-dimensional scene, they usually first scan the entire three-dimensional scene and then rotate their head to carefully observe the details in the three-dimensional scene. Moreover, for a static three-dimensional scene, when the user's pose is the same, the images of the three-dimensional scene viewed through the XR device are the same.

[0030] Therefore, based on the above considerations, the present application proposes a three-dimensional scene rendering method, which is applied to a wearable device (such as an XR device). After starting and running the wearable device, first calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device and store it in the memory. By obtaining the current pose of the wearable device, if the current three-dimensional scene rendering data corresponding to the current pose is stored in the memory, directly obtain the current three-dimensional scene rendering data from the memory to render the three-dimensional scene, without having to calculate the current three-dimensional scene rendering data again, thereby reducing the computational load. If the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, calculate the current three-dimensional scene rendering data corresponding to the current pose, then use the current three-dimensional scene rendering data to render the three-dimensional scene, and store the current three-dimensional scene rendering data in the memory for subsequent reuse, so that the current three-dimensional scene rendering data can be directly utilized later without having to calculate the three-dimensional scene rendering data again, thereby reducing the computational load.

[0031] Figure 1The figure shows a schematic flowchart of a three-dimensional scene rendering method provided by an embodiment of the present application, which is executed by a wearable device (such as an XR device). The wearable device may be a wearable device including one or more processors, and the processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, which is not limited herein. The one or more processors included in the wearable device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs, which is not limited herein. As Figure 1 shown, the method includes the following steps:

[0032] Step 110: Start the wearable device, calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in the memory.

[0033] For the convenience of introduction, the present application will take an XR glasses as an example of the wearable device for introduction. It can be understood that when a user views a three-dimensional scene through the eyes, the perspective is different, and the picture of the three-dimensional scene viewed is also different. For example, when the user stands at the same position and rotates the head, the perspective will change accordingly, and the picture of the three-dimensional scene viewed will also be different; if the user keeps the head still and moves the body, the picture of the three-dimensional scene viewed will also be different. The rotation of the user's head or the movement of the body will cause the pose of the XR glasses to be different, where the pose includes position and attitude. For a user wearing XR glasses, if he stands at the same position and rotates the head, the position in the pose of the XR glasses is the same but the attitude is different; if he keeps the head still and moves the body, the position in the pose of the XR glasses is different but the attitude is the same; of course, if the user moves the body and rotates the head, both the position and the attitude in the pose of the XR glasses are different. Therefore, if the pose of the XR glasses is different, the picture of the three-dimensional scene viewed by the user through the XR glasses is also different, that is, there is a one-to-one correspondence between the picture of the three-dimensional scene viewed by the user through the XR glasses and the pose of the XR glasses.

[0034] In this step, after the user wears the XR glasses at time t1 and starts the XR glasses to run, the initial pose p1 of the XR glasses at time t1 is obtained, and the initial 3D scene rendering data corresponding to the initial pose p1 is calculated, so that the 3D scene can be directly drawn using the initial 3D scene rendering data subsequently. Moreover, the initial 3D scene rendering data corresponds to the initial pose p1 of the XR device at time t1. It should be noted that those skilled in the art can calculate the 3D scene rendering data according to relevant existing technologies. Therefore, the specific implementation of calculating the 3D scene rendering data will not be elaborated in this application. Herein, the memory is the memory in the XR glasses. It is worth noting that for the convenience of subsequently using the 3D scene rendering data stored in the memory, the correspondence between the 3D scene rendering data and the pose of the XR glasses can be stored in the memory simultaneously, or attribute information including the pose corresponding to the 3D scene rendering data can be added to each stored 3D scene rendering data.

[0035] Step 120: Obtain the current pose of the wearable device at the current moment.

[0036] Among them, since the XR glasses need to display the picture of the 3D scene corresponding to the user's current view for the user to watch, in this step, by obtaining the current pose of the XR glasses at the current moment, the user's current view can be determined, so as to correctly display the picture of the 3D scene corresponding to the current view subsequently.

[0037] Step 130: Determine whether there is 3D scene rendering data corresponding to the current pose stored in the memory. If yes, go to step 150; if not, go to step 140.

[0038] As introduced above, there is a one-to-one correspondence between the pose of the XR glasses and the picture of the 3D scene, that is, there is a one-to-one correspondence between the pose of the XR glasses and the 3D scene rendering data. Therefore, in this step, according to the correspondence between the 3D scene rendering data and the pose stored in the memory, it can be determined whether there is 3D scene rendering data corresponding to the current pose stored in the memory.

[0039] Step 140: Calculate the current 3D scene rendering data corresponding to the current pose, and update the 3D scene rendering data stored in the memory with the current 3D scene rendering data.

[0040] Among them, if the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, then in this step, in order to enable the XR device to display the picture of the three-dimensional scene corresponding to the current pose, the current three-dimensional scene rendering data corresponding to the current pose is calculated, so that the picture of the three-dimensional scene can be drawn and displayed using the current three-dimensional scene rendering data subsequently. For example, if the current pose is p2, but only the three-dimensional scene rendering data corresponding to p1 is stored in the memory, then step 140 needs to be executed to calculate the current three-dimensional scene rendering data corresponding to the current pose p2; another example is that if the current pose is p3, but only the three-dimensional scene rendering data corresponding to p1 and p2 is stored in the memory, step 140 also needs to be executed to calculate the three-dimensional scene rendering data corresponding to the current pose p3.

[0041] Step 150: Render the three-dimensional scene according to the three-dimensional scene rendering data stored in the memory corresponding to the current pose.

[0042] Among them, in this step, the three-dimensional scene rendering data corresponding to the current pose is used for screen rendering, so that the screen of the XR glasses displays the picture of the three-dimensional scene corresponding to the user's current perspective. It should be noted that those skilled in the art can implement rendering the three-dimensional scene using the three-dimensional scene rendering data according to the relevant prior art, so the relevant details are not elaborated in this application.

[0043] Specifically, if the judgment result of step 130 is that the three-dimensional scene rendering data corresponding to the current pose is stored in the memory, for example, if the current pose is p2 and the three-dimensional scene rendering data corresponding to p1 and p2 is stored in the memory, then step 150 can be executed to render the three-dimensional scene according to the three-dimensional scene rendering data corresponding to p2 stored in the memory; if the judgment result of step 130 is that the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, for example, if the current pose is p2 and only the three-dimensional scene rendering data corresponding to p1 is stored in the memory, then step 140 can be executed first to calculate the three-dimensional scene rendering data corresponding to the current pose p2 and update the three-dimensional scene rendering data stored in the memory with the three-dimensional scene rendering data corresponding to p2, and then step 150 is executed to render the three-dimensional scene according to the three-dimensional scene rendering data corresponding to p2 stored in the memory.

[0044] In the embodiments of the present application, in view of the usage characteristics and habits of users for XR glasses, first, the initial three-dimensional scene rendering data corresponding to the initial pose is stored in the memory, and then it is determined whether the three-dimensional scene rendering data corresponding to the current pose of the XR glasses is stored in the memory. If so, the corresponding three-dimensional scene rendering data is directly used to draw the three-dimensional scene, avoiding recalculating the three-dimensional scene rendering data corresponding to the current pose, thereby reducing the amount of calculation; if not, after calculating the current three-dimensional scene rendering data corresponding to the current pose, the current three-dimensional scene rendering data is used to draw the three-dimensional scene, and the current three-dimensional scene rendering data is stored in the memory for subsequent use, avoiding recalculating the three-dimensional scene rendering data when the subsequent pose of the XR glasses is the same as the current pose, thereby reducing the amount of calculation, and by reusing the three-dimensional scene rendering data, the rendering frame rate is improved.

[0045] Moreover, when a user usually uses an XR glasses to view a three-dimensional scene, after starting and running the XR glasses, the user usually looks around and scans the three-dimensional scene first, and then rotates the head to carefully observe the images of the three-dimensional scene corresponding to different perspectives. Therefore, in the embodiments of the present application, in view of the usage characteristics and habits of users for XR glasses, during the process that the user looks around and scans the three-dimensional scene once, the three-dimensional scene rendering data corresponding to different poses can be stored in the memory. Then, when the user rotates the head (i.e., makes the XR glasses in different poses) to carefully observe the images of the three-dimensional scene corresponding to different perspectives, the three-dimensional scene rendering data corresponding to different poses can be directly obtained from the memory to draw the three-dimensional scene, avoiding recalculating the three-dimensional scene rendering data, greatly reducing the amount of data calculation, reducing resource consumption, reducing the heat generated by the XR glasses, increasing the battery life of the XR glasses, and enhancing the user experience.

[0046] In addition, the embodiments of the present application do not require additional configuration of eye tracking accessories, reducing the requirements for hardware and lowering the cost.

[0047] Figure 2 The flowchart of the three-dimensional scene drawing method provided by another embodiment of the present application is shown, as Figure 2 shown, the method includes the following steps:

[0048] Step 210: Start and run the wearable device, calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in the memory.

[0049] Step 220: Obtain the current pose of the wearable device at the current moment.

[0050] Step 230: Determine whether the three-dimensional scene rendering data corresponding to the current pose is stored in the memory. If so, go to step 270; if not, go to step 240.

[0051] Among them, steps 210 to 230 are the same as steps 110 to 130. Therefore, for the principles and specific implementation manners of steps 210 to 230, reference can be made to steps 110 to 130, and details will not be elaborated here.

[0052] Step 240: Determine whether the wearable device has moved through the pose perception module in the wearable device. If so, go to step 250; if not, go to step 260.

[0053] When a user views a three-dimensional scene using an XR glasses, the main actions are rotating the head, and rarely moving the body. Therefore, in order to improve the resource utilization rate of the memory and avoid resource waste, in the embodiments of the present application, the three-dimensional scene rendering data stored in the memory is the three-dimensional scene rendering data corresponding to different poses at the same position of the XR glasses. For example, if the position of the XR glasses is position A during the time period from t2 to t3, the three-dimensional scene rendering data stored in the memory is the three-dimensional scene rendering data corresponding to position A. Among them, usually a pose perception module is provided in the XR glasses, such as an inertial measurement unit (IMU), or an accelerometer, a gyroscope, etc. The pose perception module can detect the pose of the XR glasses in real time. Therefore, through the pose perception module, it is possible to determine whether the XR glasses have moved based on the current pose of the XR glasses at the current moment and the historical pose of the XR glasses at the previous moment. If the current position in the current pose is different from the historical position in the historical pose, it can be determined that the XR glasses have moved.

[0054] Step 250: Clear the three-dimensional scene rendering data stored in the memory.

[0055] As introduced above, since the storage resources of the memory are limited, in order to improve the resource utilization rate of the memory, the three-dimensional scene rendering data stored in the memory is the three-dimensional scene rendering data corresponding to different poses at the same position of the XR glasses. Therefore, if it is determined in step 240 that the XR glasses have moved, in this step, all the three-dimensional scene rendering data stored in the memory is cleared, so as to store the three-dimensional scene rendering data corresponding to the current position of the XR glasses subsequently.

[0056] Step 260: Calculate the current three-dimensional scene rendering data corresponding to the current pose, and store the current three-dimensional scene rendering data in the memory.

[0057] Among them, if the XR glasses have moved, after step 250 clears the three-dimensional scene rendering data stored in the memory, in this step, calculate the current three-dimensional scene rendering data corresponding to the current pose, and directly store the current three-dimensional scene rendering data in the memory for subsequent use.

[0058] Step 270: Render a three-dimensional scene according to the three-dimensional scene rendering data stored in the memory corresponding to the current pose.

[0059] Among them, Step 270 is the same as Step 150. Therefore, for the principle and specific implementation of Step 270, reference can be made to Step 150 and will not be elaborated here.

[0060] The pose includes position and attitude. When the user rotates the head, the attitude of the XR glasses will change. When the user moves the body, the position of the XR glasses will change. Whether the position of the XR glasses changes or the attitude changes, the pose of the XR glasses will change. The images of the three-dimensional scenes corresponding to different poses are different. Moreover, usually when the user is using the XR glasses, after the user moves from the historical position to the current position, the probability of returning to the historical position later is relatively small. For example, after the user moves forward 1 meter, the probability of moving backward 1 meter later is relatively small. Therefore, in the embodiments of the present application, only the three-dimensional scene rendering data corresponding to the XR glasses at the same position is stored in the memory. When the XR glasses move and the position changes, the three-dimensional scene rendering data stored in the memory is cleared in time, and the three-dimensional scene rendering data corresponding to the historical position is no longer stored, thereby improving the utilization rate of the storage resources of the memory.

[0061] In order to enable the XR glasses to reflect the changes of the dynamic three-dimensional scene in real time, in the embodiments of the present application, the three-dimensional scene rendering method further includes the following steps:

[0062] Step a1: If the three-dimensional scene corresponding to the current pose is a dynamic scene, obtain the clearing time when the three-dimensional scene rendering data stored in the memory was last cleared. Among them, if the three-dimensional scene rendering data stored in the memory has not been cleared, the clearing time is the startup and running time of the wearable device.

[0063] Among them, a dynamic scene refers to a scene in which the content of the image in the three-dimensional scene changes even when the pose of the XR glasses remains unchanged. For example, if a vehicle is included in the three-dimensional scene and the vehicle will have a relative displacement with the road during driving, it belongs to a dynamic three-dimensional scene. Or for the augmented reality three-dimensional scene with virtual images superimposed on the real scene, if someone suddenly enters the real scene at a certain moment, then even if the pose of the XR glasses does not change, the three-dimensional scene will change; or if there are continuously moving objects in the real scene, it also belongs to a dynamic scene.

[0064] Step a2: Determine whether the interval duration between the current time and the clearing time reaches a preset duration. If so, go to Step a3; if not, go to Step a1.

[0065] Among them, the preset duration can be set as needed. For example, it can be set to 0.5 s, 1 s, 2 s, etc.

[0066] Step a3: Clear the three-dimensional scene rendering data stored in the memory.

[0067] In the embodiments of the present application, since the three-dimensional scene rendering data stored in the memory corresponds to the pose of the XR glasses, that is, the three-dimensional scene rendering data corresponding to a certain pose stored in the memory is fixed. However, for a dynamic three-dimensional scene, even if the current pose of the XR glasses does not change, the content of the three-dimensional scene corresponding to its current pose will change. Therefore, in the embodiments of the present application, for a dynamic three-dimensional scene, regardless of whether the pose of the XR glasses changes, the three-dimensional scene rendering data stored in the memory will be cleared regularly, and then the three-dimensional scene rendering data stored in the memory will be updated regularly, so that the XR glasses can more real-timely reflect the changes of the dynamic three-dimensional scene.

[0068] In order to improve the display effect and the utilization rate of the storage resources in the memory, in the embodiments of the present application, the three-dimensional scene rendering data stored in the memory is also created according to the resolution of the screen of the wearable device. The three-dimensional scene rendering method further includes the following steps:

[0069] Step b1: Obtain the current resolution of the screen of the wearable device at the current moment.

[0070] Among them, in order to improve the user experience, in some XR glasses, users can set the resolution of the screen display by themselves to meet the different needs of different users.

[0071] Step b2: Determine whether the current resolution is consistent with the resolution corresponding to the three-dimensional scene rendering data stored in the memory. If so, go to step b1; if not, go to step b3.

[0072] Among them, in the embodiments of the present application, since the three-dimensional scene rendering data stored in the memory is created according to the resolution of the screen of the XR glasses, that is to say, there is a corresponding relationship between the three-dimensional scene rendering data stored in the memory and the resolution of the screen. Therefore, after obtaining the resolution corresponding to the three-dimensional scene rendering data stored in the memory in this step, it can be determined whether it is consistent with the current resolution.

[0073] Step b3: Clear the three-dimensional scene rendering data stored in the memory.

[0074] Combined with the actual usage situation of users using XR glasses, during the process of using XR glasses, users usually do not frequently change the resolution of the screen display. Therefore, in the embodiments of the present application, in order to improve the utilization rate of storage resources in the memory, by determining whether the resolution corresponding to the three-dimensional scene rendering data stored in the memory is consistent with the current resolution, if not, the three-dimensional scene rendering data stored in the memory is promptly cleared, and then the three-dimensional scene rendering data corresponding to the current resolution is calculated and stored in the memory. Compared with the method of storing three-dimensional scene rendering data corresponding to multiple resolutions in the memory, the amount of data stored in the memory can be reduced, thereby improving the utilization rate of storage resources in the memory.

[0075] Moreover, by the above settings, the three-dimensional scene rendering data stored in the memory corresponds to the current resolution. Then, when using the three-dimensional scene rendering data stored in the memory to draw a three-dimensional scene and display it on the screen of the XR glasses, the displayed three-dimensional scene can be matched with the current resolution of the screen of the XR glasses, and the display effect is better, thereby improving the user experience.

[0076] In some embodiments, storing the initial three-dimensional scene rendering data in the memory in step 210 includes the following steps c1 to step c2.

[0077] Step c1: Create a target texture map and store the target texture map in the memory, where the target texture map is a cube texture map, a two-dimensional texture map, or a three-dimensional texture map.

[0078] Among them, the target texture map created in this step is a blank texture map without any data filled in it.

[0079] Step c2: Fill the initial three-dimensional scene rendering data into the target texture map.

[0080] Storing the current three-dimensional scene rendering data in the memory in step 260 includes: filling the current three-dimensional scene rendering data into the target texture map.

[0081] Among them, in the embodiments of the present application, only the case where the target texture map is a cube texture map is taken as an example for introduction. To better introduce the cube texture map, Figure 3 shows a schematic diagram of the visible area of the user wearing the XR glasses and the cube texture map in the embodiments of the present application. As Figure 3 shown in (a) of, after the user wears the XR glasses, the shaded part corresponds to the visible area. The visible area is only a small area in the three-dimensional scene. When the user's head rotates, the visible area will sweep across the three-dimensional scene. As Figure 3As shown in (b), the cube map includes six faces: front, back, left, right, top, and bottom. If the user's visible area at time t5 is used as the reference, the 3D scene rendering data within the visible area corresponding to time t5 is filled into the "front" face of the cube map. If the user does not move at time t6 and rotates their head upward, the 3D scene rendering data within the visible area corresponding to time t6 is filled into the "top" face of the cube map. If the user does not move at time t7 and rotates their head downward, the 3D scene rendering data within the visible area corresponding to time t7 is filled into the "bottom" face of the cube map, and so on. After the user rotates their head once, all six faces of the cube texture map are filled with 3D scene rendering data, and the 3D scene rendering data filled on each face corresponds to the same position but different postures of the XR glasses.

[0082] In an embodiment of the present application, a cube texture map is used to store three-dimensional scene rendering data. After the three-dimensional scene rendering data is filled into the cube texture map, based on the characteristics of the cube texture map, the 720-degree three-dimensional scene rendering data corresponding to when the user is in the same position can be completely saved, thereby forming a closed rendering texture cube with the three-dimensional scene rendering data facing inward, so that 720 degrees of different three-dimensional scene rendering data can be directly collected through the cube texture map subsequently.

[0083] In order to avoid waste of resources, based on the above embodiment, in the embodiment of the present application, step c2 includes the following steps d1 to d3.

[0084] Step d1: Divide the target texture map into multiple regions.

[0085] For example, if the target texture map is a cube texture map, the cube texture map can be divided into six regions corresponding to the front, back, left, right, top, and bottom faces, respectively. Alternatively, the cube texture map can be divided into more than six regions, with each face being divided into multiple regions.

[0086] Step d2: Fill the initial three-dimensional scene rendering data into the area corresponding to the initial pose among the multiple areas.

[0087] Among them, taking the target texture map as a cube texture map as an example, the 720-degree space formed by the six faces of the cube texture map corresponds one to one to the 720-degree three-dimensional scene seen by the user by rotating the head when the user is in the same position. Therefore, in this step, the area in the target texture map corresponding to the initial posture can be determined according to the initial posture, and then the initial three-dimensional scene rendering data can be filled into the area corresponding to the initial posture.

[0088] Step d3: Marking the region filled with the three-dimensional scene rendering data among the multiple regions as filled.

[0089] Filling the current 3D scene rendering data into the target texture map includes the following steps e1 to e2.

[0090] Step e1: determining an unfilled area that is not identified as filled among the areas corresponding to the current three-dimensional scene rendering data in the plurality of areas.

[0091] It can be understood that when the user is in the same position, if the user rotates his head by a small amplitude, the three-dimensional scene image seen before the user rotates his head and the three-dimensional scene image seen after the user rotates his head will overlap, that is, the three-dimensional scene rendering data corresponding to the postures before and after the head rotation have the same part.

[0092] Here, to better illustrate the unfilled area, Figure 4 FIG. 1 shows a schematic diagram of a cube texture map provided by another embodiment of the present application. Figure 4 As shown, the cube texture map is Figure 3 Cube texture in (b) Figure 1 The same, including the front, back, left, right, top and bottom six faces, among which the "front" face in the cube texture map is divided into 25 areas, namely Figure 4 The "right" face of the cube texture map is also divided into 25 regions, namely Figure 4 For example, at time t3, the posture p3 of the XR glasses corresponds to Figure 4 At time t4, the user does not change his position and rotates his head to the right. At time t4, the posture p4 of the XR glasses corresponds to areas 21 to 35 in the target texture map, and there is overlap between areas 11 to 25 and areas 21 to 35. At time t3, the XR glasses have filled the three-dimensional scene rendering data corresponding to posture p3 into areas 11 to 25 in the cube texture map, that is, areas 11 to 25 in the cube texture map have been marked as filled. The unfilled area in this step is area 26 to 35 in the cube texture map.

[0093] Step e2: Fill the unfilled area with the three-dimensional scene rendering data corresponding to the unfilled area in the current three-dimensional scene rendering data.

[0094] In the embodiments of the present application, by dividing the target texture map into multiple regions, after each filling of the three-dimensional scene rendering data into the target texture map, the corresponding regions are marked as filled. Thus, the unfilled regions without the three-dimensional scene rendering data can be directly and quickly identified according to the marks. After calculating the three-dimensional scene rendering data corresponding to different poses subsequently, only the three-dimensional scene rendering data corresponding to the unfilled regions is filled into the unfilled regions. Compared with the case of filling all the three-dimensional scene rendering data into the target texture map, the situation of repeatedly filling the three-dimensional scene rendering data into the same region can be effectively avoided, thereby avoiding resource waste and improving the efficiency of filling the three-dimensional scene rendering data into the target texture map.

[0095] Figure 5 shows a schematic structural diagram of a three-dimensional scene drawing device provided by an embodiment of the present application. As Figure 5 shown, the device 300 is applied to a wearable device. The device 300 includes: a startup and operation module 301, an acquisition module 302, a judgment module 303, an update module 304, and a drawing module 305. Among them, the startup and operation module 301 is used to start and operate the wearable device, calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in the memory. The acquisition module 302 is used to acquire the current pose of the wearable device at the current moment. The judgment module 303 is used to judge whether there is three-dimensional scene rendering data corresponding to the current pose stored in the memory. The update module 304 is used to calculate the current three-dimensional scene rendering data corresponding to the current pose and update the three-dimensional scene rendering data stored in the memory with the current three-dimensional scene rendering data if there is no three-dimensional scene rendering data corresponding to the current pose stored in the memory. The drawing module 305 is used to draw a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory.

[0096] The three-dimensional scene drawing device provided in this embodiment is used to execute the technical solution of the three-dimensional scene drawing method in the foregoing method embodiments. Its implementation principle and technical effects are similar and will not be elaborated here.

[0097] It should be noted that the three-dimensional scene drawing device provided in this embodiment further includes other modules for executing the respective steps of the above three-dimensional scene drawing method embodiments, which will not be elaborated one by one here.

[0098] Figure 6 shows a schematic structural diagram of a wearable device provided by an embodiment of the present application. The specific implementation of the wearable device is not limited in the specific embodiments of the present application.

[0099] As Figure 6As shown, the wearable device 400 may include: a processor 402 and a memory 404.

[0100] Among them, the memory 404 is used to store a computer program 406. The memory 404 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory. The computer program 406 may include computer-executable instructions.

[0101] The processor 402 is used to execute the computer program 406 to implement the three-dimensional scene rendering method embodiment described above.

[0102] The processor 402 may be a central processing unit (CPU), or a specific application integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the wearable device 400 may be of the same type of processor, such as one or more CPUs; or they may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0103] The embodiments of the present application provide a computer-readable storage medium. The storage medium stores executable instructions. When the executable instructions run on the wearable device, the wearable device is caused to execute the three-dimensional scene rendering method in any of the above method embodiments.

[0104] The embodiments of the present application provide a computer program. The computer program can be called by a processor to cause the wearable device to execute the three-dimensional scene rendering method in any of the above method embodiments.

[0105] The embodiments of the present application provide a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions run on a computer, the computer is caused to execute the three-dimensional scene rendering method in any of the above method embodiments.

[0106] In several embodiments provided in the present application, if any function is implemented in the form of a software functional module / unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, all or part of the technical solution of the present application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be an electronic device such as a personal computer, a server, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store computer program codes.

[0107] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings provided herein. Based on the above description, the structure required to construct such systems will be apparent. In addition, the embodiments of the present application are not directed to any particular programming language. It should be understood that the content of the present application described herein can be implemented using various programming languages, and the descriptions made above with respect to specific languages are for the purpose of disclosing the best mode of the present application.

[0108] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a claim listing several devices, several units or modules of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0109] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A three-dimensional scene rendering method, applied to a wearable device, characterized in that, The method includes: Start running the wearable device, calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in the memory; Obtain the current pose of the wearable device at the current moment; Determine whether three-dimensional scene rendering data corresponding to the current pose is stored in the memory; If the three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, calculate the current three-dimensional scene rendering data corresponding to the current pose, and update the three-dimensional scene rendering data stored in the memory with the current three-dimensional scene rendering data; Draw a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory; If the three-dimensional scene corresponding to the current pose is a dynamic scene, obtain the clearing time when the three-dimensional scene rendering data stored in the memory was last cleared, where if the three-dimensional scene rendering data stored in the memory has not been cleared, the clearing time is the start running time of the wearable device; Determine whether the interval duration between the current moment and the clearing time reaches a preset duration; If the interval duration reaches the preset duration, clear the three-dimensional scene rendering data stored in the memory.

2. The method according to claim 1, wherein The calculating the current three-dimensional scene rendering data corresponding to the current pose and updating the three-dimensional scene rendering data stored in the memory with the current three-dimensional scene rendering data includes: Judge whether the wearable device moves through the pose perception module in the wearable device; If the wearable device does not move, calculate the current three-dimensional scene rendering data corresponding to the current pose, and store the current three-dimensional scene rendering data in the memory.

3. The method according to claim 2, wherein After judging whether the wearable device moves through the pose perception module in the wearable device, the method further includes: If the wearable device moves, clear the three-dimensional scene rendering data stored in the memory; Calculate the current three-dimensional scene rendering data corresponding to the current pose, and store the current three-dimensional scene rendering data in the memory.

4. The method according to claim 1, wherein The three-dimensional scene rendering data stored in the memory is also created according to the resolution of the screen of the wearable device; The method further includes: Obtain the current resolution of the screen of the wearable device at the current moment; Determine whether the current resolution is consistent with the resolution corresponding to the three-dimensional scene rendering data stored in the memory; If the current resolution is not consistent with the resolution corresponding to the three-dimensional scene rendering data stored in the memory, clear the three-dimensional scene rendering data stored in the memory.

5. The method according to claim 2 or 3, characterized in that, The storing the initial three-dimensional scene rendering data in the memory includes: Create a target texture map, and store the target texture map in the memory, where the target texture map is a cube texture map, a two-dimensional texture map or a three-dimensional texture map; Fill the initial three-dimensional scene rendering data into the target texture map; The storing the current three-dimensional scene rendering data in the memory includes: Fill the current three-dimensional scene rendering data into the target texture map.

6. The method according to claim 5, wherein The filling the initial three-dimensional scene rendering data into the target texture map includes: Divide the target texture map into multiple regions; Fill the initial three-dimensional scene rendering data into the region corresponding to the initial pose among the multiple regions; Mark the regions in the multiple regions that have been filled with three-dimensional scene rendering data as filled; The filling the current three-dimensional scene rendering data into the target texture map includes: Determine the unfilled regions in the regions corresponding to the current three-dimensional scene rendering data among the multiple regions that are not marked as filled; Fill the three-dimensional scene rendering data corresponding to the unfilled regions in the current three-dimensional scene rendering data into the unfilled regions.

7. A three-dimensional scene rendering device, applied to a wearable device, characterized in that, The device includes: A startup and operation module, configured to start and operate the wearable device, calculate the initial three-dimensional scene rendering data corresponding to the initial pose of the wearable device, and store the initial three-dimensional scene rendering data in a memory; An acquisition module, configured to acquire the current pose of the wearable device at the current moment; A judgment module, configured to judge whether three-dimensional scene rendering data corresponding to the current pose is stored in the memory, and further configured to, if the three-dimensional scene corresponding to the current pose is a dynamic scene, acquire the clearing time when the three-dimensional scene rendering data stored in the memory was last cleared, wherein, if the three-dimensional scene rendering data stored in the memory has not been cleared, the clearing time is the startup and operation time of the wearable device, judge whether the interval duration between the current moment and the clearing time reaches a preset duration, and if the interval duration reaches the preset duration, clear the three-dimensional scene rendering data stored in the memory; An update module, configured to, if three-dimensional scene rendering data corresponding to the current pose is not stored in the memory, calculate the current three-dimensional scene rendering data corresponding to the current pose, and update the three-dimensional scene rendering data stored in the memory with the current three-dimensional scene rendering data; A rendering module, configured to render a three-dimensional scene according to the three-dimensional scene rendering data corresponding to the current pose stored in the memory.

8. A wearable device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the three-dimensional scene rendering method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the three-dimensional scene rendering method according to any one of claims 1 to 6.

Images