Map establishing method and host
By simulating the movement path and posture of a virtual tracking device in a digital environment model using a host computer, rendering viewpoint images and generating spatial maps, the problem of time-consuming, labor-intensive, and unstable virtual reality scene map creation in existing technologies is solved, achieving efficient and stable map generation.
Patent Information
- Application Number
- CN202510132728.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-12-11
- Filing Date
- 2025-02-06
- Publication Date
- 2025-12-05
AI Technical Summary
In existing technologies, creating spatial maps for virtual reality scenes is time-consuming, labor-intensive, and of inconsistent quality. In particular, when multiple players experience the same real-world services in the same scene, specific players need to move around to create maps, resulting in low efficiency and inconsistent map quality.
The system reads a digital environment model of a real-world scene from the host computer, simulates the movement path and posture of a virtual tracking device, renders images from multiple perspectives, and builds a spatial map based on these images. It then uses SLAM technology to generate a high-quality spatial map.
High-quality spatial maps can be generated quickly and stably without human intervention, reducing the time and manpower costs of map creation and improving map stability.
Smart Images
Figure CN121074286A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a mechanism for establishing environment information, and particularly to a method and a host for establishing a map. BACKGROUND
[0002] In the prior art, it is quite common for multiple virtual reality (VR) players to experience the same reality service (e.g., a VR game) in the same scene (e.g., a room with specific furnishings, etc.).
[0003] In order to smoothly implement the above-mentioned application, a space map of the scene needs to be established by a certain tracking device first, and then the established map is shared to the game device (e.g., the head-mounted display (HMD) of other players) corresponding to each player by the tracking device.
[0004] For example, before starting to experience the reality service, a certain specific player can move around in the scene with an HMD, and the HMD can perform a tracking technology such as simultaneous localization and mapping (SLAM) during the movement of the specific player, so as to establish a space map (e.g., a SLAM map) of the scene.
[0005] However, the above-mentioned way of establishing a space map not only consumes a lot of physical strength of the specific player, but also takes a lot of time. For example, for a scene with an area of 30m x 10m, the specific player may need to move around in the scene for 40 minutes to establish a suitable space map.
[0006] In addition, because different players move around in the scene in different ways, the quality of the established space map may also be unstable. SUMMARY
[0007] Therefore, the present application provides a method and a host for establishing a map, which can solve the above-mentioned technical problems.
[0008] The method for establishing a map provided by the present application is executed by a host, and includes: reading a digital environment model corresponding to a real-world scene; determining a movement path of a virtual tracking device in the digital environment model; determining a plurality of poses for implementing the movement path in the digital environment model; rendering a plurality of perspective images based on the plurality of poses, wherein each of the poses corresponds to a corresponding one of the perspective images, and each of the perspective images corresponds to a perspective of the virtual tracking device for taking a picture of the digital environment model when presenting the corresponding one of the poses; and establishing a space map corresponding to the real-world scene based on the plurality of perspective images.
[0009] An embodiment of the present application provides a host comprising a storage circuit and a processor. The storage circuit stores program codes. The processor is coupled to the storage circuit and configured to access the program codes to perform: reading a digital environment model corresponding to a real-world scene; determining a movement path of a virtual tracking device in the digital environment model; determining a plurality of poses to implement the movement path in the digital environment model; rendering a plurality of perspective images based on the plurality of poses, wherein the plurality of poses respectively correspond to the plurality of perspective images, and each of the perspective images corresponds to a perspective of the virtual tracking device taking a picture of the digital environment model when rendering a corresponding one of the poses; and establishing a spatial map corresponding to the real-world scene based on the plurality of perspective images. BRIEF DESCRIPTION OF DRAWINGS
[0010] Figure 1 FIG. 1 is a schematic diagram of a host according to an embodiment of the present application.
[0011] Figure 2 FIG. 2 is a flowchart of a method of establishing a map according to an embodiment of the present application.
[0012] Figure 3 FIG. 3 is a top view of a digital environment model according to an embodiment of the present application.
[0013] Figure 4 FIG. 4 is a schematic diagram of a plurality of perspective images according to an embodiment of the present application. Figure 3
[0014] FIG. 5 is a schematic diagram of planning a movement path according to an embodiment of the present application. Figure 5
[0015] FIG. 6 is a schematic diagram of a plurality of specified actions according to an embodiment of the present application. Figure 6
[0016] FIG. 7 is a schematic diagram of planning a movement path according to an embodiment of the present application. Figure 7 Figure 3 Figure 6 FIG. 8 is a schematic diagram of planning a movement path according to an embodiment of the present application. DETAILED DESCRIPTION
[0017] Reference will now be made to the exemplary embodiments of the present application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.
[0018] Reference is made to Figure 1 FIG. 1 is a schematic diagram of a host according to an embodiment of the present application.
[0019] In various embodiments, host 100 may be any intelligent device and / or computer device capable of providing visual content for reality services, such as virtual reality (VR) services, augmented reality (AR) services, mixed reality (MR) services, and / or extended reality (XR) services, but the invention is not limited thereto. In some embodiments, host 100 may be an HMD capable of displaying / providing visual content (e.g., AR / VR / MR content) for wearer / user to view, but is not limited thereto.
[0020] exist Figure 1 In this system, the host 100 includes a storage circuit 102 and a processor 104. The storage circuit 102 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or other similar device or combination thereof, and may be used to record multiple program codes or modules.
[0021] Processor 104 is coupled to storage circuit 102 and may be a general purpose processor, special purpose processor, conventional processor, digital signal processor, multiple microprocessors, one or more microprocessors incorporating a digital signal processor core, controller, microcontroller, application specific integrated circuit (ASIC), field programmable gate array (FPGA), any other type of integrated circuit, state machine, processor based on advanced RISC machine (ARM), and the like.
[0022] In an embodiment of the present invention, the processor 104 may access the modules and program code recorded in the storage circuit 102 to implement the map creation method proposed in the present invention, the details of which are described below.
[0023] Please refer to Figure 2 This is a flowchart illustrating a map creation method according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 1 The host 100 executes, and the following is the configuration. Figure 1 Component description shown Figure 2 Details of each step.
[0024] In step S210, the processor 104 reads the digital environment model corresponding to the real-world scene.
[0025] In the present embodiment, the real-world scene is, for example, any real scene for which a spatial map (e.g., a SLAM map) is to be established, such as a space for a plurality of players to experience a reality service together, such as various rooms, sites, etc., but can not be limited thereto.
[0026] In addition, the digital environment model is, for example, a 3D space model corresponding to the real-world scene. In one embodiment, the digital environment model can completely correspond to the real-world scene. For example, the furnishings, spatial proportions, colors, lights, structures of the digital environment model can be completely identical to those of the real-world scene.
[0027] In some embodiments, the digital environment model can also be referred to as a digital twin of the real-world scene.
[0028] In general, a digital twin is a digital simulation technology that creates an accurate digital replica of a physical object (such as the above-mentioned real-world scene) or system in a virtual environment. It is not only a static digital representation, but also dynamically updated to reflect the real-time state of the physical object, helping to monitor, simulate, and predict. Digital twins are composed of physical objects, digital models, data connections, and data analysis technologies, can obtain real-time data through sensors and analyze them to provide decision-making. Digital twins have a wide range of applications, including industrial manufacturing (such as monitoring production processes and predicting equipment maintenance), urban planning (such as intelligent city management), medical health (such as human body system simulation), and automotive and aviation (such as performance analysis). Digital twins can deeply understand the running state of physical systems, predict fault risks, and thus reduce costs, improve efficiency, and accelerate innovation.
[0029] Please refer to Figure 3 which is a top view of a digital environment model according to an embodiment of the present application.
[0030] In Figure 3 , the processor 104 can read a digital environment model 300, for example, in step S210, where the digital environment model 300 is, for example, a digital twin of a certain real-world scene, but can not be limited thereto.
[0031] After that, in step S220, the processor 104 determines the movement path 310 of the virtual tracking device in the digital environment model 300.
[0032] In embodiments of the present application, the virtual tracking device is, for example, a virtual device that can perform tracking techniques in a virtual space, such as the digital environment model 300, to achieve the desired tracking function. In an embodiment, the virtual tracking device is virtual itself (e.g., a virtual HMD), and can exist in the digital environment model 300 to perform the relevant tracking techniques (e.g., inside-out tracking and / or outside-in tracking).
[0033] In some embodiments, the virtual tracking device can be provided with a virtual tracking camera, where the virtual tracking camera can have a range of taking pictures, and the range of taking pictures can capture different portions in the digital environment model 300 corresponding to the pose of the virtual tracking device.
[0034] In some embodiments, the virtual tracking device can perform SLAM in the digital environment model 300, for example, to establish a spatial map corresponding to the digital environment model 300. In embodiments of the present application, since the digital environment model 300 corresponds to the real-world scene described above, the spatial map established by the virtual tracking device will also be a spatial map corresponding to the real-world scene described above.
[0035] In Figure 3 , the movement path 310 is, for example, a path simulating the movement of the virtual tracking device in the digital environment model 300 when establishing the spatial map.
[0036] In some embodiments, to make the established spatial map more perfect, the movement path 310 can be designed in a more complex manner, such as Figure 3 shown, but can not be limited thereto.
[0037] It should be appreciated that, although Figure 3 the movement path 310 shown appears to be a movement trajectory simulating the movement of the virtual tracking device on a single plane, the actual movement path 310 can also involve multi-directional movement, multi-directional tilting, multi-directional rotation, and the like, but can not be limited thereto.
[0038] In step S230, the processor 104 determines a plurality of poses implementing the movement path 310 in the digital environment model 300.
[0039] In embodiments of the present application, the mentioned poses can be presented in the form of six degrees of freedom data, for example. That is, each pose can include translation and rotation, but can not be limited thereto.
[0040] As mentioned previously, the movement path 310 can substantially involve multi-directional movement, multi-directional pitching, multi-directional rotating, or the like behavior of the virtual tracking device, all of which can be represented by a corresponding series of poses. In other words, if the virtual tracking device is simulated to move according to the poses, the overall movement trajectory of the virtual tracking device in the digital environment model 300 can form the movement path 310, but can not be limited thereto.
[0041] In step S240, the processor 104 renders a plurality of perspective images based on the plurality of poses, wherein the plurality of poses respectively correspond to the plurality of perspective images, and each of the perspective images corresponds to a perspective at which the virtual tracking device takes a picture of the digital environment model 300 when presenting the corresponding pose.
[0042] For example, the plurality of poses can include an i-th pose (i is an index value), the plurality of perspective images can include an i-th perspective image corresponding to the i-th pose, and the i-th perspective image corresponds to a perspective at which the virtual tracking device takes a picture of the digital environment model 300 when presenting the corresponding i-th pose.
[0043] That is, when the virtual tracking device is simulated to present the i-th pose in the digital environment model 300, a virtual tracking camera of the virtual tracking device can be understood as taking a picture of the digital environment model 300 at a certain perspective. That is, a certain portion of the digital environment model 300 will be intercepted in the picture-taking range of the virtual tracking device, and the image intercepted by the virtual tracking camera at this time can be understood as the i-th perspective image.
[0044] Since the behavior of the virtual tracking device is simulated by the processor 104, the i-th perspective image, the processor 104 can render a corresponding perspective image based on the i-th pose as the i-th perspective image when the virtual tracking device presents the i-th pose, but can not be limited thereto.
[0045] Accordingly, for each pose determined in step S230, the processor 104 can render a corresponding perspective image according to the above-mentioned teachings.
[0046] Please refer to Figure 4 which is a schematic diagram of the plurality of perspective images according to Figure 3 illustrated.
[0047] In Figure 4In some embodiments, the perspective images 411-415 are perspective images corresponding to five of the plurality of poses, respectively. For example, when the virtual tracking device is simulated to assume pose 1 in the digital environment model 300, the processor 104 can render the perspective image 411; when the virtual tracking device is simulated to assume pose 2 in the digital environment model 300, the processor 104 can render the perspective image 412; when the virtual tracking device is simulated to assume pose 3 in the digital environment model 300, the processor 104 can render the perspective image 413; when the virtual tracking device is simulated to assume pose 4 in the digital environment model 300, the processor 104 can render the perspective image 414; and when the virtual tracking device is simulated to assume pose 5 in the digital environment model 300, the processor 104 can render the perspective image 415.
[0048] As can be seen from the perspective image 415, when the virtual tracking device is simulated to assume pose 5 in the digital environment model 300, the perspective image 415 can capture the door, the painting, the sofa, the statue, the floor, the lamp, and the like in the digital environment model 300, but can not be limited thereto.
[0049] In step S250, the processor 104 establishes a spatial map corresponding to the real-world scene based on the plurality of perspective images.
[0050] In one embodiment, the processor 104 can perform SLAM based on the plurality of perspective images to establish a spatial map (SLAM map) corresponding to the real-world scene.
[0051] For example, after obtaining the perspective images 411-415, the processor 104 can perform SLAM to determine information such as keyframes and / or map points in the SLAM map, but can not be limited thereto. Figure 4 For example, the processor 104 can detect feature points from each perspective image and perform feature point matching on the feature points in different perspective images. Then, the processor 104 can generate three-dimensional map points of the digital environment model 300 through triangulation.
[0052] In the process of performing SLAM, the processor 104 can select and store keyframes according to changes in the pose of the virtual tracking device or the scene within the perspective image range to reduce the computational burden and provide stable relocation basis. Through these steps, the processor 104 applying SLAM can gradually construct an accurate spatial map using different perspective images and achieve dynamic positioning and stable environment modeling, but can not be limited thereto.
[0053]
[0054] By the above technical means, the establishment of the space map can be completed without consuming human labor. Moreover, the quality stability of the space map can be improved.
[0055] In some embodiments, Figure 2 The method of the present application can also be executed on a computing device with higher computing capability (e.g. a personal computer, a server, etc.). In this case, since these computing devices can render the plurality of perspective images with better efficiency, the time required for establishing the space map can be effectively shortened.
[0056] In an embodiment, the host 100 (e.g. the HMD of a player) and at least one other host (e.g. the HMD of another player) are located in the real-world scene, and after the space map corresponding to the real-world scene is established, the processor 104 can share the space map corresponding to the real-world scene to the at least one other host, wherein the host 100 and the at least one other host provide the same reality service.
[0057] That is, after the host 100 establishes the space map in the manner of Figure 2 , the host 100 can share the established space map to other hosts located in the same space / venue / room, so as to enable these hosts to provide the same reality service with the same space map.
[0058] In another embodiment, the host 100 (e.g. a server) can share the space map corresponding to the real-world scene to the at least one other host after the space map corresponding to the real-world scene is established.
[0059] That is, the embodiments of the present application can share the established space map to other hosts (e.g. the HMDs of different players) located in the real-world space after the method of Figure 2 is executed by a computing device with higher computing capability (e.g. a server), but can not be limited thereto.
[0060] Please refer to Figure 5 , which is a schematic diagram of planning a moving path according to an embodiment of the present application.
[0061] In Figure 5 , for the digital environment model 50, the processor 104 can divide the digital environment model 50 into a plurality of blocks 51 (e.g. the blocks numbered 1 to 24 shown) and plan the order of passing through the plurality of blocks 51, for example.
[0062] In the present embodiment, the processor 104 can plan to pass through the blocks numbered 1 to 24 in sequence (indicated by the arrows shown), but can not be limited thereto.
[0063] In embodiments of the present application, the plurality of blocks 51 includes a first block (e.g., one of the blocks numbered 1 to 24), and the movement path includes a first path segment located in the first block. In addition, the plurality of poses can include a plurality of first poses for implementing the first path segment, and the plurality of first poses are used to simulate a plurality of specified actions sequentially performed by the virtual tracking device in the first block.
[0064] Please refer to Figure 6 , which is a schematic diagram of a plurality of specified actions according to embodiments of the present application.
[0065] In Figure 6 , the specified action 1 is, for example, a user (e.g., the specific player mentioned earlier) holding a tracking device (e.g., an HMD) in a block rotates a circle in place in the block. The specified action 2 is, for example, the user holding the tracking device moves the tracking device up and down while rotating a circle in place in the block. The specified action 3 is, for example, the user holding the tracking device walks around the block.
[0066] Accordingly, for the first path segment located in the first block, the processor 104 can simulate a plurality of specific poses corresponding to the virtual tracking device being sequentially used to perform the specified action 1, the specified action 2, and the specified action 3 in the first block, and then use the plurality of specific poses as a plurality of first poses for implementing the first path segment, but can not be limited thereto.
[0067] For example, for the block numbered 1, the processor 104 can simulate a plurality of specific poses corresponding to the virtual tracking device being sequentially used to perform the specified action 1, the specified action 2, and the specified action 3 in the block numbered 1, and then use the plurality of specific poses as a plurality of poses for implementing the path segment located in the block numbered 1.
[0068] For example, for the block numbered 2, the processor 104 can simulate a plurality of specific poses corresponding to the virtual tracking device being sequentially used to perform the specified action 1, the specified action 2, and the specified action 3 in the block numbered 2, and then use the plurality of specific poses as a plurality of poses for implementing the path segment located in the block numbered 2.
[0069] For other numbered blocks, the processor 104 can determine a plurality of poses corresponding to the path segment located in each block according to the above principle.
[0070] After that, the plurality of poses corresponding to the blocks numbered 1 to 24, respectively, can be integrated to form the movement path mentioned in step S220, but can not be limited thereto.
[0071] Please refer to Figure 7 , which is a schematic diagram of a plurality of specified actions according to embodiments of the present application. Figure 3 and Figure 6A schematic diagram of a planned movement path is shown.
[0072] In Figure 7 In the context, the digital environment model 300 is divided into blocks numbered 1 to 9, and the virtual tracking device is assumed to simulate the movement of the specified actions 1 to 3 in sequence in each block, thereby forming a trajectory such as the movement path 700, but not limited thereto.
[0073] In summary, the method for establishing a map proposed by the embodiments of the present application can simulate multiple poses of the virtual tracking device in the digital environment model, render corresponding perspective images, and then establish a spatial map corresponding to the digital environment model and the real-world scene based on the rendered perspective images. In this way, the spatial map corresponding to the real-world scene can be established in a more labor-saving, time-saving and stable manner.
[0074] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method of building a map, performed by a host, characterized by, comprising: reading a digital environment model corresponding to a real-world scene; determining a movement path of a virtual tracking device in the digital environment model; determining a plurality of poses that implement the movement path in the digital environment model; rendering a plurality of perspective images based on the plurality of poses, wherein the plurality of poses respectively correspond to the plurality of perspective images, and each of the perspective images corresponds to a perspective of the virtual tracking device taking a picture of the digital environment model when presenting a corresponding one of the poses; and establishing a spatial map corresponding to the real-world scene based on the plurality of perspective images.
2. The method of claim 1, further comprising: segmenting the digital environment model into a plurality of segments, and planning an order of passing through the plurality of segments, wherein the plurality of segments comprises a first segment, the movement path comprises a first path segment located in the first segment, the plurality of poses comprises a plurality of first poses that implement the first path segment, and the plurality of first poses are used to simulate a plurality of specified actions performed by the virtual tracking device in the first segment in the order.
3. The method of claim 1, wherein the plurality of poses comprises an i-th pose, the plurality of perspective images comprises an i-th perspective image corresponding to the i-th pose, and the i-th perspective image corresponds to a perspective of the virtual tracking device taking a picture of the digital environment model when presenting the corresponding i-th pose, where i is an index value.
4. The method of claim 1, wherein establishing the spatial map corresponding to the real-world scene based on the plurality of perspective images comprises: performing simultaneous localization and mapping based on the plurality of perspective images to establish the spatial map corresponding to the real-world scene.
5. The method of claim 1, wherein the host and at least one other host are located in the real-world scene, and after establishing the spatial map corresponding to the real-world scene, the method further comprises: sharing the spatial map corresponding to the real-world scene to the at least one other host, wherein the host and the at least one other host provide the same reality service.
6. The method of claim 1, wherein after establishing the spatial map corresponding to the real-world scene, the method further comprises: sharing the spatial map corresponding to the real-world scene to at least one other host located in the real-world scene.
7. A host, characterized by comprising: a storage circuit that stores program code; and a processor coupled to the storage circuit and configured to access the program code to perform: reading a digital environment model corresponding to a real-world scene; determining a movement path of a virtual tracking device in the digital environment model; determining a plurality of poses that implement the movement path in the digital environment model; rendering a plurality of perspective images based on the plurality of poses, wherein the plurality of poses respectively correspond to the plurality of perspective images, and each of the perspective images corresponds to a perspective of the virtual tracking device taking a picture of the digital environment model when presenting a corresponding one of the poses; and establishing a spatial map corresponding to the real-world scene based on the plurality of perspective images. establish a spatial map corresponding to the real-world scene based on the plurality of perspective images.
8. The host of claim 7, wherein the processor is further configured to: differentiate the digital environment model into a plurality of blocks and plan an order through the plurality of blocks, wherein the plurality of blocks includes a first block, the movement path includes a first path segment located in the first block, the plurality of poses includes a plurality of first poses to implement the first path segment, and the plurality of first poses are used to simulate a plurality of specified actions performed in sequence by the virtual tracking device in the first block.
9. The host of claim 7, wherein the plurality of poses includes an i-th pose, the plurality of perspective images includes an i-th perspective image corresponding to the i-th pose, and the i-th perspective image corresponds to a perspective at which the virtual tracking device takes a picture of the digital environment model when presenting the corresponding i-th pose, where i is an index value.
10. The host of claim 7, wherein the processor is configured to: perform simultaneous localization and mapping based on the plurality of perspective images to establish the spatial map corresponding to the real-world scene.