Body tracking method, body tracking system and host

Abstract

The invention provides a body tracking method, a body tracking system and a host. The body tracking method comprises the following steps: acquiring an environment map of a real world around a user; obtaining, via a tracker, tracker data, where the tracker is adapted to be mounted on a foot or leg of the user; determining a ground shape of a foot position of the user based on the environment map and the tracker data; determining gait information of the feet based on the ground shape; and determining a body posture of the user based on the gait information.

Description

Technical Field

[0001] The present invention relates to a body tracking method; in particular, the present disclosure relates to a body tracking method, a body tracking system and a host. Background Art

[0002] In order to bring an immersive experience to users, technologies related to extended reality (XR), such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), are being continuously developed. AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter a new virtual world and experience a different life. MR technology combines the real world and the virtual world. In addition, in order to bring a complete immersive experience to users, visual content, audio content, or other sensory content can be provided through one or more devices. Summary of the invention

[0003] The present invention is directed to a body tracking method, a body tracking system and a host, thereby providing a more accurate body tracking method for various environments in the real world.

[0004] According to an embodiment of the present invention, a body tracking method includes: obtaining an environmental map of the real world around a user; obtaining tracker data via a tracker, wherein the tracker is suitable for being installed on a foot or a leg of the user; determining a ground shape of a user's foot position based on the environmental map and the tracker data; determining gait information of the foot based on the ground shape; and determining the user's body posture based on the gait information.

[0005] According to an embodiment of the present invention, a body tracking system includes a camera, a tracker, and a host. The camera is configured to obtain an environmental image of an environment surrounding a user in the real world. The tracker is configured to obtain tracker data and is suitable for being installed on a foot or a leg of the user. The host includes a storage circuit and a processor. The storage circuit is configured to store program code. The processor is coupled to the storage circuit and accesses the program code to execute: based on the environmental image, obtaining an environmental map of the real world surrounding the user in real time; obtaining tracker data via the tracker; determining the ground shape of the user's foot position based on the environmental map and the tracker data; determining the gait information of the foot based on the ground shape; determining the user's body posture based on the gait information.

[0006] According to an embodiment of the present invention, the host includes a storage circuit and a processor. The storage circuit is configured to store program code. The processor is coupled to the storage circuit and accesses the program code to execute: obtaining an environmental map of the real world around the user; obtaining tracker data via a tracker, wherein the tracker is suitable for being installed on the foot or leg of the user; determining the ground shape of the user's foot position based on the environmental map and the tracker data; determining the gait information of the foot based on the ground shape; and determining the user's body posture based on the gait information.

[0007] In summary, according to the body tracking method, the body tracking system and the host, no matter what shape the ground is, the body posture can be accurately determined, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A as well as Figure 1B is a schematic diagram of two body tracking scenarios of the present disclosure according to some embodiments.

[0009] Figure 2 is a schematic diagram of a body tracking system of the present disclosure according to an embodiment.

[0010] Figure 3A is a schematic diagram of a body tracking scenario of the present disclosure according to an embodiment.

[0011] Figure 3B is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment.

[0012] Figure 4A is a schematic diagram of a body tracking scenario of the present disclosure according to an embodiment.

[0013] Figure 4B is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment.

[0014] Figure 5 is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment. DETAILED DESCRIPTION

[0015] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0016] Body tracking can be used to track the movement of a user in the real world, and the results of body tracking can be used to determine the movement of an avatar representing the user in the virtual world. It is worth mentioning that the ground in the real world may not be completely flat. That is, there may be some ups and downs on the ground. Therefore, when the user moves in the real world, the user may occasionally step on a higher ground or a lower ground. However, when the ground in the real world is not level, body tracking may not be able to accurately determine the user's body pose. Therefore, those skilled in the art seek to provide a more accurate body tracking method for various environments in the real world.

[0017] Figure 1A as well as Figure 1B is a schematic diagram of two body tracking scenarios of the present disclosure according to some embodiments. Figure 1A as well as Figure 1B , the head mounted device (HMD) 110 may be mounted on the user's head, and the tracker 120 may be mounted on different parts of the user's body. Figure 1A as well as Figure 1B As shown, the tracker 120 may be installed on the user's left wrist (left hand), right wrist (right hand), waist (torso), left leg (left foot), and right leg (right foot). However, the present disclosure is not limited thereto.

[0018] First, refer to Figure 1A . Figure 1A A body tracking scenario 100A without scene understanding is depicted. Specifically, the tracker 120 is configured to collect tracker data about the user's movements. The tracker data may include the position and orientation of the user's body parts, such as the head, torso, and limbs. Then, based on the tracker data, body tracking may be performed to determine the user's body posture.

[0019] However, if the ground around the user in the real world is not level, the user's body posture may not be accurately determined based on the tracker data. This is because body tracking assumes that the ground is level and bases its calculations on this assumption. As a result, the body posture may appear unnatural, and as a result, the user's avatar represented in the virtual world using the body posture may look unnatural. For example, if the user is standing on a slope, body tracking may determine that the user is leaning forward. However, in reality, the user may simply be standing upright on the slope.

[0020] Now refer to Figure 1B . Figure 1BA body tracking scenario 100B with scene understanding is depicted. Specifically, scene understanding is used to better understand the environment around the user in the real world. The better understanding includes information about the layout of the environment, and includes the location and orientation of objects in the environment, which can be referred to as scene information.

[0021] Furthermore, body tracking can be performed not only based on tracker data, but also based on scene information to more accurately determine the user's body posture. That is, body tracking can take into account the fact that the ground is level or not. In this way, the body posture will be more natural, and the avatar representing the user in the virtual world using the body posture can look more real. For example, if the user is standing on a slope, body tracking will determine that the body posture is standing upright on the slope, rather than leaning forward. In addition, if the user is standing on stairs, body tracking will determine that the body posture is standing on the surface of the stairs, rather than the feet floating above the stairs or penetrating the stairs.

[0022] Simply put, scene understanding is an important aspect of tracking technology, which can more accurately represent the user's movements and interactions with the environment. Without scene understanding, body tracking may misinterpret the user's body posture due to the uneven ground, resulting in a distorted representation of the body posture. The distorted representation may not only affect the user experience, but also damage the overall accuracy and reliability of the tracking system. By incorporating scene understanding into the body tracking method, the environment can be analyzed and key features such as objects, ground surfaces, and obstacles can be identified. In other words, scene understanding can enable the body tracking method to provide more accurate and realistic body tracking data.

[0023] Figure 2 is a schematic diagram of a body tracking system of the present disclosure according to an embodiment. Figure 2 , the body tracking system 200 may include a host 210, a camera 220, and a tracker 230. In one embodiment, Figure 2 As shown, the host 210, the camera 220 and the tracker 230 can be separately provided. In another embodiment, the host 210 can be integrated with the camera 220 or the tracker 230. However, the present disclosure is not limited thereto. In addition, for the sake of simplicity, the body tracking system 200 is depicted as including the tracker 230. However, the number of trackers 230 in the body tracking system 200 can be one or more. That is, the present disclosure does not limit the number of trackers 230.

[0024] In addition, the tracker 230 may be the same as the tracker 120. That is, the tracker 230 may be adapted to be mounted on a body part of the user so as to determine the posture of the body part. In addition, the HMD 110 may further include the tracker 230 so as to determine the head data of the user. The head data may include the position and orientation of the head of the user. That is, body tracking may be performed based on the head data to determine the head posture of the user.

[0025] In various embodiments, the host 210 may be any smart device and / or computer equipment. In some embodiments, the host 210 may be any electronic device capable of providing reality services (e.g., AR / VR / MR services, etc.). In some embodiments, the host 210 may be implemented as an XR device, such as a pair of AR / VR glasses and / or HMD 110. In some embodiments, the host 210 may be a computer and / or a server, and the host 210 may provide the calculation results (e.g., AR / VR / MR content) to a display device so that the display device may display the calculation results to the user. However, the present disclosure is not limited thereto.

[0026] In some embodiments, the HMD 110 may include a display module, and the display module may include, for example, an organic light emitting diode (OLED) display device, a mini LED display device, a micro LED display device, a quantum dot (QD) LED display device, a liquid crystal display (LCD) device, a tiled display device, a foldable display device, or an electronic paper display (EPD). However, the present disclosure is not limited thereto.

[0027] The host 210 includes a storage circuit 212 and a processor 214. The storage circuit 212 is one or a combination of a fixed or mobile random access memory (RAM), a read-only memory (ROM), a flash memory, a hard disk, or any other similar device, and it can record a plurality of modules and / or program codes executed by the processor 214.

[0028] The processor 214 may be coupled to the memory circuit 212, and the processor 214 may be, for example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, or the like.

[0029] In some embodiments, the host 210 may be further coupled to a camera 220. The camera 220 may be configured to obtain an environmental image of the environment surrounding the user in the real world, wherein multiple environmental images may form a panorama and / or cubemaps, but the present disclosure is not limited thereto. The camera 220 may be, for example, a complementary metal oxide semiconductor (CMOS) camera, a charge coupled device (CCD) camera, a light detection and ranging (LiDAR) device, a radar, an infrared sensor, an ultrasonic sensor, a depth camera, other similar devices, or a combination of these devices. In some embodiments, the camera 220 may be disposed on top of the HMD 110, the tracker 120, wearable glasses (e.g., AR / VR goggles), an electronic device, other similar devices, or a combination of these devices. However, the present disclosure is not limited thereto.

[0030] In some embodiments, host 210 may be further coupled to tracker 230. Tracker 230 may include a gyroscope, an accelerometer, an inertial measurement unit (IMU) sensor, other similar devices, or a combination of these devices. However, the present disclosure is not limited thereto.

[0031] In some embodiments, the host 210, the camera 220 and / or the tracker 230 may further include a communication circuit, and the communication circuit may include a wired network module, a wireless network module, a Bluetooth module, an infrared module, a radio frequency identification (e.g., RFID) module, a Zigbee network module, or a near field communication (NFC) network module, but the present disclosure is not limited thereto. That is, the host 210 may communicate with an external device (e.g., the camera 220, the tracker 230, a display, etc.) via wired communication or wireless communication.

[0032] In an embodiment of the present disclosure, the processor 214 may implement the body tracking method provided by the present disclosure by accessing the modules and / or program codes stored in the storage circuit 212, which will be further discussed below.

[0033] Figure 3A is a schematic diagram of a body tracking scenario of the present disclosure according to an embodiment. Figure 3A In FIG. 3 , body tracking scenario 300A includes scenarios 301-304. Figure 1A Body tracking scenario 100A in Figure 1B Similar to the body tracking scenario 100B in FIG. 1 , the tracker 120 may be installed on the wrists of the user's hands, the waist of the torso, and the feet of the user's feet. However, the present disclosure is not limited thereto.

[0034] Reference Figure 3AIn scenario 301, scene understanding may be performed to obtain an environment map of the environment surrounding the user in the real world. That is, the environment map may be obtained based on a scene understanding algorithm, and the environment map may include multiple scene meshes. In one embodiment, the scene mesh is a digital representation of the 3D geometry of the environment. For example, multiple scene meshes together may be like a digital blueprint that captures the position and shape of objects (including the ground) in the environment.

[0035] In one embodiment, the environment map may be obtained in real time based on the scene understanding algorithm via the camera 220. For example, the camera 220 may be integrated with the HMD 110, or one of the tracker 120 and the camera 220 may be configured to obtain an environment image of the environment. However, the present disclosure is not limited thereto. In another embodiment, the environment map may be pre-stored in a map database, and the environment map may be obtained from the map database instead of obtaining the environment map in real time.

[0036] In scenario 302, after obtaining the environment map, the foot location of the user's foot in the environment can be determined based on the environment map and the tracker data of the tracker 120 on the foot. In one embodiment, the tracker 120 on the foot can be called a foot tracker, and the tracker data of the foot tracker can be called foot tracker data. Therefore, since the position of the foot in the environment is known, and the ground location (groundlocation) of the ground stepped on by the foot may be the same as the foot location, the ground shape (ground shape) of the ground stepped on by the foot can be determined based on the scene grid and the ground location of the environment map. In addition, based on the ground shape, it can be determined how the foot is placed on the ground. In other words, gait information (foot step information) can be determined based on the ground shape. For example, gait information can include the shape of the foot stepping on the ground position (e.g., position and orientation) and the position and orientation relationship (position and orientation relationship) between the foot and the ground. In other words, the gait information may include the user's foot posture, which is determined based on the shape of the ground at the user's foot position, and the ground shape is determined based on the environment map and the tracker data.

[0037] In one embodiment, a foot step model may be pre-trained to determine gait information. That is, different types of ground shapes, such as uphill, downhill, stairs, flat ground, etc., may be used to pre-train the gait model. Therefore, by inputting the ground shape into the gait model, gait information may be determined based on the gait model. However, the present disclosure is not limited thereto.

[0038] It is noteworthy that gait information can be used to optimize another foot posture (e.g., part of the body posture) of the user determined based on foot tracking (e.g., part of body tracking). In one embodiment, foot tracking can be performed based on foot tracker data from a foot tracker and / or other tracker data from other trackers 120 on other body parts to determine another foot posture. For example, foot tracking can be configured to obtain the position and orientation of the foot based on an inverse kinematics algorithm using some or all of the tracker data. In other words, the foot posture can be determined based on the tracker data. Then, based on the foot posture determined according to the tracker data and the foot posture determined according to the ground shape, an optimized foot posture can be obtained to ensure that the foot posture looks real and at the same time ensure that the foot and the ground shape of the ground match. In other words, the foot posture determined based on foot tracking can be optimized based on gait information. More details of the optimization of the foot posture will be explained in detail with reference to scenario 303 below.

[0039] In scenario 303, after the gait information is determined, body tracking can be performed based on the gait information and tracker data from all trackers 120 (including foot tracker data from foot trackers 120 on the feet, other tracker data from other trackers 120 on other body parts, and / or head data from HMD 110). Based on body tracking, body posture can be determined. For example, body posture (including foot posture) is obtained using an inside-out algorithm, such as a simultaneous localization and mapping (SLAM) algorithm based on tracker data (such as camera images). Specifically, since the gait information has been determined, the foot posture can be determined based on the gait information and / or the foot tracker data. In addition, the foot posture is part of the body posture. Therefore, the foot posture can be used as a clue to solve the rest of the body posture (for example, trunk posture, hand posture, head posture... etc. posture). In other words, the body posture can be determined from the bottom to the top, based on the gait information, starting from the foot posture. That is, the body posture can be determined starting from the user's feet, and the body posture can be optimized based on the gait information.

[0040] In scenario 304, after the body posture is determined, the body posture may be output to determine and display the avatar posture of the avatar representing the user in the virtual world. In one embodiment, the virtual environment in the virtual world may be the same as the environment in the real world. That is, the virtual environment may be a true reproduction of the real world environment, so that the user can perform various actions in the virtual world, just as if the user is in the real world. For example, when the user steps on the stairs in the real world, the avatar may also step on the virtual stairs in the virtual world. It is worth noting that since the body posture is correctly determined, the avatar posture can be more natural, thereby increasing the immersive experience.

[0041] In another embodiment, the virtual environment in the virtual world may be different from the environment in the real world. That is, objects and ground surfaces in different areas of the real world may be different from virtual objects and virtual ground surfaces in different areas of the virtual world. For example, when a user steps on the stairs in the real world, the avatar may step on the virtual ground in the virtual world. When the user in the real world steps on the flat ground, the avatar may step on the virtual stairs in the virtual world. It is worth noting that since the body posture is determined from the foot posture, the avatar posture can also be determined from the virtual foot posture. Therefore, even if the ground shape of the ground in the real world is different from the virtual ground shape of the virtual ground, the virtual character can still step on the virtual ground instead of floating above the virtual ground or penetrating the virtual ground. In other words, the ground can be aligned with the virtual ground without a complex analysis process, thereby improving the user experience. However, in another embodiment, in an AR application, the avatar can be configured to represent the user's real posture in the displayed real world (i.e., virtual world). That is, the user can see the displayed real feet (i.e., the virtual feet of the avatar) stepping on the displayed real ground (i.e., virtual ground) via the display device. In other words, the real feet displayed by the virtual character may still be able to step on the real displayed ground, rather than floating above the real displayed ground.

[0042] It is worth mentioning that the foot position is determined based on the environment map and the foot tracker data. In other words, it is not necessary to obtain images of the foot and the ground to determine the relationship between the foot and the ground. Therefore, the requirements for body tracking can be reduced, thereby improving the performance of body tracking.

[0043] Figure 3B FIG. 1 is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment. Figure 3B In the body tracking method 300B, the body tracking method 300B includes Figure 3AThe method of this embodiment can be performed via the exemplary steps S310-S340 corresponding to the scenarios 301-304 in FIG. Figure 2 However, the present disclosure is not limited thereto. In addition, the implementation details of the body tracking method 300B can be referred to Figure 3A The description of the present invention is sufficient to provide sufficient teachings, suggestions and embodiments of the implementation method, which will not be repeated here.

[0044] Reference Figure 2 , Figure 3A as well as Figure 3B . In step S310, as shown in scenario 301, the camera 220 can obtain an environmental map in real time. In step S320, as shown in scenario 302, gait information can be determined by the processor 214 based on the environmental map and the foot position information. For example, the foot position information may include the foot position determined based on the foot tracker data and / or the foot position determined based on the image obtained by the tracker 120. It should be noted that the image does not have to include both the foot and the ground surface. The image only needs to contain enough information to identify the foot position of the foot in the environmental map.

[0045] In step S330, as shown in scenario 303, the user's body posture can be determined based on the environment map and gait information via the processor 214. In step S340, as shown in scenario 304, the body posture can be displayed via the HMD 110. In this way, regardless of the shape of the ground, the body posture can be accurately determined, thereby improving the user experience.

[0046] Figure 4A is a schematic diagram of a body tracking scenario of the present disclosure according to an embodiment. Figure 4B FIG. 1 is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment. Figure 4A In FIG. 4 , body tracking scenario 400A is an exemplary embodiment of scenario 302 or step S320 for determining gait information. Figure 4B In the method 400B for tracking the body, the method 400B includes steps S410-S450. Steps S410 and S450 can be respectively Figure 3B The steps S310 and S340 in FIG. 3 are the same, and the steps S420-S440 may be exemplary embodiments of steps for determining the details of the scenario 302 or step S320 of the gait information. However, the present disclosure is not limited thereto.

[0047] In addition, the implementation details of the body tracking method 400B can be referred to Figure 3A The description above is sufficient to provide sufficient teachings, suggestions and embodiments of the implementation method, which will not be repeated here.

[0048] Reference Figures 2 to 4B In step S410, an environment map is obtained. Furthermore, the environment map may be obtained based on a scene understanding algorithm, and the environment map may include a plurality of scene grids. Each scene grid may belong to an object in the environment surrounding the user or the ground of an area in the environment. For example, Figure 4A As shown in scenario 401 in FIG. 4 , a user may be standing on stairs, and a scene mesh may be configured to represent the stairs.

[0049] In step S420, the tracker data is obtained from the tracker 120 and / or the head data is obtained from the tracker 230 of the HMD 110. The processor 214 may be configured to determine the posture of each of the tracker 120 and / or the HMD 110 based on the tracker data or the head data, respectively. For example, the tracker posture may be determined for each of the plurality of trackers 120 and / or the HMD posture may be determined for the HMD 110. And, as shown in scenario 401, since each of the plurality of trackers 120 and the HMD 110 is suitable for being worn on a part of the user, the posture of each of the plurality of trackers 120 and the HMD 110 may be considered as the posture of the said part of the user.

[0050] For example, the HMD posture may be considered as the user's head posture. The tracker posture of the tracker on the user's left hand may be considered as the user's left hand posture. The tracker posture of the tracker on the user's right hand may be considered as the user's right hand posture. The tracker posture of the tracker on the user's torso may be considered as the user's torso posture. The tracker posture of the tracker on the user's left foot may be considered as the user's left foot posture. The tracker posture of the tracker on the user's right foot may be considered as the user's right foot posture. However, the present disclosure is not limited thereto.

[0051] In step S430, after determining the posture of each of the multiple trackers 120 and the HMD 110, the position of each of the multiple trackers 120 and the HMD 110 can be determined based on the environment map. For example, the position of the left foot in the environment map can be determined based on the left foot posture and the environment map. Similarly, the position of the right foot in the environment map can be determined based on the right foot posture and the environment map... and so on. In addition, based on the foot position (e.g., the left foot position or the right foot position), the user position of the user in the environment map can be determined (e.g., the same as or close to the foot position). That is, the foot position or the user position can be configured to represent the ground of an area where the user can be located in the environment. Based on the foot position or the user position and the environment map, the ground shape of the ground (where the user is) can be determined. As Figure 4AAs shown in scenario 402 in FIG. 4 , each of the plurality of scene grids may be configured to represent a ground shape. That is, the processor 214 may be configured to determine one of the plurality of scene grids in the environment map as a ground shape based on the tracker data.

[0052] In step S440, gait information may be determined based on the scene grid (representing the shape of the ground). For example, the gait information may include the position and orientation of the foot, and the position and orientation relationship between the foot and the ground. In other words, the foot posture may also be determined based on the gait information. It is worth noting that, since the foot posture is determined based on the scene grid, the foot posture may be properly placed on the ground, rather than the foot floating above the ground or penetrating the ground.

[0053] In step S450, body tracking may be performed based on the foot posture, and the (complete) body posture of the user may be determined based on the body tracking. In this way, the body posture may be accurately determined regardless of the shape of the ground, thereby improving the user experience.

[0054] Figure 5 FIG. 1 is a schematic flow chart of a body tracking method of the present disclosure according to an embodiment. Figure 5 In the embodiment, the body tracking method 500 includes steps S510-S530. Steps S510-S530 may be exemplary embodiments of steps for determining details in the scenario 303, step S330 or step S450 of the body posture. However, the present disclosure is not limited thereto.

[0055] In addition, the implementation details of the body tracking method 500 can be referred to Figure 3A The description of the present invention is sufficient to provide sufficient teachings, suggestions and embodiments of the implementation method, which will not be repeated here.

[0056] In step S510, the foot posture can be determined based on the gait information. It should be noted that the foot posture can be used as a clue to solve the rest of the body posture (e.g., torso posture, hand posture, head posture, etc.). In other words, the body posture can be determined based on the gait information, starting from the foot posture.

[0057] In step S520, the head posture may be determined based on the head data in the tracker 230 in the HMD 110. It should be noted that one end of the user's body posture may be the foot posture, and the other end may be the head posture. In other words, the height or outline of the body posture may be determined based on the foot posture and the head posture.

[0058] In step S530, body tracking can be performed based on the foot posture and the head posture to determine the (complete) body posture of the user. Since the height or contour of the body posture is determined, the body posture determined by body tracking can be more accurate, thereby increasing the immersive experience.

[0059] In summary, according to the body tracking methods 300B, 400B, 500, the body tracking system 200 and the host 210, since the gait information is determined based on the ground shape of the environment, the foot posture can be properly placed on the ground, rather than the foot floating on the ground or penetrating the ground. In this way, regardless of the ground shape, the body posture can be accurately determined, thereby improving the user experience.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A body tracking method, characterized in that: include: Obtain a real-world environment map around the user; obtaining tracker data via a tracker, wherein the tracker is adapted to be mounted on a foot or a leg of the user; determining a ground shape at a location of the user's foot based on the environment map and the tracker data; determining gait information of the foot based on the ground shape; as well as Based on the gait information, a body posture of the user is determined.

2. The body tracking method according to claim 1, characterized in that: Also includes: Based on a scene understanding algorithm, the environment map is obtained, wherein the environment map includes a plurality of scene grids.

3. The body tracking method according to claim 2, characterized in that: Also includes: Based on the tracker data, one of the plurality of scene meshes is determined to be the ground shape.

4. The body tracking method according to claim 1, characterized in that: Also includes: The environment map is obtained in real time via a camera arranged on the tracker.

5. The body tracking method according to claim 1, characterized in that: Also includes: The body posture is determined starting from the feet of the user.

6. The body tracking method according to claim 1, characterized in that: Also includes: Determining foot posture based on the tracker data; determining a head posture of the user's head based on head data from a head-mounted device, wherein the head-mounted device is adapted to be mounted on the head; as well as The body posture is determined based on the foot posture and the head posture.

7. The body tracking method according to claim 1, characterized in that: Also includes: Based on the gait information, performing body tracking; as well as Based on the body tracking, the body pose is determined.

8. The body tracking method according to claim 1, characterized in that: Also includes: inputting the ground shape into a gait model; as well as The gait information is determined based on the gait model.

9. The body tracking method according to claim 1, characterized in that: Also includes: Based on an inside-out algorithm and the tracker data, obtaining the body posture; as well as Based on the gait information, the body posture is optimized.

10. A body tracking system, characterized in that: include: a camera configured to obtain an environmental image of an environment surrounding a user in the real world; a tracker configured to obtain tracker data and adapted to be mounted on a foot or a leg of the user; as well as Host, including: a storage circuit configured to store program code; and A processor is coupled to the memory circuit and accesses the program code to execute: Based on the environmental image, obtaining an environmental map of the real world surrounding the user in real time; From the tracker, obtaining tracker data; determining a ground shape at a location of the user's foot based on the environment map and the tracker data; determining gait information of the foot based on the ground shape; and Based on the gait information, a body posture of the user is determined.

11. A host, characterized in that: include: a storage circuit configured to store program code; as well as A processor is coupled to the memory circuit and accesses the program code to execute: Obtain a real-world environment map around the user; obtaining tracker data via a tracker, wherein the tracker is adapted to be mounted on a foot or a leg of the user; determining a ground shape at a location of the user's foot based on the environment map and the tracker data; determining gait information of the foot based on the ground shape; as well as Based on the gait information, a body posture of the user is determined.

Images