Virtual Reality Simulation Using Surface Tracking

Abstract

This disclosure provides a system for rendering a virtual reality (VR) scene, and a method and computer program product for determining a follower pose in a VR simulator during a simulation step. In one example, the method includes: (1) calculating one or more current candidate poses using input parameters, wherein each of the current candidate poses is a time projection of a follower pose along a corresponding sweeping direction toward a leader pose, and wherein an obstacle is located between the follower pose and the leader pose; (2) selecting a target pose from the one or more current candidate poses; (3) refining the target pose using physical constraints and the input parameters, wherein the physical constraints utilize the surface of the obstacle; and (4) rendering a new follower pose based on the refined target pose.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. non-provisional application No. 16 / 888,408, entitled "Virtual Reality Simulation Using Surface Tracking," filed May 29, 2020, by Gordon Yeoman et al., which claims the benefit of U.S. provisional application No. 62 / 854,850, entitled "Virtual Reality Simulation Using Surface Tracking," filed May 30, 2019, by Gordon Yeoman et al., which is jointly assigned with this application and incorporated herein by reference. Technical Field

[0003] This application generally relates to virtual reality simulation, and more specifically to simulating the movement of an object in a virtual reality environment when the object interacts with obstacles. Background Technology

[0004] In virtual reality (VR) systems, sensors are used to track the user's movements so that those movements can be replicated within a simulated environment. Objects, people, animals, plants, and other obstacles displayed by the VR simulator in the simulated environment typically do not exist in the user's physical space—for example, the physical area surrounding the user. Therefore, the user can make movements or actions in their physical space that disrupt the illusion of the simulated environment. For example, a user might push their hand forward without encountering an obstacle in their physical space, while in a simulated environment, their hand would pass through a wall in front of them. This unrealistic situation in the simulated environment degrades the user's virtual experience. Summary of the Invention

[0005] In one aspect, this disclosure provides a method for determining a follower pose in a virtual reality (VR) simulator during a simulation step. In one example, the method includes: (1) calculating one or more current candidate poses using input parameters, wherein each of the current candidate poses is a temporal projection of a follower pose along a corresponding sweeping direction toward a leader pose, and wherein an obstacle is located between the follower pose and the leader pose; (2) selecting a target pose from the one or more current candidate poses; (3) refining the target pose using physics-based constraints and the input parameters, wherein the physics-based constraints utilize the surface of the obstacle; and (4) rendering a new follower pose based on the refined target pose.

[0006] In another aspect, this disclosure provides a computer program product having a series of operational instructions stored on a non-transitory computer-readable medium, which, when executed, directs a data processing device to perform operations to determine a follower pose in a VR simulator during a simulation step. In one example, the operations include: (1) calculating one or more candidate poses, each of which is a temporal projection of a follower pose along a corresponding sweeping direction toward a leader pose, and wherein an obstacle is located between the follower pose and the leader pose; (2) selecting a target pose from one or more candidate poses; (3) refining the target pose using physics-based constraints; and (4) rendering a new follower pose based on the refined target pose.

[0007] In another aspect, this disclosure provides a system for rendering VR scenes. In one example, the system includes: (1) a VR controller operable to send user parameters corresponding to a user's movement; and (2) a VR simulator operable to receive and use the user parameters and application parameters to render a scene of a simulated environment based on the interaction between a leader pose and obstacles using one or more movement algorithms, wherein the leader pose represents the user's movement in the simulated environment, and obstacles are at least partially located between the follower pose and the leader pose in the simulated environment. Attached Figure Description

[0008] The following description, taken in conjunction with the accompanying drawings, now refers to:

[0009] Figure 1A This is a diagram illustrating an example candidate pose analysis in a 2D environment;

[0010] Figure 1B This is a diagram illustrating an example of candidate pose analysis in a 3D environment;

[0011] Figure 2 This is a diagram visually representing an example method used to determine the final pose;

[0012] Figure 3 This is a diagram visually representing the method for analyzing the final pose ghosting.

[0013] Figure 4 It is a visual representation of the detailed pose of the target;

[0014] Figure 5 This is a flowchart illustrating an example of a method for determining the type of movement algorithm used to move objects in a simulated environment;

[0015] Figure 6This is a flowchart illustrating an example of a method for determining a target pose when there is an obstacle between the follower's pose and the leader's pose; and

[0016] Figure 7 This is a diagram illustrating a sample virtual reality surface tracking system. Detailed Implementation

[0017] VR simulators can depict various types of settings, locations, scenes, etc., provided to users via VR displays (i.e., simulated environments) and allow users to perform a wide variety of actions and movements (i.e., movement) that are replicated in the simulated environment within physical space. VR displays can be VR headsets, glasses, or another type of display capable of showing two-dimensional (2D) or three-dimensional (3D) simulated environments. Movements made by the user can be captured by one or more VR controllers, such as headsets that track eye and head movements, handheld controllers, cameras, haptic devices, and other types of body and sensor controllers.

[0018] Because VR simulators can create simulated environments that differ from the user's physical environment (e.g., the physical space around the user), users can initiate movements in the simulated environment that are not realistic in the physical environment, such as moving their hands through obstacles in the simulated environment. Obstacles can be objects, people, animals, plants, fluids, or other types of physics-based objects that can prevent or alter the movement of another object. A wall, as described above, is an example of an obstacle that can exist within a simulated environment.

[0019] While simulating unrealistic movement (e.g., showing a user's hand moving through a solid wall) can disrupt a user's VR experience, preventing the user from moving within a simulated environment can also negatively impact the VR experience by causing confusion. For example, when a user's hand is pushed forward in their physical space, the simulated environment might mimic the user's hand stopping an inch in front of the user against a wall. Another unsatisfactory solution is to allow the user's hand to move through obstacles within the simulated environment, while objects the user might be holding or carrying are stopped by the obstacles. This type of movement handling can be described as joint picking with penetration avoidance. Therefore, the user's virtual experience can be improved by refining the interaction between objects and obstacles in the simulated environment.

[0020] This disclosure provides systems and methods for controlling the movement of an object in a virtual environment to prevent the object from penetrating obstacles in the virtual environment. Instead of penetrating obstacles, the features disclosed herein involve moving the object along the surface of the obstacle in a manner that appears and seems believable to the user. This disclosure discloses methods for calculating potential movements approximating the user's movement (tracked by sensors) using a combination of rigid body algorithms and geometric queries, while maintaining physics-based constraints on obstacles within the simulated environment. Constraints are factors that control the movement of an object in a simulated environment. Similar to guardrails on a highway, constraints control movement along a specified path. Physics-based constraints may include constraints on the moving object or person (e.g., velocity, rate, and momentum), constraints on the interaction with the surface of the obstacle (e.g., surface rigidity, bounce, and texture), external constraints, rigid body constraints (e.g., a strong solid does not bend under small forces), and other physics-based constraints available within a VR simulator. An example of an external constraint could be that a hand cannot be forced to bend unnaturally when impacting an obstacle. A simulated hand attached to an arm may conform to constraints imposed by finger, wrist, and arm positioning, torque, and flexibility parameters. Constraints can follow rules established by the user, the application, or the VR simulator. Constraints do not necessarily follow real-life experiences; physics-based constraints can be established by the application running the simulation environment. The application could be a video game application.

[0021] When creating a simulated environment, a VR simulator balances the user's requested movement with constraints imposed by obstacles, such as forces satisfying the user's movement and contact forces from obstacles preventing movement, such as pushing a hand against a wall. A follower-leader pose structure is a technique that can be used according to the principles of this disclosure. A follower pose is the position, orientation, and orientation of an object at its starting point. A leader pose is the position, orientation, and orientation of an object at the point where the user requests movement. For example, at a point within the simulation, a character's hand is held above a tabletop. This position and orientation of the hand is the follower pose. The user moves their hand downwards. Within the simulated environment, the leader pose is the simulated hand now located and oriented below the tabletop (e.g., the destination of the movement). The follower pose is the position, orientation, and orientation of an object presented to the user for viewing on a VR display. The leader pose is used to track the movement of the object and is typically not presented to the user.

[0022] A VR simulator can move a follower pose until it encounters (e.g., intersects) an obstacle (in this case, a tabletop). The direction of the follower pose movement is the sweep direction. The result is that the follower pose is now positioned against the obstacle, such as a blocking pose. This type of solution can be restrictive. For example, if the user's simulated hand is near the edge of the tabletop, a better visual experience for the user would be to have the simulated hand move slightly to one side and glide around the corner of the tabletop, such as a stroking pose—where the movement stroks the surface of the obstacle. A surface stroking is the follower pose's movement just outside the surface of the obstacle, moving along the surface of the obstacle, while a leader pose can be just above, on, or below the surface of the obstacle. For example, a user tracks their hand on the surface of a car. This provides a smoother visual experience for the user.

[0023] The combination of blocking and stroking poses allows the follower's pose to follow the leader's pose in a physically plausible manner within the simulated environment. The blocking pose, implemented using the blocking algorithm, keeps the follower's pose stationary at the initial blocking point of the obstacle. Using the blocking algorithm alone may not be sufficient to follow the lateral and sideways movement of the leader's pose. The stroking pose, implemented using the stroking algorithm, allows the follower's pose to track the leader's lateral movement without maintaining the surface boundaries of the obstacle. Therefore, the movement satisfies the constraints of both the blocking and stroking algorithms.

[0024] In some respects, using a blocking algorithm instead of a stroking algorithm can be advantageous, such as when the size of the obstacle is equal to or smaller than the follower's pose. For example, it might be more efficient to stop (block) a moving hand at a small control knob rather than attempting to move around (stroke) the control knob. This can be determined using a sweeping pattern ratio, where the size of the moving object (such as a hand) is compared to the size of the obstacle (such as a control knob). Technically, the sweeping pattern ratio can be defined as the radius of the boundary sphere of the comparable obstacle (placed at the top of the ratio) versus the radius of the boundary sphere of the follower's geometry. If the sweeping pattern ratio is greater than the sweeping pattern parameter, such as 1.0 or another real number, then both blocking and stroking algorithms can be used. If the sweeping pattern ratio is equal to or less than the sweeping pattern parameter, then the blocking algorithm can be used without the stroking algorithm. This aspect can reduce the computational complexity of the simulated movement.

[0025] VR simulators can be configured to execute individual steps of a publicly disclosed method for moving objects within a simulated environment. As the user moves, the VR simulator can generate a visual representation in the simulated environment in small steps, known as simulation steps. The number of simulation steps can depend on several factors, such as the amount of available computing system resources (the more resources available, the more steps can be generated in a given time period), the visual quality set by the application (higher visual quality requires more computing resources to be used), and the number of updates to the visual representation over a duration (e.g., seconds) requested by the application or user (the more updates, the more computing resources may be used).

[0026] VR simulators can use parameters such as frames per second (FPS) to indicate how many visual representations (i.e., scenes) will be generated per second for output to a VR display. The typical range for FPS is 60 to 120 FPS, and some VR simulators can use lower or higher FPS. This means a VR simulator can output between 60 and 120 scene updates per second. The number of simulation steps a VR simulator uses during a duration (e.g., seconds) can be defaulted to FPS. Other factors, such as VR simulator optimization and available computing resources, can adjust the number of simulation steps to be higher or lower. As a VR simulator increases simulation steps, new information can be calculated using the user's currently requested movement, and the new scene can be output to the VR display.

[0027] VR simulators can generate scenes, i.e., render scenes, or use a separate rendering system. A renderer generates images or scenes based on scene data representing a 2D or 3D simulation environment and can output the rendered scene to a display device or storage device. A renderer can be implemented as a software application, a processor, a computing system that receives instructions and outputs scenes, or a combination thereof. The processor can be a parallel processor (such as a graphics processing unit (GPU)) or a serial processor (such as a central processing unit (CPU)). A combination of one or more processors can also be used.

[0028] For example, the rendering process can be divided between a general-purpose CPU and a graphics processing subsystem, with the GPU at the center of the architecture. The CPU performs high-level operations, such as determining the position, motion, and collisions of objects and obstacles in a given scene. Based on these high-level operations, the CPU generates a set of rendering commands and data that define the desired rendered image. The rendering commands and data can define the scene geometry, lighting, shadows, texturing, motion, and / or camera parameters. Based on this set of rendering commands and data, the graphics processing subsystem can create one or more rendered scenes.

[0029] Renderers can employ ray tracing rendering techniques. Ray tracing is an example of a technique used to render a scene in the rendering pipeline, especially when the scene contains complex lighting. Ray tracing describes any number of techniques used to efficiently resolve the visibility of a straight line along any two arbitrary points in a scene, where visibility information is used to resolve light transmission and light interactions with materials in the scene. For example, ray tracing can extend lines (i.e., rays) from the user's viewpoint to objects within the scene, and then determine second rays from that object to a light source or to another object in the scene. These rays can then be used to determine the amount of light illuminating objects (including indirect lighting and reflections), the angle of the light relative to the user's viewpoint, and how to represent the color, texture, and other properties of objects and obstacles to the user.

[0030] The processing of rays used for ray tracing rendering can be performed in parallel by a single processor or by multiple processors (such as different GPUs), and is applicable to both local and remote computation. Therefore, rendering can be performed by a cloud-based renderer that delivers frames of rendering to remote users. Parallelism as used herein includes at least partially parallelism and indicates that processing is occurring in parallel paths, but not necessarily simultaneously.

[0031] A cloud-based renderer can be implemented on one or more servers within a cloud computing platform, including one or more processors and additional components such as memory and communication interfaces. The graphics processor can receive the rendered scene from the cloud-based renderer and encode the scene into one or more video streams, which are then provided to a video transmitter for transmission over a communication network to computing devices such as VR displays. Through this cloud-based technology, users can receive video streams and enjoy virtual graphics experiences on their computing devices. For example, video game applications can provide input to a cloud-based renderer for rendering and streaming the game to one or more users' computing devices. Input can also be received from VR controllers for rendering and game streaming. Game streaming allows users to control and play the game rendered remotely from their input devices (such as VR controllers). The cloud-based rendering platform and game streaming allow users to compete remotely with others in video games via a communication network.

[0032] Regardless of whether a cloud-based renderer or a renderer located near a VR display or controller is used to perform the simulation, the movement of an object can be represented by two independent poses within the simulation environment. The follower pose typically remains outside the obstacle surface and represents the original position and orientation of the moving object for the current simulation step. The leader pose represents the result of motion without constraints. Furthermore, this disclosure utilizes candidate poses, which are latent poses determined through geometric queries (such as sweeping). The candidate poses are used to compute an approximate target pose, which is then refined through rigid body physics simulation to produce a refined target pose. Similar to the candidate poses, geometric queries are used to compute the approximate target pose. The movement of the follower pose towards the refined target pose is achieved through rigid body physics simulation. Rigid body physics simulation can utilize force factors, joint drive factors, and velocity factors in its algorithm to move the follower pose to the refined target pose. Physics engines (such as...) Figure 7 The physics engine 740 can be used to provide rigid body physics simulations.

[0033] In a single simulation step performed by a VR simulator, the follower pose moves toward the target pose. Depending on the size of the simulation step, the speed of movement, and the different constraints used, the follower pose may fail to reach the target pose and thus stop part of the process toward the target pose. The follower pose can catch up with the leader pose, such as when the movement is fast enough within the constraints of the simulation environment that the follower pose fails to reach the leader pose. In subsequent simulation steps, a new leader pose and a corresponding new candidate pose can be determined, and thus a new target pose can be selected. For example, if a user moves their fist very quickly, the scene may update in such a way that the fist portion is pointing toward where the user has moved their fist. In subsequent simulation steps, the fist's current position can be used to select a new target pose.

[0034] In the simulation phase, the VR simulator can drive the follower's pose by moving the leader's pose according to four computational stages. The first stage can be a sweeping movement from the follower's pose to the leader's pose, recording the intersections and geometric normals (i.e., intersecting normals) of any obstacles blocking the sweep. This allows for approaching candidate poses close to the leader's pose while maintaining surface blocking constraints from obstacles.

[0035] The first stage can be more efficient by selecting candidate poses near the leader pose, rather than the closest candidate pose. The goal is to identify and select target poses from the pool of candidate poses that are sufficient to satisfy various constraints with fewer sweeps. Selecting only the target pose closest to the leader pose from the candidate poses can lead to visual artifacts, such as when another candidate pose is selected or if cyclic flips occur between two or more similarly close candidate poses. Creating a weighted average of candidate poses can also lead to visual artifacts, such as jitter. Jitter can be reduced by applying additional temporal smoothing techniques.

[0036] In the first stage, one or more candidate poses can be calculated, where each candidate pose is a time projection of the follower pose along the corresponding sweeping direction. The sweeping direction is first selected to calculate the candidate poses.

[0037] By sweeping towards the leader's pose from multiple directions, samples of the obstacle surface surrounding the leader's pose can be selected from the intersecting normals and directions of the original sweep (from the follower's pose). To increase temporal coherence, the normals can be blended with those from previous simulation steps, where the number of previous simulation steps to review can be determined by the second-stage algorithm, the simulation environment, the application, or the user, such as the past two or five simulation steps. These sampling sweeps can reveal multiple candidate poses that position the follower's pose close to the outer surface of the obstacle. Each sweep provides one candidate pose. The selected sweep direction can include a direction along the original sweep direction (blocking algorithm), a direction perpendicular to the intersecting normal, and other mixed or averaged directions around the normal (smoothing algorithm). A selected sweep direction can be along the intersecting normal. After selecting the sweep direction, candidate poses are computed by sweeping along the selected sweep direction. Computing candidate poses may include processes that smooth the transition from the follower's pose to the candidate pose. Figure 1A and Figure 1B Visual representations of examples showing different sweeping directions are provided.

[0038] In the second stage, an approximate target pose is determined. To calculate the approximate target pose, a distance to the leader pose is calculated for each candidate pose. These distances are used as weights to blend the sampling sweep directions of the candidate poses. The approximate target pose can be minimized in exchange for temporal coherence. To increase temporal coherence, the blended sweep directions can be mixed with the corresponding sweep directions from previous simulation steps, where the number of simulation steps to be reviewed can be specified. The blended sweep directions can then be used as the final sweep directions for calculating the final candidate pose (referred to as the approximate target pose).

[0039] In the third stage, the approximate target pose is modified to provide a refined target pose. Rigid body algorithms can be used to refine the approximate target pose (such as...). Figure 4 (as indicated in the diagram), such that it is positioned close to or at the surface of the obstacle. This can be achieved by introducing a system of articulated rigid bodies to represent the geometry of the follower's posture and simulating the system forward in time using a rigid body algorithm until a stable state is achieved. More specifically, a jointed rigid body ensemble can be driven against the surface normal by using joint drives that prevent lateral movement (e.g., stroking movement) relative to the surface normal.

[0040] Follower-to-target pose analysis can represent movement in 2D or 3D. Additionally, the approximate target pose can be rotated along one or more dimensional axes, for example, under an applied rotational force, to reduce the distance between the approximate target pose and the leader pose. The rotational aspect of the applied rotational force can be modeled using rigid body physics to propel the approximate target pose with a linear force that generates a rotational force upon contact with an obstacle, altering the orientation of the approximate target pose relative to the obstacle's surface. The rotated approximate target pose becomes the refined target pose. The linear movement of the rigid body can be restricted to movement along the direction of the rotational force. By projecting the rigid body into the future through one or more simulation steps, the equilibrium of the refined target pose can be calculated as if it had been pushed against the obstacle's surface. For example, an outstretched hand with its fingertips pointing towards an obstacle can be rotated through a series of simulation steps so that the palm rests against the obstacle's surface, rather than the fingertips. This provides more realistic movement to improve the visual quality of the user experience.

[0041] The simulated environment may aim to maintain the leader's orientation within a refined target pose. Angular driving forces can be used to push or rotate the refined target pose in the direction of the leader's orientation. The simulated environment can use application-provided parameters to balance linear / rotational forces and angular driving forces on the refined target pose. For example, touching a touchscreen in a simulated environment might result in angular driving forces dominating to maintain the leader's orientation, as the user might want to touch the screen with their fingertips, while rotational forces might dominate when a simulated door is pushed open, where the user might want to push the door with their palm. In some aspects, the orientation may also be constrained by external constraints. For example, the hand may or may not be able to rotate to face the door due to the positioning, torque, and flexibility parameters of the fingers, wrist, arm, and body connected to the hand.

[0042] In the fourth stage, the refined target pose calculated for the steady state can be used for the follower pose. In some aspects, the fourth-stage algorithm allows for lateral movement of the refined target pose (overshoot movement) when overshoot makes it easier for the follower pose to traverse obstacles (such as edges), for example, to prevent the follower pose from getting stuck on obstacles. On the other hand, the contact friction between the obstacle surface and the follower pose can be reduced, such as to zero, to eliminate side effects from rolling, sticking, or other friction-related forces that can affect the refined target pose. In other aspects, contact recovery can be reduced when the refined target pose interacts with obstacles to avoid contact bounce. The friction coefficient parameter and contact recovery coefficient parameter can be tuned to different values ​​to achieve the results expected in the simulated environment.

[0043] For a single simulation step, the fourth stage can utilize rigid body joint actuation to advance the articulated whole representing the follower's posture in time. More specifically, the follower posture can be driven from its current state toward a refined target posture and can be passed forward as a new follower posture to the next simulation step.

[0044] When the leader's pose moves from a blocked position to an unblocked position, a direct tracking movement algorithm can be used to minimize computational resources. To utilize the direct tracking algorithm, the follower's pose can be in an approximate orientation of the leader's pose. A transition algorithm can be used to provide a smooth transition from the current follower's pose to the current leader's pose. This algorithm can use direct tracking values ​​(e.g., time parameters) such that the follower's pose continues to track the leader's pose using the blocking and stroking algorithms described herein (e.g., the first-stage to fourth-stage algorithm) until a specific time has elapsed, after which the follower's pose can switch to the direct tracking algorithm. Alternatively, the direct tracking values ​​can include a constant velocity parameter, which can be used to move the follower's pose toward the leader's pose. This prevents sudden jumps in the follower's pose.

[0045] Ghosting threshold can also be used. When the leader pose has moved through an obstacle and is not currently inside the obstacle, and there is an obstacle between the follower pose and the leader pose, the ghosting threshold can indicate whether the follower pose will move directly to the leader pose using a fast-motion algorithm (e.g., instantaneous movement), or whether the follower pose will stop at the surface of the obstacle. Figure 3 An example using a ghosting threshold is provided.

[0046] The ghosting threshold defines the thickness or distance at which a follower pose can travel through an obstacle to fast-move towards the leader pose. Ghosting thickness refers to the distance traveled through the obstacle, which may differ from the distance between the follower and leader poses. When the ghosting thickness through the obstacle is equal to or less than the ghosting threshold, the follower pose is allowed to fast-move towards the leader pose, for example, by ignoring the obstacle. Other factors may prevent this fast-move, such as external constraints. When the ghosting thickness through the obstacle is greater than the ghosting threshold, the follower pose is prevented from moving towards the leader pose.

[0047] This method can use one or more parameters, or combinations thereof, as input to determine the final refined target pose in various aspects. The parameters can be grouped by describing a set of parameters that can be used to compute the sweeps of candidate poses, such as the number of candidate poses, the mixed sweep angles to be utilized, and the number of previous simulation steps used in time smoothing. A second set of parameters describes a rigid body algorithm that can be used to compute the refined target pose, such as the physical values ​​and parameters to be used, such as the values ​​of gravity and magnetic fields. A third set of parameters describes a rigid body algorithm that drives the follower pose from its current position to the refined target pose; for example, the distance the follower pose is moved within a simulation step may depend on the FPS parameter.

[0048] Now turn to the attached diagram. Figure 1A This is an illustration, referred to as candidate pose analysis 100, visually representing an example of a method for analyzing candidate poses in a 2D environment including obstacle 110. Candidate pose analysis 100 shows three sweeping directions in addition to the initial sweeping direction between the leader pose 115 and the original follower pose 120. In such... Figure 1B In the 3D environment shown, additional sweep directions can be analyzed. For example, ten sweeps can be used for a 3D environment. Performing a large number of sweeps is computationally impractical. Therefore, the number of sweep directions to be analyzed can be based on historical data and provide a balance between results and processing time. Regardless of the shape of obstacle 100, the same number of sweep directions can be used. For example, the same number of sweep directions can be used for convex and concave obstacles. Alternatively, the number of sweeps to be used can vary based on the shape of the obstacle.

[0049] A sweep is performed by projecting the shape or geometry of the original follower pose 120 onto its current orientation along a vector from the original follower pose 120 to the leader pose 115. The leader pose 115 is the requested position of the person or object, such as a user pushing their hand forward through an obstacle 110. The original follower pose 120 is the position of the person or object being manipulated or moved (such as an avatar's hand).

[0050] The leader pose 115 is within the surface constraints of obstacle 110, and the original follower pose 120 is above obstacle 110. The original follower pose 120 may not be directly moved to the position of the leader pose 115, as this would violate the physics-based constraints of obstacle 110 and degrade the user's visual experience, for example, breaking the VR illusion. Therefore, a temporary follower pose and sweep direction are used to calculate candidate poses from which the target pose can be selected.

[0051] Temporary follower poses 122, 124, and 126, used to generate candidate poses along a selected sweeping direction, can be determined by moving the leader pose 115 vertically from the intersecting normal 142 until the leader pose 115 is completely outside the surface of the obstacle 110. The intersecting normal 142 is the normal on the same side of the obstacle 110 as the original follower pose 120. The temporary follower poses 122, 124, and 126 generated outside the geometry of the obstacle 110 can be calculated by calculating the boundary spheres around the obstacle 110 and the boundary spheres around the original follower pose 120, adding the radii of the two boundary spheres, and placing the temporary follower poses 122, 124, and 126 outside the sum of the radii.

[0052] Candidate poses can be calculated by sweeping temporary follower poses 122, 124, and 126 back toward the leader pose 115 along a selected sweeping direction perpendicular to the intersection normal 142 until the corresponding temporary follower pose intersects the surface of obstacle 110. Surface normals 144 and 146 can be determined from the intersecting surface. Through sweeping, temporary follower pose 122 generates a calculated candidate pose 132 associated with the intersection normal 142, temporary follower pose 124 generates a calculated candidate pose 134 associated with the surface normal 144, and follower pose 126 generates a calculated candidate pose 136 associated with the surface normal 146. The negative sweeping direction represented by the surface normal 148 is ignored because the original follower pose 120 is close to the surface normal 142, and the candidate pose relative to the surface normal 148 would cause the candidate pose to ghost through the obstacle. Figure 3 The text further describes the ghosting scene.

[0053] Therefore, the resulting candidate poses are calculated candidate pose 132, calculated candidate pose 134, and calculated candidate pose 136. Through additional analysis, one of the calculated candidate poses 132, 134, and 136 is selected as the target pose. The target pose can be used as an approximate target pose in its current orientation or further refined and used as a target for movement of the original follower pose 120.

[0054] Figure 1BThis is an illustration, referred to as candidate pose analysis 160, visually representing an example of a method for analyzing candidate poses in a 3D environment. Candidate pose analysis 160 includes an obstacle 162, shown as a solid cuboid, a leader pose 164 shown within the obstacle 162, and a follower pose 166. Using the current position of the leader pose 164, ten sweep directions are determined for ten candidate poses. Sweep 170 shows the direction of the main sweep from the follower pose 166 to the leader pose 164 and is used as the first candidate sweep direction. Sweep 172 is the surface normal of the obstacle 162 at the intersection of sweep 170 and the obstacle 162. Sweep 172 is used as the second candidate sweep direction.

[0055] Sweeps 174, 176, 178, and 180 form a plane with sweep 172 as its normal. These five sweep directions can also be used to define a Cartesian coordinate system for the remaining analysis. Sweeps 174 and 178 are perpendicular to the plane formed by sweeps 172, 176, and 180, with their corresponding vectors in opposite directions. Similarly, sweeps 180 and 176 are perpendicular to the plane formed by sweeps 172, 174, and 178, with their corresponding vectors in opposite directions.

[0056] Sweep 182 is calculated by averaging the sweep directions of sweeps 172, 174, and 180. This effectively positions sweep 182 at a 45-degree angle to the Cartesian sweep direction. This allows candidate poses to sample an approximate hemisphere defined by obstacle 162 and sweep 172, where the hemisphere can be modified using the surface shape of obstacle 162. Sweeps 184, 186, and 188 are similarly calculated using other corresponding Cartesian sweeps. Sweep 184 is calculated based on the average of sweeps 172, 174, and 176. Sweep 186 is calculated based on the average of sweeps 172, 176, and 178. Sweep 188 is calculated based on the average of sweeps 172, 178, and 180.

[0057] Figure 2 These are illustrations of four diagrams (collectively referred to as Final Attitude Analysis 200) visually representing the method used to determine the final attitude. The four diagrams illustrate a four-step simulation over time. Figure 201 shows the original follower attitude 230, Figure 202 shows the first intermediate simulation step with follower attitude 232, Figure 203 shows the second intermediate simulation step with follower attitude 234, and Figure 204 shows the final simulation step with follower attitude 236. Figures 201, 202, 203, and 204 represent the same simulation environment with the leader attitude 210 positioned within the surface boundary of obstacle 212 and the refined target attitude 220 calculated through the process described herein.

[0058] Figure 201 shows the original follower pose 230 and the refined target pose 220 selected from the previously determined candidate poses. Figure 202 shows the follower pose 232 being moved toward the refined target pose 220 using, for example, blocking and stroking algorithms. The follower pose 232 does not infringe on the surface of the obstacle 212. Furthermore, the follower pose 232 has been rotated by rotational forces applied by, for example, rigid body interactions.

[0059] Figure 203 shows the follower pose 234 rotating back to the refined target pose 220. Since the obstacle 212 is no longer between the follower pose 234 and the refined target pose 220, the angular driving force is now greater than the rotational force applied in Figure 202. Figure 204 shows the follower pose 236 rejoining the refined target pose 220. Physics engine operations (such as pose driving) can be used for final pose analysis 200.

[0060] For the final pose analysis 200, follower poses and target poses can be treated as individual objects or as a system for creating constrained bodies or components of complex shapes. For example, a follower pose can represent a hand composed of multiple parts, each with its own target pose. The number of target poses required depends on the number of parts representing the hand. The thumb can be represented by two parts, each finger by three parts, and the palm is a single part of the hand. Therefore, 15 follower poses and target poses will be used to determine the final pose and realistically define how the fingers of the hand will react on the surface of obstacle 212.

[0061] Figure 3 These are two diagrams (collectively referred to as Ghost Final Posture Analysis 300) visually representing the method of analyzing the final pose ghosting. Figure 310 shows the follower pose 312, the leader pose 314, and the obstacle 316 located between them. The obstacle 316 has a ghosting thickness represented by the dashed line 318.

[0062] Figure 320 illustrates follower pose 312, leader pose 314, and obstacle 315 located between them. Obstacle 315 has a ghosting thickness represented by dashed line 319. Using a ghosting threshold compared to the ghosting thickness represented by dashed lines 318 and 319, a determination can be made in Figure 310 or 320 to either fast-motion follower pose 312 to leader pose 314 or to move follower pose 312 using both blocking and stroking algorithms. In Figures 310 and 320, the distance between follower pose 312 and leader pose 314 is the same and is generally not used for ghosting analysis. For example, the value of dashed line 318 can be less than the ghosting threshold, and fast-motion algorithms can be used in Figure 310. In contrast, the value of dashed line 319 can be greater than the ghosting threshold, and fast-motion algorithms are not used. In aspects where ghosting analysis indicates that fast-motion algorithms can be used (i.e., the ghosting distance is less than the ghosting threshold), other constraints (such as external constraints) can be used to prevent the use of fast-motion algorithms.

[0063] Figure 4 These are illustrations of two figures collectively referred to as the refined target pose analysis 400, visually representing the application of a rotational force to the target pose, such as a hand rotating from fingertips to palm against a table. Figure 410 illustrates an approximate target pose 420 along the sweeping direction 424 for a leader pose 426 shown within obstacle 430. Figure 440 illustrates the refined target pose 450 resulting from the application of a rotational force to the approximate target pose 420. In this example, the rotational force is greater than the angular driving force to keep the refined target pose 450 in the same orientation as the leader pose 426. Other constraints may be considered when determining the refined target pose 450, such as surface friction (measured by the coefficient of friction), contact recovery (measured by the coefficient of recovery), external constraints (such as the articulated connection of the follower pose), rigid body constraints, and other environmental constraints (such as simulated environments such as external space, water, and other environments).

[0064] For the refined target pose analysis 400, the target pose can be treated as a single object or as a system of constrained bodies as discussed above with respect to the final pose analysis 200. Thus, the approximate target pose 420 and the refined target pose 450 can be represented by multiple systems constrained together to realistically define how the fingers of the hand will rest on the surface of the obstacle 430.

[0065] Figure 5This is an illustration of a flowchart illustrating an example of a method 500 for determining the type of a movement algorithm used to move an object in a simulated environment. The object could be, for example, a user's hand moving in their physical space. Method 500 determines the type of one or more movement algorithms used for movement from follower pose to leader pose in order to replicate actual hand movement in the simulated environment. The type of movement algorithm can be direct, blocking, stroking, fast-moving, transitional, or a combination of one or more of these movement algorithms. Method 500 can be performed by a computational system capable of analyzing object movement within a simulated environment while employing physics-based constraints, such as... Figure 7 The computing system shown. Method 500 begins at step 505.

[0066] In step 510, it is determined whether the movement from the follower's pose to the leader's pose is obstructed by an obstacle. If the result of the determination step is no, i.e., the movement is not obstructed, method 500 proceeds to step 520, where a direct movement algorithm is used. When switching from another type of movement algorithm to a direct movement algorithm, parameters can be used to determine whether an additional movement algorithm should be used for the transition. For example, when switching from an obstruction / touch movement algorithm to a direct movement algorithm, a time delay, a distance parameter, or both can be used to determine the transition algorithm. The time delay parameter can specify a time value, the number of simulation steps, the number of frames, and other values ​​that can be used to delay during the movement algorithm switch to provide a smoother visual movement transition. The distance parameter can be used to indicate how far an object can travel before switching from an obstruction / touch movement algorithm to a direct movement algorithm.

[0067] If the result of decision step 510 is yes (i.e., an obstacle exists), method 500 proceeds to decision step 530. In decision step 530, it is determined whether a swiping movement algorithm is needed. This determination can be based on the size of the obstacle and the size of the moving object. A comparison between the swiping pattern ratio and the swiping pattern parameter can be used for size analysis. The swiping pattern ratio is the ratio of the size of the obstacle to the size of the moving object (or the size of the boundary sphere of the corresponding object and obstacle), i.e., an analysis of the size of the moving object compared to the size of the obstacle. The swiping pattern ratio can indicate whether a blocking algorithm or a combination of a blocking algorithm and a swiping algorithm can be used to determine the movement of the follower's posture toward the leader's posture.

[0068] When it is determined that a stroking motion algorithm is needed, method 500 proceeds to step 540, where both the blocking and stroking algorithms are used. The result of step 530 can be "yes" if the sweeping mode ratio is "greater than" the sweeping mode parameter. Therefore, the obstacle has sufficient size compared to the moving object to ensure the use of both the blocking and stroking algorithms, such as a hand touching a tabletop.

[0069] When it is determined that the stroking movement algorithm is not needed, method 500 proceeds to step 550, where a blocking algorithm is used. The result of determination step 530 can be "No" if the swiping mode ratio is 'less than or equal to' the swiping mode parameter. Therefore, the obstacle does not have sufficient size compared to the moving object, and the blocking algorithm itself is utilized. For example, a hand touching a control knob. At the end of steps 520, 540, and 550, method 500 proceeds to step 560 and ends.

[0070] When creating a simulated environment, standard tracking algorithms (such as direct movement algorithms) can be used to move the follower pose to the leader pose when there are no obstacles between the two poses. Standard direct movement algorithms cannot be used in situations where there are obstacles or part of an obstacle between the follower and leader poses. Figure 6 This is a flowchart illustrating an example of a method 600 for determining a target pose when there are obstacles between the follower and leader poses. Method 600 calculates multiple candidate poses, selects a target pose, and refines the target pose using environmental constraints. Method 600 can be executed by a computational system capable of analyzing object movement within a simulated environment while employing physics-based constraints, such as... Figure 7 The computing system shown. Method 600 begins at step 605.

[0071] In step 610, one or more sweeping directions are calculated. The surface normals of obstacles can be used as a guide to calculate the sweeping directions from the follower's pose to the leader's pose. Figure 1A and Figure 1B Examples of calculated sweep directions for 2D and 3D simulation environments are provided.

[0072] In step 620, a sweeping direction is used to determine candidate poses. The sweeping direction can be used to place candidate poses close to the surface of an obstacle without violating the surface's physically based constraints. In some aspects, candidate poses can utilize blending and averaging algorithms. For example, blending and averaging algorithms can be used in scenes where several candidate poses are placed close to or overlapping each other due to the irregular shape of the obstacle's surface. Blending and averaging algorithms can produce candidate poses that improve the visual experience. Furthermore, candidate poses can be blended with similarly positioned candidate poses from past simulation steps to provide temporal consistency between simulation steps. Temporal consistency can improve the smoothness of the visual experience when objects move.

[0073] In step 630, an approximate target pose is selected from the candidate poses. The approximate target pose selection can utilize information from previous simulation steps, as well as other factors, to determine the best-fit candidate pose. Temporal coherence (e.g., a reduction in jitter or jumping of objects in a VR scene) is one factor that can be used when selecting an approximate target pose to maintain the user's VR visual experience. Temporal coherence can be enhanced by blending the sweep directions of candidate poses from previous simulation steps with the sweep directions of candidate poses calculated during the current simulation step. One or more additional factors can also be used in the analysis to select the approximate target pose. For example, one or more sweep directions can be blended to calculate candidate poses from which an approximate target pose can be selected. Additional blending can provide a smoother movement perceived by the user. Further blending can also be used with sweep directions. Two or more sweep directions can be blended together to determine the final sweep direction. This can be useful when the leader pose is moving rapidly, and small changes in direction between the two poses can be blended and smoothed during simulation steps as the follower pose continues to catch up.

[0074] In step 640, the approximate target pose is refined into a refined target pose. The approximate target pose can be refined by applying rotational forces, angular driving forces, and other constraints. The refined target pose can produce an adjusted position and an adjusted orientation. The refined target pose can be oriented to mimic the leader pose unless another force or constraint prevents this orientation.

[0075] Refinement can include adding lateral movement to obtain a refined target pose. Lateral movement can provide spring- or damper-type adjustment logic, which can allow some flexibility in the movement, such as around small edges on obstacles. Lateral movement can also help provide a smoother visual experience across multiple simulation steps by providing temporal coherence. Lateral movement can include overshoot movement, causing the target pose to overshoot (e.g., move past) the candidate pose due to obstacles and environmental constraints.

[0076] The coefficients of friction and contact recovery for moving objects and obstacles can be adjusted. For example, when a moving object touches the surface of an obstacle, the corresponding coefficients can be reduced to zero to eliminate additional complexity. Alternatively, the coefficients can be partially reduced to decrease the impact of those constraints on the overall process.

[0077] The approximate target posture can also be rotated and oriented by rotational forces as it pushes against the surface of an obstacle, such as a hand rotating from fingertips to palm against a door. The approximate target posture can also have applied angular driving forces that tend to force the approximate target posture to better approximate the orientation of the leader posture, such as maintaining a hand orientation with fingertips against an obstacle. The rotational and angular driving forces are balanced by the VR simulator. As each force dominates, the approximate target posture can change its orientation. External constraints are another example of refinement that can be applied to the approximate target posture. External constraints can be hinged connections to the object, hydrodynamics, and other simulated environmental factors.

[0078] An approximate target pose can be represented as a single rigid graph or a multipart representation of an articulated rigid body. In a multipart representation, the approximate target pose can act as a flexible object and respond as a compliant hand, where the finger joints move when the hand presses down on the surface of an obstacle.

[0079] In step 650, the follower pose is moved to a refined target pose. The amount of movement by which the follower pose is moved to the refined target pose may depend on the user's movement speed and the size of the simulation step (e.g., measured in FPS). Other simulated environmental constraints (such as if the follower pose is moving in the air or underwater) may also affect the amount of movement. The follower pose may be partially or fully moved to the refined target pose. The follower pose and target pose may be represented as rigid or flexible objects of a system with constrained bodies. A new follower pose may be rendered based on the refined target pose. For example, the refined target pose or a state between the previous follower pose and the refined target pose may be rendered as the new follower pose. Method 600 continues to step 660 and ends.

[0080] Figure 7 This is an illustration showing a block diagram of an example VR surface tracking system 700. The VR surface tracking system 700 can be used as part of a VR simulator to track the movement of an object in a simulated environment and adjust the object's movement when it encounters obstacles within the simulated environment. The VR surface tracking system 700 includes a VR controller 710, a VR display 715, and a VR simulator 720. The VR controller 710, VR display 715, and VR simulator 720 can be separate devices communicatively coupled via wired or wireless connections. For example, the VR simulator 720, or at least a portion thereof, can be implemented on a cloud-based platform and used to provide game streaming. The VR simulator 720 can also be combined with the VR display 715 and VR controller 710 to form a single integrated VR device capable of performing the described processes and functions.

[0081] VR controller 710 is capable of detecting user movement and transmitting that movement to VR simulator 720. VR controller 710 can be part of VR display 715, such as a VR headset capable of detecting eye and head movements. VR controller 710 can also be one or more separate controllers, such as handheld sensors, one or more cameras, motion detectors, and other tactile and body sensor devices. Therefore, VR controller 710 represents one or more VR controllers.

[0082] VR display 715 can receive and display simulated scenes from VR simulator 720. VR display 715 can be one or more of various displays, monitors, and other devices capable of displaying 2D or 3D scenes. VR display 715 can be associated with data storage devices such as hard drives, cloud storage, servers, USB storage, and other storage media.

[0083] The VR simulator 720 can be implemented as hardware, software, or a combination thereof. The VR simulator 720 can be a software application that executes on a GPU, CPU, or other processor or is embedded in the circuitry of a computing system. The VR simulator 720 can utilize zero, one, or more such processors. The VR simulator 720 can also be packaged as a software application, for example, in an application library, dynamic link library (DLL), application function, application module, graphics library or driver, ROM module, and code segment.

[0084] The VR simulator 720 can be located in a server, data center, cloud environment, or other computing system (e.g., laptop computer, tablet computer, and smartphone). The VR simulator 720 can be located near, at a certain distance from, the VR controller 710 and the VR display 715, and is part of, or a combination thereof, the VR controller 710 or the VR display 715. Communication between the VR simulator 720, the VR controller 710, and the VR display 715 can be through various conventional transmission standards and can be wired, wireless, or other conventional transmission means.

[0085] The VR simulator 720 includes a pose analyzer 730, a physics engine 740, and a renderer 750. The VR simulator can receive input parameters from the VR controller 710, such as parameters related to user movement, and input parameters from the application, such as occlusion threshold, FPS, lateral overshoot parameters, ghosting threshold, and other parameters. The application can be a video game application. In some aspects, the application executes on the VR controller 710 and receives application parameters from the VR controller 710. In other aspects, the application can execute on the VR simulator and receive application parameters from the VR controller. In still other aspects, the application can execute on another computing system and receive application parameters from that other computing system. For example, the VR simulator 720 can receive simulated environmental parameters, rigid body physics values, scene geometry, ghosting threshold, lateral movement values, target FPS, time delay, distance parameters, and other input values ​​and parameters from an appropriate application source.

[0086] The pose analyzer 730 can determine the follower and leader poses for the current simulation step using scene geometry, the simulated environment, and user movement. Analyzing the VR geometry can detect whether obstacles are obstructing the follower pose, allowing movement to the leader pose using a direct tracking algorithm. The pose analyzer 730 can then use one or more movement algorithms (such as blocking algorithms, stroking algorithms, fast-motion algorithms, transition algorithms, and direct tracking algorithms) to compute candidate poses for the requested movement. The pose analyzer 730 can select a candidate pose as an approximate target pose and refine the target pose by applying zero, one, or more constraints and forces to the approximate target pose. The result can be provided to the renderer 750 for refining the follower pose. The pose analyzer 730 can also use methods disclosed herein (such as...) Figure 5 and Figure 6 The pose analyzer 730 (represented in the figure) provides a refined target pose. The pose analyzer 730 can work with the physics engine 740 to calculate the refined target pose. The physics engine 740 can be a conventional physics engine used in VR simulators.

[0087] The renderer 750 receives refined target pose information and other supporting information from the pose analyzer 730 and renders the scene for the current simulation step. The renderer 750 can determine how much movement the follower's pose has undergone in the current simulation step. Using the speed of movement, simulated environmental factors, and other movement factors, the follower's pose can be rendered partially toward the leader's pose or entirely in the leader's pose. The renderer 750 can then output the scene to the VR display 715 for viewing.

[0088] Renderer 750 can be a cloud-based renderer that provides game streaming to VR display 715. One or more components of VR simulator 720 can also be cloud-based. For example, pose analyzer 730, physics engine 740, and renderer 750 can be implemented on one or more cloud-based servers.

[0089] A portion of the aforementioned apparatus, system, or method may be embodied in or executed by various digital data processors or computers, wherein these computers are programmed or store executable programs of sequences of software instructions to perform one or more steps of the methods. The software instructions of such a program may represent algorithms and are encoded in a machine-executable form on a non-transitory digital data storage medium (e.g., magnetic disk or optical disk, random access memory (RAM), magnetic hard disk, flash memory, and / or read-only memory (ROM)) to enable various types of digital data processors or computers to perform one, more, or all steps of one or more of the methods or functions, systems, or apparatuses described herein.

[0090] A digital data processor or computer may include one or more GPUs, one or more CPUs, one or more other processor types, or a combination thereof. Digital data processors and computers may be located close to each other, close to the user, in a cloud environment, a data center, or in a combination thereof. For example, some components may be located near the user, and some components may be located in a cloud environment or a data center.

[0091] A GPU can be implemented on a single semiconductor substrate and included in a system with one or more other devices, such as additional GPUs, memory, and a CPU. A GPU can be included on a graphics card that includes one or more memory devices and is configured to interface with the computer's motherboard. A GPU can be an integrated GPU (iGPU) located in the same location as a CPU on a single chip. The processor or computer can be part of a GPU rack located in a data center. A GPU rack can be a high-density (HD) GPU rack that includes high-performance GPU compute nodes and storage nodes.

[0092] Various portions of the disclosed embodiments may relate to computer storage products having a non-transitory computer-readable medium having program code thereon for performing various computer-implemented operations that embody part of an apparatus, device, or steps for performing the methods set forth herein. As used herein, "non-transitory" refers to all computer-readable media other than transient propagation signals. Examples of non-transitory computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROM discs; magneto-optical media such as floppy disks; and hardware devices specifically configured to store and execute program code, such as ROM and RAM devices. Examples of program code include both machine code generated by a compiler and files containing higher-level code that can be executed by a computer using an interpreter.

[0093] In interpreting this disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to an element, component, or step in a non-exclusive manner, indicating that the mentioned element, component, or step may be present, utilized, or combined with other elements, components, or steps not expressly mentioned.

[0094] Those skilled in the art to whom this application relates will understand that other and further additions, deletions, substitutions, and modifications can be made to the described embodiments. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting, as the scope of this disclosure will be limited only by the claims. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. While any methods and materials similar to or equivalent to those described herein may be used to practice or test this disclosure, a limited number of exemplary methods and materials are described herein.

[0095] It should be noted that, as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural indicators unless the context clearly indicates otherwise.

Claims

1. A method for determining follower pose in a virtual reality (VR) simulator during a simulation step, comprising: Calculate one or more current candidate poses using input parameters, wherein each of the current candidate poses is a time projection of the follower pose along a corresponding sweeping direction toward the leader pose, and wherein an obstacle is located between the follower pose and the leader pose in the simulated environment. Select the target pose from one or more current candidate poses; The target pose is refined using physics-based constraints and the input parameters, wherein the physics-based constraints utilize the surface of the obstacle; as well as The new follower pose is rendered based on the refined target pose.

2. The method of claim 1, wherein the corresponding sweeping direction is determined using the intersecting normal corresponding to the follower's posture and a direction perpendicular to the intersecting normal.

3. The method of claim 1, wherein the leader posture is located within the boundary of the surface of the obstacle.

4. The method of claim 1, wherein the VR simulator is executed at least in part on one or more graphics processing units (GPUs).

5. The method of claim 1, wherein the input parameters are received from the VR controller and the software application.

6. The method of claim 1, wherein the input parameters include one or more of the following: the follower's posture, the leader's posture, the obstacle, the coefficient of friction of the surface, the contact recovery coefficient of the surface, external constraints, values ​​from rigid body algorithms, or direct tracking values.

7. The method according to claim 1, wherein when the sweeping mode ratio is equal to or less than the sweeping mode parameter, the calculation utilizes a blocking algorithm, and when the sweeping mode ratio is greater than the sweeping mode parameter, the calculation utilizes both the blocking algorithm and the stroking algorithm, wherein, The sweeping pattern ratio is the ratio of the radius of the boundary sphere of the obstacle to the radius of the boundary sphere of the follower geometry.

8. The method of claim 1, wherein the selection further comprises: The previous candidate poses calculated in the previous simulation step are mixed with the one or more current candidate poses.

9. The method of claim 1, wherein the selection further comprises: The corresponding sweeping direction from each of one or more of the candidate poses is mixed, and the candidate pose using the mixed corresponding sweeping direction is one of one or more candidate poses.

10. The method of claim 1, wherein the refinement further comprises: When the target posture approaches the surface of the obstacle, the target posture is rotated under the applied rotational force.

11. The method of claim 10, wherein the refinement further comprises: The applied rotational force is reduced by the angular driving force of the target posture.

12. The method of claim 1, wherein the refinement utilizes one or more of the following: external constraints, lateral movement, the coefficient of friction of the surface of the obstacle, or the contact recovery coefficient of the surface of the obstacle.

13. The method of claim 1, wherein the rendering further comprises: When the obstacle is no longer between the follower's pose and the leader's pose, switch to the direct tracking algorithm.

14. The method of claim 1, wherein the target pose and the refined target pose represent a flexible object of a plurality of constrained bodies.

15. The method of claim 14, wherein each of the plurality of constrained bodies has a unique target pose and a unique refined target pose.

16. A computer program product having a series of operational instructions stored on a non-transitory computer-readable medium, the series of operational instructions, when executed, directing a data processing device to perform operations to determine a follower's posture in a virtual reality (VR) simulator during a simulation step, the operations including: Calculate one or more candidate poses, each of which is a time projection of a follower pose along a corresponding sweeping direction toward a leader pose, and wherein an obstacle is located between the follower pose and the leader pose in the simulated environment. Select the target pose from one or more candidate poses; The target pose is refined using physics-based constraints; as well as Render new follower poses based on refined target poses.

17. A system for rendering virtual reality (VR) scenes, comprising: A VR controller that can be operated to send user parameters corresponding to the user's movements; as well as A VR simulator operable to receive and use the user parameters and application parameters to render a scene of a simulated environment using one or more movement algorithms based on the interaction between a leader pose and an obstacle, wherein the leader pose represents the user's movement in the simulated environment, and the obstacle is at least partially located between the follower pose and the leader pose in the simulated environment.

18. The system of claim 17, wherein the VR simulator utilizes one or more of a graphics processing unit (GPU) or a central processing unit (CPU).

19. The system of claim 17, wherein the operation of the VR simulator is encapsulated as one or more of an application library, a dynamic link library (DLL), a function, a module, a graphics library, a ROM module, and a code segment.

20. The system of claim 17, further comprising: A VR display operable to receive the scene from the VR simulator.

21. The system of claim 17, wherein the VR simulator further comprises: A pose analyzer operable to calculate candidate poses using the follower pose, the leader pose, previously calculated candidate poses, the user parameters, and the application parameters; as well as A renderer operable to generate the scene using the output of the pose analyzer.

22. The system of claim 17, wherein the VR simulator is located in a cloud environment.

23. The system according to claim 17, wherein the movement algorithm includes a blocking algorithm, a stroking algorithm, a direct algorithm, a transition algorithm, and a fast-motion algorithm.

Images