A tactile retargeting method for multi-physics agents based on agent importance

Abstract

The present invention proposes a tactile redirection method for multiple physical agents based on agent importance. The method can select corresponding physical agents and physical agent placement positions for users to pick up virtual objects and place virtual objects in a virtual reality scene with multiple physical agents and multiple virtual objects. First, two scene parameters are defined: effective agent distance and reset area radius. Then, the agent importance is calculated based on the selected virtual object and the physical agent that is valid in the current scene state, and then the physical agent with the lowest agent importance is selected. Finally, the agent importance is calculated based on the selected virtual object placement position and the candidate physical agent placement positions and for all candidate physical agent placement positions, and then the position with the highest agent importance is selected as the placement position. The results show that this method can significantly reduce the number of resets required, the time to complete the interaction, and the hand movement distance in continuous redirection operations.

Description

Technical Field

[0001] The invention relates to a tactile redirection method of a multi-physical agent based on agent importance, and belongs to the fields of virtual reality, tactile feedback and human perception. Background Art

[0002] Virtual reality (VR) technology generates multisensory stimulation through computer simulation and acts on users. At the same time, virtual reality receives feedback from users to enable users to interact with the virtual world. With the advancement of rendering technology, the visual experience presented to users through head-mounted displays (HMDs) has become more and more realistic, and this improvement has enhanced the sense of immersion. However, when users try to touch virtual objects, if there is no corresponding physical object to provide tactile feedback, users may feel that they have passed through the virtual objects, which will interfere with the sense of immersion. Therefore, tactile feedback is crucial to enhance the user's sense of immersion because it supplements visual information. However, the virtual world is dynamic and complex, and it is often difficult to replicate the same number and distribution of physical objects in the virtual world. Therefore, researchers have proposed a technology called tactile redirection to solve the matching problem between virtual and physical objects. The tactile retargeting method takes advantage of the advantages of vision in human perception and provides tactile feedback for virtual objects by redirecting the user's real hand to the corresponding real object. This is achieved by establishing a mapping between virtual space and physical space. However, in order to prevent the offset between the virtual hand and the real hand from accumulating as the number of interactions increases, causing the user to clearly perceive the offset of the virtual hand relative to the real hand, the user needs to perform a reset operation after each interaction to eliminate the offset between the virtual hand and the real hand.

[0003] Previous tactile relocalization methods can be divided into two categories. The first category of methods focuses on static virtual scenes. Some methods are suitable for scenes with only one physical agent, and when the user touches a virtual object in the virtual scene, they redirect the user's hand to this physical agent. Other methods are suitable for scenes with multiple physical agents, and select an agent as the virtual object that the user wants to touch when considering the distribution or shape of the physical agents and virtual objects. The second category of methods focuses on dynamic virtual scenes. It allows virtual objects to move to other locations in the virtual world, so it is necessary to select physical agents to represent virtual objects and determine the physical location where the physical agents are placed. However, only simple rules are used to guide the selection and placement of agents, such as using the nearest physical agent to represent the virtual object and placing the agent at the location where the sum of the distances to all virtual objects is the smallest. Since the above methods do not make full use of the distribution relationship between physical agents and virtual objects in the scene, there are some irrationalities in the selection of physical agents and placement positions, resulting in a large number of resets during user interaction, which makes users feel tired. Summary of the invention

[0004] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and provide a tactile redirection method for multiple physical agents based on agent importance, a novel virtual reality interactive method that provides tactile feedback in real time, and can use the position information of objects and operators obtained by tracking the scene, as well as the distribution of virtual objects and physical agents in the current scene, to determine the most suitable physical agent or agent placement position for the operator, and provide a more efficient and better experience virtual reality system with tactile feedback. In order to solve the above problems, a tactile redirection method for multiple physical agents based on agent importance is proposed. The method can support multi-person collaboration or single-person operation of the tactile redirection system, support the existence of multiple physical agents in the scene, and allow the position of virtual objects to change. The method of the present invention improves the user's comfort and sense of reality by providing a more efficient virtual reality tactile redirection experience with tactile feedback. The technical solution adopted by the present invention is as follows: Figure 3 As shown, the steps include:

[0005] Step 1: Get the position of the virtual object and virtual hand in the virtual scene, and use the locator to get the position of the physical agent and real hand in the real world, align the virtual scene with the physical world, so that the virtual hand and the real hand coincide. Then update the position coordinates of the virtual object, virtual hand, physical agent and real hand in real time;

[0006] Step 2: Based on the positions of the virtual objects and physical agents in the current virtual scene obtained in step 1, the scene parameters are calculated: the effective agent distance and the reset area radius. First, the effective agent distance is calculated according to its distribution, and then the reset area radius is calculated according to the effective agent distance and the current redirection gain threshold.

[0007] Step 3: The user selects the virtual object to be picked up. Based on the selected virtual object, the effective proxy distance in step 1 is used to determine the candidate physical proxy, and the proxy importance is calculated for each candidate physical proxy. The candidate physical proxy with the lowest proxy importance is selected as the physical proxy of the current virtual object. Then, the user's virtual hand and real hand are guided to the virtual object and the physical proxy respectively using the tactile redirection method.

[0008] Step 4: After the user's virtual hand picks up the virtual object (i.e., the real hand picks up the physical agent) in step 3, the user selects the location where the virtual object will be placed, and based on the selected virtual object placement location, the effective agent distance in step 1 is used to determine the candidate placement area of ​​the physical agent, and then the candidate area is discretized into a certain number of candidate placement locations using a grid of a certain size, and the agent importance is calculated for each candidate location, and the location with the highest agent importance is selected as the placement location of the physical agent, and then the user's virtual hand and real hand are guided to the virtual object placement location and the physical agent placement location respectively using the same tactile redirection method;

[0009] In step 2: the scene parameter calculation algorithm is specifically implemented as follows:

[0010] Step (1) First calculate the effective proxy distance. The definition of the effective proxy distance is: the shortest distance such that the number of virtual objects and physical agents whose distance is less than the effective proxy distance is greater than a certain proportion of the total number of virtual objects. We use the binary search method to find the effective proxy distance, and continuously narrow the search range until the size of the search range is less than the preset lower limit;

[0011] Step (2) calculates the reset area radius. The definition of the reset area radius is to satisfy the minimum radius that satisfies the condition that if the gain of a certain redirection operation exceeds the threshold, then the probability of the operation occurring in the reset area is greater than a preset value. The reset area radius is also calculated using a binary search method. When verifying whether a certain radius meets the condition, the effective agent distance calculated in step (1) is taken as the distance between the virtual object and the physical agent. At the same time, a circle is drawn with the center of the virtual object and the physical agent as the center and the radius to be verified as the radius. Points are evenly selected on the circle. Each sampling point is taken as a possible redirection end point of the virtual object and the physical agent. It is determined whether the redirection operation exceeds the threshold when each sampling point is used as the end point. If the number of sampling points exceeding the threshold exceeds a certain proportion of the total number of sampling points, it is considered that the radius value meets the definition.

[0012] Step (3) Finally, each time the user completes the pick-up-and-place operation, the state of each object in the scene needs to be updated, and the value of the scene parameter needs to be updated according to the latest acquired state.

[0013] In step 3 and step 4: the proxy importance calculation algorithm is specifically implemented as follows:

[0014] First, the proxy importance is the weighted sum of three proxy factors. First is the proxy closeness, which is a factor that describes the number of virtual objects within the effective proxy distance of a physical proxy and the sum of the distances from the physical proxy; second is the reset probability, which describes the probability of reset occurring in the next redirection operation within the reset area centered at the midpoint between the current physical hand and the virtual hand, starting from the current physical hand and virtual hand; finally, the proxy penalty term is the gain value of the current redirection operation. The greater the gain value, the greater the penalty term. Also, if the operation gain exceeds the threshold, an additional penalty will be imposed. Additionally, the calculation of proxy importance can be extended to scenarios involving multiple users. In this scenario, virtual objects may have different ownerships, and only the user with the ownership of the current virtual object can move the virtual object. For this situation, we further propose a calculation method for weighted proxy importance. The basic proxy importance calculation method is as described above. The difference is that for each factor that constitutes the proxy importance, the ownership of the virtual object involved in the current factor calculation and its position relative to the virtual hand in the current operation are multiplied by a weight factor. Specifically, for virtual objects that do not belong to the user in the current operation, if they are located outside the reset area centered at the virtual hand, they are multiplied by the weight w1 (0 < w1 < 1), and those within the reset area are multiplied by the weight w2 (w1 < w2 < 1). In addition, we also count the number of physical proxies within the effective distance of the virtual objects respectively belonging to all users. The virtual objects belonging to the user with the largest number of such physical proxies will be additionally multiplied by the weight w3 (0 < w3 < 1) in the calculation.

[0015] The advantages of the present invention compared with the prior art are as follows:

[0016] (1) The present invention proposes the definitions of two scenario parameters and realizes the calculation of the scenario parameters. By introducing the effective proxy distance and the radius of the reset area, we effectively describe whether there is a proxy relationship between the virtual object and the physical proxy and the probability of reset occurring in the next operation. These two parameters can be used as the basis for decision-making to optimize the number of resets, thereby improving the user interaction efficiency.

[0017] (2) The present invention proposes a calculation algorithm for proxy importance. By defining three constituent factors: proxy closeness, reset probability, and penalty term, it describes the proxy importance of a physical proxy for a certain virtual object to the entire scenario. Based on this importance value, operations beneficial to reducing the number of user resets can be made, thereby improving the user interaction efficiency.

[0018] (3) The present invention extends the redirection interaction to scenarios involving multiple participants, and introduces two weight terms according to the characteristics of multi-user interaction, further expanding the calculation method of proxy importance. When multiple users operate, the present invention can significantly reduce the number of resets for each user, and at the same time make the number of resets of different users relatively balanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flow chart for realizing the method of the present invention;

[0020] Figure 2 It is a rendering of the virtual world and the real world in the operation of the present invention;

[0021] Figure 3 Calculate schematics for scene parameters;

[0022] Figure 4 This is a schematic diagram of the components of the agent importance in the acquisition stage;

[0023] Figure 5 Schematic diagram of the factors that make up the agent’s importance during the placement phase;

[0024] Figure 6 Schematic diagram of the factors that determine agent importance in a multi-person situation. DETAILED DESCRIPTION

[0025] The present invention is further described below in conjunction with other drawings and specific embodiments.

[0026] like Figure 1 As shown, the process of the present invention is divided into four steps. The first step is to obtain the coordinates of the hand and the object in the virtual world and the real world, align the virtual world with the real world, and continuously update the coordinates of the objects in the virtual world and the real world through the tracker; the second step is to calculate the scene parameter value according to the current scene distribution of the virtual world and the real world; the third step is to select the virtual object to be picked up, determine the candidate physical agent and calculate the agent importance, and finally select the agent and complete the tactile redirection; the fourth step is to select the location where the virtual object will be placed, determine the candidate physical agent placement location and calculate the agent importance, and finally select the placement location and complete the tactile redirection.

[0027] Figure 2A scenario is presented in which two users wearing HMDs play chess in virtual reality. The two users operate alternately, and each user needs to first select the virtual chess piece to be picked up, and then follow the virtual hand seen in the HMD to pick up the corresponding virtual chess piece. The algorithm will redirect the user's real hand to the selected physical chess piece. The placement process is similar. The main innovations of the present invention are: (1) A tactile orientation pipeline for a dynamic virtual reality scene with multiple physical agents is proposed and implemented, which can satisfy one or more users to complete interactive operations requiring tactile redirection. (2) Two scene parameters describing the distribution status of objects in the virtual world and the real world are proposed and designed. (3) A method for describing the importance of a physical agent to the currently selected virtual object or the placement position of a physical agent to the placement position of the currently selected virtual object is proposed and implemented, which further improves the rationality of the distribution position of physical agents and virtual objects in the entire scene and reduces the occurrence of reset operations.

[0028] Step 1: Correspondence Figure 1 In the coordinate acquisition and alignment part, the positions of the virtual objects and virtual hands in the virtual scene are acquired, and the positions of the physical proxy and real hands in the real world are acquired using the locator, and the virtual scene is aligned with the physical world so that the virtual hand and the real hand coincide with each other. Then the position coordinates of the virtual objects, virtual hands, physical proxies, and real hands are updated in real time.

[0029] Step 2: Correspondence Figure 1 In the scene parameter generation part, the scene parameters are calculated based on the positions of the virtual objects and physical agents in the current virtual scene obtained in step 1: the effective agent distance and the reset area radius. First, the effective agent distance is calculated according to its distribution, and then the reset area radius is calculated based on the effective agent distance and the current redirection gain threshold.

[0030] Specific implementation details such as Figure 3 As shown, the scene parameter calculation algorithm is specifically implemented as follows:

[0031] (1) First, calculate the effective proxy distance D. The definition of the effective proxy distance D is: the shortest distance such that the logarithm of the virtual object-physical proxy whose distance is less than the effective proxy distance is greater than a certain ratio r of the total number of virtual objects. In practice, r is 0.8. We use the binary search method to find the effective proxy distance, and continuously narrow the search range until the size of the search range is less than the preset lower limit e. In practice, e is 0.01, such as Figure 3As shown in a, there are 4 virtual objects V2, V6, V7, and V9 in the effective proxy distance of R2, and there are 4 virtual objects V1, V2, V3, and V9 in the effective proxy distance of R1. Therefore, the total number of virtual objects in the effective proxy area of ​​the physical proxy is 8 (because V2 is in the effective proxy areas of R1 and R2 at the same time, it needs to be recorded twice), and the total number of virtual objects is 9, 8>9*0.8, so this distance is valid;

[0032] (2) Next, calculate the reset area radius R. The definition of the reset area radius R is to satisfy the minimum radius that satisfies the condition that if the gain of a certain redirection operation exceeds the threshold, then the probability of the operation occurring in the reset area is greater than a preset value r'. In practice, r' is taken as 0.5. The reset area radius is calculated using the binary search method. When verifying whether a certain radius meets the condition, take the effective agent distance D calculated in (1) as the distance between the virtual object and the physical agent. At the same time, use the center of the virtual object and the physical agent as the center of the circle and the radius to be verified to draw a circle, and evenly select points on the circle. Take each sampling point as a possible redirection end point of the virtual object and the physical agent, and judge whether the redirection operation exceeds the threshold when each sampling point is used as the end point. If the number of sampling points exceeding the threshold exceeds r' times the total number of sampling points, it is considered that the radius value meets the definition, such as Figure 3 As shown in (b), let R1 and V1 be the starting positions of the real hand and the virtual hand, and each sampling point be a possible end point. The light-colored sampling point represents that the redirection operation gain corresponding to the end point does not exceed the threshold, while the dark-colored sampling point represents that the redirection operation gain corresponding to the end point exceeds the threshold. The number of light-colored points is 16, the number of dark-colored points is 12, and the total number is 28. 16>28*0.5, so the radius is valid.

[0033] (3) Finally, each time the user completes the pick-up-and-place operation, the state of each object in the scene needs to be updated, and the value of the scene parameter needs to be updated according to the latest state obtained.

[0034] Step 3: The user selects the virtual object to be picked up. Based on the selected virtual object, the effective agent distance in step 1 is used to determine the candidate physical agents, and the agent importance is calculated for each candidate physical agent. The candidate physical agent with the lowest agent importance is selected as the physical agent of the current virtual object. Then, the tactile redirection method is used to guide the user's virtual hand and real hand to the virtual object and physical agent respectively.

[0035] The agent importance calculation algorithm is specifically implemented as follows:

[0036] First, the proxy importance is the weighted sum of three proxy factors I = a*PC+b*PR+c*PN. The first is the proxy closeness PC. The proxy closeness is a factor that describes the number of virtual objects within the effective proxy distance of the physical proxy and the sum of the distances from the physical proxy, such as Figure 4 As shown, Figure 4 In a, since there are two virtual objects V2 and V3 within the effective proxy distance of R2 and only one virtual object V4 within the effective proxy distance of R1, the proxy importance of R2 is large. In 4b, when R3, which is closer to V2 and V3, appears around V2 and V3, the proxy importance of R2 decreases, which makes the proxy importance of R1 exceed R2. Secondly, the reset probability PR describes the probability of the next redirection operation being reset in the reset area centered on the midpoint of the current physical hand and the virtual hand starting from the current physical hand and the virtual hand. However, since there are infinite placement positions for virtual objects that may not be traversed, this item is set to 0 when picking up objects. Finally, the proxy penalty item PN is the gain value of the current redirection operation. The larger the gain value, the larger the penalty item. At the same time, if the operation gain exceeds the threshold, an additional penalty will be added. Since the physical agent with the lowest proxy importance is selected in step 3, the weight sparseness of the penalty item should be positive. In practice, the values ​​of a, b, and c are assigned to 2, 5, and 1 respectively.

[0037] Step 4: After the user's virtual hand picks up the virtual object (that is, the real hand picks up the physical agent) in step 3, the user selects the location where the virtual object will be placed. At the same time, based on the selected virtual object placement location, the effective agent distance in step 1 is used to determine the candidate placement area of ​​the physical agent. Then, a grid of a certain size is used to discretize the candidate area into a certain number of candidate placement locations. In practice, the grid size is selected to be 3 cm, and the agent importance is calculated for each candidate location. The location with the highest agent importance is selected as the placement location of the physical agent. Then, the user's virtual hand and real hand are guided to the virtual object placement location and the physical agent placement location respectively using the tactile redirection method. The calculation process of the agent importance is similar to that in step 3, using the formula I = a*PC+b*PR+c*PN. The process of calculating the agent density PC is consistent with that in step 3. The influence of the PC factor is as follows: Figure 5 As shown in a, a relevant introduction is given; PR describes the probability of reset in the next redirection operation within the reset area centered at the midpoint of the current physical hand and the virtual hand, as shown in Figure 5As shown in Figure b, in the case on the left, since the operation gain with the placement position L2 and the virtual object placement position V1' as the end point is less than that of L1, L2 has a higher proxy importance. However, when there are virtual object V2 and physical agent R2 in the scene, since V2 and R2 are within the reset area of the placement position, and it is not necessary to reset when starting from L1-V1' to reach R2-V2, but it is necessary to reset when starting from L2-V1', so the proxy importance of L1 becomes larger than that of L2 at this time; in addition, the calculation of proxy importance can be extended to scenarios with multiple users. In this scenario, virtual objects may have different ownerships, and only the user with the current ownership of the virtual object can move the virtual object. For this situation, we further propose a calculation method for weighted proxy importance. The basic proxy importance calculation method is as described above. The difference is that for each factor that constitutes the proxy importance, the ownership of the virtual object involved in the current factor calculation and its position relative to the virtual hand in the current operation are multiplied by a weight factor. Specifically, for virtual objects that do not belong to the user in the current operation, if they are outside the reset area centered on the virtual hand in the current operation, they are multiplied by weight w1 (0 < w1 < 1), and in practice, w1 is set to 0.5, while those within the reset area are multiplied by weight w2 (w1 < w2 < 1), and in practice, w2 is set to 0.8. In addition, we will also count the number of physical agents within the effective distance of the virtual objects respectively belonging to all users. The virtual objects belonging to the user with the largest number will be additionally multiplied by weight w3 (0 < w3 < 1) in the calculation, and in practice, w3 is set to 0.6; this weighted method is better than the strategy of not considering virtual object ownership and only considering virtual objects belonging to the current user. An example showing the effectiveness of this method is as Figure 6 shown. When the user holds V1 and R1 in one hand and is about to place the virtual object at V1', the strategy of treating all virtual objects equally without considering virtual object ownership will choose L2 because L1, L2, and L3 respectively have V2V3, V5V6, and V2V4 within the effective proxy distance. When the number of proxyable virtual objects is the same, since the sum of the distances between L2 and V5 and V6 is closer, L2 is chosen as the placement position; while the strategy that only considers the virtual objects owned by the user in the current operation will only consider making the placement position close to V2 and V3, so L3 will be chosen; and when using the weighted proxy importance method, since V1, V2, V3, V5, and V6 are all outside the reset area of user two, they have a weight less than that of V4, so L3 close to V2 and V4 will be chosen as the placement position. It can be seen that when user one completes the placement operation, if V5 or V6 is taken next, then none of these three methods will cause a reset, but if V4 is taken next, then those two comparison strategies will both require a reset, while the method using weighted proxy importance does not require a reset.

[0038] The technical contents not elaborated in detail in the present invention belong to the common knowledge of those skilled in the art.

[0039] Although the above describes the illustrative specific embodiments of the present invention to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A tactile redirection method for multi-physics agents based on agent importance, characterized in that: The steps include: Step 1: Get the positions of the virtual object and virtual hand in the virtual scene, and use the locator to get the positions of the physical agent and real hand in the real world, align the virtual scene with the physical world, so that the virtual hand and the real hand coincide with each other; then update the position coordinates of the virtual object, virtual hand, physical agent and real hand in real time; Step 2: Based on the positions of the virtual objects and physical agents in the current virtual scene obtained in step 1, the scene parameters are calculated: the effective agent distance and the reset area radius. First, the effective agent distance is calculated according to the position distribution of the virtual objects and the physical agents, and then the reset area radius is calculated according to the effective agent distance and the currently set redirection gain threshold. Step 3: The user selects the virtual object to be picked up. Based on the selected virtual object, the effective agent distance in step 2 is used to determine the candidate physical agents, and the agent importance is calculated for each candidate physical agent. The candidate physical agent with the lowest agent importance is selected as the physical agent of the current virtual object. Then, the user's virtual hand and real hand are guided to the virtual object and the physical agent respectively using the tactile redirection method. Step 4: After the user's virtual hand picks up the virtual object in step 3, that is, the real hand picks up the physical agent, the user selects the position where the virtual object will be placed, and according to the selected virtual object placement position, the effective agent distance in step 2 is used to determine the candidate placement area of ​​the physical agent, and then the candidate placement area is discretized into a certain number of candidate placement positions using a grid of a certain size, and the agent importance is calculated for each candidate placement position, and the position with the highest agent importance is selected as the placement position of the physical agent, and then the user's virtual hand and real hand are guided to the virtual object placement position and the physical agent placement position respectively using the same tactile redirection method; The definition of the reset area radius is to satisfy the minimum radius that satisfies the condition that if the gain value of a certain redirection operation does not exceed the redirection gain threshold, then the probability that the redirection operation occurs in the reset area is greater than a preset value; The definition of effective proxy distance is: the shortest distance that makes the number of virtual object-physical proxy pairs that satisfy the distance less than the effective proxy distance greater than a certain proportion of the total number of virtual objects; The specific implementation of the proxy importance calculation algorithm is as follows: First, the proxy importance is the weighted sum of three proxy factors. The first is the proxy density, which describes the number of virtual objects within the effective proxy distance of the physical proxy and the sum of the distances from the physical proxy. Secondly, the reset probability describes the probability of a reset occurring in the next redirection operation within the reset area centered on the midpoint of the current physical hand and the virtual hand, starting from the current physical hand and the virtual hand. Finally, the proxy penalty term is the gain value of the current redirection operation. The larger the gain value, the larger the penalty term. At the same time, if the gain value of the redirection operation exceeds the redirection gain threshold, an additional penalty will be added.

2. The tactile redirection method of multi-physics agents based on agent importance according to claim 1, characterized in that: In step 2: the scene parameter calculation algorithm is specifically implemented as follows: Step (1) first calculate the effective proxy distance; use the binary search method to find the effective proxy distance, and continuously narrow the search range until the size of the search range is less than the preset lower limit; Step (2) Next, calculate the radius of the reset area. Use binary search to calculate the radius of the reset area. When verifying whether a certain radius meets the conditions, take the effective proxy distance calculated in step (1) as the distance between the virtual object and the physical agent. At the same time, use the center of the virtual object and the physical agent as the center and the radius to be verified to draw a circle. Evenly select points on the circle, and use each sampling point as a possible redirection end point of the virtual object and the physical agent. When each sampling point is used as the end point, determine whether the gain value of this redirection operation exceeds the redirection gain threshold. If the number of sampling points that do not exceed the threshold exceeds a certain proportion of the total number of sampling points, it is considered that the radius meets the definition. Step (3) Each time the user completes the pick-up-and-place operation, the state of each object in the scene needs to be updated, and the value of the scene parameter needs to be updated according to the latest acquired state.

3. The tactile redirection method of multi-physics agents based on agent importance according to claim 2, characterized in that: In some scenarios, virtual objects have different ownerships. Only the user who owns the current virtual object can move the virtual object. In view of this situation, a method for calculating weighted proxy importance is further proposed. For each factor that constitutes the proxy importance, the ownership of the virtual object involved is calculated according to the current factor and the position relative to the virtual hand in the current operation is multiplied by the weight factor. Specifically, if the virtual object that does not belong to the user in the current operation is outside the current reset area centered on the virtual hand, it is multiplied by the weight w1, 0 < w1 < 1; and if it is within the reset area, it is multiplied by the weight w2, w1 < w2 < 1; the number of physical proxies within the effective proxy distance of the virtual objects of all users is counted, and the virtual object of the user with the largest number is additionally multiplied by the weight w3 in the calculation, 0 < w3 < 1. <w3 < 1。

Images