Information processor

Abstract

To provide a technology capable of realizing both reduction in oscillation of a position (designated position) designated by a display item, and high followability of the designated position relative to movement of an operation body.SOLUTION: An information processor of the present invention has: determination means for determining a designated position by a display item according to the position and posture of an operating body; first acquisition means for acquiring information on a distance from the operating body to the designated position; second acquisition means for acquiring information on a movement amount of the designated position; and processing means for performing an averaging process of the designated position with an intensity based on the distance and the movement amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, and more particularly to a technique for improving the operability of an operation for specifying a position using a display item. [Background technology]

[0002] There is a technology known that allows users to experience a computer-generated virtual world (virtual space) as if it were the real world (real space). This technology is called virtual reality (VR). The technology that augments the real world with virtual objects (Computer Graphics:CG) is also known, and this technology is called Augmented Reality (AR) technology. VR and AR technologies are used, for example, in various games and simulations.

[0003] In VR and AR technologies, a display item (virtual object) called a ray is sometimes used to specify a position in the three-dimensional space (virtual space or augmented (mixed) reality space) experienced by the user. A ray is a virtual object like a light ray that extends from the operating body to an object in three-dimensional space, and at least one of the starting point and direction of the ray changes with the change in the movement of the operating body (at least one of the position and / or orientation). When at least one of the starting point and / or direction of the ray changes, the end point of the ray (the position specified by the ray) also changes.

[0004] Patent Document 1 discloses a technique for displaying a cursor on a display, moving the cursor in accordance with the movement of an input pen away from the display, and averaging the position of the cursor with greater strength the farther the input pen is from the display. By averaging the position of the cursor with greater strength the farther the input pen is from the display, it is possible to reduce shaking of the cursor caused by shaking hands. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2004-133566 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology disclosed in Patent Document 1, when a user wants to move the cursor a large distance, the averaging process causes the cursor movement to lag behind the movement of the input pen (deteriorating the cursor's ability to track the movement of the input pen), causing the user to feel sluggish.

[0007] The present invention has an object to provide a technique that can achieve both reduction in shaking of a position specified by a display item (specified position) and high trackability of the specified position to the movement of an operating body. [Means for solving the problem]

[0008] A first aspect of the present invention is an information processing device characterized by having a determination means for determining a designated position by a display item in accordance with the position and posture of an operating body, a first acquisition means for acquiring information on the distance from the operating body to the designated position, a second acquisition means for acquiring information on the amount of movement of the designated position, and a processing means for performing an averaging process of the designated position with an intensity based on the distance and the amount of movement.

[0009] In a second aspect of the present invention, a designated position is determined by a display item according to the position and posture of an operating object. a step of determining a distance from the operating body to the designated position, a step of obtaining information on the distance from the operating body to the designated position, a step of obtaining information on the amount of movement of the designated position, and a step of performing averaging processing of the designated position with a strength based on the distance and the amount of movement.

[0010] A third aspect of the present invention is a program for causing a computer to function as each of the means of the information processing device. Effect of the Invention

[0011] According to the present invention, it is possible to reduce shaking of a position specified by a display item (specified position) while ensuring high followability of the specified position to the movement of an operating body. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a functional block diagram of a system according to a first embodiment. [Diagram 2] FIG. 1 is a hardware block diagram of an information processing device according to first and second embodiments. [Diagram 3] 5 is a schematic diagram for explaining the operation of an averaging processor according to the first embodiment. FIG. [Figure 4] 4 is a flowchart showing the operation of the information processing device according to the first embodiment. [Diagram 5] FIG. 11 is a functional block diagram of a system according to a second embodiment. [Figure 6] 10 is a schematic diagram for explaining a specific example of a fluctuation angle according to the second embodiment. FIG. [Figure 7] 10 is a flowchart showing the operation of the information processing device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Embodiment 1) A first embodiment of the present invention will be described below. In the first embodiment, a position (designated position) designated by a display item (virtual object) is determined based on the position and posture of an operating object. In the first embodiment, the operating object is a user's finger (index finger, second finger), but the operating object is not limited to a finger and may be, for example, a hand, an arm, a controller, a stylus, or a pen. In the first embodiment, the display item is a ray, but the display item is not limited to a ray and may be, for example, a pointer (cursor) placed at a designated position. By displaying a display item that designates a position, a user can select (designate) an object placed at a position that is out of reach and operate the object.

[0014] A ray is a virtual object like a light ray. The start point of the ray is set to the position of the operating body, and the end point of the ray (the position specified by the ray) is set to the position where the light ray contacts an object in three-dimensional space (the three-dimensional space experienced by the user). Therefore, the ray extends from the operating body to the object in the three-dimensional space. At least one of the start point and the direction of the ray changes depending on the change in the movement (at least one of the position and the posture) of the operating body. The end point of the ray also changes due to the change in at least one of the start point and the direction of the ray. Another display item (virtual object) may be displayed at the end point of the ray so that the user can recognize the end point of the ray. The display item indicating the end point of the ray is, for example, a pointer (cursor) having a shape of a circle or an arrow.

[0015] Fig. 1 is a block diagram showing a functional configuration of a system according to embodiment 1. The system in Fig. 1 includes an information processing device 1000, a head mounted display (HMD) 1200, and an image display device 1300. For example, the information processing device 1000 is a personal computer (PC) or a server, and the image display device 1300 is a PC monitor, a television device, a smartphone, or a tablet terminal. A part or all of the processing performed by the information processing device 1000 may be performed by the HMD 1200 or the image display device 1300.

[0016] The information processing device 1000 has a captured image acquisition unit 1010, a position and orientation measurement unit 1020, a virtual data storage unit 1030, a virtual space generation unit 1130, a ray information calculation unit 1050, a distance calculation unit 1060, a movement amount calculation unit 1070, a movement direction calculation unit 1080, an averaging processing unit 1090, and a mixed reality image generation unit 1140. These multiple functional units are connected to each other via a path 1110.

[0017] The HMD 1200 has an imaging unit 1210 and an image display unit 1220. The HMD 1200 is connected to the information processing device 1000, and data communication (transmission and reception of data) can be performed between the information processing device 1000 and the HMD 1200. In addition, an image display device 1300 is also connected to the information processing device 1000, and data communication can also be performed between the information processing device 1000 and the image display device 1300. The connection between the information processing device 1000, the HMD 1200, and the image display device 1300 is not particularly limited, and may be a wired connection or a wireless connection.

[0018] The information processing device 1000 will be described.

[0019] The captured image acquisition unit 1010 acquires an image of real space (real space image) captured by the imaging unit 1210 of the HMD 1200 , and outputs it to the position and orientation measurement unit 1020 , the ray information calculation unit 1050 , and the mixed reality image generation unit 1140 .

[0020] The position and orientation measurement unit 1020 detects feature points from the real space image acquired by the captured image acquisition unit 1010, and measures (detects, estimates) the position and orientation of the HMD 1200 based on the detection result of the feature points, and creates an environmental map around the HMD 1200. The position and orientation measurement unit 1020 outputs position and orientation information (information indicating the position and orientation) of the HMD 1200 and the environmental map to the ray information calculation unit 1050 and the mixed reality image generation unit 1140. For example, the position and orientation measurement unit 1020 acquires the position and orientation information and the environmental map using VSLAM (Visual Simultaneous Localization and Mapping). VSLAM is a known technology that simultaneously estimates the self-position of a camera and creates an environmental map in an unknown environment. Note that the method is not limited to the method using VSLAM, and the position and orientation information and the environmental map may be acquired by, for example, a method using infrared light, a method using ultrasonic waves, or a method using an inertial sensor.

[0021] The virtual data storage unit 1030 stores data relating to the virtual space (virtual space relation data), and outputs the data to the ray information calculation unit 1050 and the virtual space generation unit 1130. The virtual space relation data includes, for example, data relating to virtual objects that configure the virtual space (shape information and position / orientation information), and data relating to a light source that irradiates light into the virtual space (type information and position / orientation information).

[0022] The ray information calculation unit 1050 determines the start point, end point, and direction of the ray according to the position and posture of the index finger (second finger) of the user. In the first embodiment, the ray information calculation unit 1050 recognizes (detects) the joint points of the user's hand from the real space image acquired by the captured image acquisition unit 1010, and measures (detects, estimates) the position and posture of the hand (including the position and posture of the index finger) based on the detection result of the joint points. The ray information calculation unit 1050 maps the coordinates indicating the position and posture of the hand to a global coordinate system based on the position and posture information of the HMD 1200 acquired by the position and posture measurement unit 1020 and the environmental map, and calculates the start point and direction of the ray. Then, the ray information calculation unit 1050 calculates the end point (position specified by the ray) where the ray contacts the virtual object in the virtual space based on the virtual space relationship data output from the virtual data storage unit 1030 and the start point and direction of the ray. The ray information calculation unit 1050 outputs ray information (start point and end point of the ray) to the distance calculation unit 1060, the movement amount calculation unit 1070, the movement direction calculation unit 1080, and the averaging processing unit 1090. The method of recognizing the joint points of the hand is as follows. There is no particular limitation, and for example, a trained model trained by known machine learning may be used to recognize the joint points of the hand. The position and posture of the hand may be measured using a device such as a controller. The end point of the ray may be a position where the ray contacts a real object in a mixed (augmented) reality space.

[0023] The distance calculation unit 1060 calculates the distance from the start point to the end point of the ray (the length of the ray, the distance from the user's index finger to the end point of the ray) based on the start point (start point coordinates) and end point (end point coordinates) of the ray acquired by the ray information calculation unit 1050. Then, the distance calculation unit 1060 outputs information on the calculated distance to the averaging processing unit 1090.

[0024] The movement amount calculation unit 1070 calculates the movement amount of the end point of the ray based on the end point (end point coordinates) of the latest ray and the end point of the past ray (e.g., the previous time) acquired by the ray information calculation unit 1050. Then, the movement amount calculation unit 1070 outputs information on the calculated movement amount to the averaging processing unit 1090.

[0025] The moving direction calculation unit 1080 calculates the moving direction of the end point of the ray based on the end point (end point coordinates) of the latest ray and the end point of the past ray (e.g., the previous time) acquired by the ray information calculation unit 1050. Then, the moving direction calculation unit 1080 outputs information on the calculated moving direction to the averaging processing unit 1090.

[0026] The averaging processor 1090 acquires information on the distance from the start point to the end point of the ray from the distance calculation unit 1060, acquires information on the movement amount of the end point of the ray from the movement amount calculation unit 1070, and acquires information on the movement direction of the end point of the ray from the movement direction calculation unit 1080. Then, the averaging processor 1090 performs averaging processing of the end points of the ray with an intensity based on the acquired information. For example, the averaging processor 1090 determines whether or not to perform averaging processing based on the movement amount and movement direction of the end point of the ray, and determines the intensity of the averaging processing according to the distance from the start point to the end point of the ray.

[0027] In the first embodiment, the averaging processor 1090 judges whether or not the conditions that the movement amount of the end point of the ray is smaller than a threshold and the movement direction of the end point of the ray is not constant are satisfied. If the condition is not satisfied, the averaging processor 1090 does not perform the averaging process, and if the condition is satisfied, the averaging processor 1090 performs the averaging process with an intensity according to the distance from the start point to the end point of the ray. If the averaging processor 1090 does not perform the averaging process, it outputs the latest ray information acquired from the ray information calculation unit 1050 to the virtual space generation unit 1130, and if the averaging process is performed, it outputs the latest averaged ray information to the virtual space generation unit 1130. Note that, since the movement amount of the end point of the ray is likely to be large as the distance from the start point to the end point of the ray is longer, a larger value may be used as the threshold value for the movement amount as the distance is longer.

[0028] The averaging processor 1090 acquires ray information (starting point and ending point of the ray) from the ray information calculation unit 1050. The averaging processor 1090 converts the latest ending point (ending point coordinates) acquired from the ray information calculation unit 1050 into the center of gravity (average position) of multiple ending points acquired during the reference period up to the present, by averaging processing. As a result, averaged ray information in which the ending points are converted is obtained. In the first embodiment, the averaging processor 1090 sets a longer reference period as the distance from the starting point to the ending point of the ray is longer, and a shorter reference period as the distance is shorter. As a result, when the distance from the starting point to the ending point of the ray is long, the number of ending points used in the averaging process is increased, and the averaging process is strengthened. When the distance from the starting point to the ending point of the ray is short, the number of ending points used in the averaging process is reduced, and the averaging process is weakened.

[0029] Although the information on the movement amount of the end point of the ray and the information on the movement direction of the end point of the ray are obtained separately, it is also possible to obtain one piece of information that combines both of these pieces of information. For example, it is also possible to obtain a motion vector of the end point of the ray. Then, a frequency characteristic analysis of the motion vector is performed to obtain The motion of the ray's end point may be estimated.

[0030] A specific example of the operation of the averaging processor 1090 will be described with reference to FIGS. 3(A) to 3(C).

[0031] FIG. 3A shows a state where a user selects or operates a virtual object 302 with a ray 303. The ray 303 extends from the tip (start point 304) of the user's index finger 301 in the direction in which the index finger 301 extends, and reaches a virtual object 302. In the first embodiment, the ray 303 and a pointer 305 (cursor) indicating the end point of the ray 303 (the contact position between the virtual object 302 and the ray 303) are displayed. The distance d is the distance from the tip of the index finger 301 (start point 304 of the ray 303) to the pointer 305 (end point of the ray 303). In the case of the distance d, six end points at different times are used in the averaging process. It is not necessary that one of the ray 303 and the pointer 305 is displayed.

[0032] Fig. 3B shows a change in the end point of the ray 303. In Fig. 3B, the end point moves from position 311 to position 312, from position 312 to position 313, from position 313 to position 314, from position 314 to position 315, and from position 315 to position 316 at predetermined time intervals. Position 316 is the latest end point. Motion vector V11 indicates the amount of movement and direction of movement from position 311 to position 312, motion vector V12 indicates the amount of movement and direction of movement from position 312 to position 313, and motion vector V13 indicates the amount of movement and direction of movement from position 313 to position 314. Motion vector V14 indicates the amount of movement and direction of movement from position 314 to position 315, and motion vector V15 indicates the amount of movement and direction of movement from position 315 to position 316.

[0033] FIG. 3B shows a case where a user wants to select or operate a virtual object, and the end point is shaking due to camera shake. In FIG. 3B, the movement amount of the end point is smaller than the threshold, and the movement direction of the end point is irregular. Therefore, the latest end point is converted from position 316 to the center of gravity 317 of positions 311-316 by an averaging process.

[0034] FIG. 3C shows another change in the end point of the ray 303. In FIG. 3C, the end point moves from position 321 to position 322, from position 322 to position 323, from position 323 to position 324, from position 324 to position 325, and from position 325 to position 326 at predetermined time intervals. Position 326 is the latest end point. Motion vector V21 indicates the amount of movement and direction of movement from position 321 to position 322, motion vector V22 indicates the amount of movement and direction of movement from position 322 to position 323, and motion vector V23 indicates the amount of movement and direction of movement from position 323 to position 324. Motion vector V24 indicates the amount of movement and direction of movement from position 324 to position 325, and motion vector V25 indicates the amount of movement and direction of movement from position 325 to position 326.

[0035] Fig. 3C shows a case where the user wants to move the ray 303 in one direction by a large amount (want to move the end point in one direction by a large amount). In Fig. 3C, the amount of movement from position 325 to position 326 (the latest end point) is greater than the threshold, and positions 321 to 326 are aligned in one direction (the movement direction of the end point is constant). Therefore, averaging processing is not performed.

[0036] Returning to the explanation of Fig. 1, the virtual space generation unit 1130 acquires ray information (the latest ray information acquired from the ray information calculation unit 1050 or the latest averaged ray information) from the averaging processing unit 1090, and acquires virtual space relation data from the virtual data holding unit 1030. Then, based on this acquired information (data), the virtual space generation unit 1130 generates a virtual space including a ray and a pointer (cursor) indicating the end point of the ray. The virtual space generation unit 1130 outputs data of the generated virtual space (virtual space data) to the mixed reality image generation unit 1140.

[0037] The mixed reality image generating unit 1140 acquires virtual space data from the virtual space generating unit 1130, and acquires position and orientation information and an environmental map of the HMD 1200 from the position and orientation measuring unit 1020. Then, the mixed reality image generating unit 1140 generates an image of the virtual space (virtual space image) based on the acquired information (data). Furthermore, the mixed reality image generating unit 1140 acquires a real space image from the captured image acquiring unit 1010, and generates an image of the mixed reality space (mixed reality image) by synthesizing (superimposing) the virtual space image on the real space image. The mixed reality image generating unit 1140 outputs the generated mixed reality image to the image display unit 1220 of the HMD 1200. Note that known techniques can be used to generate a virtual space image that represents an appearance from an arbitrary position, and to generate a mixed reality space.

[0038] The HMD 1200 will be described. The imaging unit 1210 captures a real space and outputs a real space image, which is a captured image, to the captured image acquisition unit 1010 of the information processing device 1000. The image display unit 1220 has, for example, a liquid crystal display panel or an organic EL display panel, and displays a mixed reality image generated by the mixed reality image generation unit 1140 of the information processing device 1000. The image display unit 1220 may directly project an image onto the retina of the user using a laser. The mixed reality image generation unit 1140 may output the mixed reality image to the image display unit 1220 and the image display device 1300, and the mixed reality image may be displayed by the image display unit 1220 and the image display device 1300. Although one image display unit 1220 is shown in FIG. 1, the HMD 1200 has an image display unit for the right eye (right display unit) and an image display unit for the left eye (left display unit) as the image display unit 1220. The right display unit is provided to display an image to the right eye of a user wearing the HMD 1200 on the head, and the left display unit is provided to display an image to the left eye of a user wearing the HMD 1200 on the head. To enable a user to view an image in stereo, stereo display may be performed in which two images with parallax are displayed on two image display units (right display unit and left display unit), respectively.

[0039] FIG. 2 is a block diagram showing a hardware configuration of the information processing device 1000. The CPU 2010 performs overall control of each device connected via the bus 2000. The CPU 2010 reads and executes a program stored in a read-only memory (ROM) 2020. Various programs such as an operating system, a processing program of a flowchart described later, and a device driver are stored in the ROM 2020. These programs are temporarily stored in a random access memory (RAM) 2030 and are executed by the CPU 2010 as appropriate. The input I / F 2040 inputs a signal from an external device (e.g., an imaging device or an operating device) to the information processing device 1000. The input I / F 2040 may appropriately convert the signal from the external device into a signal in a format that can be processed by the information processing device 1000. The output I / F 2050 outputs a signal to an external device (e.g., a display device). The signal output to the external device is, for example, a signal generated by the information processing device 1000. The output I / F 2050 may appropriately convert a signal to be output to an external device into a signal in a format that can be processed by the external device.

[0040] 1 are realized by, for example, a CPU 2010 expanding a program (a processing program of a flowchart to be described later) stored in a ROM 2020 into a RAM 2030 and executing the program. Note that some or all of the functional units of the information processing device 1000 shown in FIG. 1 may be realized by hardware (e.g., a processing circuit).

[0041] FIG. 4 is a flowchart showing the operation of the information processing device 1000.

[0042] In step S401, the position and orientation measurement unit 1020 acquires a real space image from the imaging unit 1210 via the captured image acquisition unit 1010, and measures the position and orientation of the HMD 1200 and creates an environmental map around the HMD 1200 based on the real space image. Proceed to step S402.

[0043] In step S402, the ray information calculation unit 1050 measures the position and orientation of the index finger based on the real space image, the information acquired in step S401 (the position and orientation information of the HMD 1200 and the environment map), and the virtual space relationship data, and calculates the start point and end point of the ray. Then, the process proceeds to step S403.

[0044] In step S403, distance calculation unit 1060 calculates the distance from the start point to the end point of the ray calculated in step S402 (the distance from the index finger to the end point), and then proceeds to step S404.

[0045] In step S404, the movement amount calculation unit 1070 calculates the movement amount of the end point between the end point of the latest ray calculated in step S402 and the end point of the previous ray. Furthermore, the movement direction calculation unit 1080 calculates the movement direction of the end point between the end point of the latest ray and the end point of the previous ray. Then, the process proceeds to step S405.

[0046] In step S405, the averaging processor 1090 determines whether the movement amount calculated in step S404 is smaller than a threshold value. If the movement amount is smaller than the threshold value, the process proceeds to step S406, otherwise, the process proceeds to step S408. Note that although the process proceeds to step S408 if the movement amount is equal to the threshold value, the process may proceed to step S406 if the movement amount is equal to the threshold value.

[0047] In step S406, the averaging processor 1090 determines whether the moving direction of the ray is constant based on the moving direction calculated in step S404 (and the past moving direction). For example, the averaging processor 1090 determines whether a state in which the amount of change in the moving direction from the previous time is smaller than a threshold continues for a predetermined number of times or more. If the moving direction of the ray is constant (if a state in which the amount of change in the moving direction from the previous time is smaller than a threshold continues for a predetermined number of times or more), the process proceeds to step S408; otherwise, the process proceeds to step S407.

[0048] In step S407, the averaging processor 1090 performs averaging processing on the end points of the rays with the intensity according to the distance calculated in step S403, and then the process proceeds to step S408.

[0049] In step S408, the virtual space generation unit 1130 generates a virtual space including a ray and a pointer indicating the end point of the ray based on the latest ray information and virtual space relation data. If step S407 is not passed, the ray information acquired in step S402 (ray information indicating the start point and end point calculated in step S402) is used. If step S407 is passed, averaged ray information obtained by converting the end point of the ray information acquired in step S402 by averaging processing is used. Then, the process proceeds to step S409.

[0050] In step S409, the mixed reality image generation unit 1140 generates a virtual space image based on the virtual space generated in step S408 and the information acquired in step S401 (position and orientation information of the HMD 1200 and the environmental map). Furthermore, the mixed reality image generation unit 1140 acquires a real space image from the imaging unit 1210 via the captured image acquisition unit 1010, and generates a mixed reality image by synthesizing (superimposing) the virtual space image on the real space image. The mixed reality image generation unit 1140 outputs the generated mixed reality image to the image display unit 1220 of the HMD 1200 (and the image display unit 1220). Then, the process proceeds to step S410.

[0051] In step S410, the CPU 2010 determines whether or not a termination condition for the operation in Fig. 4 is satisfied. The termination condition is, for example, a condition that the user issues an instruction to terminate the operation in Fig. 4. If the termination condition is satisfied, the operation in Fig. 4 is terminated, and if not, step S410 is executed. Proceed to S401.

[0052] As described above, according to the first embodiment, it is possible to reduce shaking of a position specified by a display item (specified position) while ensuring high followability of the specified position to the movement of the operating body.

[0053] Although the averaging process is performed with the intensity based on the distance from the operating body (start point of the ray) to the designated position (end point of the ray), the movement amount of the designated position, and the movement direction of the designated position, the method of the averaging process (method of determining the intensity) is not limited to this. For example, the averaging process may be performed with the intensity based on the distance and the movement amount without considering the movement direction. It may be determined based on the movement amount whether or not to perform the averaging process, and the intensity of the averaging process may be determined according to the distance. Step S406 in FIG. 4 may be omitted. In other words, if the movement amount is larger than the threshold, the averaging process may not be performed, and if the movement amount is smaller than the threshold, the averaging process may be performed with the intensity according to the distance.

[0054] (Embodiment 2) A second embodiment of the present invention will be described below. Note that, in the following, a description of the same points as in the first embodiment (for example, the same configuration and processing as in the first embodiment) will be omitted, and only points different from the first embodiment will be described. In the second embodiment, the fluctuation angle of the ray direction (amount of change in the direction from the operating body to the specified position) is further taken into consideration. The direction of the ray may be calculated from the start point (start point coordinates) and end point (end point coordinates) of the ray, but in the second embodiment, the ray information indicates the start point, end point, and direction of the ray.

[0055] Fig. 5 is a block diagram showing the functional configuration of a system according to embodiment 2. The system in Fig. 5 includes an information processing device 5000, an HMD 1200, and an image display device 1300. The information processing device 5000 includes a captured image acquisition unit 1010, a position and orientation measurement unit 1020, a virtual data storage unit 1030, a virtual space generation unit 1130, a ray information calculation unit 1050, a distance calculation unit 1060, a movement amount calculation unit 1070, a movement direction calculation unit 1080, a fluctuation angle calculation unit 5100, an averaging processing unit 5090, and a mixed reality image generation unit 1140. These multiple functional units are connected to each other via a path 1110.

[0056] The fluctuation angle calculation unit 5100 calculates the fluctuation angle of the ray direction based on the latest ray direction and the past (e.g., previous) ray direction acquired by the ray information calculation unit 1050. Then, the fluctuation angle calculation unit 5100 outputs information on the calculated fluctuation angle to the averaging processing unit 5090.

[0057] A specific example of a fluctuation angle calculated by the fluctuation angle calculation unit 5100 will be described with reference to FIG. 6. Here, it is assumed that the distance from the tip of the index finger (start point of the ray) to the pointer (end point of the ray) is constant. FIG. 6 shows a state in which a user selects or operates a virtual object 602 with a ray. The user moves the index finger 601 from downward to upward. A ray 606 is the previous ray, which extends from the tip of the index finger 601 (start point 607) in the direction in which the index finger 601 extends, and reaches the virtual object 602. A pointer 608 (cursor) is the previous pointer, which indicates the end point of the ray 606 (the contact position of the virtual object 602 and the ray 606). A ray 603 is the latest ray, which extends from the tip of the index finger 601 (start point 604) in the direction in which the index finger 601 extends, and reaches the virtual object 602. A pointer 605 (cursor) is the latest pointer, and indicates the end point of the ray 603 (the contact position between the virtual object 602 and the ray 603). A ray 609 is obtained by translating the ray 606 so that a start point 607 of the ray 606 coincides with a start point 604 of the ray 603. The variation angle calculation unit 5100 calculates, for example, the angle Θ between the ray 603 and the ray 609 (ray 606) as the variation angle. When the user wishes to move the ray greatly, the variation angle Θ is likely to be large compared to other cases.

[0058] The averaging processor 5090, like the averaging processor 1090 of the first embodiment, acquires information on distance (ray length) from the distance calculation unit 1060, acquires information on the movement amount of the ray from the movement amount calculation unit 1070, and acquires information on the movement direction of the ray from the movement direction calculation unit 1080. Furthermore, the averaging processor 5090 acquires information on the fluctuation angle of the ray direction from the fluctuation angle calculation unit 5100. Then, the averaging processor 5090 performs averaging processing of the end points of the ray with the strength based on the acquired information. For example, the averaging processor 5090 determines whether to perform averaging processing based on the movement amount of the end point of the ray, the movement direction of the end point of the ray, and the fluctuation angle of the ray direction, and determines the strength of the averaging processing according to the distance from the start point to the end point of the ray.

[0059] In the second embodiment, the averaging processor 5090 judges whether or not the following conditions are satisfied: the amount of movement of the end point of the ray is smaller than a threshold, the direction of movement of the end point of the ray is not constant, and the variation angle of the direction of the ray is smaller than a threshold. If the conditions are not satisfied, the averaging processor 5090 does not perform the averaging process, and if the conditions are satisfied, the averaging processor 5090 performs the averaging process with an intensity according to the distance from the start point to the end point of the ray. If the averaging processor 5090 does not perform the averaging process, it outputs the latest ray information acquired from the ray information calculation unit 1050 to the virtual space generation unit 1130, and if the averaging process is performed, it outputs the latest averaged ray information to the virtual space generation unit 1130.

[0060] 7 is a flowchart showing the operation of the information processing device 5000. Steps S701 to S704 are the same as steps S401 to S404 in the first embodiment (FIG. 4). After step S704, the process proceeds to step S711. Note that in step S702, the direction of the ray is also calculated.

[0061] In step S711, the fluctuation angle calculation unit 5100 calculates the angle between the ray direction calculated in step S702 and the previous ray direction as the fluctuation angle of the ray direction. Then, the process proceeds to step S712.

[0062] In step S712, the averaging unit 5090 judges whether the fluctuation angle calculated in step S711 is smaller than a threshold value. If the fluctuation angle is smaller than the threshold value, the process proceeds to step S705, otherwise, the process proceeds to step S708. Note that, although the process proceeds to step S708 if the fluctuation angle is equal to the threshold value, the process may proceed to step S705 if the fluctuation angle is equal to the threshold value.

[0063] Steps S705 to S710 are the same as steps S405 to S410 in the first embodiment (FIG. 4).

[0064] As described above, according to the second embodiment, by further taking into consideration the fluctuation angle of the ray direction (the amount of change in direction from the operating body to the specified position), it is possible to realize operations that match the user's intentions with higher accuracy (further improving operability).

[0065] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.

[0066] For example, the above-mentioned various thresholds may be fixed values ​​determined in advance, values ​​designated by the user (changeable values), or values ​​determined by the information processing device (changeable values). Alternatively, an optical see-through HMD that allows direct observation may be used.

[0067] In addition, the present invention can be applied to various scenes. For example, the present invention can be applied to a scene where a smartphone or a tablet terminal is used instead of an HMD. The present invention can also be applied to a scene where a position on a display surface of a display device such as an electronic whiteboard is specified.

[0068] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0069] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) A determination means for determining a designated position by a display item according to a position and a posture of an operating object; A first acquisition means for acquiring information on a distance from the operating object to the designated position; A second acquisition means for acquiring information on the amount of movement of the designated position; a processing means for performing an averaging process on the designated position with a strength based on the distance and the amount of movement; 13. An information processing device comprising: (Configuration 2) The display item is a ray extending from the operating object to the specified position. 2. The information processing device according to configuration 1. (Configuration 3) The display item is a pointer that is placed at the specified position. 2. The information processing device according to configuration 1. (Configuration 4) The processing means determines whether or not to perform the averaging process based on the amount of movement, and determines the strength of the averaging process in accordance with the distance. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) The processing means includes: If the amount of movement is greater than a threshold, the averaging process is not performed. When the amount of movement is smaller than the threshold, the averaging process is performed with a strength according to the distance. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) A third acquisition means for acquiring information on a moving direction of the designated position, The processing means performs the averaging process with a strength based on the distance, the amount of movement, and the direction of movement. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 7) The processing means determines whether or not to perform the averaging process based on the amount of movement and the direction of movement, and determines the strength of the averaging process in accordance with the distance. 7. The information processing device according to configuration 6. (Configuration 8) The processing means includes: The averaging process is not performed when the amount of movement is greater than a threshold value and when the direction of movement is constant, When the amount of movement is smaller than the threshold and the direction of movement is not constant, The averaging process is performed with a strength according to 8. The information processing device according to configuration 6 or 7. (Configuration 9) A fourth acquisition means for acquiring information on a change amount of a direction from the operating object to the specified position, The processing means performs the averaging process with a strength based on the distance, the amount of movement, the direction of movement, and the amount of change. 8. The information processing device according to configuration 6 or 7. (Configuration 10) The processing means determines whether or not to perform the averaging process based on the amount of movement, the direction of movement, and the amount of change, and determines the strength of the averaging process in accordance with the distance. 10. The information processing device according to configuration 9. (Configuration 11) The processing means includes: not performing the averaging process when the amount of movement is greater than a first threshold, when the direction of movement is constant, and when the amount of change is greater than a second threshold; When the amount of movement is smaller than the first threshold, the direction of movement is not constant, and the amount of change is smaller than the second threshold, the averaging process is performed with a strength according to the distance. 11. The information processing device according to configuration 9 or 10. (method) determining a designated position by a display item according to a position and a posture of an operating object; acquiring information on a distance from the operating object to the designated position; acquiring information on the amount of movement of the designated position; performing an averaging process on the specified position with a strength based on the distance and the amount of movement; 13. An information processing method comprising: (program) 12. A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 11. [Explanation of symbols]

[0070] 1000, 5000: Information processing device 1050: Ray information calculation unit 1060: Distance calculation unit 1070: Movement amount calculation unit 1090,5090: Averaging processing unit 2010: CPU

Claims

1. A determination means for determining a designated position by a display item according to a position and a posture of an operating object; a first acquisition means for acquiring information on a distance from the operating object to the designated position; A second acquisition means for acquiring information on the amount of movement of the designated position; a processing means for performing an averaging process on the designated position with a strength based on the distance and the amount of movement; 13. An information processing device comprising:

2. The display item is a ray extending from the operating object to the specified position.

2. The information processing apparatus according to claim 1,

3. The display item is a pointer that is placed at the specified position.

2. The information processing apparatus according to claim 1,

4. The processing means determines whether or not to perform the averaging process based on the amount of movement, and determines the strength of the averaging process in accordance with the distance.

2. The information processing apparatus according to claim 1,

5. The processing means includes: If the amount of movement is greater than a threshold, the averaging process is not performed. When the amount of movement is smaller than the threshold, the averaging process is performed with a strength according to the distance.

2. The information processing apparatus according to claim 1,

6. The apparatus further includes a third acquisition means for acquiring information on a moving direction of the designated position, The processing means performs the averaging process with a strength based on the distance, the amount of movement, and the direction of movement.

2. The information processing apparatus according to claim 1,

7. The processing means determines whether or not to perform the averaging process based on the amount of movement and the direction of movement, and determines the strength of the averaging process in accordance with the distance.

7. The information processing apparatus according to claim 6,

8. The processing means includes: The averaging process is not performed when the amount of movement is greater than a threshold value and when the direction of movement is constant, When the amount of movement is smaller than the threshold value and the direction of movement is not constant, the averaging process is performed with a strength according to the distance.

7. The information processing apparatus according to claim 6,

9. A fourth acquisition means for acquiring information on a change amount of a direction from the operating object to the designated position, The processing means performs the averaging process with a strength based on the distance, the amount of movement, the direction of movement, and the amount of change.

7. The information processing apparatus according to claim 6,

10. The processing means determines whether or not to perform the averaging process based on the amount of movement, the direction of movement, and the amount of change, and determines the strength of the averaging process in accordance with the distance.

10. The information processing apparatus according to claim 9,

11. The processing means includes: not performing the averaging process in each of the cases where the amount of movement is greater than a first threshold, the direction of movement is constant, and the amount of change is greater than a second threshold; When the amount of movement is smaller than the first threshold, the direction of movement is not constant, and the amount of change is smaller than the second threshold, the averaging process is performed with a strength according to the distance.

10. The information processing apparatus according to claim 9,

12. determining a designated position by a display item according to a position and a posture of an operating object; acquiring information on a distance from the operating object to the designated position; acquiring information on the amount of movement of the designated position; performing an averaging process on the specified position with a strength based on the distance and the amount of movement; 13. An information processing method comprising:

13. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 11.

Images