Information processing device, information processing method and computer-readable non-volatile storage medium

Abstract

Problem: The goal is to enable more flexible and intuitive rotation control of a virtual object. Solution: Provided is an information processing device comprising a control unit that dynamically determines a mode of rotation of a displayed or projected virtual object based on a positional relationship between the virtual object and an operating object used to operate the virtual object.

Description

Technical field

[0001] The present disclosure relates to an information processing device, information processing method and a computer-readable non-volatile storage medium. State of the art

[0002] In recent years, technologies have been developed to improve the functionality of CG (Computer Graphics) models (hereinafter also referred to as virtual objects). For example, patent literature 1 discloses a technology for rotating the virtual objects in a three-dimensional space according to a user's line of sight. List of cited documents Patent literature

[0003] Patent literature 1: JP 6939801 Disclosure of the invention Technical problem

[0004] However, the intersection point between a direction estimated from the line of sight or the like and the virtual object lies on a surface of the virtual object. Therefore, with the technology disclosed in patent literature 1, it is difficult to define a center of rotation in the virtual object. Furthermore, there is a need for additional flexible modifications to the rotational motion patterns of the virtual object, such as a change in the center of rotation of the virtual object. Solution to the problem

[0005] According to one aspect of the present disclosure, a video processing device is provided which includes the following: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object.

[0006] Furthermore, according to another aspect of the present disclosure, an information processing procedure is provided which includes the following: Dynamically determining a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object.

[0007] Furthermore, according to yet another aspect of the present disclosure, a computer-readable non-volatile storage medium is provided which stores a program to cause a computer to function as an information processing device, the information processing device comprising the following: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object. Brief description of the drawings Fig. 1 An explanatory view illustrating how a center of rotation of a virtual object 30 according to an embodiment of the present disclosure is determined based on a positional relationship between the virtual object 30 and an operating object 20 according to the embodiment of the present disclosure. Fig. 2 An explanatory view illustrating a configuration example of an information processing system 1 according to the embodiment. Fig. 3 A block diagram illustrating an example of the functional configuration of an information processing device 10 and a control device 24 according to the embodiment. Fig. 4 A table showing examples of how the center of rotation and a rotation scaling factor of the virtual object 30 are determined by a control unit 150 according to the embodiment. Fig. 5 An explanatory view illustrating how a rotation amount is determined based on a change in the angle of the control object 20 according to the embodiment. Fig. 6 An explanatory view illustrating how the amount of rotation is determined based on an angular velocity of the operating object 20 according to the embodiment. Fig. 7 An explanatory view illustrating an example of how a rotation axis is determined according to the embodiment. Fig. 8 An explanatory view illustrating an example of a rotation process in which the axis of rotation is defined by a single control object 20 according to the embodiment. Fig. 9 An explanatory view illustrating a deterioration of the user experience, wherein the deterioration is possible if a default point is set as the pivot center 35, if a user 50 with the control object 20 is present in the virtual object 30 according to the embodiment. Fig. 10 An explanatory view illustrating an advantage achieved by defining a reference position of the control object 20 as the center of rotation 35 if the user 50 is located with the control object 20 in the virtual object 30 according to the embodiment. Fig. 11 A view illustrating a control in a case where the control unit 150 according to the embodiment causes the virtual object 30 to rotate by 180° in response to a change in the angle of the hand 22, the change exceeding a threshold (100° in the Fig. 11 (example shown). [ Fig. 12] A flowchart showing an example of a control operation by the control unit 150 according to the embodiment. Fig. 13 A block diagram illustrating an example of a hardware configuration of an information processing device 90 according to the embodiment. Method(s) of carrying out the invention

[0008] A suitable embodiment of the present disclosure is described in more detail below with reference to the accompanying drawings. It should be noted that in this description and in the drawings, components with substantially the same functional configurations are designated with the same reference numerals in order to avoid redundant descriptions.

[0009] Furthermore, in this description and in the drawings, letter suffixes or similar may be added to the reference symbols to distinguish between a multitude of configurations of the same type. Conversely, if there is no need to distinguish between a multitude of configurations of the same type, the letter suffixes or similar may be omitted to describe features common to all of a multitude of configurations of the same type.

[0010] It should be noted that the description follows in the order below. 1. Design 1.1. Overview 1.2. System configuration example 1.3. Details 1.4. Process flow 2. Example of a hardware configuration 3. Summary 1. Design 1.1. Overview

[0011] For example, in mechanical computer-aided design (CAD) environments and similar applications, changes in the viewpoint of virtual objects being designed are frequently performed. Operations for performing viewpoint changes include operations for rotating the virtual objects.

[0012] In order to make changes in viewpoint even more flexible, it is important in this context to change the rotation centers of the virtual objects according to user requirements.

[0013] For example, if the centers of the virtual objects are defined as rotation centers, it is possible to fold the virtual objects stably and easily.

[0014] If any point is set as the center of rotation, the user can focus on the surroundings of that point.

[0015] Furthermore, the rotation processes described above are not only required in the field of mechanical CAD.

[0016] For example, when using or designing virtual objects intended for use in AR (Augmented Virtuality), VR (Virtual Reality), XR (Extended Reality / Cross Reality) and the like, the rotation processes described above are also necessary.

[0017] Furthermore, in AR, VR and XR, it is assumed that the rotation operations on the virtual objects are carried out, for example, with the user's hand or by means of a device held by the user.

[0018] Although manual manipulation is advantageous because it allows for fine rotation of virtual objects, the limited range of motion of the hand can make it difficult to rotate virtual objects extensively.

[0019] The technical scope according to the embodiment of the present disclosure was developed with a focus on the circumstances described above in order to make rotation operations on the virtual objects more flexible and intuitive to perform.

[0020] To enable this, patterns of rotational movements of the virtual objects can be dynamically determined according to the embodiment of the present disclosure based on the positional relationships between the virtual objects and the control objects used to operate the virtual objects.

[0021] Examples of factors affecting rotational movements are the centers of rotation of virtual objects.

[0022] A control unit 150 (see Fig. 3) An information processing device 10 according to this embodiment can determine at least the centers of rotation of the virtual objects based on the positional relationships between the virtual objects and the control objects.

[0023] Fig. Figure 1 is an explanatory view illustrating how a center of rotation of a virtual object 30 according to the embodiment of the present disclosure is determined based on a positional relationship between the virtual object 30 and an operating object 20 according to the embodiment of the present disclosure.

[0024] It should be noted that, according to this embodiment, the control object 20 is an object that gives instructions for rotating the virtual object 30.

[0025] The control object 20 according to this embodiment can, for example, be a user's hand 22.

[0026] In one in the upper part of Fig. In the case shown, the virtual object 30, which is controlled by the control unit 150 of the display, and the user's hand 22 are outside of contact with each other.

[0027] If it is determined that the virtual object 30 and the control object 20 are out of contact with each other, the control unit 150 according to this embodiment can set a rotation center 35 of the virtual object 30 to a standard point that is preset in the virtual object 30.

[0028] The default point can be a center point, a center of gravity, an origin of a coordinate system, or any other point of the virtual object 30.

[0029] Even in the state where the virtual object 30 and the control object 20 have no contact with each other, it is possible, by setting the rotation center 35 to the standard point, to, for example, rotate the virtual object 30 stably and easily as described above.

[0030] In one in the lower part of Fig. In the case shown, the virtual object 30 and the user's hand 22 are in contact with each other.

[0031] If it is determined that the virtual object 30 and the control object 20 are in contact with each other, the control unit 150 can determine the center of rotation 35 of the virtual object 30 based on the position of the control object 20 according to this embodiment.

[0032] For example, if it is determined that the virtual object 30 and the control object 20 are in contact with each other, the control unit 150 can define the rotation center 35 of the virtual object 30 at a contact point between the virtual object 30 and the control object 20 or at a reference position of the control object 20.

[0033] For example, the reference position can be a preset position for each of the control objects 20, such as the tip of the index finger if the control object 20 is the user's hand 22, or the center of the middle phalanx of the index finger if the control object 20 is a ring-type control device 24 (see Fig. 2) is.

[0034] In the case that the virtual object 30 and the control object 20 are in contact with each other, by setting the center of rotation 35 to the contact point or the reference position, it is possible for the user to concentrate on the surroundings of the arbitrary point as described above.

[0035] The figure above outlines how the center of rotation 35 of the virtual object 30 is determined according to this embodiment.

[0036] The control described above makes it possible to perform the rotation operations on the virtual object 30 more flexibly and intuitively. 1.2. Configuration example of a system

[0037] The following describes a configuration example of an information processing system 1 according to this embodiment.

[0038] Fig. Figure 2 is an explanatory view showing the configuration example of the information processing system 1 according to this embodiment.

[0039] The information processing system 1 according to this embodiment includes at least the information processing device 10, which controls the display or projection of the virtual object 30.

[0040] The information processing device 10 according to this embodiment can, for example, be a head-mounted display (HMD) worn by a user 50.

[0041] The user 50 performs the rotation operation on the virtual object 30, which is displayed by the information processing device 10, via the control object 20.

[0042] As described above, according to this embodiment, the control object 20 can, for example, be the hand 22 of the user 50.

[0043] Alternatively, the operating object 20 could be, for example, the control device 24.

[0044] In a Fig. In the case shown in Figure 2, the control device 24 can be a ring-shaped device attached to the middle segment between the first and second joints of the index finger of the hand 22.

[0045] User 50 may be permitted to rotate the virtual object 30 by operating an input unit 220 (see Fig. 3) the control device 24.

[0046] The following refers to Fig. 3 An example of the functional configuration of the information processing device 10 and the control device 24 according to this embodiment is described.

[0047] Fig. Figure 3 is a block diagram illustrating the example of a functional configuration of the information processing device 10 and the control device 24 according to the embodiment. Information processing device 10

[0048] The information processing device 10 according to this embodiment is a computer that controls the display or projection of the virtual object.

[0049] The information processing device 10 can be the HMD, a PC (personal computer), a smartphone, a tablet or the like.

[0050] As an example, how in Fig. As shown in Figure 3, the information processing device 10 according to this embodiment can include a communication unit 110, a detection unit 120, a display unit 130, an audio input / output unit 140 and the control unit 150. Communication unit 110

[0051] The communication unit 110 according to this embodiment performs wireless or wired communication with the control device 24. For example, the

[0052] Communication unit 110 from the control device 24 provides information about operations at the input unit 220 (for example, whether the operations are carried out or not, duration of the operations and intensity of the operations). Recording unit 120

[0053] The detection unit 120 according to this embodiment detects a position and a posture of the operating object 20.

[0054] If the operating object 20 is, for example, the hand 22, the detection unit 120 can detect a position and posture of the hand 22 based on feature sizes and finger bones.

[0055] Alternatively, for example, if the operating object 20 is the control device 24 designed as a ring according to Fig. 2 is, the detection unit 120 detects the position and attitude of the control device 24 using, for example, feature points of the control device 24 or directly detects the position and attitude of the hand 22 as described above.

[0056] To perform position detection as described above, the detection unit 120 includes a ToF (Time of Flight) sensor and the like. Display unit 130

[0057] The display unit 130 according to this embodiment is a display that shows various visual information, including the virtual object 30, under the control of the control unit 150.

[0058] It should be noted that in a case where the virtual object 30 is projected, the information processing device 10 may include a projection unit instead of or in addition to the display unit 130. Audio input / output unit 140

[0059] The audio input / output unit 140 according to this embodiment outputs various acoustic information under the control of the control unit 150.

[0060] The audio input / output unit 140 can, for example, output voice guidance regarding the rotation process on the virtual object 30. Control unit 150

[0061] The control unit 150 according to this embodiment controls each of the units of the information processing device 10.

[0062] Furthermore, according to this embodiment, the control unit 150 performs the display control of the virtual object 30.

[0063] In particular, one of the features of the control unit 150 according to this embodiment dynamically determines a pattern of the rotational movement of the virtual object 30 based on the positional relationship between the virtual object 30 and the control object 20.

[0064] As described above, factors of the rotational movement of the virtual object 30 according to this embodiment include the center of rotation 35 of the virtual object 30.

[0065] In addition, the factors of the rotational motion of the virtual object 30 according to this embodiment can include a rotation amount, a rotation scaling factor, a rotation speed, an axis of rotation and the like.

[0066] The functions of the control unit 150 according to this embodiment are implemented by various processors. Details of the functions of the control unit 150 according to this embodiment are described below. Control device 24

[0067] The control device 24 according to this embodiment is an example of the control object 20.

[0068] The control device 24 can, for example, control the in Fig. 2. The device shown is a ring-shaped device, or a VR controller, a game controller, or the like.

[0069] As in Fig. As shown in Figure 3, the control device 24 according to this embodiment comprises at least one communication unit 210 and one input unit 220. Communication unit 210

[0070] The communication unit 210 according to this embodiment performs wireless or wired communication with the information processing device 10.

[0071] For example, the communication unit 210 transmits information about the operations at the input unit 220 to the information processing device 10 (for example, whether the operations are being carried out, the duration of the operations and the intensity of the operations). Input unit 220

[0072] The input unit 220 according to this embodiment receives input operations by the user 50.

[0073] The input unit 220 can, for example, include buttons, a lever, a joystick, a slider, and the like.

[0074] The configuration example of information processing system 1 according to this embodiment is described above. It should be noted that the above references to Fig. 2 and Fig. The configuration described in section 3 is merely an example, and therefore the configuration of the information processing system 1 according to this embodiment is not limited to such an example.

[0075] In particular, the functions of the information processing device 10 described above can be implemented through interaction between a large number of devices.

[0076] The configuration of the information processing system 1 according to this embodiment can be flexibly varied according to specifications, practical use, and the like. 1.3. Details

[0077] Next, the functions of the information processing device 10 according to this embodiment will be described in more detail.

[0078] As described above, one of the features of the control unit 150 according to this embodiment is to dynamically determine the pattern of the rotational movement of the virtual object 30 based on the positional relationship between the virtual object 30 and the control object 20.

[0079] The control unit 150 according to this embodiment can dynamically determine the center of rotation and the rotation scaling factor of the virtual object 30 based on the positional relationship between the virtual object 30 and the control object 20.

[0080] Fig. Figure 4 shows examples of how the center of rotation and the rotation scaling factor of the virtual object 30 are determined by the control unit 150 according to this embodiment.

[0081] First, an example of the determination is described in the case where the control object 20 is the hand 22 of the user 50.

[0082] In this case, a method of rotation on the virtual object 30 can be a rotation of the hand 22.

[0083] The user 50 rotates his hand 22 while the hand 22 is in contact with the virtual object 30 or is not in contact with it, in order to control the rotation process on the virtual object 30.

[0084] At this time, the position and posture of hand 22 are continuously recorded by the detection unit 120.

[0085] If it is determined that the virtual object 30 and the hand 22 are in contact at the time of the start of the rotation process, the control unit 150 sets the rotation center of the virtual object 30 to the contact point between the virtual object 30 and the hand 22 or to the reference position preset on the hand 22.

[0086] If, however, it is determined that the virtual object 30 and the hand 22 are not in contact with each other at the time of the start of the rotation process, the control unit 150 sets the rotation center of the virtual object 30 to the default point preset in the virtual object 30.

[0087] Additionally, the control unit 150 determines the rotation scaling factor of the virtual object 30 based on the positional relationship between the virtual object 30 and the control object 20.

[0088] In particular, if it is determined that the virtual object 30 and the control object 20 are not in contact with each other, the control unit 150 can set the rotation scaling factor of the virtual object 30 higher than in a case where it is determined that the virtual object 30 and the control object 20 are in contact with each other.

[0089] For example, the control unit 150 can be installed in a Fig. In the case shown in step 4, if it is determined that the virtual object 30 and the hand 22 are in contact with each other at the time of the start of the rotation process, adjust the rotation scaling factor so that it is equal to the amount of rotation of the hand 22.

[0090] At this point, an angle of hand 22, which was detected by the detection unit 120, can be used to input the rotation amount.

[0091] The control described above enables an operating procedure that closely resembles actual movements, such as actually grasping and rotating an object, by setting a rotation procedure and the rotation scaling factor, wherein the procedure and the rotation scaling factor cause a rotation that corresponds to the rotation of hand 22 relative to the center of rotation, where the center of rotation is a point with which hand 22 is in contact.

[0092] If, on the other hand, it is determined that the virtual object 30 and the hand 22 are not in contact with each other at the time of the start of the rotation process, the control unit 150 can set the rotation scaling factor to twice the amount of rotation of the hand 22.

[0093] In this case too, the angle of hand 22, which was detected by the detection unit 120, can be used to input the amount of rotation.

[0094] It is assumed that the situation in which the virtual object 30 and the hand 22 are not in contact with each other does not serve the purpose of viewing the virtual object 30 while simultaneously rotating it slightly. Therefore, in this situation, it is desirable that the center of rotation be set to the default point preset in the virtual object 30 and that the rotation scaling factor be increased according to the rotation of the hand 22, so that the virtual object 30 can be viewed during extensive rotation.

[0095] It should be noted that the control described above, which is carried out in the case described above where the control object 20 is the hand 22, can also be carried out in the case where the control object 20 is the control device 24 designed as a ring. This is because the operating sensation in the case of the control device 24 designed as a ring corresponds to that of the hand 22.

[0096] However, the control device 24 according to this embodiment is not limited to the ring type. The control device 24 according to this embodiment can, for example, be a VR controller or a game controller with a joystick or the like.

[0097] The operating experience in the case where the control device 24 includes the joystick and the user 50 performs the rotational operations on the virtual object 30 via the joystick differs from the operating experience in the case where the rotational operations are performed manually 22. Furthermore, it is assumed that the case in which the user 50 performs the rotational operations on the virtual object 30 via the joystick is not intended for viewing the virtual object 30 while simultaneously performing fine rotations.

[0098] Therefore, in the event that the virtual object 30 is rotated via the joystick, the control unit 150 can set the rotation center of the virtual object 30 at the standard point, regardless of the positional relationship between the virtual object 30 and the control device 24.

[0099] Furthermore, in the case described above, the control unit 150 can use an angular velocity to input the amount of rotation, so that the rotation scaling factor depends on the tilt of the joystick.

[0100] This is because the joystick is particularly well suited for capturing both the extent of the operation (tilt angle) and the duration of the operation (tilt duration).

[0101] In this context, with reference to Fig. 5 and Fig. 6. The difference between determining the amount of rotation based on the angle and determining the amount of rotation based on the angular velocity is described.

[0102] Fig. Figure 5 is an explanatory view showing how the amount of rotation is determined based on a change in the angle of the control object 20.

[0103] As in Fig. As shown in Figure 5, when using the angle to input the rotation amount, the virtual object 30 is also rotated by 90° in response to a 90° rotation of hand 22 (if the rotation scaling factor is 1). At this point, the duration of the hand 22 rotation and the duration of the virtual object 30 rotation do not necessarily have to be synchronized.

[0104] Fig. Figure 6 is an explanatory view showing how the amount of rotation is determined based on the angular velocity of the control object 20.

[0105] As in Fig. As shown in Figure 6, when the angular velocity is used to input the rotation amount, the virtual object 30 is rotated according to the time it takes for the hand 22 to rotate. At this point, the angle of the hand 22 and the angle of the virtual object do not necessarily have to be synchronized.

[0106] As described above, according to this embodiment, the control unit 150 can determine the amount of rotation of the virtual object 30 based on the angle or angular velocity of the controlled object.

[0107] It should be noted that the control unit 150 can decide whether the angle or the angular velocity is used to input the amount of rotation, based on, for example, the types of the control object 20, whether the virtual object 30 and the control object 20 are in contact with each other, the properties of applications and the properties of the virtual object 30.

[0108] Additionally, regardless of whether the angle or angular velocity is used to input the amount of rotation, the control unit 150 can use a sound, a vibration, a visual effect or the like to indicate that the rotation of the virtual object 30 has reached a straight angle of 45°, 90°, 180° or the like.

[0109] The following describes how the axis of rotation is determined according to this embodiment.

[0110] The factors of the rotational movement of the virtual object 30 according to this embodiment can include the axis of rotation of the virtual object 30.

[0111] It is assumed that when performing rotation operations on the virtual object 30, there are many cases in which it is not only desired to set the axis of rotation with only three degrees of freedom (yaw / roll / pitch), but also that the axis itself should be set to a predetermined axis.

[0112] Therefore, according to this embodiment, the control unit 150 can determine the axis of rotation of the virtual object 30 based on the positional relationship between the virtual object 30 and the control object 20.

[0113] For example, according to this embodiment, the control unit 150 can determine the axis of rotation of the virtual object 30 based on the positional relationship between the virtual object 30 and a first control object and cause the virtual object 30 to rotate in response to operation by a second control object.

[0114] Fig. Figure 7 is an explanatory view showing an example of how the axis of rotation is determined according to this embodiment.

[0115] In a Fig. In the case shown in Figure 7, the user 50 sets the axis of rotation of the virtual object 30 by assuming a pointing position with their left hand 22L (example of the first control object) and then grasping and moving the virtual object 30 with their right hand 22R (an example of the second control object). This allows the user 50 to rotate the virtual object 30.

[0116] At this point, the control unit 150 determines an extension direction of the index finger of the left hand 22L as the axis of rotation and causes the virtual object 30 to rotate exclusively around the determined axis of rotation based, for example, on an angle of the right hand 22R.

[0117] The control described above allows for easy switching between rotating the virtual object 30 with three degrees of freedom and rotating the virtual object 30 about the predetermined axis of rotation, as well as easily setting the predetermined axis of rotation. Additionally, regarding rotation about the predetermined axis, performing this operation with the right hand 22R offers the advantage that even if the pointing position is accidentally assumed with the left hand 22L and the rotation about the predetermined axis is initiated, the rotation will not occur unintentionally, provided the operation with the right hand 22R is not performed.

[0118] It should be noted that the rotation operations for setting the axis of rotation do not necessarily have to be performed with both hands of the user 50 (two operating objects 20), as in the Fig. 7 shown in the example, and the rotation operations for setting the axis of rotation can also be performed with one hand (one operating object 20).

[0119] Fig. Figure 8 is an explanatory view showing an example of the rotation process in which the axis of rotation is determined by the single operating object 20 according to this embodiment.

[0120] It should be noted that Fig. Figure 8 illustrates a case in which the single control object 20 is the user's left hand 22L 50.

[0121] In the Fig. In the case shown in Figure 8, the user 50 first brings the left hand 22L into contact with the virtual object 30.

[0122] At this point, the control unit 150 establishes a contact position between the virtual object 30 and the left hand 22L as the center of rotation.

[0123] Furthermore, the control unit 150 can vary a display pattern, for example, the colors of the virtual object 30, so that the user 50 perceives that the virtual object 30 and the left hand 22L are in contact with each other. Such a visual control enables the user 50 to intuitively grasp the influencing factors of the rotational movement, such as the center of rotation and the rotation scaling factor. It should be noted that the influencing factors of the rotational movement, such as the center of rotation and the rotation scaling factor, can be explicitly displayed using visual or acoustic information.

[0124] The user then gives a rotation instruction 50 by rotating the left hand 22L while keeping the left hand 22L in contact with the virtual object 30.

[0125] At this point, the control unit 150 determines the orientation direction of the index finger of the left hand 22L as the axis of rotation and causes the virtual object 30 to rotate only around the determined axis of rotation based, for example, on the angle of the left hand 22L.

[0126] Subsequently, when the user 50 releases their left hand 22L from the virtual object 30, the control unit 150 can consider this as the end of the rotational movement and terminate the rotation of the virtual object 30.

[0127] The control described above makes it possible to define both the axis of rotation and the amount of rotation of the virtual object via a single control object 20.

[0128] The following describes a control example for the case where the user 50 is in virtual reality (VR), augmented reality (XR) or the like in virtual object 30 with the control object 20.

[0129] In such a case, if the user 50 is located with the control object 20 at a position away from the standard point of the virtual object 30, the positional relationship between the user 50 and the virtual object 30 can change significantly if the virtual object 30 rotates around the standard point as its center of rotation, which can impair the user experience.

[0130] Fig. Figure 9 is an explanatory view illustrating a deterioration in user experience that may occur if the default point is set as rotation center 35, in the event that the user 50 is inside the virtual object 30 with the control object 20.

[0131] The left side in Fig. Figure 9 shows a positional relationship before a rotation, when the default point of the virtual object 30 (center of the virtual object 30 in the Fig. 9 (example shown) is defined as the center of rotation 35.

[0132] The right side in Fig. Figure 9 shows a positional relationship after rotation when the standard point of the virtual object 30 is set as the rotation center 35.

[0133] In a Fig. In the case shown in Figure 9, the positional relationship before and after the rotation around the center of rotation 35 at the standard point of the virtual object 30 changes significantly. As a result, a collision occurs between the user 50 and a structure in the virtual object 30.

[0134] In this way, if user 50 is present with the control object 20 in the virtual object 30, the user experience can be significantly impaired if the default point of the virtual object 30 is set as the center of rotation 35, because the positional relationship changes considerably.

[0135] It should be noted that the deterioration of the user experience does not only occur in the collision case described above, but also, for example, when user 50 moves outside of virtual object 30.

[0136] To avoid such a deterioration of the user experience, according to this embodiment, when it is determined that the control object 20 is located in the virtual object 30, the control unit 150 determines the rotation center 35 of the virtual object 30 based on the position of the control object 20, regardless of the positional relationship between the virtual object 30 and the control object 20.

[0137] Fig. Figure 10 is an explanatory view showing an advantage to be exploited by setting the reference position of the control object 20 as the center of rotation 35 in the case that the user 50 is located with the control object 20 in the virtual object 30.

[0138] The left side in Fig. Figure 10 shows a position relationship before a rotation when the reference position of the control object 20 is set as the rotation center 35.

[0139] The right side in Fig. Figure 10 shows a position relationship after rotation when the reference position of the control object 20 is set as the rotation center 35.

[0140] As in Fig. As shown in Figure 10, the change in the positional relationship when the reference position of the control object 20 is set as the center of rotation 35 is smaller than when the standard point of the virtual object 30 is set as the center of rotation 35. This advantageously avoids the deterioration in the user experience described above.

[0141] It should be noted that the determination of whether the user 50 is located in the virtual object 30 with the control object 20 can be made based on the presence or absence of the virtual object 30 in the up-down, right-left and forward-backward directions of the user 50.

[0142] For example, if the virtual object 30 exists in all up-down, right-left and front-back directions of user 50, it can be determined that user 50 is located in the virtual object 30.

[0143] It should be noted that, for example, there may also be a virtual object such as a roofless fence, which is why the presence or absence of virtual objects in the upper direction of user 50 does not necessarily have to be used to determine whether user 50 is located in virtual object 30 with control object 20.

[0144] Furthermore, the determination of whether the user 50 is located in the virtual object 30 with the control object 20 can be made based on the shapes and other properties of virtual objects.

[0145] Furthermore, it can be determined whether the user 50 is located in the virtual object 30 with the control object 20, based on position information of the user 50, position information of the control object 20 and display position information of the virtual object 30.

[0146] Furthermore, if it is determined that the user 50 is located in the virtual object 30 with the control object 20, a control such as limiting the direction of rotation to the yaw direction can be carried out to avoid motion sickness in the user 50.

[0147] Next, another example of control by the control unit 150 according to this embodiment will be described.

[0148] For example, according to this embodiment, if the change in the angle or angular velocity of the controlled object 20 exceeds a threshold value, the control unit 150 can rotate the virtual object 30 by a predetermined angle.

[0149] Fig. Figure 11 shows a control in a case where the control unit 150 according to this embodiment causes the virtual object 30 to react to the change in the angle of the hand 22, which has a threshold value (100° in the Fig. (11 case shown) exceeds, rotates by 180°.

[0150] From the perspective of hand mobility, it can be difficult to rotate the virtual object 30 extensively while grasping it with hand 22. However, the control described above makes it easy to perform a rotation that exceeds the hand's range of motion. Therefore, it is expected, for example, that the user 50 will be prevented from excessively twisting and injuring their wrist, and usability will be improved.

[0151] In addition, operations such as 180° rotation, 90° rotation, and the like are frequently performed. Therefore, with the operating object 20 of the control device 24, it may be possible to perform operations such as pressing a button twice to effect a 90° rotation and pressing the button three times to effect a 180° rotation.

[0152] It should be noted that, to avoid excessive twisting and injury of the user's wrist, for example, a control can be carried out which displays a comparison between a pre-registered range of motion of the wrist and a position and posture of the detected wrist.

[0153] Furthermore, a control can be implemented that continues the rotation even when the movement limits are reached and switches from the angle input mode to the angular velocity input mode.

[0154] Furthermore, a rotation scaling factor can be set in the state where hand 22 is not in contact with the virtual object 30, based on user information such as the mobility of user 50's wrist. For example, the rotation scaling factor can be set high if user 50's wrist is stiff, so that the virtual object 30 can be extensively rotated with a single operation.

[0155] Furthermore, the control unit 150 can change the amount of rotation, the rotating range / direction of rotation, and similar parameters according to the user's situation. For example, the user's range of motion varies depending on whether the user is standing or sitting, and therefore the control unit 150 can change the amount of rotation, the rotating range, and similar parameters depending on whether the user is standing or sitting.

[0156] Furthermore, if the user 50 rotates their hand at a predetermined speed or higher while the hand 22 is kept in contact with the virtual object 30, the control unit 150 can perform a control that allows the virtual object 30 to continue rotating due to inertia even after the hand 22 has been released from the virtual object 30, and stops the rotation when the virtual object 30 and the hand 22 come back into contact with each other.

[0157] Furthermore, the default point as the center of rotation of the virtual object 30, for example, does not necessarily have to be the center, center of gravity, or origin of a coordinate system mentioned in the description above, and a feature point that is closest to the control object 20 among the feature points of the virtual object 30 can be set as the center of rotation.

[0158] For example, with a virtual object 30 in the form of an airplane, a control such as snapping the feature point closest to the control object 20 can be performed, so that feature points corresponding to, for example, a cockpit, a door handle and the tips of the right and left wings can be selected as the center of rotation.

[0159] It should be noted that the feature points described above can be automatically detected, for example by edge detection, or can be registered by the user.

[0160] Furthermore, the user can be enabled to set any rotation center, including the feature points described above, via voice command.

[0161] This factor setting via voice command is not only applicable to the center of rotation, but also, for example, to the amount of rotation, the rotation scaling factor, the axis of rotation and the rotational speed.

[0162] Furthermore, commands to start and stop the rotation process according to this embodiment can be given, for example, using predefined gestures. For example, if the control object 20 is the hand 22, a command to start the rotation process can be given by tapping the thumb and forefinger.

[0163] It should be noted that if the operating object 20 is the control device 24, the start and end of the rotation process can be a predetermined operation at the input unit 220 (such as pressing a specific key).

[0164] Furthermore, according to this embodiment, the control unit 150 can provide a function to save the state of the virtual object 30 before the rotation control and to allow a return to the state of the virtual object 30 before the rotation control in response to a user 50 instruction, even after the rotation control. This enables an immediate return to a previous state and thus facilitates, for example, recovery in the event of an operator error and a comparison with a previous viewpoint.

[0165] Furthermore, during rotation control, the control unit 150 can terminate the rotation independently of the user's 50 operation if information is present that should not be displayed. For example, the control unit 150 can stop the rotation to prevent an area containing confidential information, such as design data that should not be displayed to the current user 50, from becoming visible to the user 50.

[0166] Furthermore, the control unit 150 can control the display of a manual that explains 50 functions to the user. For example, the control unit 150 can implement a control mechanism so that the manual is not displayed again once it has been shown a predetermined number of times in an initial phase.

[0167] Furthermore, if the virtual object 30 is operated simultaneously by a large number of users 50, the control unit 150 can set a priority for each of the users 50 to avoid conflicts, or implement a locking function so that other users 50 cannot perform rotational movements while a specific user 50 is performing a rotation. 1.4. Process flow

[0168] The following describes a sequence of operations by the control unit 150 according to this embodiment using an example.

[0169] Fig. Figure 12 is a flowchart showing an example of the control process by the control unit 150 according to this embodiment.

[0170] In a Fig. In the case shown in Figure 12, the control unit 150 first performs a display control of the virtual object (S101).

[0171] Subsequently, the control unit 150 records the positional relationship between the operating object 20 and the virtual object 30 based on the position and attitude of the operating object 20 (S102) recorded by the detection unit 120.

[0172] The control unit 150 then detects the rotation process via the operating object 20 (S103).

[0173] Subsequently, the control unit 150 performs the rotation control of the virtual object 30 based on the position relationship recorded in step S102 and the rotation process recorded in step S103 (S104).

[0174] Until an instruction is given to shut down the system, the control unit 150 can repeatedly execute the sequences of steps S101 to S104. 2. Example of a hardware configuration

[0175] The following describes an example of a hardware configuration of the information processing device 90 according to the embodiment of the present disclosure. Fig. Figure 13 is a block diagram showing an example of the hardware configuration of the information processing device 90 according to the embodiment of the present disclosure. The information processing device 90 can be a device with a hardware configuration corresponding to that of the information processing device 10.

[0176] As in Fig.As shown in Figure 13, the information processing device 90 comprises, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a memory 880, a drive 881, a connection port 882, and a communication unit 883. It should be noted that the hardware configuration described here is merely an example, and some of these components may be omitted. Furthermore, components other than those described here may also be included. Processor 871

[0177] The processor 871, for example, functions as a computing device or control device and controls all or parts of the respective operations of the components according to various programs stored in the ROM 872, in the RAM 873, in the memory 880 or on removable storage media 901. ROM 872 and RAM 873

[0178] The ROM 872 is a storage device, for example, for programs to be loaded by the processor 871 and data to be used for calculations. The RAM 873 temporarily or permanently stores, for example, programs to be loaded by the processor 871, various parameters that vary as needed during the execution of these programs, and the like. Host bus 874, bridge 875, external bus 876 and interface 877

[0179] The 871 processor, the 872 ROM, and the 873 RAM are connected to each other via the 874 host bus, which is capable of high-speed data transmission. The 874 host bus, in turn, is connected to the 876 external bus, which transmits data at a relatively low rate, via the 875 bridge, for example. The 876 external bus is also connected to various components via the 877 interface. Input device 878

[0180] Input device 878 includes, for example, a mouse, a keyboard, a touchscreen, buttons, a switch, a lever, and the like. Furthermore, input device 878 can include a remote control (hereinafter referred to as a remote control) that can transmit control signals by means of infrared beams or other radio waves. Input device 878 also includes a speech input device, such as a microphone. Dispensing device 879

[0181] Output device 879 is a device capable of notifying users visually or audibly of received information, and may, for example, be a display device such as a CRT (cathode ray tube), an LCD or an organic light-emitting diode display, an audio output device such as a loudspeaker or headphones, a printer, a mobile phone, a fax machine, or the like. Furthermore, output device 879, according to the present disclosure, comprises various vibration devices capable of emitting tactile stimuli. Memory 880

[0182] The 880 storage device is a device for storing various types of data. For example, a magnetic storage medium such as a hard disk drive (HDD), a semiconductor memory, an optical storage medium, a magneto-optical storage medium, or the like can be used as the 880 storage device. Drive 881

[0183] The drive 881 is a device that reads information from a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, or writes information to the removable storage medium 901. Removable storage medium 901

[0184] Removable storage medium 901 is, for example, a DVD, a Blu-ray disc (brand), an HD-DVD disc, various types of semiconductor storage media, or the like. Of course, removable storage medium 901 could also be, for example, an IC card, an electronic device, or the like, equipped with a contactless IC chip. Connection port 882

[0185] The 882 connection port is an interface for connecting an external connection system 902, for example a USB (Universal Serial Bus) port, an IEEE1394 port, SCSI (Small Computer System Interface), an RS-232C port, an optical audio port, or the like. External connection device 902

[0186] The external connection device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, or the like. Communication device 883

[0187] The communication device 883 is a communication device for establishing a connection to a network, such as a communication card for a wired or wireless LAN, Bluetooth (brand) or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), modems of various types for communication, or the like. 3. Summary

[0188] As described above, the information processing device 10 according to the embodiment of the present disclosure comprises the control unit 150, which dynamically determines a pattern of the rotational movement of the virtual object 30 to be displayed or projected based on the positional relationship between the virtual object 30 and the operating object 20, which is used to operate the virtual object 30.

[0189] This configuration allows for more flexible and intuitive rotation operations on virtual objects.

[0190] The technical scope of this disclosure is not limited to the examples described above in the suitable, detailed embodiment of this disclosure with reference to the accompanying drawings. It is evident that those skilled in the art in the field of this disclosure could have invented various alternative embodiments or modifications within the scope of the technical concept described in the claims. It is self-evident that these examples also fall within the technical scope of this disclosure.

[0191] Furthermore, the individual steps of the processes described in this disclosure do not necessarily have to be carried out in the chronological order shown in the flowchart or sequence diagram. For example, the individual execution steps of the processes to be carried out by the devices can be performed in a different order than shown or in parallel.

[0192] Furthermore, the processes described in the present disclosure, which are to be carried out by the devices, can be executed according to programs stored in a non-volatile, computer-readable storage medium. At the time of execution by a computer, the programs are loaded into RAM and then executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, flash memory, or the like. In addition, the programs can also be distributed, for example, via a network without using the storage medium.

[0193] Furthermore, the advantages described herein are merely descriptive or illustrative and are not intended to be limiting. In other words, the technology according to the present disclosure may, in addition to or instead of the advantages described above, provide other advantages which will be apparent to a person skilled in the art from the description herein.

[0194] It should be noted that the following configurations are also included in the technical scope of this disclosure. (1) Information processing device comprising: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object. (2) Information processing device according to point (1), wherein The control unit determines at least one center of rotation of the virtual object based on the positional relationship between the virtual object and the control object. (3) Information processing device according to point (2), wherein, If it is determined that the virtual object and the control object have no contact with each other, the control unit sets the rotation center of the virtual object to a default point preset in the virtual object. (4) Information processing device according to point (3), wherein The standard point encompasses a center point or center of gravity of the virtual object. (5) Information processing device according to a point from point (2) to (4), wherein, If it is determined that the virtual object and the control object are in contact with each other, the control unit determines the rotation center of the virtual object based on the position of the control object. (6) Information processing device according to point (5), wherein, wherein, if it is determined that the virtual object and the control object are in contact with each other, the control unit specifies the rotation center of the virtual object at a contact point between the virtual object and the control object or at a reference position of the control object. (7) Information processing equipment according to a point from point (2) to (6), wherein The control unit also determines a rotation scaling factor of the virtual object based on the positional relationship between the virtual object and the control object. (8) Information processing device according to point (7), wherein, If it is determined that the virtual object and the control object are not in contact with each other, the control unit sets the rotation scaling factor of the virtual object higher than in the case where it is determined that the virtual object and the control object are in contact with each other. (9) Information processing equipment according to a point from point (2) to (8), wherein The control unit also determines a rotation axis of the virtual object based on the positional relationship between the virtual object and the control object. (10) Information processing device according to point (9), wherein the control object comprises a first control object and a second control object and The control unit determines the axis of rotation of the virtual object based on the positional relationship between the virtual object and the first control object, and causes the virtual object to rotate in response to an operation by the second control object. (11) Information processing device according to point (2), wherein, If it is determined that the control object is located within the virtual object, the control unit determines the rotation center of the virtual object based on the position of the control object. (12) Information processing equipment according to a point from point (1) to (11), wherein The control unit determines the amount of rotation of the virtual object based on an angle or angular velocity of the control object. (13) Information processing device according to point (12), wherein, If a change in the angle or angular velocity of the control object exceeds a threshold, the control unit causes the virtual object to rotate by a predetermined angle. (14) Information processing equipment according to a point from point (1) to (13), wherein The control object is a user's hand. (15) Information processing equipment according to a point from point (1) to (13), wherein The object being controlled is a control device. (16) Information processing equipment according to a point from point (1) to (15), wherein The information processing device is a display mounted on the head. (17) Data processing procedures, in total: Dynamically determining a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object. (18) Computer-readable non-volatile storage medium for storing a program to enable a computer to function as an information processing device, the information processing device comprising: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object. (19) Information processing device, comprising: an input unit that receives an input operation on a virtual object; and a communication unit that transmits information about the input process, wherein a pattern of rotational movement of the virtual object is determined based on a positional relationship between the virtual object and the information processing device and the information about the input process. Reference symbol list 1 system 10 Information processing device 110 Communication unit 120 recording units 130 display unit 140 Audio input / output unit 150 control unit 20 Control object 22 Hand 24 Control device 210 Communication unit 220 input unit 30 virtual objects 35 Turning center QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 6939801

[0003]

Claims

Information processing device comprising: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and an operating object used to operate the virtual object. Information processing device according to claim 1, wherein the control unit determines at least one center of rotation of the virtual object based on the positional relationship between the virtual object and the control object. Information processing device according to claim 2, wherein, when it is determined that the virtual object and the control object have no contact with each other, the control unit sets the center of rotation of the virtual object to a standard point preset in the virtual object. Information processing device according to claim 3, wherein the standard point comprises a center point or a center of gravity of the virtual object. Information processing device according to claim 2, wherein, when it is determined that the virtual object and the control object are in contact with each other, the control unit determines the center of rotation of the virtual object based on the position of the control object. Information processing device according to claim 5, wherein, when it is determined that the virtual object and the control object are in contact with each other, the control unit determines the center of rotation of the virtual object at a contact point between the virtual object and the control object or at a reference position of the control object. Information processing device according to claim 2, wherein the control unit further determines a rotation scaling factor of the virtual object based on the positional relationship between the virtual object and the control object. Information processing device according to claim 7, wherein when it is determined that the virtual object and the control object have no contact with each other, the control unit sets the rotation scaling factor of the virtual object higher than in the case where it is determined that the virtual object and the control object are in contact with each other. Information processing device according to claim 2, wherein the control unit further determines a rotation axis of the virtual object based on the positional relationship between the virtual object and the control object. Information processing device according to claim 9, wherein the control object comprises a first control object and a second control object, and the control unit determines the axis of rotation of the virtual object based on the positional relationship between the virtual object and the first control object and causes the virtual object to rotate in response to operation by the second control object. Information processing device according to claim 2, wherein, when it is determined that the control object is located in the virtual object, the control unit determines the center of rotation of the virtual object based on the position of the control object. Information processing device according to claim 1, wherein the control unit determines the amount of rotation of the virtual object based on an angle or angular velocity of the control object. Information processing device according to claim 12, wherein if a change in the angle or angular velocity of the control object exceeds a threshold value, the control unit causes the virtual object to rotate by a predetermined angle. Information processing device according to claim 1, wherein the control object is a user's hand. Information processing device according to claim 1, wherein the control object is a control device. Information processing device according to claim 1, wherein the information processing device is a display mounted on the head. Information processing method, including: dynamically determining a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and a control object used to operate the virtual object. A computer-readable non-volatile storage medium that stores a program to cause a computer to function as an information processing device, wherein the information processing device comprises: a control unit that dynamically determines a pattern of rotational movement of a virtual object to be displayed or projected based on a positional relationship between the virtual object and an operator object used to operate the virtual object.

Images