Information processing apparatus capable of smoothly allocating functions desired by user to operation terminal, control method for information processing apparatus, storage medium, and program product

Abstract

The present invention provides an information processing apparatus, a control method of the information processing apparatus, a storage medium, and a program product capable of smoothly assigning a function desired by a user to an operation terminal. The information processing apparatus is communicatively connected to an operation terminal including an input unit that performs an input operation for an image of a virtual space and performs a specified function in response to the input operation. The information processing apparatus includes at least one processor and a memory connected to the processor, the memory storing instructions that, when executed by the processor, cause the processor to function as a position obtaining unit that obtains position information related to a position of the operation terminal, and a changing unit that changes the position information related to the position of the operation terminal. And a function changing unit capable of changing a function of the input unit to be performed by the input operation according to the position information obtained by the position obtaining unit.

Description

Technical Field

[0001] The present invention relates to an information processing device, a control method of the information processing device and a storage medium. Background Art

[0002] In recent years, the technological innovation of "virtual reality (VR)" has attracted attention. With this technological innovation, VR devices such as head-mounted displays (HMD) have become more and more popular. In the virtual space displayed on the VR device, various contents such as games and photo shooting can be executed or used, for example. In the case of executing or using the content, an operation terminal such as a controller operated by hand is used. Japanese Patent Laid-Open No. 2023-31290 discloses a game system including an HMD and a controller communicatively connected to the HMD. In the game system disclosed in Japanese Patent Laid-Open No. 2023-31290, the posture of the user holding the controller is detected. In addition, as a posture, in the case of detecting a posture in which the user brings the controller close to his or her face, an object on a line extending from the end of the barrel of a weapon held by the user in the virtual space can be displayed in an enlarged manner. In addition, Japanese Patent Laid-Open No. 2016-81408 discloses that a device other than a dedicated VR controller can be used as a controller for the HMD.

[0003] In an HMD, a user can change the position of the user's own avatar in a virtual space by operating the controller while holding (holding) the controller. In addition, a camera can also be used as a controller. In this case, since the camera is used for photography, a controller separate from the camera needs to be used to change the avatar position, that is, the user needs to switch between the camera and the separate controller. For this reason, it is desirable that a single controller can perform various functions. Summary of the invention

[0004] The present invention provides an information processing device capable of smoothly allocating functions desired by a user to an operation terminal, a control method of the information processing device, and a storage medium.

[0005] Therefore, the present invention provides an information processing device, which can be communicatively connected to an operation terminal including an input unit, the input unit performs an input operation on an image of a virtual space, and performs a specified function in response to the input operation, the information processing device includes: at least one processor; and a memory connected to the processor, the memory storing instructions, which when executed by the processor causes the processor to function as: a position obtaining unit, which obtains position information related to the position of the operation terminal; and a changing unit, which is capable of changing the function of the input unit to be performed by the input operation according to the position information obtained by the position obtaining unit.

[0006] According to the present invention, functions desired by a user can be smoothly assigned to an operation terminal.

[0007] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A , Figure 1B and Figure 1C is a diagram showing an example of a usage state of the information processing apparatus according to the first embodiment.

[0009] Figure 2A is a block diagram showing an example of a hardware configuration of an information processing device, Figure 2B is a block diagram showing an example of the hardware configuration of an operation terminal.

[0010] Figure 3 is a block diagram showing an example of a software configuration (functional configuration) of an information processing device.

[0011] Figure 4 : is a diagram showing an example of data obtained by the operation terminal position information obtaining module of the information processing device.

[0012] Figure 5A , Figure 5B and Figure 5C is a diagram showing an example of allocating functions to the operation units of the operation terminals according to the determination results obtained by the allocation determination module of the information processing device.

[0013] Figure 6 is a flowchart illustrating processing performed by the information processing apparatus.

[0014] Figure 7 is a block diagram showing an example of a software configuration (functional configuration) of an information processing apparatus according to the second embodiment.

[0015] Figure 8 is a flowchart illustrating processing performed by the information processing apparatus according to the second embodiment. DETAILED DESCRIPTION

[0016] The present invention will be described in detail below with reference to the accompanying drawings showing embodiments of the present invention.

[0017] Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the configurations described in the following embodiments are only examples, and the scope of the present invention is not limited to the configurations described in the embodiments. For example, each unit (each component) constituting the present invention can be replaced with a unit (component) having any configuration capable of performing the same function. In addition, any component can be added. In addition, any two or more than two configurations (features) of the embodiments can be combined.

[0018] Below, we will refer to Figures 1A to 6 A first embodiment of the present invention will be described. Figure 1A , Figure 1B and Figure 1C is a diagram showing an example of a usage state of the information processing apparatus according to the first embodiment. Figure 1A : is a diagram showing a state where the operation terminal is located within the viewing angle of the information processing apparatus. Figure 1B : is a diagram showing a state in which the operation terminal is located within the viewing angle of the information processing apparatus and the user is looking at the viewfinder of the operation terminal. Figure 1C : is a diagram showing a state in which the operation terminal is located outside the viewing angle of the information processing device. The information processing device 101 is a head-mounted display (HMD) that can be mounted on the head of the user 103. The information processing device 101 can be used by being communicatively connected to the operation terminal 102. Hereinafter, the state in which the information processing device 101 and the operation terminal 102 are communicatively connected to each other may be referred to as a "communication state". The operation terminal 102 is a camera device such as a single-lens reflex camera or a mirrorless camera, and includes a viewfinder photography function and a live view photography function in the first embodiment. It should be noted that the operation terminal 102 is not limited to a camera device, and may be, for example, a smartphone or a mobile controller, etc. As Figure 1A As shown in , the user 103 wears the information processing device 101 on his or her head (hereinafter referred to as the "wearing state") and places the operation terminal 102 in front of his or her eyes. At this time, the operation terminal 102 becomes a state in which the operation terminal 102 is located within the visual field of the information processing device 101. Figure 1B As shown in , the user 103 in the wearing state is looking at the viewfinder of the operation terminal 102. At this time, the operation terminal 102 changes to a state where the operation terminal 102 is located within the viewing angle of the information processing device 101 and the user 103 is looking at the viewfinder of the operation terminal 102. Figure 1C As shown in FIG. 1 , the user 103 in the wearing state places the operation terminal 102 in front of his or her chest. At this time, the operation terminal 102 becomes a state where the operation terminal 102 is located outside the visual angle of the information processing device 101.

[0019] Figure 2A is a block diagram showing an example of a hardware configuration of an information processing device, Figure 2B is a block diagram showing an example of the hardware configuration of the operation terminal. Figure 2AAs shown in FIG. 1 , the information processing device 101 includes a central processing unit (CPU) 201, a read-only memory (ROM) 202, a random access memory (RAM) 203, a sensor 204, an input detection unit 205, a communication interface (communication I / F) 206, a recording unit 207, a photographing unit (camera unit) 208, a display unit 209, and an operation unit 210. These hardware included in the information processing device 101 are communicatively connected to each other via a bus 211. The CPU 201 is a system control unit (computer) that controls the entire information processing device 101. The ROM 202 is a read-only memory that stores programs and parameters that do not need to be changed, such as basic programs and initial data. The RAM 203 is a memory for temporarily storing input information, and calculation results in information processing and image processing. The sensor 204 detects the position and posture of the information processing device 101. The sensor 204 is not particularly limited, and may be, for example, a gyro sensor. The input detection unit 205 accepts input of required data. The required data is not particularly limited, and may be, for example, an operation performed on the operation unit 212 of the operation terminal 102 received through the communication I / F 206, etc. The communication I / F 206 is an interface unit capable of transmitting and receiving data to and from the operation terminal 102 and the cloud. Connection forms input / output through the communication I / F 206 include local connection through a universal serial bus (USB) or Bluetooth (registered trademark), etc., and Internet connection through Ethernet or Wi-Fi (registered trademark), etc.

[0020] The recording unit 207 is a device capable of writing and reading various types of information. The recording unit 207 is not particularly limited, and may be, for example, a hard disk or a memory card built into or external to the information processing device 101, or a memory card, a removable disk, or an IC card, etc. that can be installed or removed from the information processing device 101. In addition, the recording unit 207 includes (stores) an information processing program for causing the CPU 201 to execute the various units and modules of the information processing device 101 (a control method for the information processing device). The information processing program is read out from the recording unit 207, loaded into the RAM 203, and executed by the CPU 201. It should be noted that the information processing program may be stored in the ROM 202. In addition, the recording unit 207 is capable of recording the required data used by the information processing program. The photographing unit 208 is capable of photographing images. The photographing unit 208 is capable of performing image recognition on the photographed real video image, and tracking the hand joints of the user 103 or the hand movements of the user 103. In addition, the photographing unit 208 photographs the operation terminal 102 and detects the position of the operation terminal 102 based on the photographed image. The display unit 209 is a stereoscopic display corresponding to the eyes of the user 103, and displays an image of a virtual space. The operation unit 210 controls the input to the information processing device 101. The operation unit 210 is not particularly limited, and examples thereof include a power button, a menu button, a selection button, and an OK button. It should be noted that in the case of reducing the size and weight of the information processing device 101, it is preferable to separate the information processing system including the CPU 201, the ROM 202, the RAM 203, the sensor 204, the input detection unit 205, the communication I / F 206, the recording unit 207, and the operation unit 210 from the imaging and display system outside the information processing system. The information processing system is arranged as an external component (external configuration), and the imaging and display system is built into the information processing device 101.

[0021] like Figure 2B As shown in , the operation terminal 102 includes an operation unit (input unit) 212, a sensor 213, an input detection unit 214, and a communication I / F 215. These hardware included in the operation terminal 102 are communicatively connected to each other via a bus 216. The operation terminal 102 is provided with a plurality of operation units 212, and each operation unit 212 is configured with any one of a mechanical button, a mechanical dial, a touch panel, and a touch pad. In addition, by operating the operation unit 212, operation instructions, commands, etc. from the user 103 are obtained. In addition, by operating each operation unit 212 in a communication state, a desired input operation is performed on the image of the virtual space displayed on the display unit 209 of the information processing device 101. Through such an input operation, a specified function is performed.

[0022] For example, in Figure 1A In the state shown in , a field of view video image 104 similar to what the user 103 sees in the real space is displayed on the display unit 209 of the information processing device 101. The field of view video image 104 includes a virtual object 105 of the operation terminal 102 corresponding to the position, posture, and appearance of the operation terminal 102 held (held) by the hand of the user 103 in the real space. In addition, in this state, when the user 103 releases the shutter of the operation terminal 102, it is possible to take a photo or video (moving image) of the virtual space corresponding to the position of the virtual object 105. In the case of body tracking by the information processing device 101, the hand 106 of the avatar of the user 103 is displayed in the field of view video image 104 without any sense of incongruity, so that the user 103 feels as if he or she is holding the operation terminal 102 in the virtual space. In addition, the user 103 is able to change various settings such as photographic conditions on the touch panel of the operation terminal 102. Figure 1B In the state shown in , when the user 103 releases the shutter of the operation terminal 102, it is possible to take a photo or video (moving image) of the virtual space. The field of view video image 104 is displayed in full screen, as if the field of view video image 104 is being taken by looking through the viewfinder. Figure 1C In the state shown in , the user 103 can move in the virtual space by operating the operation unit 212 of the operation terminal 102. The field of view video image 108 includes a guide display 109 for moving in the virtual space. The guide display 109 indicates the operation content in the virtual space when the user 103 operates the operation unit 212 of the operation terminal 102 held by the user 103. In this way, using the information processing device 101, the function of the input from the operation terminal 102 can be allocated according to the position of the operation terminal 102.

[0023] It should be noted that the operation terminal 102 is not limited to including a plurality of operation units 212, but may include at least one operation unit 212. The sensor 213 obtains the position and posture of the operation terminal 102, and obtains changes in the position and posture of the operation terminal 102. The sensor 213 is not particularly limited, and may be, for example, an acceleration sensor. The input detection unit 214 accepts input of required data. The required data is not particularly limited, and may be, for example, instructions issued by an operation on the operation unit 212. The communication I / F 215 is an interface unit capable of sending data to the information processing device 101 and receiving data from the information processing device 101. The connection form of input / output through the communication I / F 215 includes a local connection through a universal serial bus (USB) or Bluetooth, etc., and an Internet connection through Ethernet or Wi-Fi, etc.

[0024] Figure 3 is a block diagram showing an example of a software configuration (functional configuration) of an information processing device. Figure 3 As shown in , the information processing device 101 includes an operation terminal position information acquisition module 301, an allocation judgment module 302, an allocation module 303, and a display switching module 304. The operation terminal position information acquisition module 301 (position acquisition unit) obtains position information related to the position of the operation terminal 102 (position acquisition step). In the first embodiment, the operation terminal position information acquisition module 301 can obtain the position information of the operation terminal 102 by applying image recognition processing to the image captured by the photographing unit 208 of the information processing device 101. In addition, the operation terminal position information acquisition module 301 can also obtain the position information of the operation terminal 102 from the sensor 213 of the operation terminal 102. In addition, by applying image recognition processing to the image captured by the photographing unit 208 of the information processing device 101, the operation terminal position information acquisition module 301 can determine whether the operation terminal 102 exists within the viewing angle of the photographing unit 208, that is, it can determine the position of the operation terminal 102. Figure 4 2 is a diagram showing an example of data obtained by an operation terminal position information obtaining module of an information processing device. Figure 4 As shown in , the data obtained by the operation terminal position information obtaining module includes the position information of the operation terminal 102 and whether the operation terminal 102 is within the field of view of the photographic unit 208 of the information processing device 101 as the HMD. The position information of the operation terminal 102 is expressed as (x, y, z) with the position of the photographic unit 208 as a reference point. As a result, the distance between the operation terminal 102 and the photographic unit 208 (information processing device 101) can be obtained. It should be noted that x and y are positions in the horizontal direction, and z is a position in the vertical direction. In Figure 4 In , as an example, (x, y, z) = (0, 100, 100). Whether the information is within the viewing angle is indicated by "○" indicating that the operation terminal 102 is within the viewing angle or "×" indicating that the operation terminal 102 is not within the viewing angle. Figure 4 In FIG. 1 , as an example, the information on whether or not the view is within the view is indicated by “○”. It should be noted that the information on whether or not the view is within the view can also be included in the position information of the operation terminal 102 as a part of the position information of the operation terminal 102 .

[0025] Based on the position information of the operation terminal 102, the allocation judgment module 302 can judge whether to change the function of the operation unit 212 performed by the input operation on the operation unit 212, that is, can judge which function is to be allocated to the operation unit 212 (changing step). In the first embodiment, the allocation judgment module 302 judges which function is to be allocated to the operation unit 212 in response to the length of the distance between the operation terminal 102 and the information processing device 101 and the presence or absence of the operation terminal 102 within the angle of view of the photographing unit 208.

[0026] The allocation module 303 allocates (changes) a function to the operation unit 212 according to the judgment result obtained by the allocation judgment module 302. It should be noted that in the first embodiment, the allocation judgment module 302 and the allocation module 303 (serving as a change unit) are separate components, but are not limited to this and may be one component (allocation and change module). Figure 5A , Figure 5B and Figure 5C is a diagram showing an example of allocating functions to the operation units of the operation terminals according to the determination results obtained by the allocation determination module of the information processing device. Figure 5A : is a diagram showing an example of function allocation for photography (for viewfinder photography function). Figure 5B : is a diagram showing an example of function allocation for live view photography (for live view photography function). Figure 5C 201 is a diagram showing an example of function allocation for moving angles (for moving within a virtual space). In a case where the operation terminal 102 is within the viewing angle of the photographing unit 208 and the distance between the operation terminal 102 and the information processing device 101 is less than or equal to a threshold value (see Figure 1B ), the function of the operation unit 212 used as an input unit is set to Figure 5A The functional allocation for photography is shown in FIG. Figure 5A As shown in , the operation unit 212 includes a button A, a button B, a dial A, and the like. When the button A is operated, a function for performing photography the same as photography performed by looking through the viewfinder is assigned to the button A. Furthermore, when the button B is operated, a function for zooming when performing photography is assigned to the button B. In addition, when the dial A is operated, a function for performing setting changes such as changing the settings of the photographing conditions is assigned to the dial A.

[0027] In the case where the operation terminal 102 is within the visual angle of the photographing unit 208 and the distance between the operation terminal 102 and the information processing device 101 is not less than or equal to the threshold value (see Figure 1A ), the function of the operation unit 212 is set to Figure 5B The functional allocation for live view photography is shown in FIG. Figure 5BAs shown in , the operation unit 212 includes a button A, a button B, a dial A, a touch panel, and the like. When the button A is operated, a function for performing live view photography is assigned to the button A. Furthermore, when the button B is operated, a function for performing zooming when performing live view photography is assigned to the button B. In addition, when the dial A is operated, a function for performing setting changes such as live view shooting conditions is assigned to the dial A. In addition, when the touch panel is operated, a function for performing live view photography is assigned to the touch panel.

[0028] In the case where the operation terminal 102 is not within the field of view of the photographing unit 208 (see Figure 1C ), the function of the operation unit 212 is set to Figure 5C The function assignment for moving the angle is shown in FIG. Figure 5C As shown in , the operation unit 212 includes button A, button B, dial A, etc. When button A is operated, a function for performing rightward movement is assigned to button A. Furthermore, when button B is operated, a function for performing leftward movement is assigned to button B. In addition, when dial A is operated, a function for performing direction change for changing direction is assigned to dial A.

[0029] The display switching module 304 switches the display content on the display unit 209 according to the function allocation performed by the allocation module 303. Figure 1A As shown in , in the case of the function allocation for live view photography, the virtual object 105 of the operation terminal 102 photographed by the photography unit 208 is displayed on the image of the virtual space as the background. Figure 1B As shown in , in the case of the function allocation for photography, an image of a virtual space as a background (field of view video image 107) is displayed. Figure 1C As shown in , in the case of function assignment for moving the angle, the operation unit 212 and the function (movement guide) of the operation unit 212 set or changed by the assignment module 303 are displayed on the image of the virtual space as the background.

[0030] Figure 6 is a flowchart showing the processing performed by the information processing device. Figure 6 As shown in FIG. 5 , in step S601 , the operation terminal location information obtaining module 301 obtains the location information of the operation terminal 102 .

[0031] In step S602, the allocation determination module 302 determines (judgments) whether the operation terminal 102 is located within the field of view of the photographing unit 208 based on the position information of the operation terminal 102 obtained in step S601. As a result of the determination in step S602, if it is determined that the operation terminal 102 is located within the field of view of the photographing unit 208, Figure 6The processing shown in goes to step S603. On the other hand, as a result of the determination in step S602, in the case where it is determined that the operation terminal 102 is not within the field of view of the photographing unit 208, Figure 6 The processing shown in enters step S608.

[0032] In step S603, the allocation determination module 302 determines whether the distance between the operation terminal 102 and the photographing unit 208 is less than or equal to a threshold value based on the position information of the operation terminal 102 obtained in step S601. As a result of the determination in step S603, if it is determined that the distance between the operation terminal 102 and the photographing unit 208 is less than or equal to the threshold value, Figure 6 The processing shown in goes to step S604. On the other hand, as a result of the determination in step S603, in the case where it is determined that the distance between the operation terminal 102 and the imaging unit 208 is not less than or equal to the threshold value, Figure 6 The process shown in proceeds to step S606. It should be noted that the threshold value is preferably set in advance, and can be changed appropriately.

[0033] In step S604, the allocation module 303 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5A Function allocation for photography is shown in FIG.

[0034] In step S605, the display switching module 304 switches the display content on the display unit 209 to the field of view video image 107 (see Figure 1B ). The field of view video image 107 is displayed in full screen. After executing step S605, Figure 6 The processing shown in is completed.

[0035] In step S606, the allocation module 303 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5B 2 shows the functional allocation for live view photography.

[0036] In step S607, the display switching module 304 switches the display content on the display unit 209 to the field of view video image 104 (see Figure 1A ). The field of view video image 104 includes a background image. After executing step S607, Figure 6 The processing shown in is completed.

[0037] In step S608, the allocation module 303 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5C The functional assignment for shifting the angle is shown in FIG.

[0038] In step S609, the display switching module 304 switches the display content on the display unit 209 to the field of view video image 108 (see Figure 1C ). The field of view video image 108 includes a guide display 109 for moving in the virtual space. After executing step S609, Figure 6 The processing shown in is completed.

[0039] With the information processing device 101 having the above configuration, the functions desired by the user 103 can be smoothly assigned to the operation terminal 102 according to the position of the operation terminal 102 relative to the user 103 while the operation terminal 102 maintains the state in which the operation terminal 102 can be used for photography. In other words, it is possible to make one operation terminal 102 perform various functions without the user 103 having to switch from the operation terminal 102 to another operation terminal. As a result, the user 103 can operate the operation terminal 102 while maintaining the operation terminal 102 to, for example, capture an image and change the position of the user's own avatar in the virtual space.

[0040] Below, we will refer to Figure 7 and Figure 8 The second embodiment of the present invention is described centering on the differences from the first embodiment described above, and descriptions of the same matters will be omitted. The second embodiment is the same as the first embodiment described above except that functions are assigned to the operation unit 212 of the operation terminal 102 according to the line of sight of the user 103.

[0041] Figure 7 1 is a block diagram showing an example of a software configuration (functional configuration) of an information processing device according to the second embodiment. Figure 7As shown in , the information processing device 101 includes an operation terminal position information acquisition module 701, a user line of sight information acquisition module (line of sight acquisition unit) 702, an allocation judgment module 703, and an allocation module 704. As with the operation terminal position information acquisition module 301, the operation terminal position information acquisition module 701 (position acquisition unit) obtains position information related to the position of the operation terminal 102. The user line of sight information acquisition module 702 obtains line of sight information related to the line of sight of the user 103 who is in a wearing state and operates the operation terminal 102. There is no particular limitation on the method of obtaining the line of sight information, and an example of the method includes eye tracking, which detects (tracks) the movement of the user's eyeball by using a camera or an infrared sensor built in the information processing device 101, and uses the detection result. The allocation judgment module 703 determines which function is to be allocated to the operation unit 212 based on the position information of the operation terminal 102 and the line of sight information of the user 103. In the second embodiment, the allocation judgment module 703 judges which function is to be allocated to the operation unit 212 according to the presence or absence of the operation terminal 102 within the viewing angle of the photographing unit 208 and the presence or absence of the operation terminal 102 in the direction of the line of sight of the user 103. The allocation module 704 allocates the function to the operation unit 212 according to the judgment result obtained by the allocation judgment module 703. In the second embodiment, in the case where the operation terminal 102 is located within the viewing angle of the photographing unit 208 and the operation terminal 102 is located in the direction of the line of sight of the user 103, the allocation module 704 sets the allocation of the function of the operation unit 212 to the operation terminal 102. Figure 5A In addition, when the operation terminal 102 is located within the visual angle of the photographing unit 208 and the operation terminal 102 is not located in the visual direction of the user 103, the allocation module 704 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5C The assignment of functions for shifting angles is shown in FIG.

[0042] Figure 8 1 is a flowchart showing the processing performed by the information processing device according to the second embodiment. Figure 8 As shown in FIG. 8 , in step S801 , the operation terminal location information obtaining module 701 obtains the location information of the operation terminal 102 .

[0043] In step S802 , the user sight line information obtaining module 702 obtains the sight line information (sight line position) of the user 103 .

[0044] In step S803, the allocation determination module 703 determines whether the operation terminal 102 is located within the field of view of the photographing unit 208 based on the position information of the operation terminal 102 obtained in step S801. As a result of the determination in step S803, if it is determined that the operation terminal 102 is located within the field of view of the photographing unit 208, Figure 8 The processing shown in goes to step S804. On the other hand, as a result of the determination in step S803, in the case where it is determined that the operation terminal 102 is not within the field of view of the photographing unit 208, Figure 8 The processing shown in is completed.

[0045] In step S804, the allocation judgment module 703 judges whether the operation terminal 102 is located in the direction of the line of sight of the user 103 (whether the operation terminal 102 is located in front of the line of sight of the user 103). This judgment is made based on the position information of the operation terminal 102 obtained in step S801 and the line of sight information of the user 103 obtained in step S802. Then, as a result of the judgment in step S804, in the case where it is judged that the operation terminal 102 is located in the line of sight of the user 103, Figure 8 The processing shown in goes to step S805. On the other hand, as a result of the determination in step S804, in the case where it is determined that the operation terminal 102 is not located in the direction of the line of sight of the user 103, Figure 8 The processing shown in enters step S806.

[0046] In step S805, the allocation module 704 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5A Function allocation for photography is shown in FIG.

[0047] In step S806, the allocation module 704 sets the allocation of the function of the operation unit 212 of the operation terminal 102 to Figure 5C The functional assignment for shifting the angle is shown in FIG.

[0048] As described above, in the second embodiment, the information processing device 101 can change the function of the operation unit 212 according to the position information of the operation terminal 102 and the line of sight information of the user 103. As a result, the functions desired by the user 103 can be assigned to the operation terminal 102 more smoothly, thereby improving the usability of the operation terminal 102.

[0049] Although the preferred embodiments of the present invention are described above, the present invention is not limited to the above-described embodiments, and various modifications and changes within the scope of the gist of the present invention are possible. The information processing device 101 is not limited to a configuration in which the CPU 201, ROM 202, RAM 203, sensor 204, input detection unit 205, communication I / F 206, recording unit 207, photographing unit 208, display unit 209, and operation unit 210 are integrated into a single device, that is, a configuration in which the information processing device 101 is formed as one unit. For example, the CPU 201, ROM 202, and RAM 203 may be configured as one unit (one server), and the sensor 204, input detection unit 205, communication I / F 206, recording unit 207, photographing unit 208, display unit 209, and operation unit 210 may be configured as one unit. In addition, the information processing device 101 may include a switching module (switching unit) capable of switching between a plurality of operation modes of the information processing device 101 within an image of a virtual space. The operation mode is not particularly limited, and examples of the operation mode include a mode suitable for moving in a dark environment. In addition, using the display unit 209, the image of the virtual space can be changed to an image including a guide or auxiliary message, etc. according to each operation mode. As a result, the usability of the information processing device 101 is improved.

[0050] Other embodiments

[0051] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0052] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

[0053] This application claims the benefit of priority from Japanese Application No. 2023-155922, filed on September 21, 2023, which is hereby incorporated by reference in its entirety.

Claims

1. An information processing device that is communicably connected to an operation terminal including an input unit that performs an input operation for an image of a virtual space and performs a specified function in response to the input operation, the information processing device comprising: at least one processor; as well as a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to: a position obtaining unit, which obtains position information related to the position of the operation terminal; as well as a changing unit capable of changing a function of the input unit to be performed by the input operation according to the position information obtained by the position obtaining unit.

2. The information processing device according to claim 1, wherein: The position obtaining unit obtains the distance between the operation terminal and the information processing device as the position information, and The changing unit can change a function of the input unit according to the length of the distance.

3. The information processing device according to claim 2, wherein: A camera including a viewfinder photography function and a live view photography function is used as the operation terminal, and The changing unit changes the function of the input unit to be used for the viewfinder photography function when the distance is less than or equal to a threshold, and changes the function of the input unit to be used for the live view photography function when the distance is not less than or equal to the threshold.

4. The information processing device according to claim 1, further comprising: A camera unit, configured to take a picture of the operation terminal; and wherein the position obtaining unit obtains the position of the operation terminal within the viewing angle of the camera unit as the position information, and The changing unit can change the function of the input unit according to the position of the operation terminal within the angle of view of the imaging unit.

5. The information processing device according to claim 4, wherein: A camera including a viewfinder photography function and a live view photography function is used as the operation terminal, and The changing unit changes the function of the input unit for a viewfinder photography function or a live view photography function when the position of the operation terminal is within the field of view of the camera unit, and changes the function of the input unit for movement within the virtual space when the position of the operation terminal is not within the field of view of the camera unit.

6. The information processing device according to claim 1, wherein: causing the processor to further function as a sight line obtaining unit that obtains sight line information related to a sight line of a user operating the operation terminal, and The changing unit can change the function of the input unit according to the position information and the line of sight information obtained by the line of sight obtaining unit.

7. The information processing device according to claim 6, wherein: A camera including a viewfinder photography function and a live view photography function is used as the operation terminal, and When the line of sight information indicating that the operation terminal is located in front of the user's line of sight is obtained, the changing unit changes the function of the input unit to be used for the viewfinder photography function; when the line of sight information indicating that the operation terminal is not located in front of the user's line of sight is obtained, the changing unit changes the function of the input unit to be used for moving in the virtual space.

8. The information processing device according to claim 1, further comprising: A display unit is configured to display an image of the virtual space.

9. The information processing device according to claim 8, wherein: causing the processor to further function as a switching unit capable of switching between a plurality of operation modes of the information processing device within the image of the virtual space, and The display unit changes the image of the virtual space to an image corresponding to each operation mode.

10. The information processing device according to claim 8, further comprising: A camera unit, configured to take a picture of the operation terminal; and The display unit is capable of displaying the virtual object of the operation terminal photographed by the camera unit on the image of the virtual space.

11. The information processing device according to claim 8, wherein: The display unit can display the function of the input unit changed by the changing unit on the image of the virtual space.

12. The information processing device according to claim 8, wherein: The information processing device is a head-mounted display.

13. The information processing device according to any one of claims 1 to 12, wherein: One of a camera, a smartphone, and a mobile controller is used as the operation terminal.

14. The information processing device according to any one of claims 1 to 12, wherein: The operation terminal includes a plurality of the input units, and at least one of the plurality of the input units is configured with a mechanical button, a mechanical dial, a touch panel, and a touch pad.

15. A control method for controlling an information processing device, the information processing device being communicably connected to an operation terminal including an input unit, the input unit performing an input operation for an image of a virtual space and performing a designated function in response to the input operation, The control method comprises: a location obtaining step of obtaining location information related to the location of the operation terminal; as well as a changing step capable of changing a function of the input unit performed by the input operation according to the position information obtained in the position obtaining step. 16 . A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 15 .

17. A computer program product comprising a program for causing a computer to execute the control method according to claim 15.

Images