System, information processing device, processing method, and program

Abstract

This system comprises: a display; a measurement unit that measures a direction of a controller being present with respect to the display; and a processing unit that displays on the display an image including a first object moved according to a user input to the controller without being moved according to the measured direction and a second object presenting information related to at least one of a user operating the controller and the first object. The processing unit changes at least one of a display position and an orientation of the second object according to the measured direction.

Description

[Technical Field]

[0001] The present disclosure relates to a system, an information processing device, a processing method, and a program. [Background technology]

[0002] Conventionally, information processing devices that display menu screens and game-related screens have been known (see, for example, Japanese Patent Application Laid-Open No. 2019-197585). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-197585 Summary of the Invention [Problem to be solved by the invention]

[0004] The object is to improve the visibility of the displayed image and / or improve the operability of the controller compared to the above-mentioned known devices. [Means for solving the problem]

[0005] A system according to one embodiment includes a display, a measurement unit that measures a direction in which a controller is present relative to the display, and a processing unit that displays on the display an image including a first object that is not moved in accordance with the measured direction but is moved in accordance with a user input to the controller, and a second object that presents information about at least one of a user operating the controller and the first object. The processing unit changes at least one of a display position and an orientation of the second object in accordance with the measured direction.

[0006] According to this configuration, the second object is displayed at a display position and / or orientation according to the direction in which the controller (and the user operating the controller) is located relative to the display, thereby improving the visibility of the second object for the user. Furthermore, the user can operate the second object while viewing it, which is displayed at a display position and / or orientation according to the direction in which the controller (and the user operating the controller) is located relative to the display, thereby improving the operability of the controller for the user.

[0007] The processing unit may change the correspondence between the user input to the controller and the operation content for the first object according to the measured direction. With this configuration, the operation content of the first object corresponding to the user input to the controller is changed according to the direction in which the controller (and the user operating the controller) is present relative to the display, thereby improving the operability of the controller for the user.

[0008] The controller may include a direction input unit. The processing unit may change the correspondence between the direction input to the direction input unit and the movement direction of the first object on the screen. With this configuration, even if the same user input is made to the direction input unit, the operation content for the first object changes depending on the direction in which the controller (and the user operating the controller) is located relative to the display, thereby improving the operability of the controller for the user.

[0009] The controller may include a sensor that detects movement. The processing unit may change the correspondence between the direction of the detected movement and the direction of movement of the first object on the screen. With this configuration, even if a user input causes the same movement of the controller, the operation content for the first object is changed depending on the direction in which the controller (and the user operating the controller) is present relative to the display, thereby improving the operability of the controller for the user.

[0010] The display may be rectangular. The processing unit may maintain the correspondence as long as it is determined that the controller is present within a range corresponding to one side of the display, and may change the correspondence when it is determined that the controller is present within a range corresponding to another side of the display. With this configuration, by maintaining and changing the correspondence in association with each side of the rectangular display, it is possible to make the user's recognition clearer and improve the operability of the controller for the user.

[0011] The processing unit may change the correspondence after changing the display position or orientation of the second object. With this configuration, by changing the display position or orientation of the second object, the user can recognize that the system has detected a change in the direction in which the controller (and the user operating the controller) is located relative to the display. Since the correspondence is then changed, it is possible to reduce the likelihood that the user will feel uncomfortable when operating the controller.

[0012] The processing unit may select a display position of the second object from among a plurality of positions predetermined as display positions of the second object, according to the measured direction. With this configuration, the second object is displayed at one of the plurality of predetermined positions, so that even if the controller (and the user operating the controller) moves, the possibility that the display position of the second object will change over time can be reduced. This can reduce the possibility that visibility for the user will be reduced.

[0013] When displaying the second object for each of a plurality of users, the processing unit may assign the display position of the second object for each user from among a plurality of positions according to the orientation of the controller associated with each user. With this configuration, the display position of the second object can be determined according to the relative positional relationship or arrangement order between the controllers, thereby reducing the sense of discomfort felt by the user even in a limited display space.

[0014] When the display position of the second object corresponding to the measured direction is different from the current display position of the second object, the processing unit may change the display position of the second object to the display position corresponding to the measured direction after a predetermined condition is met. With this configuration, even if the controller (and the user operating the controller) moves frequently, the display position of the second object can be prevented from changing frequently. This can reduce the sense of discomfort felt by the user.

[0015] The predetermined condition may be satisfied when a predetermined time has elapsed. With this configuration, it is possible to determine whether the predetermined condition is satisfied by measuring the time.

[0016] When displaying second objects for three or more users, the processing unit may not only rearrange adjacent second objects but also change the display positions of the second objects in any order according to the measured directions. This configuration makes it possible to display second objects that reflect the arrangement order of the controllers (and the users operating the controllers).

[0017] The first object may not only not be moved according to the measured direction, but also not change its orientation. With this configuration, when multiple users are watching the display and playing a game or application, the display position and orientation of the first object are maintained even if the controller (and the user operating the controller) moves, so it is possible to maintain the operability of the controller while reducing any sense of discomfort felt by the user.

[0018] An information processing device according to another embodiment includes a display, a measurement unit that measures the direction in which a controller is located relative to the display, and a processing unit that displays on the display an image including a first object that is not moved in accordance with the measured direction but is moved in accordance with a user input to the controller, and a second object that presents information about at least one of the user operating the controller and the first object. The processing unit changes at least one of the display position and orientation of the second object in accordance with the measured direction.

[0019] A processing method according to yet another embodiment includes the steps of measuring a direction in which a controller is located relative to a display; displaying on the display an image including a first object that is not moved in accordance with the measured direction but is moved in accordance with user input to the controller, and a second object that presents information about at least one of the user operating the controller and the first object; and changing at least one of the display position and orientation of the second object in accordance with the measured direction.

[0020] According to yet another embodiment, there is provided a program executed on a computer having a display, the program causing the computer to perform the steps of measuring a direction in which a controller is present relative to the display, displaying on the display an image including a first object that is not moved in accordance with the measured direction but is moved in accordance with user input to the controller, and a second object that presents information about at least one of a user operating the controller and the first object, and changing at least one of a display position and an orientation of the second object in accordance with the measured direction. [Effects of the Invention]

[0021] According to the present disclosure, it is possible to further improve the visibility of the displayed image and / or the operability of the controller. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing an example of a hardware configuration of a game device in the system according to the present embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of a hardware configuration of a controller of the system according to the present embodiment. [Figure 4] 1A and 1B are diagrams for explaining the principle of direction measurement in the system according to the present embodiment. [Figure 5] 1 is a schematic diagram showing an example of an antenna module in which a plurality of antenna elements are arranged in one direction. [Figure 6] 1 is a schematic diagram showing an example of an antenna module in which a plurality of antenna elements are arranged in two directions. [Figure 7] FIG. 2 is a schematic diagram showing an example of the configuration of a short-range communication unit in the system according to the present embodiment. [Figure 8] FIG. 2 is a schematic diagram showing an example of a configuration of a frame transmitted by a controller of the system according to the present embodiment. [Figure 9]FIG. 10 is a schematic diagram showing an example of a screen output by the game device of the system according to the present embodiment. [Figure 10] FIG. 10 is a schematic diagram showing another example of a screen output by the game device of the system according to the present embodiment. [Figure 11] 11 is a diagram for explaining an example of a correspondence relationship between a user input to a controller and the operation content of the operation on the example screen shown in FIG. 10. FIG. [Figure 12] 11 is a diagram for explaining an example of a correspondence relationship between a user input to a controller and the operation content of the operation on the example screen shown in FIG. 10. FIG. [Figure 13] 11 is a diagram for explaining another example of the correspondence between a user input to the controller and the operation content of the operation on the example screen shown in FIG. 10. FIG. [Figure 14] FIG. 10 is a schematic diagram showing an example of a user operation definition held by a game device in the system according to the present embodiment. [Figure 15] FIG. 10 is a diagram illustrating an example of a selection process of a user operation definition provided in the game device of the system according to the present embodiment. [Figure 16] FIG. 10 is a schematic diagram showing an example of a screen on which information presentation objects are displayed at a plurality of predetermined positions in the system according to the present embodiment. [Figure 17] 10A and 10B are diagrams illustrating an example of a process for changing the display of an information presentation object in the system according to the present embodiment. [Figure 18] FIG. 2 is a schematic diagram showing an example of a plurality of predetermined positions at which an information presentation object is displayed in the system according to the present embodiment. [Figure 19] 10 is a flowchart showing a processing procedure executed by the game device of the system according to the present embodiment. [Figure 20] 20 is a flowchart showing the procedure of the direction measurement process shown in FIG. 19. DETAILED DESCRIPTION OF THE INVENTION

[0023] The present embodiment will be described in detail with reference to the drawings, in which the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0024] [A. Configuration example] First, an example of the configuration of a system 1 according to the present embodiment will be described.

[0025] In the following description, a game device will be used as an example of an information processing device, but the information processing device is not limited to a game device and any computer such as a smartphone, tablet, personal computer, etc. Furthermore, the information processing device is not limited to a portable device and may be a stationary device.

[0026] 1 is a schematic diagram showing an example of the configuration of a system 1 according to the present embodiment. Referring to FIG. 1, system 1 includes a game device 100 and one or more controllers 200.

[0027] In this specification, the term "controller" is intended to encompass any device that accepts user input, and is not limited to game controllers, but also encompasses general-purpose input devices such as keyboards, mice, and pen tablets, as well as operating devices used for specific purposes.

[0028] The game device 100 exchanges data with each of the controllers 200 using wireless signals. That is, the controllers 200 transmit wireless signals in response to user inputs.

[0029] The controller 200 may be attachable to the game device 100. In this embodiment, the controllers 200 are attached to both sides of the game device 100. When the controller 200 is attached to the game device 100, the game device 100 may be electrically connectable to the controller 200. At this time, data may be exchanged via wired communication. Note that even when the controller 200 is attached to the game device 100, data may be exchanged via wireless communication.

[0030] For convenience of explanation, differences in structure and function between the controllers 200 will not be mentioned, but the structure and function of the controller 200 may differ depending on the side (left / right) on which it is attached to the game device 100.

[0031] The game device 100 has a display 106 for displaying any image, and a touch panel 108 for accepting user input.

[0032] The game device 100 has an antenna module 124 for receiving wireless signals from the controller 200. The antenna module 124 may be disposed at any position on the game device 100, but is preferably disposed so as to be parallel to the display surface of the display 106, for example.

[0033] Each of the controllers 200 has an operation unit 210 that accepts user input. The operation unit 210 is composed of, for example, push buttons, a cross key, an operation lever, etc. In the example shown in FIG. 1 , the operation unit 210 includes a direction input unit 212 and an operation button 214.

[0034] One example of the directional input unit 212 is an analog stick. In another embodiment, the directional input unit 212 may be a slide pad, a touch pad, a cross key, or four buttons corresponding to each direction. In yet another embodiment, the directional input unit 212 may be an optical sensor. For example, it may be an optical sensor in a mouse or an optical sensor in a pen-type controller. In yet another embodiment, the directional input unit 212 may be a camera or a subject captured by a camera, and may recognize a directional input by image recognition of the subject captured by the camera.

[0035] 2 is a schematic diagram showing an example of a hardware configuration of game device 100 of system 1 according to the present embodiment. Referring to Fig. 2, game device 100 includes processor 102, memory 104, display 106, touch panel 108, storage 110, near-field communication unit 120, antenna module 124, wireless communication unit 126, speaker 128, microphone 130, gyro sensor 132, first controller interface 134, second controller interface 136, cradle interface 138, and memory card interface 140.

[0036] The processor 102 is a processing entity that executes the processes provided by the game device 100. The memory 104 is a storage device accessible by the processor 102, and is, for example, a volatile storage device such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The storage 110 is, for example, a non-volatile storage device such as a flash memory.

[0037] The processor 102 performs the processing described below by reading a program stored in the storage 110, expanding it into the memory 104, and executing it. The storage 110 stores, for example, an application program 112 consisting of instruction codes for performing any information processing, and a system program 114 that provides libraries and the like necessary for program execution.

[0038] Processor 102 executes the processes required for game device 100. In the following description, attention is particularly focused on the processes for generating images to be displayed or output on the display. That is, processor 102 corresponds to a processing unit that displays images on display 106.

[0039] The application program 112 includes a user operation definition 116, which defines the correspondence between a user input to the controller 200 and the operation content for an operation object, as will be described later. The user operation definition 116 includes a plurality of types of correspondence.

[0040] The short-range communication unit 120 transmits and receives wireless signals to and from one or more controllers 200. Any wireless method such as Bluetooth (registered trademark), ZigBee (registered trademark), wireless LAN (IEEE802.11), or infrared communication can be used for the short-range communication unit 120. In the following description, an example in which Bluetooth is used as the wireless method for the short-range communication unit 120 is shown.

[0041] The antenna module 124 receives wireless signals transmitted from one or more controllers 200. The antenna module 124 may be arranged as an antenna for the short-range communication unit 120 to transmit and receive wireless signals, or the antenna module 124 may be arranged in addition to a normal antenna for the short-range communication unit 120 to transmit and receive wireless signals.

[0042] The near field communication unit 120 has a direction measurement unit 122 that measures the direction in which the controller 200 is located relative to the display 106 (i.e., the direction in which the controller 200 is located as seen from the game device 100). More specifically, the direction measurement unit 122 measures the direction in which the controller 200 that transmitted the wireless signal is located, based on a wireless signal from the controller 200 received by the antenna module 124. Note that the function provided by the direction measurement unit 122 may be provided by the near field communication unit 120, or may be provided by cooperation between the near field communication unit 120 and the processor 102. Details of the measurement process by the direction measurement unit 122 will be described later.

[0043] The wireless communication unit 126 exchanges data using wireless signals with a wireless repeater connected to the Internet, etc. The wireless communication unit 126 can employ any wireless system, such as a wireless LAN (IEEE802.11) or a public wireless line (4G, 5G, etc.).

[0044] The speaker 128 generates any sound around the game device 100. The microphone 130 collects sounds occurring around the game device 100.

[0045] The gyro sensor 132 detects the orientation of the game device 100 . When the controller 200 is attached to the game device 100, the first controller interface 134 and the second controller interface 136 exchange data with the attached controller 200.

[0046] Cradle interface 138 exchanges data with a cradle (not shown) when game device 100 is placed on the cradle.

[0047] The memory card interface 140 reads data stored in a removable memory card 142 from the memory card 142, and writes data to the memory card 142. The memory card 142 may store an application program or the like.

[0048] 3 is a schematic diagram showing an example of a hardware configuration of controller 200 of system 1 according to the present embodiment. Referring to FIG. 3, controller 200 includes a processor 202, a memory 204, an operation unit 210, an acceleration sensor 206, a short-range communication unit 220, and a main body communication unit 230.

[0049] The processor 202 loads a program into the memory 204 and executes it to realize the processing required by the controller 200 .

[0050] The operation unit 210 generates a signal according to a user input. The acceleration sensor 206 is a sensor that detects the movement of the controller 200, and generates a signal according to the acceleration occurring in the controller 200.

[0051] The short-range communication unit 220 transmits and receives wireless signals to and from the game device 100 . The main body communication unit 230 exchanges data with the game device 100 when the controller 200 is attached to the game device 100 .

[0052] In this specification, the term "processor" not only refers to a processing circuit that executes processing according to instruction codes written in a program, such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), but also includes hardwired circuits such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Hardwired circuits such as an ASIC or FPGA have pre-formed circuits corresponding to the processing to be executed. Furthermore, in this specification, "processor" also includes circuits that integrate multiple functions, such as a system on chip (SoC), and also includes a combination of a processor in the narrow sense and a hardwired circuit.

[0053] [B.Direction measurement] Next, a description will be given of direction measurement that can be performed by the system 1 according to the present embodiment.

[0054] In system 1 according to the present embodiment, game device 100 has a function of measuring the direction in which controller 200 is located, based on a wireless signal received from controller 200. More specifically, game device 100 calculates the direction in which controller 200 is located, based on a phase difference that occurs when wireless signals are received by a plurality of antenna elements that are arranged at separate locations.

[0055] 4 is a diagram for explaining the principle of direction measurement of system 1 according to the present embodiment. Referring to FIG. 4, antenna module 124 has a plurality of antenna elements 125-1 and 125-2 (hereinafter, sometimes collectively referred to as "antenna elements 125").

[0056] Because the distance between the game device 100 and the controller 200 is sufficiently long compared to the wavelength of the wireless signal, the wireless signal transmitted from the controller 200 can be regarded as a plane wave. Therefore, equiphase surface 240 of the wireless signal transmitted from the controller 200 is orthogonal to a line connecting the controller 200 and the center O of antenna element 125-1 and antenna element 125-2 (a line that forms an angle θ with the line connecting antenna element 125-1 and antenna element 125-2). Note that angle θ is the angle at which the wireless signal is incident on antenna module 124, and is also referred to as the angle of arrival.

[0057] 4, antenna element 125-1 intersects with equiphase surface 240 of phase φ1, and antenna element 125-2 intersects with equiphase surface 240 of phase φ4. That is, a phase difference Δφ of |phase φ1−phase φ4| occurs between the radio signal received by antenna element 125-1 and the radio signal received by antenna element 125-2. This phase difference Δφ depends on angle θ and inter-element distance d.

[0058] More specifically, if the wavelength of the radio signal is λ, the following relational expression holds: Δφ=2π×(d×cos(θ) / λ) When this relational expression is rearranged with respect to the angle θ (arrival angle), it can be expressed as follows:

[0059] θ=cos -1 ((Δφ×λ) / (2π×d)) Here, since the wavelength λ of the radio signal and the inter-element distance d are known, the direction (angle θ) in which the controller 200 is located can be calculated based on the phase difference Δφ occurring in the radio signals received by the two antenna elements 125.

[0060] The antenna module 124 may have two or more antenna elements 125, but using more antenna elements 125 can improve measurement accuracy.

[0061] FIG. 5 is a schematic diagram showing an example of an antenna module 124 in which a plurality of antenna elements 125 are arranged in one direction. The antenna module 124 shown in FIG. 5 has four antenna elements 125-1 to 125-4 arranged in a line along the X-axis. With such an arrangement of the antenna elements 125, it is possible to measure the angle of arrival (one-dimensional) with respect to the X-axis. More specifically, by using two antenna modules 124, it is possible to measure the angle of arrival of a wireless signal transmitted from the controller 200 as seen from the center of the two antenna modules 124. Note that any two adjacent antenna elements 125 may be selected, or two adjacent antenna elements 125 may be selected sequentially. Alternatively, in other embodiments, any two antenna elements that are not adjacent to each other may be selected.

[0062] In the example shown in Figure 5, angle θ1 can be measured by selecting antenna element 125-1 and antenna element 125-2, angle θ2 can be measured by selecting antenna element 125-2 and antenna element 125-3, and angle θ3 can be measured by selecting antenna element 125-3 and antenna element 125-4.

[0063] From each measured angle, the position (or distance) of the controller 200 can be measured in addition to the direction of the controller 200 that transmitted the wireless signal. Note that the measured direction and position are both relative values ​​with respect to the display 106. Therefore, in this specification, the process of measuring the "direction" can include the process of measuring the "position."

[0064] Fig. 6 is a schematic diagram showing an example of antenna module 124 in which multiple antenna elements 125 are arranged in two directions. Antenna module 124 shown in Fig. 6 has 4 x 4 antenna elements 125-11 to 125-44 arranged along the X-axis and Y-axis, respectively. With antenna elements 125 arranged in this way, the angle of arrival with respect to the X-axis and Y-axis (two-dimensional) can be measured.

[0065] More specifically, by using two antenna modules 124 arranged in the same row on the X axis, it is possible to measure the component of the arrival angle of a wireless signal transmitted from the controller 200 relative to the X axis. Similarly, by using two antenna modules 124 arranged in the same row on the Y axis, it is possible to measure the component of the arrival angle of a wireless signal transmitted from the controller 200 relative to the Y axis.

[0066] In the example shown in Figure 6, by selecting antenna elements 125-11 and 125-12, angle θx, which is the component of the arrival angle relative to the X axis, can be measured, and by selecting antenna elements 125-34 and 125-44, angle θy, which is the component of the arrival angle relative to the Y axis, can be measured.

[0067] As in FIG. 5, by performing measurements multiple times with different combinations of antenna elements 125, it is possible to measure the position (or distance) of controller 200 in addition to the direction of controller 200.

[0068] To perform the above-described direction measurement, it is necessary to receive the same radio signal using multiple antenna elements 125. Multiple receiving circuits may be provided, or a common receiving circuit may be configured to sequentially switch between the multiple antenna elements 125 to be used for reception.

[0069] Fig. 7 is a schematic diagram showing an example of the configuration of short-range communication unit 120 of system 1 according to the present embodiment. Fig. 7 shows an example in which direction measurement unit 122 is implemented as a part of the configuration of short-range communication unit 120.

[0070] 7, short-range communication unit 120 includes multiplexer 1221, detector 1222, differentiator 1223, delay element 1224, angle calculation unit 1225, control unit 1226, and decoder 1227. Direction measurement unit 122 mainly includes differentiator 1223, delay element 1224, angle calculation unit 1225, and control unit 1226.

[0071] The multiplexer 1221 selects one antenna element 125 from the plurality of antenna elements 125 in accordance with a selection command from the control unit 1226 .

[0072] The detector 1222 decodes the radio signal received by the antenna element 125 connected via the multiplexer 1221, and outputs the decoded signal.

[0073] Differentiator 1223 calculates the phase difference between the signals output from detector 1222. The signal output from detector 1222 is directly input to one end of differentiator 1223, and the signal output from detector 1222 is input to the other end of differentiator 1223 via delay element 1224. The delay time of delay element 1224 is set according to the selection time by multiplexer 1221. In other words, differentiator 1223 receives a signal obtained by decoding the radio signal received by currently selected antenna element 125 and a signal obtained by decoding the radio signal received by the antenna element 125 selected immediately before.

[0074] Angle calculation section 1225 calculates an angle (arrival angle) from the phase difference calculated by differentiator 1223. In angle calculation section 1225, inter-element distance d and wavelength λ are set in advance.

[0075] The control unit 1226 outputs a selection command to the multiplexer 1221, and performs statistical processing (for example, averaging or outlier removal) on the angles sequentially calculated by the angle calculation unit 1225 in response to the selection command, thereby outputting a measurement result indicating the direction in which the controller 200 is located. The measurement result may include the distance to the controller 200 in addition to the one-dimensional angle or two-dimensional angle indicating the direction in which the controller 200 is located.

[0076] The decoder 1227 reconstructs a frame from the signal output from the detector 1222. The decoder 1227 also outputs, to the control unit 1226, identification information for identifying the transmission source of the wireless signal, based on the information included in the frame.

[0077] FIG. 8 is a schematic diagram showing an example of a configuration of a frame transmitted by controller 200 of system 1 according to the present embodiment.

[0078] 8, a frame 250 includes a preamble 251, a destination address 252, data 253, a CRC 254, and direction measurement data 256. The preamble 251, the destination address 252, the data 253, and the CRC 254 correspond to an actual frame 255.

[0079] The direction measurement data 256 includes a plurality of constant values ​​(usually "1"). Since the values ​​included in the direction measurement data 256 do not change over time, the radio signal is a sine wave whose phase and amplitude do not change over time. This sine wave is used to perform the direction measurement described above.

[0080] Since destination address 252 includes identification information for identifying the controller 200 that transmitted the wireless signal, it is possible to measure the direction for each controller 200 when multiple controllers 200 are connected to game device 100. That is, based on the information included in destination address 252, it is possible to identify which controller 200 the wireless signal is from, and then measure the direction to which the identified controller 200 is located.

[0081] [C. Example of screen display using the measurement results of direction measurement] Next, some examples of screen displays using the measurement results of the above-mentioned direction measurement will be described.

[0082] In system 1 according to the present embodiment, game device 100 can generate an image based on the direction in which controller 200 is positioned relative to display 106.

[0083] Fig. 9 is a schematic diagram showing an example of a screen output by game device 100 of system 1 according to the present embodiment. Fig. 9 shows an example in which game device 100 is supported by stand 144 and placed so that display 106 faces sideways or diagonally upward, and one or more users operate controller 200 while viewing an image displayed on display 106. Note that the usage mode shown in Fig. 9 may hereinafter be referred to as the "standing mode."

[0084] 9(A), an image 450 displayed on the display 106 of the game device 100 includes controllable objects 401 to 404 that are moved in response to a user input to the controller 200. That is, user A controls controllable object 401 using controller 200A, user B controls controllable object 402 using controller 200B, user C controls controllable object 403 using controller 200C, and user D controls controllable object 404 using controller 200D.

[0085] The image 450 further includes information presenting objects 411-414 that present information about at least one of the user and / or the control object. In the example shown in Fig. 9, the information presenting objects 411-414 include the player names and scores of the control objects 401-404 controlled by the corresponding users.

[0086] In this way, the game device 100 (processor 102) displays an image including the control object and the information presentation object on the display 106.

[0087] In this specification, the term "controllable object" refers to an object that corresponds to the first object and is controlled in response to a user input to the controller 200 (or the touch panel 108, etc.). The controllable object may also be referred to as a player character. The controllable object is moved in response to a user input to the controller 200, but is not moved in response to the measured direction. However, the display position and / or orientation of the controllable object may not change in response to the measured direction, or may be changed. In this way, in addition to not moving in response to the measured direction, the orientation of the controllable object may also not change.

[0088] In this specification, the term "information presentation object" refers to an object that corresponds to a second object and presents information about at least one of the user operating the controller 200 and the controlled object. In other words, the information presentation object encompasses objects for providing necessary information to the user when the game device 100 progresses through an application. The information presentation object includes, for example, a user name, a player name assigned to the user, an arrow indicating the direction in which the controller 200 (or the user operating the controller 200) is located, and a status value of the controlled object (e.g., stamina value, experience value, etc.). In this way, the information presentation object can be considered not as an object that affects the progress of the game or application being executed, but as an object that simply provides information.

[0089] Note that the display position and / or orientation of the information presentation object is not changed in response to user input. However, in another embodiment, the display position and / or orientation of the information presentation object may be changed in response to user input. Note that the display position and / or orientation being changed in response to user input includes a case where the display position and / or orientation of the information presentation object is associated with a control object and the display position and / or orientation of the information presentation object is changed in response to a change in the display position and / or orientation of the control object in response to user input, and a case where the display position and / or orientation of the information presentation object is changed independently of the control object. The former includes, for example, an arrow indicating the control object operated by the user, which is displayed at a predetermined display position and / or orientation relative to the control object.

[0090] In this embodiment, the image displayed on the display 106 may include an operation object and an information presentation object, but the display position and / or orientation of the information presentation object changes depending on the measured direction.

[0091] 9(A), the display positions of information presentation objects 411 to 414 are image 450 that reflects the directions of users A to D (controllers 200A to 200D) relative to display 106. That is, from the left side of display 106, controller 200A, controller 200B, controller 200C, and controller 200D are present in this order, and corresponding to this positional relationship, information presentation object 411, information presentation object 412, information presentation object 413, and information presentation object 414 are displayed in this order on display 106 from the left side.

[0092] 9(B) shows, as an example, a state in which the positions of user B and user C are swapped. As a result, the directions in which the controllers 200B and 200C are present change, and the game device 100 displays, on the display 106, an image 451 in which the display positions of the information presentation object 412 and the information presentation object 413 have been changed.

[0093] In this way, the game device 100 changes the display position of the information presentation object depending on the measured direction in which the controller 200 exists.

[0094] Fig. 10 is a schematic diagram showing another example of a screen output by game device 100 of system 1 according to the present embodiment. Fig. 10 shows an example in which controller 200 is placed with display 106 facing upward, and one or more users operate controller 200 while viewing an image displayed on display 106. Note that the usage mode shown in Fig. 10 may hereinafter be referred to as "flat-lay mode."

[0095] In the usage pattern shown in Fig. 10, users are present around display 106. In the example shown in Fig. 10, it is assumed that users A to D are present around display 106, and each user operates controller 200 to play a game or the like.

[0096] 10, an image 452 displayed on the display 106 of the game device 100 includes controllable objects 421-424 that are operated in response to a user input to the controller 200. The image 452 further includes information presenting objects 431-434 that present information about at least one of the user and / or the controllable object. In the example shown in Fig. 10, the information presenting objects 431-434 include the player name assigned to the corresponding user and the score of the corresponding user.

[0097] The display positions and orientations of the information presentation objects 431 to 434 correspond to the positions of the users A to D (controllers 200A to 200D) relative to the display 106.

[0098] More specifically, information presentation object 431 including information for user A is arranged at a display position and orientation corresponding to the direction in which controller 200A is located. Similarly, information presentation object 432 including information for user B is arranged at a display position and orientation corresponding to the direction in which controller 200B is located, information presentation object 433 including information for user C is arranged at a display position and orientation corresponding to the direction in which controller 200C is located, and information presentation object 434 including information for user D is arranged at a display position and orientation corresponding to the direction in which controller 200D is located.

[0099] The position and orientation of the entire game screen displayed on the display 106 does not change depending on the direction in which the controller 200 is located.

[0100] In this way, the game device 100 changes the display position and orientation of the information presentation object depending on the measured direction in which the controller 200 exists.

[0101] Note that Figure 9 shows an example of changing the display position of an information presentation object according to the measured direction, and Figure 10 shows an example of changing the display position and orientation of an information presentation object according to the measured direction, but it is also possible to change only the orientation of the information presentation object (while maintaining its display position) according to the measured direction, or to change only the display position while maintaining the orientation of the information presentation object.

[0102] In this way, the display position and / or orientation of the information presentation object changes depending on the measured direction of the controller 200, thereby improving the visibility of the information presentation object for the user operating the controller 200. Furthermore, the user can operate the controller 200 while referring to the information presentation object, improving operability.

[0103] [D. Changing operation contents according to the measurement results of direction measurement] Next, several examples will be described in which the operation content of the operation object corresponding to the user operation is changed in accordance with the measurement result of the above-described direction measurement.

[0104] The game device 100 may change the correspondence between the user input to the controller 200 and the operation content for the control object, depending on the measured direction. That is, the game device 100 may change the interpretation of the user operation corresponding to the user input to the controller 200, depending on the measured direction. More specifically, the operation of the control object corresponding to the same user operation on the controller 200 may be changed depending on the relative relationship between the position of the controller 200 (or the position of the user operating the controller 200) and the display 106.

[0105] Figures 11 and 12 are diagrams for explaining an example of the correspondence between a user input to controller 200 and the operation content of the operation on the example screen shown in Figure 10. With reference to Figure 11, for example, it is assumed that operation object 421 included in image 452 can be operated from controller 200A.

[0106] In the example shown in FIG. 11, user A holding controller 200A is present within a range corresponding to the lower side of display 106 (the "lower" side indicated by directional indicator 10).

[0107] User A holds controller 200A in a landscape orientation. In this state, when user A performs an input operation of pressing directional input unit 212 of controller 200A upward (operation direction 261), game device 100 interprets this as an operation to move control object 421 in direction 426. Furthermore, in the example shown in FIG. 11 , in order to move control object 421 in direction 427, user A needs to perform an input operation of pressing directional input unit 212 of controller 200A rightward (operation direction 262).

[0108] 12, it is assumed that user A holding controller 200A moves within the range corresponding to the left side of display 106 (the "left" side indicated by directional indicator 10). In response to user A's movement, the display position and orientation of information presentation object 431 corresponding to user A change.

[0109] In this state, when the user A performs an input operation of pressing the directional input unit 212 of the controller 200A upward (operation direction 261), the game device 100 interprets this as an operation to move the operation object 421 in direction 427, not direction 426. In addition, in the example shown in Fig. 12, in order to move the operation object 421 in direction 426, the user A needs to perform an input operation of pressing the directional input unit 212 of the controller 200A leftward (operation direction 263).

[0110] In this way, the game device 100 determines different operation contents for the control object in response to the same user input to the direction input unit 212 of the controller 200. In other words, the game device 100 changes the correspondence between the direction input to the direction input unit 212 and the movement direction of the control object on the screen.

[0111] Fig. 13 is a diagram illustrating another example of the correspondence between a user input to controller 200 and the operation content of the operation on the example screen shown in Fig. 10. With reference to Fig. 13, for example, it is assumed that operation object 421 included in image 452 can be operated from controller 200A.

[0112] 13(A), user A holding controller 200A is present within an area corresponding to the lower side of display 106 (the "lower" side indicated by directional indicator 10). When user A performs an input operation to swing controller 200A forward, acceleration sensor 206 of controller 200A outputs a signal corresponding to the user input by user A. Then, game device 100 interprets the user input as an operation to move control object 421 in direction 426.

[0113] 13(B), it is assumed that user A holding controller 200A moves within a range corresponding to the left side of display 106 (the "left" side indicated by directional indicator 10). In response to user A's movement, the display position and orientation of information presentation object 431 corresponding to user A change.

[0114] In this state, when user A performs an input operation of swinging controller 200A forward, acceleration sensor 206 of controller 200A outputs a signal corresponding to the user input by user A. Then, game device 100 interprets the user input as an operation to move control object 421 in direction 427, not direction 426.

[0115] In this way, different operation contents are determined for the control object in response to the same user input to the controller 200. In other words, the game device 100 changes the correspondence between the detected direction of movement of the controller 200 and the moving direction of the control object on the screen.

[0116] 14 is a schematic diagram showing an example of user operation definitions 116 held by game device 100 of system 1 according to the present embodiment. Referring to Fig. 14, for example, four types of user operation definitions 116 may be prepared corresponding to the sides of display 106. Each definition includes operation contents for control objects respectively assigned to a plurality of directions (e.g., upward, downward, leftward, rightward) that can be input to direction input unit 212, operation contents for control objects respectively assigned to operation buttons 214 (e.g., A button, B button, X button, Y button), and operation contents for control objects respectively assigned to detection signals (e.g., forward swing, backward swing) of acceleration sensor 206.

[0117] The game device 100 selects one of the multiple definitions depending on the measured direction of the controller 200.

[0118] 15 is a diagram illustrating an example of a selection process of user operation definitions 116 held by game device 100 of system 1 according to the present embodiment. Referring to Fig. 15, each definition included in user operation definitions 116 may be assigned in association with each side of rectangular display 106.

[0119] For example, if the user is within a range corresponding to the lower side of display 106 (the "lower" side shown in directional indicator 10), definition 1 may be selected; if the user is within a range corresponding to the right side of display 106 (the "right" side shown in directional indicator 10), definition 2 may be selected; if the user is within a range corresponding to the upper side of display 106 (the "upper" side shown in directional indicator 10), definition 3 may be selected; and if the user is within a range corresponding to the left side of display 106 (the "left" side shown in directional indicator 10), definition 4 may be selected.

[0120] In this way, the game device 100 may change the correspondence between the user input to the controller 200 and the operation content for the control object, depending on the measured direction. That is, the game device 100 may maintain the correspondence as long as it is measured that the controller 200 is present within a range corresponding to one side of the display 106, and may change the correspondence when it is measured that the controller 200 is present within a range corresponding to another side of the display 106. Changing the interpretation of the intention of the user operation corresponding to the user input to the controller 200, depending on the measured direction, allows the user to perform an operation that is more in line with their intuition.

[0121] Note that, when the correspondence is changed by moving the controller 200, the display position and / or orientation of the information presentation object may be changed first. That is, the game device 100 may change the correspondence between the user input to the controller 200 and the operation content for the operation object after changing the display position and / or orientation of the information presentation object.

[0122] By changing the correspondence relationship after changing the display position and / or orientation of the information presentation object, the system 1 implicitly notifies the user that it has recognized that the user's position has changed, and then changes the interpretation of the user input to the controller 200. Therefore, the user can foresee that the interpretation of the user input will be changed, and can continue to perform operations in line with their intuition.

[0123] In another embodiment, the correspondence between the user input to the controller 200 and the operation content for the operation object may be changed according to the measured direction simultaneously with or before the change in the position and / or orientation of the information presentation object.

[0124] The correspondence between the user input to the controller 200 and the operation content for the control object means a rule for determining in which direction the user input to the controller 200 (such as an input operation of pressing the direction input unit 212 upward or an operation of swinging the controller 200) will indicate on the screen displayed on the display 106. In other words, because the relative relationship between the display 106 and the controller 200 can change, it may be determined depending on the situation as to in which direction an arbitrary user input to the controller 200 should move the corresponding control object on the displayed screen.

[0125] 14, the correspondence between the user input to the controller 200 and the operation content for the control object may be maintained for the operation buttons 214. In other words, regardless of the measured direction of the controller 200, the correspondence may not be changed for the operation buttons 214 other than the direction input unit 212 and the acceleration sensor 206.

[0126] 14 shows an example in which operation contents for operation objects respectively assigned to a plurality of directions (for example, upward, downward, leftward, and rightward) that can be input to the direction input unit 212 are defined, but the interpretation of the input direction to the direction input unit 212 by the game device 100 may be changed. That is, the meaning indicated by the signal corresponding to the user operation output from the direction input unit 212 may be changed on the game device 100 side, for example.

[0127] For example, in definition 1, when the directional input unit 212 is pressed upward, the operation is interpreted as an upward direction, and when the directional input unit 212 is pressed downward, the operation is interpreted as a downward direction, whereas in another definition 2, when the directional input unit 212 is pressed upward, the operation is interpreted as a leftward direction, and when the directional input unit 212 is pressed downward, the operation is interpreted as a rightward direction. In this case, the correspondence between the interpreted operation direction (upward, downward, leftward, rightward) and the movement direction of the control object may be uniquely determined. As a result, the correspondence between the operation input to the controller 200 and the operation content for the control object changes depending on the measured direction of the controller 200.

[0128] In addition, not only can the interpretation of the operation direction be changed when the operation on the direction input unit 212 is completed (pressed state), but the interpretation of the operation direction during the process of the operation on the direction input unit 212 can also be changed in the same way.

[0129] In another embodiment, the input direction to the direction input unit 212 may be interpreted after adding a correction amount according to the position of the controller 200 for each position where the controller 200 is located. For example, if the signal according to a user operation output from the direction input unit 212 indicates the angle at which the user operation was performed, four correction amounts may be prepared: 0° (no correction), +90°, +180°, and +270° (or −90°). In this case, a corresponding correction amount may be selected according to the position where the controller 200 is located, and the input direction may be interpreted after the signal output from the direction input unit 212 is corrected by the selected correction amount.

[0130] In the present embodiment, an example has been shown in which the correspondence between the user input to the controller 200 and the operation content for the control object is changed when the controller 200 moves from a range corresponding to one side of the display 106 to a range corresponding to another side, but in another embodiment, the correspondence between the user input to the controller 200 and the operation content for the control object may be changed even when the controller 200 is in a range corresponding to the same side of the display 106. For example, when the user (controller 200) is on the left side in front of the display 106, pressing the direction input unit 212 upward may instruct movement in the upper right direction of the screen, and when the user is on the right side in front of the display 106, pressing the direction input unit 212 upward may instruct movement in the upper left direction of the screen.

[0131] In yet another embodiment, the correspondence between the user input to the controller 200 and the operation content for the operation object does not need to be changed depending on the measured direction.

[0132] [E. Display position of information presentation object] As described above, the display position of the information presentation object may be changed depending on the measured direction. In this case, the display position of the information presentation object may be dynamically determined depending on the measured direction, or may be appropriately selected from a plurality of predetermined positions.

[0133] 16 is a schematic diagram showing an example screen on which information presentation objects are displayed at a plurality of predetermined positions in system 1 according to the present embodiment. In the example shown in FIG. 16, the directions in which four controllers 200 exist are measured, and four information presentation objects (information presentation objects 411 to 414) are displayed in accordance with the measurement results.

[0134] The four information presentation objects may be displayed at predetermined positions. That is, four positions may be determined in advance as positions at which the information presentation objects are to be displayed. By determining the positions at which the information presentation objects are to be displayed in advance, it is possible to prevent the display positions of the information presentation objects from fluctuating depending on the measured direction, thereby suppressing a decrease in visibility.

[0135] When an information presentation object is displayed at a predetermined position, the position at which the information presentation object corresponding to each controller 200 is displayed may be determined according to the relative positional relationship between the controllers 200 whose directions have been measured.

[0136] In this way, the game device 100 may select a display position of the information presentation object from among a plurality of positions predetermined as display positions of the information presentation object in accordance with the measured direction. Note that, although the example shown in Fig. 16 shows an example in which four information presentation objects 411 to 414 are displayed at four positions, when a smaller number of information presentation objects are to be displayed, the same number of positions as the number of information presentation objects to be displayed may be selected from the four positions.

[0137] 16, users A to C and user D are present apart from each other, but information presentation objects 411 to 414 included in image 450 are arranged at equal intervals. In this way, when displaying information presentation objects for multiple users, the game device 100 assigns a display position of the information presentation object for each user from among multiple positions according to the orientation of the controller 200 associated with each user.

[0138] More specifically, the game device 100 measures the direction in which each of the multiple controllers 200 is located, estimates the arrangement order of the controllers 200 based on the measured direction, and determines the display position of the information presentation object corresponding to each controller 200 (user) according to the estimated arrangement order.

[0139] Even when there are a plurality of controllers 200 (users), the position at which the information presentation object is displayed is determined based on the arrangement order of the controllers 200, thereby maintaining visibility for a plurality of users.

[0140] When the display position of the information presentation object corresponding to the measured direction differs from the current display position of the information presentation object, the display position of the information presentation object may be changed to the display position corresponding to the measured direction after a predetermined condition is met. That is, the display position of the information presentation object may be changed every time the measured direction changes, or a certain amount of buffer time may be provided. This prevents a decrease in visibility due to frequent changes in the display position of the information presentation object even when the controller 200 is moved from its original position for a short period of time.

[0141] In this way, the game device 100 may change the display position of the information presentation object after the measured direction satisfies the condition for changing the display position of the information presentation object and after a predetermined condition is met. Conditions for changing the display position of the information presentation object include when the side of the display 106 corresponding to the measured direction is different from the side of the currently selected display 106, or when multiple controllers 200 correspond to the same side of the display 106 and the relative relationship between the multiple controllers 200 has changed.

[0142] As will be described later, whether the predetermined condition is met may be determined by the passage of a predetermined time, or may be determined by the movement of the controller 200, or the like.

[0143] 17A and 17B are diagrams illustrating an example of a process for changing the display of an information presentation object in system 1 according to the present embodiment. In the example shown in Fig. 17A, from the left as one faces display 106, controller 200A, controller 200B, controller 200C, and controller 200D are present in this order, and corresponding to this positional relationship, information presentation object 411, information presentation object 412, information presentation object 413, and information presentation object 414 are displayed on display 106 in this order from the left.

[0144] For example, as shown in FIG. 17(B), assume that user A holding controller 200A moves to the right of user D holding controller 200D (to the "right" side indicated by direction indicator 10). The change in the relative relationship between user A (controller 200A) and users B to D (controllers 200B to 200D) allows determination that the condition for changing the display position of the information presentation object is satisfied. That is, in FIG. 17(B), the display position of the information presentation object corresponding to the measured direction is different from the current display position of the information presentation object.

[0145] After a predetermined time has elapsed since the state shown in Fig. 17(B) was reached, the display order of the information presentation objects 411 to 414 is changed as shown in Fig. 17(C). That is, from the left side of the display 106, the controllers are present in the order of controller 200B, controller 200C, controller 200D, and controller 200A, and corresponding to this positional relationship, the information presentation objects 412, information presentation object 413, information presentation object 414, and information presentation object 411 are displayed on the display 106 in this order from the left side.

[0146] It should be noted that, while user A is moving, there may be a state where user A is between user B and user C, but unless this state continues for a predetermined period of time, it will not be reflected in the change in the position of the information presentation object.

[0147] The time point at which the determination of whether the state has continued for a predetermined time can be started can be set arbitrarily. For example, the determination may be started from the time point at which the position where the controller 200A is present begins to change, or from the time point at which it is determined that the controller 200A is present between the controller 200B and the controller 200C (i.e., the time point at which the arrangement order of the controllers 200 changes).

[0148] The length of the predetermined time may be a fixed value or a variable value. If a variable value is used, the length may be dynamically changed depending on, for example, the progress of the game or the frequency of movements of the controller 200.

[0149] When three or more users are playing a game or application, successively changing the display position of an information presentation object in response to changes in the user's position can result in a decrease in visibility. However, by providing a certain amount of buffer time, the state in which the user's movement has stabilized is reflected on the screen, thereby preventing a decrease in visibility.

[0150] Note that a factor other than time may be used as a condition for changing the display position of the information presentation object. For example, a condition may be that the temporal fluctuation (variation) of the measured value of the direction or position of the controller 200 falls within a predetermined range. In another embodiment, a condition may be that the movement of the controller 200 falls within a predetermined range (for example, the controller 200 can be considered to be stationary) based on the detection value of the acceleration sensor 206 (or a gyro sensor, not shown) of the controller 200. In yet another embodiment, a plurality of the above-described conditions may be combined. For example, the display position of the information presentation object may be changed when a certain state continues for a predetermined time and the controller 200 can be considered to be stationary.

[0151] 18 is a schematic diagram showing an example of a plurality of predetermined positions at which an information presentation object is displayed in system 1 according to the present embodiment. In the example shown in FIG. 18, a plurality of positions (indicated by dashed lines) at which an information presentation object can be displayed may be set within an image displayed in the flat mode.

[0152] In the example shown in FIG. 18, multiple positions are predefined for each side of the display 106, and even if multiple users (controllers 200) are present in the range corresponding to the same side, an information presentation object can be displayed for each user (controller 200).

[0153] In another embodiment, only the area in which the information presentation object is displayed may be set. For example, in the example shown in Fig. 18, the position of the information presentation object may be freely set according to the position of the controller 200 as long as it is on the screen edge side on each side. In yet another embodiment, the information presentation object may be displayed at any position on the screen.

[0154] [F. Display position and orientation of operation objects] In the above description, an example has been shown in which the display position and orientation of the operation object included in the image displayed on the display 106 do not change depending on the direction in which the controller 200 is present.

[0155] That is, the game device 100 may maintain the display position and orientation of the control object independently of the measured direction. Maintaining the display position and orientation of the control object can eliminate the possibility of causing discomfort to users when multiple users are playing a game or application.

[0156] In another embodiment, only the orientation of the control object may be changed while maintaining the display position of the control object according to the direction of the controller 200. By changing only the orientation of the control object according to the situation, it is possible to improve the visibility for the user.

[0157] In yet another embodiment, a movement operation may be performed on the control object and / or a direction change operation may be performed on the control object depending on the direction in which the controller 200 is present.

[0158] [G. Processing Procedures] Next, an example of a processing procedure executed by system 1 according to the present embodiment will be described.

[0159] 19 is a flowchart showing a processing procedure executed by game device 100 of system 1 according to the present embodiment. Each step shown in FIG. 19 is typically realized by processor 102 of game device 100 executing application program 112.

[0160] 19, game device 100 determines whether or not application program 112 being executed supports generation of an image based on a direction (step S100). If application program 112 does not support generation of an image based on a direction (NO in step S100), game device 100 does not measure the direction of controller 200, and generates an image including an operation object and an information presentation object according to predetermined settings (step S102). The generated image is output to display 106.

[0161] Game device 100 determines whether an instruction to end the application program has been given (step S104). If an instruction to end the application program has not been given (NO in step S104), the processing from step S102 onwards is repeated. If an instruction to end the application program has been given (YES in step S104), the processing ends.

[0162] If the generation of an image based on a direction is supported (YES in step S100), game device 100 determines whether or not the currently running application program 112 is requesting that an image be generated based on a direction (step S106). If the currently running application program 112 is not requesting that an image be generated based on a direction (NO in step S106), the process proceeds to step S126 and thereafter.

[0163] If the currently running application program 112 requests generation of an image based on the direction (YES in step S106), the game device 100 measures the direction of the controller 200 (step S108). Then, the game device 100 determines whether the position at which the information presentation object should be displayed in accordance with the measured direction (hereinafter also referred to as the "planned display position of the information presentation object") is different from the current display position of the information presentation object (step S110). In other words, the game device 100 determines whether the position at which the information presentation object should be displayed, which is determined based on the measured direction, is different from the current display position of the information presentation object. display It is determined whether the position matches the position specified.

[0164] If the planned display position of the information presentation object does not match the current display position of the information presentation object (NO in step S110), the game device 100 determines whether the planned display position of the information presentation object has been maintained the same for a predetermined time period (step S112).

[0165] If the planned display positions of the information presentation objects are maintained the same for a predetermined time period (YES in step S112), the game device 100 determines the display position and orientation for each information presentation object based on the measured direction of each controller 200 (step S114), and also determines the display position and orientation for each control object in accordance with user input (step S116). Then, an image including the control object and the information presentation object is generated (step S118). The generated image is output to the display 106. Then, the processing from step S126 onwards is executed.

[0166] If the planned display positions of the information presentation objects match the current display positions of the information presentation objects (YES in step S110), the game device 100 maintains the current display positions and orientations of each information presentation object (step S120), and determines the display positions and orientations of each operation object in accordance with the user input (step S122). Then, an image including the operation object and the information presentation object is generated (step S124). The generated image is output to the display 106. Then, the processing from step S126 onwards is executed.

[0167] If the planned display position of the information presentation object has not been maintained the same for the predetermined time (NO in step S112), the process proceeds to step S120 and subsequent steps.

[0168] Game device 100 determines whether an instruction to end the application program has been given (step S126). If an instruction to end the application program has not been given (NO in step S126), the processing from step S106 onwards is repeated. If an instruction to end the application program has been given (YES in step S126), the processing ends.

[0169] Fig. 20 is a flowchart showing the procedure of the direction measurement process shown in Fig. 19. Referring to Fig. 20, the game device 100 extracts two adjacent antenna elements 125 from among the antenna elements 125 to be used (step S200). The game device 100 selects one of the two extracted antenna elements 125 (step S202) and receives a wireless signal at the selected antenna element 125 (step S204). Next, the game device 100 selects the other of the two extracted antenna elements 125 (step S206) and receives a wireless signal corresponding to the same frame at the selected antenna element 125 (step S208).

[0170] The game device 100 then calculates the phase difference between the wireless signal received in step S204 and the wireless signal received in step S208 (step S210), and calculates an angle indicating the direction in which the controller 200 is located based on the calculated phase difference (step S212). Furthermore, the game device 100 adds identification information for identifying the controller 200 that is the sender of the wireless signals received by the two antenna elements 125, and stores the calculated angle (step S214).

[0171] The game device 100 determines whether or not a predetermined measurement completion condition is met (step S216). The predetermined measurement completion condition includes a condition such as measurement for a predetermined time period or a predetermined number of measurements.

[0172] If the predetermined measurement completion condition is not met (NO in step S216), the processes from step S200 onwards are repeated.

[0173] If a predetermined measurement completion condition is met (YES in step S216), the game device 100 calculates the direction for each controller 200 by statistically processing one or more angles calculated for each stored controller 200 (step S218), and then the process returns.

[0174] [H. Other Forms] The division of responsibilities among the various processing entities is not limited to the above description. For example, the image generation process may be performed by the processor 102 of the game device 100, or a computing resource other than the game device 100 may be used. Typically, a computing resource on a cloud that can communicate with the game device 100 may generate the image. In this case, the game device 100 transmits to the computing resource a signal indicating a user operation received from the controller 200 and information indicating the direction of the controller 200, receives the image from the computing resource, and displays the image on the display 106 or an external display. I Furthermore, instead of a computing resource on the cloud, any computing resource that can communicate over a local network may be used.

[0175] In the above description, an example has been described in which the direction of the controller 200 is measured using a wireless signal transmitted by the controller 200, but the direction may be measured using other methods. For example, infrared rays or ultrasonic waves may be used.

[0176] Furthermore, instead of a configuration in which the game device 100 receives a wireless signal transmitted by the controller 200 and measures the direction, a configuration in which the controller 200 receives a wireless signal transmitted by the game device 100 and measures the direction may be employed. In this case, by transmitting information indicating the direction measured by the controller 200 to the game device 100, the direction information can be reflected in the generation of an image.

[0177] In the above description, an example has been described in which the position and orientation of the entire game screen displayed on the display 106 are not changed depending on the direction in which the controller 200 is present. However, in another embodiment, the position and orientation may be changed. For example, when two users (two controllers 200) are present on either side of the game device 100 in the flat mode, the game screen may be rotated as the game progresses. As an example, depending on each user's turn (operation), the game screen itself (the actual game screen, which may exclude, for example, various interface displays surrounding the game screen) may be rotated, for example, upside down, to make it easier for the user whose turn it is to operate. Alternatively, when users (four controllers 200) are present surrounding the game device 100 on four sides, the game screen may be rotated in four directions. Note that, when an information presentation object is included in the game screen, the information presentation object may be rotated together with the game screen without changing its relative position and orientation with respect to the game screen, or the relative position and orientation of the information presentation object with respect to the game screen may be changed as the game screen is rotated.

[0178] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0179] 1 System, 10 Directional indicator, 100 Game device, 102, 202 Processor, 104, 204 Memory, 106 Display, 108 Touch panel, 110 Storage, 112 Application program, 114 System program, 116 User operation definition, 120, 220 Near-field communication unit, 122 Direction measurement unit, 124 Antenna module, 125 Antenna element, 126 Wireless communication unit, 128 Speaker, 130 Microphone, 132 Gyro sensor, 134 First controller interface, 136 Second controller interface, 138 Cradle interface, 140 Memory card interface, 142 Memory card, 144 Stand, 200, 200A, 200B, 200C, 200D Controller, 206 Acceleration sensor, 210 Operation unit, 212 Direction input unit, 214 Operation button, 230 Main body communication unit, 240 equiphase surface, 250, 255 frame, 251 preamble, 252 destination address, 253 data, 256 direction measurement data, 261, 262, 263 operation direction, 300 external display, 401, 402, 403, 404, 421, 422, 423, 424 operation object, 411, 412, 413, 414, 431, 432, 433, 434 information presentation object, 426, 427 direction, 450, 451, 452 image, 1221 multiplexer, 1222 detector, 1223 difference calculator, 1224 delay element, 1225 angle calculation unit, 1226 control unit, 1227 decoder, d distance between elements.

Claims

1. The display and a measurement unit that measures the directions of the plurality of controllers relative to the display; a processing unit that displays, on the display, an image including a plurality of first objects that are not moved in accordance with the measured directions but are moved in accordance with user inputs to the controllers, and a plurality of second objects that present information about at least one of the user operating each controller and the first object corresponding to the controller; the processing unit determines at least one of a display position and an orientation of the second object corresponding to each controller in accordance with a direction measured for the controller and a direction measured for a controller other than the controller; When the display position of the second object determined for any controller differs from the current display position of the second object corresponding to that controller, after a predetermined condition is met, the system changes at least one of the display position and orientation of the second object corresponding to that controller to at least one of the determined display position and orientation.

2. The system according to claim 1 , wherein the predetermined condition includes a state in which the display position of the second object determined for any controller continues to be different from the current display position of the second object corresponding to that controller for a predetermined period of time.

3. The processing unit When three second objects corresponding to a first user, a second user, and a third user are displayed, When a state changes from a first state in which the directions of the first controller operated by the first user, the second controller operated by the second user, and the third controller operated by the third user are measured in this order, to a second state in which the directions of the second controller, the first controller, and the third controller are measured in this order, to a third state in which the directions of the second controller, the third controller, and the first controller are measured in this order, if the time in the second state is less than a predetermined time, 3. The system according to claim 1, wherein the display on the display is changed from a first display state in which the second object corresponding to the first user, the second object corresponding to the second user, and the second object corresponding to the third user are displayed in that order, to a third display state in which the second object corresponding to the second user, the second object corresponding to the third user, and the second object corresponding to the first user are displayed in that order, without passing through a second display state in which the second object corresponding to the second user, the second object corresponding to the first user, and the second object corresponding to the third user are displayed in that order.

4. the controller includes a sensor for detecting movement; The system according to claim 1 , wherein the processing unit determines whether the predetermined condition is met based on a detected value of the sensor.

5. The system of claim 4 , wherein the predetermined condition includes the controller being within a predetermined range of motion.

6. the controller includes a sensor for detecting movement; 2. The system according to claim 1, wherein the predetermined conditions include a state in which the display position of the second object determined for any controller is different from the current display position of the second object corresponding to that controller for a predetermined period of time, and a state in which the movement of the controller is within a predetermined range.

7. The system of any one of claims 3 to 5, wherein the first object is controlled in response to movements of a corresponding controller.

8. measuring the orientation of each of the plurality of controllers relative to the display; displaying on the display an image including a plurality of first objects that are not moved in accordance with the measured directions but are moved in accordance with user inputs to the controllers, and a plurality of second objects that present information about at least one of the user operating each controller and the first object corresponding to the controller; determining at least one of a display position and an orientation of the second object corresponding to each controller according to a direction measured for the controller and a direction measured for a controller other than the controller; and when the display position of the second object determined for any controller differs from the current display position of the second object corresponding to that controller, changing at least one of the display position and orientation of the second object corresponding to that controller to at least one of the determined display position and orientation after a predetermined condition is met.

9. A program executed on a computer having a display, the program causing the computer to: means for acquiring information regarding the directions in which a plurality of controllers are present relative to the display; means for moving, in response to a user input to each controller, each of a plurality of first objects that are not moved in response to each of the acquired directions; a means for determining, in accordance with the acquired information, at least one of a display position and an orientation of a plurality of second objects presenting information relating to at least one of a user operating each controller and the first object corresponding to the controller; means for generating an image including the plurality of first objects and the plurality of second objects; a program that, when a display position of the second object determined for any controller differs from a current display position of the second object corresponding to that controller, functions as a means for changing at least one of the display position and orientation of the second object corresponding to that controller to at least one of the determined display position and orientation after a predetermined condition is met.

Images