System, portable electronic device, processing method, and program

Abstract

This system comprises a portable electronic device including a display and an interface for outputting an image to an external display separate from the display, a controller that receives a user operation, a processing unit that generates an image according to a user operation performed on the controller, and a measurement unit that measures a direction of existence of the controller relative to the portable electronic device. The processing unit generates an image on the basis of the measured direction when the image is output to the display, whereas the processing unit restricts generation of the image based on the measured direction when the image is output to the external display.

Description

[Technical Field]

[0001] The present disclosure relates to a system, a portable electronic device, a processing method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, devices have been known that have a display and also output images to an external display device separate from the display (see, for example, JP 2019-197585 A). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-197585 Summary of the Invention [Problem to be solved by the invention]

[0004] The object is to further improve operability and / or entertainment value compared to the known devices mentioned above. [Means for solving the problem]

[0005] A system according to one embodiment includes a portable electronic device including a display and an interface for outputting an image to an external display separate from the display, a controller that accepts user operations, a processing unit that generates an image in response to the user operations on the controller, and a measurement unit that measures the direction in which the controller is located relative to the portable electronic device, wherein the processing unit generates an image based on the measured direction when the image is output to the display, and limits the generation of the image based on the measured direction when the image is output to the external display.

[0006] According to this configuration, an image is generated based on the direction in which the controller (and the user operating the controller) is positioned relative to the portable electronic device, thereby providing the user with an intuitive operation.

[0007] Furthermore, when an image is output to an external display, even if the direction in which the controller is present relative to the portable electronic device is measured, the positional relationship between the external display and the portable electronic device is not determined, and therefore the measured direction may not reflect the direction in which the actual controller (and the user operating the controller) is present. Therefore, when an image is output to an external display, by restricting the generation of an image based on the measured direction, it is possible to reduce the possibility of the user feeling uncomfortable.

[0008] The controller may transmit a wireless signal in response to a user operation. The measurement unit may calculate the direction in which the controller is located based on a phase difference that occurs when the wireless signals are received by a plurality of antenna elements disposed at different locations. With this configuration, the direction in which the controller is located can be measured using a wireless signal (including, for example, information about a user operation) exchanged between the portable electronic device and the controller, thereby achieving direction measurement while keeping costs down.

[0009] The measurement unit may disable measurement of the direction in which the controller is located when an image is output to an external display. When an image is output to an external display, image generation based on the measured direction is restricted, making direction measurement unnecessary. Disabling direction measurement can prevent unnecessary increases in processing resources, communication bandwidth, and the like required for direction measurement.

[0010] The processing unit may not restrict image generation based on the measured direction when a predetermined condition is met, even when the image is output to an external display. For example, a condition may be employed to ensure that the measured direction reflects the direction in which the real controller (and the user operating the controller) is located. This allows for intuitive operation for the user, even when the image is output to an external display.

[0011] The measurement unit may change which of the multiple antenna elements is used to calculate the direction in which the controller is located, depending on at least one of the attitude of the portable electronic device and the application program executed by the processing unit. With this configuration, by appropriately selecting the antenna element to be used depending on the situation, it is possible to suppress an increase in processing load and improve measurement accuracy.

[0012] The measurement unit may be configured to use some of the multiple antenna elements when the display is in a first position where it is placed sideways or diagonally upward, and / or when an application program corresponding to the first position is being executed.

[0013] In the first position, it is assumed that the controller (and the user operating the controller) is positioned directly facing the display, so some of the antenna elements are used so that the direction can be measured within the range in which the controller is present. By using some of the antenna elements in this way, it is possible to suppress an increase in the processing load.

[0014] The measurement unit may be configured to use all of the multiple antenna elements when the device is in a second position in which the display is placed facing upwards, or when an application program corresponding to the second position is being executed.

[0015] In the second position, since the controller (and the user operating the controller) may be present around the display, all antenna elements are used so that the direction of the controller can be measured no matter where the controller is located, thereby making it possible to more reliably measure the direction of the controller.

[0016] The multiple antenna elements include at least two antenna elements arranged along each of two directions that define the display surface of the display, and some of the antenna elements used may be antenna elements in a single row among the multiple antenna elements. With this configuration, the direction (i.e., angle) of the controller can be calculated based on the direction in which the antenna elements are lined up (corresponding to one of the sides of the display), thereby improving measurement accuracy compared to, for example, using multiple antenna elements that are aligned in a direction different from either of the two directions that define the display surface.

[0017] The row of antenna elements may be the upper row of antenna elements among the plurality of antenna elements. This configuration can reduce noise such as reflected components from a table on which the portable electronic device is placed.

[0018] The measurement unit may change the antenna element used to calculate the direction in which the controller is located from among the multiple antenna elements, depending on the previously measured direction in which the controller is located. With this configuration, by selecting an antenna element that can measure the direction in which the controller is actually located, it is possible to omit measurement of areas in which the controller is not located. This makes it possible to suppress an increase in processing load and improve measurement accuracy.

[0019] The measurement unit may change the frequency with which it calculates the direction in which the controller is located, depending on the previously measured direction in which the controller is located. With this configuration, by changing the frequency with which it calculates the direction depending on the direction in which the controller actually is located, it is possible to suppress an increase in processing load and reduce power consumption.

[0020] The processing unit may be configured to generate an image according to a predetermined setting independent of the measured direction when switching from a state in which the image is being output on the display to a state in which the image is being output on an external display. When switching from a state in which the image is being output on the display to a state in which the image is being output on an external display, it is highly likely that the positional relationship between the portable electronic device and the controller has changed. Therefore, if the image is output as it is while being output on the display, it may cause a sense of discomfort to the user. This configuration reduces the possibility of causing such a sense of discomfort to the user.

[0021] The processing unit may be configured to start generating an image based on the measured direction after a predetermined time has elapsed since the image is switched from being output to an external display to being output to the display.

[0022] The processing unit may be configured to start generating an image based on the measured direction when the portable electronic device's movement falls within a predetermined range after the image is switched from being output to an external display to being output to the display.

[0023] The processing unit may be configured to start generating an image based on the measured direction when the temporal change in the relative positional relationship between the portable electronic device and the controller falls within a predetermined range after the state in which the image is output to the external display is switched to the state in which the image is output to the display.

[0024] When switching from a state in which an image is output to an external display to a state in which an image is output to the display, it is highly likely that the positional relationship between the portable electronic device and the controller has changed, and therefore if the image is output as is while it is being output to the external display, it may cause the user to feel uncomfortable. Each of the above-mentioned configurations can reduce the possibility of causing such discomfort to the user.

[0025] According to another embodiment, there is provided a portable electronic device capable of communicating with a controller that accepts user operations. The portable electronic device includes a display, an interface for outputting an image to an external display separate from the display, a processing unit that generates an image in response to a user operation on the controller, and a measurement unit that measures the direction of the controller relative to the portable electronic device. The processing unit generates an image based on the measured direction when the image is output to the display, and limits the generation of the image based on the measured direction when the image is output to the external display.

[0026] According to yet another embodiment, there is provided a processing method executed by a portable electronic device including a display and an interface for outputting an image to an external display separate from the display, the processing method including the steps of communicating with a controller that accepts user operations, determining a direction of the controller relative to the computer, and generating an image in response to the user operations on the controller, the step of generating the image including, if the image is to be output to the display, generating the image based on the determined direction, and, if the image is to be output to the external display, limiting the generation of the image based on the determined direction.

[0027] According to yet another embodiment, there is provided a program executed on a computer including a display and an interface for outputting an image to an external display separate from the display, the program causing the computer to perform the steps of communicating with a controller that accepts user operations, determining an orientation of the controller relative to the portable electronic device, and generating an image in response to the user operations on the controller, the step of generating the image including, if the image is to be output to the display, generating the image based on the determined orientation, and, if the image is to be output to the external display, limiting generation of the image based on the determined orientation. [Effects of the Invention]

[0028] According to the present disclosure, operability and / or interest can be further improved. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing an example of a hardware configuration of a game device in the system according to the present embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of a hardware configuration of a controller of the system according to the present embodiment. [Figure 4] FIG. 1 is a schematic diagram showing an example of a usage form of a system according to an embodiment of the present invention. [Figure 5] FIG. 10 is a schematic diagram showing another example of a usage form of the system according to the present embodiment. [Figure 6] 1A and 1B are diagrams for explaining the principle of direction measurement in the system according to the present embodiment. [Figure 7] 1 is a schematic diagram showing an example of an antenna module in which a plurality of antenna elements are arranged in one direction. [Figure 8] 1 is a schematic diagram showing an example of an antenna module in which a plurality of antenna elements are arranged in two directions. [Figure 9] FIG. 2 is a schematic diagram showing an example of the configuration of a short-range communication unit in the system according to the present embodiment. [Figure 10] FIG. 2 is a schematic diagram showing an example of a configuration of a frame transmitted by a controller of the system according to the present embodiment. [Figure 11] FIG. 10 is a schematic diagram showing an example of a screen output by the game device of the system according to the present embodiment in a stance mode. [Figure 12] FIG. 10 is a schematic diagram showing an example of a screen output by the game device of the system according to the present embodiment in flat mode. [Figure 13]FIG. 10 is a schematic diagram showing an example of a screen output by the game device of the system according to the present embodiment in an external output mode. [Figure 14] FIG. 10 is a schematic diagram showing an example of a screen when the game device of the system according to the present embodiment is switched to an external output mode. [Figure 15] FIG. 10 is a schematic diagram showing an example of a screen when the game device of the system according to the present embodiment is switched to an external output mode. [Figure 16] 10A and 10B are diagrams for explaining an example of operation of the antenna module in the system according to the present embodiment. [Figure 17] 10 is a flowchart showing a processing procedure executed by the game device of the system according to the present embodiment. [Figure 18] 18 is a flowchart showing the procedure of the direction measurement process shown in FIG. 17. DETAILED DESCRIPTION OF THE INVENTION

[0030] The present embodiment will be described in detail with reference to the drawings, in which the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0031] [A. Configuration example] First, an example of the configuration of a system 1 according to the present embodiment will be described.

[0032] In the following description, a gaming device will be used as an example of a portable electronic device, but the portable electronic device is not limited to a gaming device and can be any computer such as a smartphone, tablet, personal computer, etc. Note that a portable electronic device can also be called a portable information processing device.

[0033] 1 is a schematic diagram showing an example of the configuration of a system 1 according to the present embodiment. Referring to FIG. 1, system 1 includes a game device 100 and one or more controllers 200.

[0034] In this specification, the term "controller" is intended to encompass any device that accepts user operations, and is not limited to game controllers, but also encompasses, for example, general-purpose input devices such as keyboards, mice, and pen tablets, as well as operating devices used for specific purposes.

[0035] The game device 100 exchanges data with each of the controllers 200 using wireless signals. That is, the controllers 200 transmit wireless signals in response to user operations.

[0036] The controller 200 may be attachable to the game device 100. In this embodiment, the controllers 200 are attached to both sides of the game device 100. When the controller 200 is attached to the game device 100, the game device 100 may be electrically connectable to the controller 200. At this time, data may be exchanged via wired communication. Note that even when the controller 200 is attached to the game device 100, data may be exchanged via wireless communication.

[0037] For convenience of explanation, differences in structure and function between the controllers 200 will not be mentioned, but the structure and function of the controller 200 may differ depending on the side (left / right) on which it is attached to the game device 100.

[0038] The game device 100 has a display 106 that displays any image, and a touch panel 108 that accepts user operations.

[0039] The game device 100 has an antenna module 124 for receiving wireless signals from the controller 200. The antenna module 124 may be disposed at any position on the game device 100, but is preferably disposed so as to be parallel to the display surface of the display 106, for example.

[0040] Each of the controllers 200 has an operation unit 210 that accepts user operations. The operation unit 210 is configured with, for example, push buttons, a cross key, an operation lever, and the like.

[0041] 2 is a schematic diagram showing an example of a hardware configuration of game device 100 of system 1 according to the present embodiment. Referring to Fig. 2, game device 100 includes processor 102, memory 104, display 106, touch panel 108, storage 110, near-field communication unit 120, antenna module 124, wireless communication unit 126, speaker 128, microphone 130, gyro sensor 132, first controller interface 134, second controller interface 136, cradle interface 138, and memory card interface 140.

[0042] The processor 102 is a processing entity that executes the processes provided by the game device 100. The memory 104 is a storage device accessible by the processor 102, and is, for example, a volatile storage device such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The storage 110 is, for example, a non-volatile storage device such as a flash memory.

[0043] The processor 102 performs the processing described below by reading a program stored in the storage 110, expanding it into the memory 104, and executing it. The storage 110 stores, for example, an application program 112 consisting of instruction codes for performing any information processing, and a system program 114 that provides libraries and the like necessary for program execution.

[0044] Processor 102 executes the processes required for game device 100. In the following description, attention is particularly focused on the processes for generating images to be displayed or output on a display. That is, processor 102 corresponds to a processing unit that generates images in response to user operations on controller 200.

[0045] The short-range communication unit 120 transmits and receives wireless signals to and from one or more controllers 200. Any wireless method such as Bluetooth (registered trademark), ZigBee (registered trademark), wireless LAN (IEEE802.11), or infrared communication can be used for the short-range communication unit 120. In the following description, an example in which Bluetooth is used as the wireless method for the short-range communication unit 120 is shown.

[0046] The antenna module 124 receives wireless signals transmitted from one or more controllers 200. The antenna module 124 may be arranged as an antenna for the short-range communication unit 120 to transmit and receive wireless signals, or the antenna module 124 may be arranged in addition to a normal antenna for the short-range communication unit 120 to transmit and receive wireless signals.

[0047] The short-range communication unit 120 has a direction measurement unit 122 that measures the direction in which the controller 200 exists relative to the game device 100 (i.e., the direction in which the controller 200 exists as seen from the game device 100). More specifically, the direction measurement unit 122 measures the direction in which the controller 200 that transmitted the wireless signal exists, based on a wireless signal from the controller 200 received by the antenna module 124. Note that the function provided by the direction measurement unit 122 may be provided by the short-range communication unit 120, or may be provided by cooperation between the short-range communication unit 120 and the processor 102. Details of the measurement process by the direction measurement unit 122 will be described later.

[0048] The wireless communication unit 126 exchanges data using wireless signals with a wireless repeater connected to the Internet, etc. The wireless communication unit 126 can employ any wireless system, such as a wireless LAN (IEEE802.11) or a public wireless line (4G, 5G, etc.).

[0049] The speaker 128 generates any sound around the game device 100. The microphone 130 collects sounds occurring around the game device 100.

[0050] The gyro sensor 132 detects the orientation of the game device 100 . When the controller 200 is attached to the game device 100, the first controller interface 134 and the second controller interface 136 exchange data with the attached controller 200.

[0051] The cradle interface 138 exchanges data with the cradle when the game device 100 is placed on the cradle, which will be described later.

[0052] The memory card interface 140 reads data stored in a removable memory card 142 from the memory card 142, and writes data to the memory card 142. The memory card 142 may store an application program or the like.

[0053] 3 is a schematic diagram showing an example of a hardware configuration of controller 200 of system 1 according to the present embodiment. Referring to FIG. 3, controller 200 includes a processor 202, a memory 204, an operation unit 210, an acceleration sensor 212, a short-range communication unit 220, and a main body communication unit 230.

[0054] The processor 202 loads a program into the memory 204 and executes it to realize the processing required by the controller 200 .

[0055] The operation unit 210 generates a signal in response to a user operation. The acceleration sensor 212 generates a signal in response to an acceleration occurring in the controller 200.

[0056] The short-range communication unit 220 transmits and receives wireless signals to and from the game device 100 . The main body communication unit 230 exchanges data with the game device 100 when the controller 200 is attached to the game device 100 .

[0057] In this specification, the term "processor" not only refers to a processing circuit that executes processing according to instruction codes written in a program, such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), but also includes hardwired circuits such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Hardwired circuits such as an ASIC or FPGA have pre-formed circuits corresponding to the processing to be executed. Furthermore, in this specification, "processor" also includes circuits that integrate multiple functions, such as a system on chip (SoC), and also includes a combination of a processor in the narrow sense and a hardwired circuit.

[0058] [B. Usage form] Next, several examples of usage patterns of system 1 according to the present embodiment will be described. As described above, a pair of controllers 200 can be attached to game device 100. A user can use game device 100 by holding game device 100 with a pair of controllers 200 attached. This type of usage pattern in which a user holds and uses game device 100 is called a "portable mode." In addition to the portable mode, the following usage patterns are also possible.

[0059] FIG. 4 is a schematic diagram showing an example of a usage form of system 1 according to the present embodiment. FIG. 4 shows a mode in which game device 100 is used with an image displayed on display 106. Referring to FIG. 4, game device 100 is supported by stand 144 and placed with display 106 facing sideways or diagonally upward, and one or more users operate controller 200 while viewing an image displayed on display 106. In the following description, for convenience, the mode in which game device 100 is used with stand 144 as shown in FIG. 4 is referred to as a "standing mode."

[0060] Alternatively, game device 100 can be used while placed on a flat surface such as a table with display 106 facing upward. In the following description, for convenience, the mode in which game device 100 is used while placed with display 106 facing upward is referred to as "flat-lay mode." Note that if application program 112 executed on game device 100 is compatible with the flat-lay mode, processing may be executed in the flat-lay mode regardless of the orientation of game device 100.

[0061] FIG. 5 is a schematic diagram showing another example of a usage form of system 1 according to the present embodiment. FIG. 5 shows a mode in which game device 100 is used with an image displayed on external display 300 separate from display 106 of game device 100. Referring to FIG. 5, with game device 100 placed on cradle 350, one or more users operate controller 200 while viewing an image displayed on external display 300. In this manner, game device 100 includes display 106 and cradle interface 138 (see FIG. 2) for outputting an image to external display 300 separate from display 106. In the following description, for convenience, the mode in which game device 100 is used with game device 100 placed on cradle 350 will be referred to as the "external output mode."

[0062] In the external output mode, communication between the game device 100 and the cradle 350 and / or between the cradle 350 and the external display 300 may be wired communication or wireless communication.

[0063] As described above, the system 1 according to the present embodiment can be used in different ways.

[0064] [C.Direction measurement] Next, a description will be given of direction measurement that can be performed by the system 1 according to the present embodiment.

[0065] In system 1 according to the present embodiment, game device 100 has a function of measuring the direction in which controller 200 is located, based on a wireless signal received from controller 200. More specifically, game device 100 calculates the direction in which controller 200 is located, based on a phase difference that occurs when wireless signals are received by a plurality of antenna elements that are arranged at separate locations.

[0066] Fig. 6 is a diagram for explaining the principle of direction measurement of system 1 according to the present embodiment. Referring to Fig. 6, antenna module 124 has a plurality of antenna elements 125-1 and 125-2 (hereinafter, sometimes collectively referred to as "antenna elements 125").

[0067] Because the distance between the game device 100 and the controller 200 is sufficiently long compared to the wavelength of the wireless signal, the wireless signal transmitted from the controller 200 can be regarded as a plane wave. Therefore, equiphase surface 240 of the wireless signal transmitted from the controller 200 is orthogonal to a line connecting the controller 200 and the center O of antenna element 125-1 and antenna element 125-2 (a line that forms an angle θ with the line connecting antenna element 125-1 and antenna element 125-2). Note that angle θ is the angle at which the wireless signal is incident on antenna module 124, and is also referred to as the angle of arrival.

[0068] 6, antenna element 125-1 intersects with equiphase surface 240 of phase φ1, and antenna element 125-2 intersects with equiphase surface 240 of phase φ4. That is, a phase difference Δφ of |phase φ1−phase φ4| occurs between the radio signal received by antenna element 125-1 and the radio signal received by antenna element 125-2. This phase difference Δφ depends on angle θ and inter-element distance d.

[0069] More specifically, if the wavelength of the radio signal is λ, the following relational expression holds: Δφ=2π×(d×cos(θ) / λ) When this relational expression is rearranged with respect to the angle θ (arrival angle), it can be expressed as follows:

[0070] θ=cos -1 ((Δφ×λ) / (2π×d)) Here, since the wavelength λ of the radio signal and the inter-element distance d are known, the direction (angle θ) in which the controller 200 is located can be calculated based on the phase difference Δφ occurring in the radio signals received by the two antenna elements 125.

[0071] The antenna module 124 may have two or more antenna elements 125, but using more antenna elements 125 can improve measurement accuracy.

[0072] FIG. 7 is a schematic diagram showing an example of an antenna module 124 in which a plurality of antenna elements 125 are arranged in one direction. The antenna module 124 shown in FIG. 7 has four antenna elements 125-1 to 125-4 arranged in a line along the X-axis. With such an arrangement of the antenna elements 125, it is possible to measure the angle of arrival (one-dimensional) with respect to the X-axis. More specifically, by using two antenna modules 124, it is possible to measure the angle of arrival of a wireless signal transmitted from the controller 200 as seen from the center of the two antenna modules 124. Note that any two adjacent antenna elements 125 may be selected, or two adjacent antenna elements 125 may be selected sequentially. Alternatively, in other embodiments, any two antenna elements that are not adjacent to each other may be selected.

[0073] In the example shown in Figure 7, angle θ1 can be measured by selecting antenna element 125-1 and antenna element 125-2, angle θ2 can be measured by selecting antenna element 125-2 and antenna element 125-3, and angle θ3 can be measured by selecting antenna element 125-3 and antenna element 125-4.

[0074] From each measured angle, the position (or distance) of the controller 200 can be measured in addition to the direction of the controller 200 that transmitted the wireless signal. Note that the measured direction and position are both relative values with respect to the game device 100. Therefore, in this specification, the process of measuring the "direction" can include the process of measuring the "position."

[0075] Fig. 8 is a schematic diagram showing an example of antenna module 124 in which multiple antenna elements 125 are arranged in two directions. Antenna module 124 shown in Fig. 8 has 4x4 antenna elements 125-11 to 125-44 arranged along the X-axis and Y-axis, respectively. With antenna elements 125 arranged in this way, the angle of arrival with respect to the X-axis and Y-axis (two-dimensional) can be measured.

[0076] More specifically, by using two antenna modules 124 arranged in the same row on the X axis, it is possible to measure the component of the arrival angle of a wireless signal transmitted from the controller 200 relative to the X axis. Similarly, by using two antenna modules 124 arranged in the same row on the Y axis, it is possible to measure the component of the arrival angle of a wireless signal transmitted from the controller 200 relative to the Y axis.

[0077] In the example shown in Figure 8, by selecting antenna elements 125-11 and 125-12, angle θx, which is the component of the arrival angle relative to the X axis, can be measured, and by selecting antenna elements 125-34 and 125-44, angle θy, which is the component of the arrival angle relative to the Y axis, can be measured.

[0078] As in FIG. 7, by performing measurements multiple times with different combinations of antenna elements 125, it is possible to measure the position (or distance) of controller 200 in addition to the direction of controller 200.

[0079] To perform the above-described direction measurement, it is necessary to receive the same radio signal using multiple antenna elements 125. Multiple receiving circuits may be provided, or a common receiving circuit may be configured to sequentially switch between the multiple antenna elements 125 to be used for reception.

[0080] 9 is a schematic diagram showing an example of the configuration of short-range communication unit 120 of system 1 according to the present embodiment. In FIG. 9, an example is shown in which direction measurement unit 122 is implemented as part of the configuration of short-range communication unit 120.

[0081] 9, short-range communication unit 120 includes multiplexer 1221, detector 1222, differentiator 1223, delay element 1224, angle calculation unit 1225, control unit 1226, and decoder 1227. Direction measurement unit 122 mainly includes differentiator 1223, delay element 1224, angle calculation unit 1225, and control unit 1226.

[0082] The multiplexer 1221 selects one antenna element 125 from the plurality of antenna elements 125 in accordance with a selection command from the control unit 1226 .

[0083] The detector 1222 decodes the radio signal received by the antenna element 125 connected via the multiplexer 1221, and outputs the decoded signal.

[0084] Differentiator 1223 calculates the phase difference between the signals output from detector 1222. The signal output from detector 1222 is directly input to one end of differentiator 1223, and the signal output from detector 1222 is input to the other end of differentiator 1223 via delay element 1224. The delay time of delay element 1224 is set according to the selection time by multiplexer 1221. In other words, differentiator 1223 receives a signal obtained by decoding the radio signal received by currently selected antenna element 125 and a signal obtained by decoding the radio signal received by the antenna element 125 selected immediately before.

[0085] Angle calculation section 1225 calculates an angle (arrival angle) from the phase difference calculated by differentiator 1223. In angle calculation section 1225, inter-element distance d and wavelength λ are set in advance.

[0086] The control unit 1226 outputs a selection command to the multiplexer 1221, and performs statistical processing (for example, averaging or outlier removal) on the angles sequentially calculated by the angle calculation unit 1225 in response to the selection command, thereby outputting a measurement result indicating the direction in which the controller 200 is located. The measurement result may include the distance to the controller 200 in addition to the one-dimensional angle or two-dimensional angle indicating the direction in which the controller 200 is located.

[0087] The decoder 1227 reconstructs a frame from the signal output from the detector 1222. The decoder 1227 also outputs, to the control unit 1226, identification information for identifying the transmission source of the wireless signal, based on the information included in the frame.

[0088] FIG. 10 is a schematic diagram showing an example of a configuration of a frame transmitted by controller 200 of system 1 according to the present embodiment.

[0089] 10, a frame 250 includes a preamble 251, a destination address 252, data 253, a CRC 254, and direction measurement data 256. The preamble 251, the destination address 252, the data 253, and the CRC 254 correspond to an actual frame 255.

[0090] The direction measurement data 256 includes a plurality of constant values (usually "1"). Since the values included in the direction measurement data 256 do not change over time, the radio signal is a sine wave whose phase and amplitude do not change over time. This sine wave is used to perform the direction measurement described above.

[0091] Since destination address 252 includes identification information for identifying the controller 200 that transmitted the wireless signal, it is possible to measure the direction for each controller 200 when multiple controllers 200 are connected to game device 100. That is, based on the information included in destination address 252, it is possible to identify which controller 200 the wireless signal is from, and then measure the direction to which the identified controller 200 is located.

[0092] [D. Example of processing using the measurement results of direction measurement] Next, some examples of processing using the measurement results of the above-mentioned direction measurement will be described.

[0093] In system 1 according to the present embodiment, game device 100 is capable of generating an image based on the direction in which controller 200 is present. However, only when the direction in which controller 200 is present can be properly measured, game device 100 may generate an image based on the measured direction. More specifically, when an image is output to display 106 (specifically, in the standing mode or flat mode), game device 100 generates an image based on the measured direction. In contrast, when an image is output to external display 300 (specifically, in the external output mode), game device 100 restricts the generation of an image based on the measured direction. Each mode will be described in detail below.

[0094] (d1: stance mode) 11 is a schematic diagram showing an example of a screen output in the standing mode by game device 100 of system 1 according to the present embodiment. Referring to Fig. 11(A), user A operates object 401 using controller 200A, user B operates object 402 using controller 200B, user C operates object 403 using controller 200C, and user D operates object 404 using controller 200D.

[0095] At this time, the display positions of objects 401 to 404 reflect the positions of users A to D (controllers 200A to 200D) relative to game device 100. That is, from the left side of game device 100, controller 200A, controller 200B, controller 200C, and controller 200D are present in this order, and corresponding to this positional relationship, objects 401, 402, 403, and 404 are displayed on display 106 in this order from the left side.

[0096] As the positional relationship between the controllers 200 changes, the image displayed on the display 106 also changes.

[0097] 11(B) shows, as an example, a state in which the positions of user B and user C have been swapped. As a result, the positions of controller 200B and controller 200C change, and the display positions of object 402 and object 403 displayed on display 106 also change.

[0098] Note that the determination of whether or not the game device 100 is in the standing mode may be made based on the detection result by the gyro sensor 132, or if it is assumed that the application program 112 to be executed is executed in the standing mode, processing in the standing mode may be executed regardless of the detection result by the gyro sensor 132.

[0099] In this way, when an image is output to the display 106 , the game device 100 outputs to the display 106 an image based on the direction in which the controller 200 is positioned relative to the game device 100 .

[0100] (d2: Flat mode) 12 is a schematic diagram showing an example of a screen output by game device 100 of system 1 according to the present embodiment in the flat-lay mode. In the standing mode shown in FIG. 11, the user is present in front of display 106, but in the flat-lay mode shown in FIG. 12, the user is present around display 106.

[0101] In the example shown in FIG. 12, it is assumed that users A to D are present around game device 100, and each user operates controller 200 to play a game or the like.

[0102] Objects 411-414 indicating information for each user are displayed on display 106 according to the positions of users A-D (controllers 200A-200D) relative to game device 100. More specifically, object 411 including information for user A is placed at a position corresponding to the direction in which controller 200A is located. Similarly, object 412 including information for user B is placed at a position corresponding to the direction in which controller 200B is located, object 413 including information for user C is placed at a position corresponding to the direction in which controller 200C is located, and object 414 including information for user D is placed at a position corresponding to the direction in which controller 200D is located.

[0103] Furthermore, each of the objects 411 to 414 is displayed in an orientation that makes it easy to view from the direction in which the corresponding user (that is, the controller 200) is located.

[0104] It should be noted that the determination of whether or not the game device 100 is in the flat-lay mode may be made based on the detection result by the gyro sensor 132, or if it is assumed that the application program 112 to be executed is executed in the flat-lay mode, processing in the flat-lay mode may be executed regardless of the detection result by the gyro sensor 132.

[0105] In this way, when an image is output to the display 106 , the game device 100 outputs to the display 106 an image based on the direction in which the controller 200 is positioned relative to the game device 100 .

[0106] (d3: External output mode) 13 is a schematic diagram showing an example of a screen output by game device 100 of system 1 according to the present embodiment in the external output mode. In the standing mode and flat mode described above, an image is displayed on display 106 of game device 100. Because the positional relationship between display 106 and antenna module 124 is fixed, an image generated based on the measured direction reflects the location of controller 200.

[0107] In contrast, in the external output mode, the image is displayed on an external display 300 separate from the display 106, so even if an image is generated based on a direction measured using the antenna module 124 as a reference, it may not be possible to properly reflect the user's location.

[0108] Therefore, in the external output mode, generation of an image based on the measured direction may be restricted. As a result, for example, as shown in Figures 13(A) and 13(B), even if the position of the user (controller 200) changes, the change in position is not reflected in the image output to external display 300.

[0109] Note that the process of restricting image generation based on the measured direction may include not only the process of not reflecting the measured direction as described above, but also the process of disabling part of the process or function based on the direction. That is, in this specification, "restricting image generation based on the measured direction" means disabling at least part of any process or function based on the measured direction that is executed in the stance mode or flat mode.

[0110] In this case, the direction measurement itself may be performed, but the measurement result may be discarded. Alternatively, the direction measurement itself may not be performed. That is, the direction measurement unit 122 of the game device 100 may disable measurement of the direction in which the controller 200 exists when an image is output to the external display 300.

[0111] However, even in the external output mode, if a predetermined condition is satisfied, the generation of an image based on the measured direction may not be restricted. As an example, if a predetermined condition is satisfied, the restriction may be lifted. The predetermined condition includes, for example, a case where the positional relationship between the external display 300 and the game device 100 (antenna module 124) is considered to be similar to the positional relationship between the display 106 and the antenna module 124.

[0112] The game device 100 may determine whether or not a predetermined condition is satisfied by any method, or may provide information to the user so that the predetermined condition is satisfied. For example, if a predetermined positional relationship between the game device 100 and the external display 300 is adopted as the predetermined condition for removing the restriction on image generation based on the direction measured in the external output mode, a screen may be provided to assist the user in performing operations so that the positional relationship can be realized.

[0113] Note that whether or not the game device 100 is in the external output mode may be determined based on the connection state of the cradle interface 138, or if it is assumed that the application program 112 to be executed is executed in the external output mode, processing in the external output mode may be executed regardless of the connection state of the cradle interface 138. Alternatively, processing associated with the external output mode may be executed in response to the user selecting "external output mode" on the game device 100.

[0114] In this way, the game device 100 may limit the generation of images based on the measured direction when the images are output to the external display 300.

[0115] (d4: Switch to external output mode) In this embodiment, when switching from the standing mode or the flat mode to the external output mode, the user places game device 100 on cradle 350. At this time, the image generated based on the direction measured immediately before placing game device 100 on cradle 350 may continue to be used as is (after placing game device 100 on cradle 350, a new image based on the direction measured may not be generated).

[0116] However, while the user carries the game device 100 to the position of the cradle 350, the positional relationship between the game device 100 held by the user and the controller 200 held by the user or the controllers 200 held by another user may change, which may result in an image being generated in a manner not intended by the user or another user. Therefore, if the image generated earlier continues to be used as is even after the game device 100 is placed on the cradle 350, the user may feel uncomfortable.

[0117] Therefore, when changing from the standing mode or the flat lay mode to the external output mode, an image may be generated according to a predetermined setting. For example, with respect to an object whose position or orientation changes depending on the direction of the controller 200, the object may be displayed in a predetermined position or orientation in the external output mode. Note that when changing from the standing mode or the flat lay mode to the external output mode, the application program 112 being executed may be temporarily stopped.

[0118] 14 and 15 are schematic diagrams showing example screens when game device 100 of system 1 according to the present embodiment is switched to the external output mode. Fig. 14 shows an example screen in the flat lay mode, and Fig. 15 shows an example screen in the external output mode.

[0119] As shown in FIG. 14, in the flat mode (or the standing mode), an image is generated based on the measured direction, and therefore the positions and orientations of the objects 411 to 414 correspond to the direction of the controller 200 relative to the game device 100.

[0120] In contrast, as shown in Fig. 15, when the mode is switched to the external output mode, image generation based on the measured direction is restricted, and the positions and orientations of the objects 411 to 414 are in accordance with predetermined settings. Specifically, in this embodiment, the objects 411 to 414 corresponding to each user are arranged in the order of user registration (user A, user B, user C, and user D in the example shown in Fig. 15) set before the game starts. The display orientations of the objects 411 to 414 are also aligned.

[0121] In this way, when the state in which an image is output to the display 106 is switched to a state in which an image is output to the external display 300, the game device 100 generates an image according to a predetermined setting independent of the measured direction.

[0122] It should be noted that if the above-mentioned predetermined conditions are satisfied immediately after switching to the external output mode, the restriction on image generation based on the measured direction may be lifted.

[0123] (d5: Switching from external output mode) When switching from the external output mode to the standing mode or the flat mode, the restriction on image generation based on the measured direction may be lifted immediately after the mode is switched. Note that when the external output mode is switched to the standing mode or the flat mode, the application program 112 that is currently running may be temporarily stopped.

[0124] Alternatively, some buffer period or grace period may be provided. In this case, the restriction on image generation based on the measured orientation may be lifted only when any starting condition is satisfied after switching from the external output mode to the standing mode or the flat mode. Examples of such starting conditions include the following:

[0125] As an example, after the state in which an image is output to the external display 300 is switched to the state in which an image is output to the display 106, the game device 100 may start generating an image based on the measured direction a predetermined time after the switch.

[0126] Alternatively, the game device 100 may be configured to start generating an image based on the measured direction when the movement of the game device 100 falls within a predetermined range after switching from a state in which the image is output to the external display 300 to a state in which the image is output to the display 106.

[0127] Alternatively, after the state in which the image is output to the external display 300 is switched to the state in which the image is output to the display 106, the game device 100 may start generating an image based on the measured direction when the temporal change in the relative positional relationship between the game device 100 and the controller 200 falls within a predetermined range.

[0128] For example, when a user removes game device 100 from cradle 350 and places it on a table or the like, the direction of controller 200 may not be measured properly while the user is adjusting the position where game device 100 is placed. Alternatively, it is conceivable that the user may be adjusting the playing location while holding controller 200. Under such circumstances, if an image is generated based on the measured direction, the image may be unintended by the user, which may cause the user to feel uncomfortable. For example, by setting the above-described start conditions, the sense of discomfort felt by the user may be reduced.

[0129] (d6: Orientation-based image generation) In this specification, generating an image based on a direction includes processing for changing the display position, display direction (display orientation), display mode (color, decoration, etc.) of an object based on the measured direction. For example, Fig. 11 shows an example in which the display position of an object is changed based on the measured direction, and Fig. 12 shows an example in which the display position and display direction of an object are changed based on the measured direction.

[0130] Furthermore, generating an image based on the direction may include a process of operating an object (player character) according to rules corresponding to the measured direction, and a process of differently interpreting a user operation on the controller 200. For example, in Fig. 12, when user A presses the up button on controller 200A, the game device 100 interprets this as an instruction for a movement operation toward the top of the screen, whereas when user D presses the up button on controller 200D, the game device 100 interprets this as an instruction for a movement operation toward the right side of the screen.

[0131] In this way, generating an image based on orientation encompasses processes that not only result in significant changes to the image itself displayed on display 106 or external display 300, but also in significant changes to the way the image is generated.

[0132] [E. Variations of Direction Measurement] Next, some variations on direction measurement will be described.

[0133] (e1: Selection of antenna elements according to usage) As described above, there are several usage patterns (for example, the three modes described above) in the system 1. Depending on the usage pattern, some or all of the antenna elements 125 included in the antenna module 124 may be used. That is, the direction measurement unit 122 of the game device 100 may change the antenna element to be used for calculating the direction in which the controller 200 is located, out of the plurality of antenna elements 125, depending on at least one of the attitude of the game device 100 and the application program 112 being executed.

[0134] Fig. 16 is a diagram illustrating an example of the operation of antenna module 124 of system 1 according to the present embodiment. Fig. 16 shows an example configuration including 4 × 4 antenna elements 125, similar to antenna module 124 shown in Fig. 8. Antenna module 124 shown in Fig. 16 may be arranged parallel to the display surface of display 106. Typically, antenna module 124 has a plurality of antenna elements 125 arranged along each of two directions that define the display surface of display 106 (in the example shown in Fig. 16, the vertical direction and the horizontal direction of the paper), with at least two antenna elements 125 in each direction.

[0135] 16(A) illustrates an example of antenna elements 125 used in the standing mode. In the standing mode, the user is present in front of display 106, and therefore it may be sufficient to measure the direction of controller 200 in one dimension. Therefore, of the multiple antenna elements 125 included in antenna module 124, one row of antenna elements 125 may be used.

[0136] At this time, since the bottom side of the game device 100 is close to the table on which the game device 100 is placed, there is a possibility that the received wireless signal may contain a reflected component. Therefore, when using a row of antenna elements 125, the upper antenna elements 125 may be used. As shown in FIG. 16(A), the upper row of antenna elements 125 among the plurality of antenna elements 125 may be used.

[0137] In this way, the direction measurement unit 122 of the game device 100 uses some of the multiple antenna elements 125 when the display 106 is placed in a position (first position) where it is facing sideways or diagonally upward, and / or when the application program 112 corresponding to that position is being executed (i.e., in the standing mode).

[0138] 16(B) illustrates antenna elements 125 used in the flat-lay mode. In the flat-lay mode, the user is present around display 106. Therefore, it may be desirable to measure the direction of controller 200 in two dimensions. That is, as shown in FIG. 16(B), all of antenna elements 125 included in antenna module 124 may be used.

[0139] In this way, the direction measurement unit 122 of the game device 100 uses all of the multiple antenna elements 125 when the game device 100 is placed in a position where the display 106 faces upward (second position) and / or when the application program 112 corresponding to that position is being executed (i.e., in flat-lay mode).

[0140] 16(C) illustrates an example of the antenna element 125 used in the external output mode. In the external output mode, the generation and display of an image based on the measured direction is basically limited. Therefore, it is sufficient to use one antenna element 125 included in the antenna module 124. In other words, it is not necessary to select the antenna elements 125 sequentially.

[0141] However, even in the external output mode, if the restriction on image generation and display based on the measured direction is lifted, multiple antenna elements 125 may be used. As an example, as shown in Fig. 16(A), a row of antenna elements 125 may be used.

[0142] In this embodiment, multiple antenna elements 125 are arranged along each of the two directions that define the display surface of display 106, but when detecting direction using a method similar to this embodiment, two or more antenna elements 125 may be arranged in a different manner.

[0143] (e2: Selection of antenna elements according to the location of the controller) After measuring the direction in which the controller 200 is located, the direction measurement unit 122 of the game device 100 may repeat the measurement of the direction using only the antenna element 125 that is suitable for measuring the direction. In other words, the direction measurement unit 122 of the game device 100 may change the antenna element 125 to be used to calculate the direction in which the controller 200 is located, among the multiple antenna elements 125, depending on the previously measured direction in which the controller 200 is located.

[0144] For example, in the flat-lay mode, depending on the measurement result that the controller 200 is present near one of the sides of the display 106, only the antenna element 125 suitable for measuring the direction in which the controller 200 is present may be used.

[0145] (e3: Change the measurement frequency according to the controller's behavior) After measuring the direction in which the controller 200 is located, the direction measurement unit 122 of the game device 100 may optimize the measurement frequency in accordance with the measurement result. In other words, the direction measurement unit 122 of the game device 100 may change the frequency at which the direction in which the controller 200 is located is calculated in accordance with the previously measured direction in which the controller 200 is located.

[0146] For example, when the controller 200 is located relatively far from the game device 100, the amount of change over time in the direction of the controller 200 relative to the game device 100 is relatively small, so the frequency of measurement can be reduced.

[0147] Furthermore, the direction measurement unit 122 of the game device 100 may change the measurement frequency depending on the degree of change over time in the direction being measured. For example, if the variation in the measurement results from multiple direction measurements performed within a predetermined period is within a predetermined range, the movement of the controller 200 may be deemed to be relatively small, and the measurement frequency may be reduced below a reference value. Conversely, if the variation in the measurement results exceeds the predetermined range, the movement of the controller 200 may be deemed to be relatively large, and the measurement frequency may be increased above the reference value.

[0148] By adjusting the measurement frequency in accordance with the behavior of the controller 200, it is possible to suppress an increase in the processing load on the game device 100, and it is also possible to expect a reduction in power consumption.

[0149] (e4:Other) Regardless of whether the game device 100 is in a flat mode or a standing mode, that is, regardless of the position of the game device 100 or the application program being executed, a certain number of antenna elements 125 (for example, all of the antenna elements 125) may always be used.

[0150] [F. Processing Procedure] Next, an example of a processing procedure executed by system 1 according to the present embodiment will be described.

[0151] 17 is a flowchart showing a processing procedure executed by game device 100 of system 1 according to the present embodiment. Each step shown in FIG. 17 is typically realized by processor 102 of game device 100 executing application program 112.

[0152] 17, game device 100 determines whether or not application program 112 being executed supports generation of an image based on a direction (step S100). If application program 112 does not support generation of an image based on a direction (NO in step S100), game device 100 does not measure the direction of controller 200, and generates an image according to predetermined settings (step S102). The generated image is output to display 106 or external display 300.

[0153] Game device 100 determines whether an instruction to end the application program has been given (step S104). If an instruction to end the application program has not been given (NO in step S104), the processing from step S102 onwards is repeated. If an instruction to end the application program has been given (YES in step S104), the processing ends.

[0154] If the game device 100 supports generation of an image based on the orientation (YES in step S100), the game device 100 determines the current usage mode (step S106). The usage mode is determined based on whether or not the device is electrically connected to the cradle 350 via the cradle interface 138, and the results of the attitude detection by the gyro sensor 132. Alternatively, the type or settings of the application program 112 being executed may also be used to determine the usage mode.

[0155] If the current usage mode is "portable mode" ("portable mode" in step S106), 102 The following processing is performed:

[0156] If the current usage mode is "standing mode" ("standing mode" in step S106), game device 100 determines the topmost row of antenna elements 125 included in antenna module 124 as the ones to be used (step S108).

[0157] The game device 100 determines whether the currently running application program 112 requests that an image be generated based on the direction (step S110). If the currently running application program 112 requests that an image be generated based on the direction (YES in step S110), the game device 100 measures the direction of the controller 200 (step S112). Then, the game device 100 generates an image based on the measured direction of each controller 200 (step S114). The generated image is output to the display 106 or the external display 300 depending on the current usage mode.

[0158] If the running application program 112 does not request that an image be generated based on the orientation (NO in step S110), the processes of steps S112 and S114 are skipped.

[0159] Next, the game device 100 determines whether or not the usage pattern has changed (step S116). If the usage pattern has changed (YES in step S116), the processing from step S106 onwards is executed.

[0160] If the usage pattern has not changed (NO in step S116), game device 100 determines whether or not an instruction to end the application program has been given (step S118). If an instruction to end the application program has not been given (NO in step S118), the processing from step S110 onwards is repeated. If an instruction to end the application program has been given (YES in step S118), the processing ends.

[0161] If the current usage mode is "flat lay mode" ("flat lay mode" in step S106), game device 100 determines all antenna elements 125 included in antenna module 124 as objects to be used (step S120), and then executes the processes from step S110 onwards.

[0162] If the current usage mode is "external output mode" ("external output mode" in step S106), the game device 100 determines whether or not a predetermined condition for releasing the restriction on image generation based on the measured direction is satisfied (step S122). If the predetermined condition for releasing the restriction on image generation based on the measured direction is satisfied (YES in step S122), the game device 100 executes the processing from step S120 onwards. Note that instead of the processing from step S120 onwards, the processing from step S108 onwards may be executed. Alternatively, in the external output mode The restrictions in A particular antenna element may be targeted for use if it is suitable for release.

[0163] If the predetermined condition for removing the restriction on image generation based on the measured direction is not satisfied (NO in step S122), the game device 100 does not measure the direction of the controller 200, and generates an image according to predetermined settings (step S124). The generated image is output to the external display 300.

[0164] Next, the game device 100 determines whether the usage pattern has changed (step S126). If the usage pattern has changed (YES in step S126), 106 The following processing is performed:

[0165] If the usage pattern has not changed (NO in step S126), game device 100 determines whether an instruction to end the application program has been given (step S128). If an instruction to end the application program has not been given (NO in step S128), the processing from step S124 onwards is repeated. If an instruction to end the application program has been given (YES in step S128), the processing ends.

[0166] Fig. 18 is a flowchart showing the procedure of the direction measurement process shown in Fig. 17. Referring to Fig. 18, the game device 100 extracts two adjacent antenna elements 125 from among the antenna elements 125 to be used (step S200). The game device 100 selects one of the two extracted antenna elements 125 (step S202) and receives a wireless signal at the selected antenna element 125 (step S204). Next, the game device 100 selects the other of the two extracted antenna elements 125 (step S206) and receives a wireless signal corresponding to the same frame at the selected antenna element 125 (step S208).

[0167] The game device 100 then calculates the phase difference between the wireless signal received in step S204 and the wireless signal received in step S208 (step S210), and calculates an angle indicating the direction in which the controller 200 is located based on the calculated phase difference (step S212). Furthermore, the game device 100 adds identification information for identifying the controller 200 that is the sender of the wireless signals received by the two antenna elements 125, and stores the calculated angle (step S214).

[0168] The game device 100 determines whether or not a predetermined measurement completion condition is met (step S216). The predetermined measurement completion condition includes a condition such as measurement for a predetermined time period or a predetermined number of measurements.

[0169] If the predetermined measurement completion condition is not met (NO in step S216), the processes from step S200 onwards are repeated.

[0170] If a predetermined measurement completion condition is met (YES in step S216), the game device 100 calculates the direction for each controller 200 by statistically processing one or more angles calculated for each stored controller 200 (step S218), and then the process returns.

[0171] [G. Other Forms] In the above description, an example has been described in which the processor 102 of the game device 100 is responsible for the process of generating an image in response to a user operation on the controller 200. However, the process of generating an image may be performed using a computing resource other than the game device 100. Typically, a computing resource on a cloud that can communicate with the game device 100 may generate an image. In this case, the game device 100 transmits a signal indicating a user operation received from the controller 200 and information indicating the direction of the controller 200 to the computing resource, receives the image from the computing resource, and outputs the image to the display 106 or the external display 300. Furthermore, instead of a computing resource on a cloud, any computing resource that can communicate over a local network may be used.

[0172] In the above description, an example has been described in which the direction of the controller 200 is measured using a wireless signal transmitted by the controller 200, but the direction may be measured using other methods. For example, infrared rays or ultrasonic waves may be used.

[0173] Furthermore, instead of a configuration in which the game device 100 receives a wireless signal transmitted by the controller 200 and measures the direction, a configuration in which the controller 200 receives a wireless signal transmitted by the game device 100 and measures the direction may be employed. In this case, by transmitting information indicating the direction measured by the controller 200 to the game device 100, the direction information can be reflected in the generation of an image in the game device 100.

[0174] In the above explanation, the standing mode and the flat-laying mode are exemplified as modes that utilize the direction in which the controller 200 is located while using the display 106 of the game device 100, but only a single mode (which may be either the standing mode or the flat-laying mode, or a mode different from these) may be implemented, or three or more modes may be implemented. When only a single mode is implemented, all antenna elements may be used in that mode to measure the direction in which the controller 200 is located.

[0175] In the above description, processing is performed according to the standing mode or flat mode determined according to application program 112, but if application program 112 includes multiple sub-applications (e.g., mini-games), the mode may be determined not only for application program 112 as a whole, but also for each sub-application.

[0176] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0177] 1 System, 100 Game device, 102, 202 Processor, 104, 204 Memory, 106 Display, 108 Touch panel, 110 Storage, 112 Application program, 114 System program, 120, 220 Near-field communication unit, 122 Direction measurement unit, 124 Antenna module, 125 Antenna element, 126 Wireless communication unit, 128 Speaker, 130 Microphone, 132 Gyro sensor, 134 First controller interface, 136 Second controller interface, 138 Cradle interface, 140 Memory card interface, 142 Memory card, 144 Stand, 200, 200A, 200B, 200C, 200D Controller, 210 Operation unit, 212 Acceleration sensor, 230 Main body communication unit, 240 Equiphase surface, 250, 255 Frame, 251 Preamble, 252 Destination address, 253 data, 256 data for direction measurement, 300 external display, 350 cradle, 401,402,403,404,411,412,413,414 object, 1221 multiplexer, 1222 detector, 1223 differentiator, 1224 delay element, 1225 angle calculation unit, 1226 control unit, 1227 decoder, O center, d distance between elements.

Claims

1. a portable electronic device including a display and an interface for outputting an image to an external display separate from the display; a controller that accepts user operations; a processing unit that generates an image in response to a user operation on the controller; a measurement unit that measures a direction in which the controller is present relative to the portable electronic device; The processing unit generates an image based on the measured direction when the image is output to the display, and limits generation of the image based on the measured direction when the image is output to the external display.

2. the controller transmits a wireless signal in response to the user operation; The system according to claim 1 , wherein the measurement unit calculates the direction in which the controller is located based on a phase difference that occurs when the wireless signal is received by a plurality of antenna elements that are disposed at separate locations.

3. The system according to claim 2 , wherein the measurement unit disables measurement of the direction in which the controller is present when the image is output to the external display.

4. The system according to claim 2 or 3, wherein the processing unit does not restrict generation of an image based on the measured direction when a predetermined condition is satisfied, even if the image is output to the external display.

5. The system according to any one of claims 2 to 4, wherein the measurement unit changes an antenna element of the plurality of antenna elements to be used to calculate the direction in which the controller is located, depending on at least one of the attitude of the portable electronic device and the application program executed by the processing unit.

6. 6. The system according to claim 5, wherein the measurement unit uses some of the plurality of antenna elements when the display is in a first position in which it is placed sideways or diagonally upward, and / or when an application program corresponding to the first position is being executed.

7. 7. The system according to claim 5, wherein the measurement unit uses all of the plurality of antenna elements when the device is in a second position in which the display is placed facing upwards, or when an application program corresponding to the second position is being executed.

8. the plurality of antenna elements include at least two antenna elements arranged along two directions that define a display surface of the display, The system according to claim 6 or 7, wherein the part of the antenna elements used is a row of antenna elements among the plurality of antenna elements.

9. The system of claim 8 , wherein the row of antenna elements is an upper row of antenna elements of the plurality of antenna elements.

10. The system according to any one of claims 5 to 9, wherein the measurement unit changes an antenna element to be used to calculate the direction in which the controller is located, among the plurality of antenna elements, depending on the direction in which the controller is located that was previously measured.

11. The system according to any one of claims 1 to 10, wherein the measurement unit changes the frequency of calculating the direction in which the controller is located, depending on a previously measured direction in which the controller is located.

12. The system according to any one of claims 1 to 11, wherein when the state in which the image is output to the display is switched to the state in which the image is output to the external display, the processing unit generates the image according to predetermined settings independent of the measured direction.

13. The system according to any one of claims 1 to 12, wherein the processing unit starts generating an image based on the measured direction after a predetermined time has elapsed since the image is switched from being output to the external display to being output to the display.

14. The system of any one of claims 1 to 13, wherein the processing unit starts generating an image based on the measured direction when the portable electronic device moves within a predetermined range after the image is switched from being output to the external display to being output to the display.

15. The system according to any one of claims 1 to 14, wherein the processing unit starts generating an image based on the measured direction when a temporal change in the relative positional relationship between the portable electronic device and the controller falls within a predetermined range after the image is switched from being output to the external display to being output to the display.

16. A portable electronic device capable of communicating with a controller that accepts user operations, The display and an interface for outputting an image to an external display other than the display; a processing unit that generates an image in response to a user operation on the controller; a measurement unit that measures a direction in which the controller is present relative to the portable electronic device; The processing unit generates an image based on the measured direction when the image is output to the display, and limits generation of the image based on the measured direction when the image is output to the external display.

17. A processing method executed by a portable electronic device including a display and an interface for outputting an image to an external display separate from the display, comprising: communicating with a controller that accepts user operations; determining a direction in which the controller is located relative to the portable electronic device; generating an image in response to a user operation on the controller; The step of generating an image comprises: generating an image based on the measured orientation if the image is to be output to the display; and if the image is to be output to the external display, limiting image generation based on the measured orientation.

18. A program executed on a computer including a display and an interface for outputting an image to an external display other than the display, the program including: communicating with a controller that accepts user operations; determining the direction in which the controller is located relative to the computer; generating an image in response to a user operation on the controller; The step of generating an image comprises: generating an image based on the measured orientation if the image is to be output to the display; and if the image is to be output to the external display, limiting generation of the image based on the measured direction.

Images