Terminal device

Abstract

To improve a technique to increase the convenience of an operating environment for user operations using a physical user interface.SOLUTION: A terminal device has: an operation unit that accepts operations by a user; a projection unit that projects images onto the operation unit; and a control unit that accepts operations by the user on the operation unit as operations to demonstrate different functions when the images are projected and when the images are not projected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a terminal device. [Background technology]

[0002] Conventionally, techniques have been proposed to improve the convenience of users who play games using terminal devices. For example, a technique is known in which a terminal device for a board game displays an image relating to the progress of the game on a table. Patent Document 1 describes a board game in which a user plays by moving a piece according to the roll of a roulette, in which a menu button is projected and displayed on the table and an operation by the user on the menu button is accepted. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185812 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in technologies that improve the convenience of user operations using a physical user interface (UI).

[0005] In view of the above circumstances, an object of the present disclosure is to provide a terminal device or the like that can improve the technology for improving the convenience of the operating environment for user operations using a physical UI. [Means for solving the problem]

[0006] A terminal device according to one embodiment of the present disclosure has an operation unit that accepts operations by a user, a projection unit that projects an image onto the operation unit, and a control unit that accepts the operation by the user on the operation unit as an operation for performing different functions when the image is projected and when it is not projected. [Effects of the Invention]

[0007] According to an embodiment of the present disclosure, it is possible to improve the technology for improving the convenience of the operating environment for user operations using a physical UI. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating a terminal device according to the present embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a terminal device according to the present embodiment. [Figure 3] 5 is a flowchart showing the operation of the terminal device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The first embodiment will be described below with reference to FIGS.

[0010] 1 shows an example of the configuration of a terminal device 10. In this embodiment, the terminal device 10 is a keyboard instrument. The keyboard instrument as the terminal device 10 includes a keyboard as an operation unit 15 that accepts operations by a user, and a projection unit 17 that projects an image onto the keyboard. However, the terminal device 10 is not limited to this, and may also be a stringed instrument such as a guitar that includes strings on the neck or knobs on the body as the operation unit 15, or a percussion instrument that includes a striking surface as the operation unit 15.

[0011] The terminal device 10 in this embodiment accepts user operations on the operation unit 15 as operations for performing different functions depending on whether an image is projected or not. The user's operations on the keyboard serving as the operation unit 15 are pressing operations. A function performed in response to a pressing operation when an image is not projected on the operation unit 15 is, for example, a function in which a keyboard instrument produces the sound of a key operated by the user (hereinafter referred to as a performance function). On the other hand, a function performed by a pressing operation when an image is projected on the operation unit 15 is a function (hereinafter referred to as a setting function) different from the performance function, including setting a tone, volume, etc., as described below. Hereinafter, a pressing operation that causes the terminal device 10 to perform such a setting function is referred to as a setting operation. If the terminal device 10 is a stringed instrument, the performance operation and the setting operation include operations such as the user plucking strings on the neck serving as the operation unit 15 or turning a knob on the body. If the terminal device 10 is a percussion instrument, the performance operation and the setting operation include operations such as the user striking the striking surface serving as the operation unit 15.

[0012] According to this embodiment, the user does not need to perform any special switching operation to activate the setting function, but can instead perform the same pressing operation as for the performance operation on the operation unit 15 to activate the setting function, which is different from the performance function. The user does not need to switch the movement of their hand, and can activate the performance function and the setting function by continuously operating the same operation unit 15, thereby improving user convenience.

[0013] 2 shows an example of the configuration of the terminal device 10. The terminal device 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input / output unit 14, an operation unit 15, a detection unit 16, and a projection unit 17.

[0014] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). The control unit 11 controls each part of the terminal device 10 and executes processes related to the operation of the terminal device 10.

[0015] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). RAM is, for example, SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory). ROM is, for example, EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 12 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores information used in the operation of the terminal device 10 and information obtained by the operation of the terminal device 10. For example, the storage unit 12 may store a system program, an application program, a database, etc. The information stored in the storage unit 12 may be updatable with information obtained from a network including the Internet via the communication unit 13.

[0016] The communication unit 13 includes at least one communication interface. The communication interface is, for example, an interface compatible with a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation) standard, or 5G (5th Generation) standard, an interface compatible with short-range wireless communication such as Bluetooth (registered trademark), or a LAN interface. The communication unit 13 receives information used in the operation of the terminal device 10 and transmits information obtained by the operation of the terminal device 10.

[0017] The input / output unit 14 includes an input interface that detects input by the user and sends the input information to the control unit 11. The input interface includes an operation unit 15. The keyboard serving as the operation unit 15 according to this embodiment has physical keys, but may also include capacitive keys, etc. The input interface may also include a microphone that accepts audio input. The input / output unit 14 includes an output interface that outputs information generated by the control unit 11 or information read from the memory unit 12 to the user. The output interface is, for example, a speaker that outputs information as sound, or a panel display that outputs information as an image, etc. The input / output unit 14 can notify the user of information acquired via the communication unit 13 by sound, a screen display, etc. Instead of being provided in the terminal device 10, the input / output unit 14 may be connected to the terminal device 10 as an external output device.

[0018] The detection unit 16 has one or more sensors that detect the state or operation of the terminal device 10, or an interface with the sensors, and sends information indicating the detection results of the sensors to the control unit 11. The sensors include various sensors that detect the state of the operation unit 15. The sensors are, for example, pressure sensors that detect the pressure of a pressing operation on a key, or displacement meters that measure the amount of displacement of a key due to a pressing operation on a key using light, current, etc. The detection unit 16 sends position information indicating the position of the key where the operation was detected to the control unit 11.

[0019] The projection unit 17 includes a light source, a light modulator, and a lens. The projection unit 17 modulates light from the light source in response to instructions from the control unit 11, and projects the light magnified through the lens onto the operation unit 15 of the terminal device 10. The projection unit 17 is supported by an arm, for example, and is provided vertically above the operation unit 15.

[0020] The functions of the terminal device 10 are realized by executing a control program according to this embodiment on a processor corresponding to the control unit 11. That is, the functions of the terminal device 10 are realized by software. The control program causes the processor to execute the operations of the terminal device 10, thereby causing the processor to function as the terminal device 10. That is, the processor functions as the terminal device 10 by executing the operations of the terminal device 10 in accordance with the control program.

[0021] The program can be stored in a non-transitory processor-readable medium. Examples of the non-transitory processor-readable medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, or a ROM. The program can be distributed by selling, transferring, or lending a portable medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) that stores the program. The program can also be distributed by storing the program in a server's storage and transferring it from the server to another processor. The program can also be provided as a program product.

[0022] The processor temporarily stores a program stored on a portable medium or transferred from a server in a main storage device. The processor then reads the program stored in the main storage device and executes processing in accordance with the read program. The processor may also read the program directly from the portable medium and execute processing in accordance with the program. The processor may also execute processing in accordance with the received program each time a program is transferred from the server to the processor. Processing may also be executed by a so-called ASP (Application Service Provider) type service that realizes functions only by issuing execution instructions and obtaining results without transferring a program from the server to the processor. Programs include information used for processing by a computer that is equivalent to a program. For example, data that is not a direct instruction to the processor but has properties that define the processor's processing falls under the category of "something equivalent to a program."

[0023] Some or all of the functions of the terminal device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. In other words, some or all of the functions of the terminal device 10 may be realized by hardware.

[0024] The operation of the terminal device 10 according to this embodiment will be described with reference to the flowchart of Fig. 3. The following procedure is started after the power of the terminal device 10 is turned on.

[0025] In S1, the control unit 11 projects an image onto the operation unit 15 using the projection unit 17. Specifically, the control unit 11 reads from the storage unit 12 information associating the image to be projected, the position on the operation unit 15 onto which the image is projected, and the setting function to be performed. The image may be text, a character design, or the like. The position onto which the image is projected may be a position corresponding to the entire operation unit 15, or a position corresponding to a portion of the operation unit 15. The position corresponding to a portion includes, for example, the position of the keyboard in the high range from the center or the low range. The information read by the control unit 11 is, for example, information associating the position of the high range keys with the volume setting function, and the position of the low range keys with the tone setting function. The control unit 11 instructs the projection unit 17 to output the image read from the storage unit 12. Based on this instruction, the projection unit 17 projects the image onto the operation unit 15.

[0026] In S2, the control unit 11 accepts a user operation on the operation unit 15. In this embodiment, when the detection unit 16 connected to the keyboard detects a pressure equal to or greater than a predetermined reference pressure, the control unit 11 accepts a user pressing operation on the keyboard. The control unit 11 acquires, from the detection unit 16, position information indicating the position on the keyboard where the pressure was detected.

[0027] In S3, the control unit 11 accepts the operation received in S2 as a setting operation and activates the setting function. Specifically, the control unit 11 determines the setting function to be activated based on the information read in S1 and the position information acquired in S2. The setting function to be activated includes setting the tone color to be output from the input / output unit 14. For example, if the information read in S1 is information associating the position of a high-pitched key with a volume setting function and the position of a low-pitched key with a tone setting function, when the position information indicates a high-pitched key position, the control unit 11 determines the setting function to be the volume setting function. When the position information indicates a low-pitched key position, the control unit 11 determines the setting function to be the tone setting function. Next, the control unit 11 references the information indicating the type of tone and increments the value of a counter each time the received pressing operation by the user is performed. The control unit 11 completes the setting of the tone color corresponding to the counter value when a preset condition is satisfied as the tone color to be emitted from the input / output unit 14. The preset condition may include, for example, the passage of a predetermined time since the acceptance of an operation, or the acceptance by the control unit 11 of a voice input of a user's utterance such as "Confirm" via the input / output unit 14. The second function to be exercised is not limited to the above, and may be any setting such as a setting of the volume output from the input / output unit 14, or a setting of the difficulty level of a game in which a performance is performed in accordance with the output melody.

[0028] The control unit 11 may read out user information, which is a setting stored in advance for each user, from the storage unit 12, and perform the setting function based on the user information. The user information may be default settings for volume, tone, game difficulty, etc. For example, when the control unit 11 refers to information indicating volume levels divided into stages from 1 to 10 in S2, and the user information indicates a default setting for volume level "5," the control unit 11 performs the setting function to set the volume to level "5."

[0029] In S4, the control unit 11 ends the projection of the image. Specifically, the control unit 11 instructs the projection unit 17 to end the output of the image. Based on this signal, the projection unit 17 ends the projection of the image.

[0030] In S5, the control unit 11 determines whether or not to project an image on the operation unit 15. The operations from S4 onwards are executed periodically, for example, at intervals of several tens of microseconds to several seconds. The control unit 11 determines to project an image if a preset condition is met, and determines not to project an image if the condition is not met. The preset condition includes, for example, the control unit 11 receiving a voice input of a user's utterance such as "project an image" via the input / output unit 14. If it is determined to project an image, the process of the control unit 11 proceeds to S6. If it is determined not to project an image, the process of the control unit 11 proceeds to S11.

[0031] In S6, similarly to S1, control unit 11 causes projection unit 17 to project an image onto operation unit 15. Control unit 11 reads information from storage unit 12 that associates the image to be projected, the position on operation unit 15 onto which the image is projected, and the setting function to be performed. Control unit 11 instructs projection unit 17 to output the image read from storage unit 12. Based on this instruction, projection unit 17 projects the image onto operation unit 15.

[0032] In S7, the control unit 11 accepts a user operation on the operation unit 15, as in S2. In this embodiment, when the detection unit 16 connected to the keyboard detects a pressure equal to or greater than a predetermined reference pressure, the control unit 11 accepts a user pressing operation on the keyboard. The control unit 11 acquires, from the detection unit 16, position information indicating the position on the keyboard where the pressure was detected.

[0033] In S8, the control unit 11 determines whether the operation received in S6 is an operation on the operation unit 15 at the position where the image was projected in S5. Specifically, based on the information read out in S6 and the position information, the control unit 11 determines whether the position where the image is projected includes the position of the keyboard where the press operation was received in S7, and if so, determines that the operation is for the position where the image is projected. If the position where the image is projected does not include the position of the keyboard where the press operation was received, the control unit 11 determines that the operation is not for the projection position. If it is determined that the operation is for the projection position, the control unit 11 proceeds to S9. If it is determined that the operation is not for the projection position, the control unit 11 proceeds to S12.

[0034] In S9, the control unit 11 accepts the operation received in S7 as a setting operation and, similarly to S3, causes the setting function to be performed. Specifically, the control unit 11 determines the setting function to be performed by referring to the information read out in S6. For example, the control unit 11 refers to information indicating a tone, and increments a counter value each time the pressing operation by the user received in S7 is increased by one. The control unit 11 completes the setting of the tone corresponding to the counter value when a preset condition is satisfied as the tone to be emitted from the input / output unit 14. As in S3, the control unit 11 may read out user information, which has been read out from the storage unit 12 and in which settings have been saved in advance for each user, and cause the setting function to be performed based on the user information.

[0035] In S10, control unit 11 ends the projection of the image. Specifically, control unit 11 sends an instruction to end the output of the image to projection unit 17. Based on the instruction, projection unit 17 ends the projection of the image. Thereafter, the processing of control unit 11 proceeds to S13.

[0036] In S11, the control unit 11 accepts a user operation on the operation unit 15. As in S2, when the detection unit 16 connected to the keyboard detects a pressure equal to or greater than a predetermined reference pressure, the control unit 11 accepts a user pressing operation on the keyboard. The control unit 11 acquires, from the detection unit 16, position information indicating the position on the keyboard where the pressure was detected.

[0037] In S12, the control unit 11 accepts the user's operation on the operation unit 15 as a performance operation and causes the performance function to be exercised. Specifically, the control unit 11 reads out a sound of a pitch corresponding to the position on the keyboard from the storage unit 12 and outputs it via the input / output unit 14. The control unit 11 outputs a sound according to the tone, volume, etc. set in S3 or S9 via the input / output unit 14.

[0038] In S13, the control unit 11 determines whether there is an instruction to end the operation from the user. If it is determined that there is no instruction to end, the operation of the control unit 11 returns to S5. If it is determined that there is an instruction to end, the operation of the control unit 11 ends. Any method can be used to determine whether there is an instruction to end. For example, the control unit 11 may receive the user's utterance of "I'm going to end" via the input / output unit 14, analyze it using any voice processing technology, and determine that there is an instruction to end the operation.

[0039] As a modified example, the terminal device 10 may communicate with another terminal device 10 used by a user other than the user (hereinafter referred to as the other party) via a network including the Internet. In this case, the control unit 11 transmits information indicating the performance function activated in S12 and the setting function activated in S3 or S9 to the other terminal device 10. The information indicating the performance function also includes position information indicating the position of the keyboard. In this modified example, each keyboard has a drive unit that moves the keyboard up and down in response to instructions from the control unit 11. The drive unit may be a motor, a solenoid device, or the like. The control unit 11 may receive information indicating the performance function and setting function activated in the other terminal device 10 transmitted from the other terminal device 10, control the drive unit based on the information, and move the keyboard of the terminal device 10 at the user's hand up and down to output sound from the input / output unit 14. By alternately transmitting and receiving the information, the user and the other party in a remote location can realize interactions such as playing an ensemble using the terminal devices 10 they each use.

[0040] For example, if the information received by the control unit 11 indicates that another terminal device 10 has performed a setting function to set the tone to "organ" and a performance function to play a key at a specified position in the tone of the organ, the control unit 11 moves the key at the specified position up and down to output the sound of the organ tone via the input / output unit 14.

[0041] In this modification, the setting function may include setting to select the other party, setting the character string of the name on the user network, setting the character string of the message to be sent to the other party, and the like.

[0042] As another variation, the control unit 11 may control the input / output unit 14 so as not to activate the performance function while the setting function is being activated. Specifically, the control unit 11 outputs a message such as "Do you want to complete the settings?" via the input / output unit 14, and controls the input / output unit 14 so as not to output sound as the performance function until a user's response to the message, such as a user's utterance of "Yes," is received via the input / output unit 14. This makes it possible to prevent a performance operation from being mistakenly accepted while a performance operation is being accepted, resulting in the performance function being activated.

[0043] As yet another modification, control unit 11 may accept a touch operation with a finger on operation unit 15 as a setting operation in addition to a pressing operation. In this case, detection unit 16 includes a distance measurement sensor for detecting the touch operation. Detection unit 16 may be a camera that can detect a touch operation by performing any image analysis processing on an image obtained by photographing operation unit 15.

[0044] Although the embodiments have been described above based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined or divided into one. [Explanation of symbols]

[0045] 10 Terminal Equipment 11 Control section 12 Storage section 13 Communications Department 14 Input / output section 15 Control section 16. Detection unit 17 Projection section

Claims

[Claim 1] an operation unit that accepts operations by a user; a projection unit that projects an image onto the operation unit; a control unit that accepts an operation by the user on the operation unit as an operation for performing a different function depending on whether the image is projected or not; A terminal device having the above configuration.

Images