Pad attachment, virtual input device, and input system

Abstract

To provide a pad attachment, a virtual input device, and an input system which can improve the degree of freedom in designing an operation pad constituting the virtual input device.SOLUTION: A pad attachment (24) comprises: a memory (60) in which device information (62) about a virtual input device (18) is stored; one or more communication elements (40, 42) for performing wireless communication with a touch device (12) via a touch sensor (82) or a planar sensor different from the touch sensor (82); and a transmission circuit (58) which transmits a transmission signal including the device information (62) read from the memory (60) via the one or more communication elements (40, 42).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a pad attachment, a virtual input device, and an input system. [Background technology]

[0002] 2. Description of the Related Art There is known a technique for displaying a virtual input device (for example, a virtual keyboard or an on-screen keyboard) on a touch device and emulating a hardware input device with software.

[0003] For example, Patent Document 1 discloses an input device that includes an operation unit that is configured to detect user operations on the touch panel device side, and an input device discrimination conductor (hereinafter referred to as "discrimination conductor") that indicates the area in which the operation is detected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-041320 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the input device disclosed in Patent Document 1, in order to correctly reflect the user's operation, it is necessary to uniquely define the shape or arrangement pattern of the discrimination conductors for each input device. Also, it is necessary to arrange the conductors in positions that do not interfere with the operation unit. Thus, the addition of discrimination conductors may impose restrictions on the layout design of the operation pad.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a pad accessory, a virtual input device, and an input system that can improve the design freedom of the operation pad that constitutes the virtual input device. [Means for solving the problem]

[0007] The pad accessory in a first aspect of the present invention is an operation pad having at least one physical operation element, and is attached to an operation pad placed on a touch device having a capacitive touch sensor, and is equipped with a memory that stores device information regarding a virtual input device including the operation pad, one or more communication elements for wireless communication with the touch device via the touch sensor or a planar sensor different from the touch sensor, and a transmission circuit that transmits a transmission signal including the device information read from the memory via the one or more communication elements.

[0008] A virtual input device according to a second aspect of the present invention comprises the above-described pad attachment and an operation pad having the physical operation element and to which the pad attachment is attached.

[0009] An input system in a third aspect of the present invention comprises a virtual input device including the pad attachment described above, an operation pad having the physical operation element and to which the pad attachment is attached, and a touch device having the touch sensor. [Effects of the Invention]

[0010] According to the present invention, it is possible to improve the degree of freedom in designing the operation pad that constitutes the virtual input device. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram of an input system according to a first embodiment of the present invention. [Figure 2] 2 is a cross-sectional view of the virtual input device shown in FIG. 1 taken along line II. [Figure 3] 3 is a block diagram showing an example of the configuration of a pad attachment in FIG. 2. FIG. [Figure 4] 2 is a block diagram showing an example of the configuration of a touch device in FIG. 1. FIG. [Figure 5] FIG. 10 is a diagram illustrating a method for sharing pad information. [Figure 6] 6 is a diagram showing an example of an operation mode of the touch IC of FIG. 4 and FIG. 5. [Figure 7] 5 is a flowchart of a touch scan operation by the touch device of FIG. 4. [Figure 8] FIG. 1 is a first diagram showing a method for setting an operation area on an operation pad. [Figure 9] FIG. 10 is a second diagram showing a method for setting an operation area on an operation pad. [Figure 10] 10A and 10B are diagrams illustrating screen transitions on the touch panel display in response to changes in the arrangement of the virtual input device. [Figure 11] 10A and 10B are diagrams illustrating operation states of a virtual input device. [Figure 12] 5 is a diagram showing an example of a data structure of the pad information of FIG. 4. FIG. [Figure 13] FIG. 10 is a plan view of a virtual input device according to a first modified example of the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating the overall configuration of an input system according to a second modified example of the first embodiment. [Figure 15] FIG. 10 is an overall configuration diagram of an input system according to a third modified example of the first embodiment. [Figure 16] FIG. 11 is a cross-sectional view of a virtual input device according to a fourth modified example of the first embodiment. [Figure 17] FIG. 10 is a diagram illustrating the overall configuration of an input system according to a second embodiment of the present invention. [Figure 18] FIG. 10 is a block diagram showing an example of the configuration of a pad attachment in the second embodiment. [Figure 19] FIG. 18 is a block diagram showing an example of the configuration of a touch device in FIG. 17. [Figure 20] 20 is a flowchart of a light emitting operation by the touch device of FIG. 19. [Figure 21] 10A and 10B are diagrams illustrating an example of a method for setting a light emitting area on a display panel. [Figure 22]10A and 10B are diagrams illustrating screen transitions on the touch panel display in response to changes in the arrangement of the virtual input device. DETAILED DESCRIPTION OF THE INVENTION

[0012] The pad attachment, virtual input device, and input system of the present invention will be described with reference to the accompanying drawings. To facilitate understanding of the description, the same components in each drawing are denoted by the same reference numerals as much as possible, and duplicated descriptions will be omitted. Furthermore, the term "part" may be replaced with other terms such as "unit," "module," "device," or "element."

[0013] The present invention is not limited to the following embodiments and modifications, and can be freely modified without departing from the spirit of the present invention. Alternatively, the respective configurations may be arbitrarily combined within the scope of no technical contradiction. Alternatively, the execution or execution order of each step constituting the flowchart may be changed within the scope of no technical contradiction.

[0014] [First embodiment] First, an input system 10 according to the first embodiment will be described with reference to FIGS.

[0015] <Overall configuration of input system 10> 1 is a diagram showing the overall configuration of an input system 10 according to a first embodiment of the present invention. The input system 10 basically comprises a touch device 14 equipped with a touch panel display 12, an electronic pen 16 which is a pen-type pointing device, and a virtual input device 18 which can be freely placed on the touch device 14.

[0016] The touch device 14 may be, for example, a tablet terminal with or without a display function, a smartphone, a laptop, or a personal computer. For example, a user can perform various operations via a graphical user interface (GUI) by touching a touch surface 20 of the touch device 14 with their finger.

[0017] The electronic pen 16 is configured to be capable of one-way or two-way communication with the touch device 14. In the example of Fig. 1, the electronic pen 16 is an "active electrostatic coupling (AES) type" (or capacitive type) stylus that actively generates a signal from electrical energy stored in the electronic pen 16 and transmits this signal to the touch device 14. For example, a user can write pictures or characters on the touch device 14 by holding the electronic pen 16 in one hand and moving the pen tip while pressing it against the touch surface 20 of the touch device 14.

[0018] The virtual input device 18 is a pseudo physical device that performs input according to a user's operation on the touch device 14. Specifically, the virtual input device 18 includes an operation pad 22 and a pad accessory 24.

[0019] FIG. 2 is a schematic cross-sectional view taken along line II of the virtual input device 18 shown in FIG. 1. The operation pad 22 has a three-dimensional solid or hollow shape. In the example of FIG. 2, the interior 26 of the operation pad 22 is filled with various fluids including air, for example, gas or liquid having a lower dielectric constant than the operation pad 22. The operation pad 22 is transparent or translucent in color and is made of a non-conductive and elastic material (for example, a resin material). The operation pad 22 elastically deforms when an external force is applied to it, and can restore its original shape when the external force is released.

[0020] The operation pad 22 has a surface 28 with concaves and convexes formed thereon and a flat back surface 30. The surface 28 of the operation pad 22 is provided with step portions 32 that divide the surface 28 into a plurality of small regions. In the example of FIG. 2, the step portions 32 form the steps of the surface 28 by concave shapes arranged in a mesh pattern, but the steps of the surface 28 may also be formed by convex shapes. Hereinafter, the convex portions formed by each small region will be referred to as operation elements 34.

[0021] This operation pad 22 is configured so that, when placed on a touch device 14 having a capacitive touch sensor 82 (Figures 4 and 5), a change in capacitance occurs at the part of the touch sensor 82 corresponding to the position pressed by the user from the surface 28 side, in response to the approach of the user or a conductive body different from the user.

[0022] The pad attachment 24 is detachably provided on the side of the operation pad 22 (on the left side in plan view in the example of FIG. 1). The pad attachment 24 has an AES communication function, similar to the electronic pen 16. Hereinafter, a signal transmitted from the touch device 14 to the pad attachment 24 will be referred to as an "uplink signal US," and a signal transmitted from the pad attachment 24 to the touch device 14 will be referred to as a "downlink signal DS."

[0023] <Pad accessory 24 configuration> Fig. 3 is a block diagram showing an example of the configuration of the pad attachment 24 of Fig. 2. The pad attachment 24 includes a first electrode 40 (corresponding to a "communication element"), a second electrode 42 (corresponding to a "communication element"), a main board 44, a power storage unit 46, and a power management integrated circuit (hereinafter, PMIC 48).

[0024] The first electrode 40 and the second electrode 42 are provided for wireless communication with the touch device 14. The first electrode 40 and the second electrode 42 are arranged apart so as to be at different positions in a plan view of the pad attachment 24.

[0025] The main board 44 is provided with a microcontroller unit (hereinafter referred to as "MCU 50"), a first switch 52, a second switch 54, a receiving circuit 56, and a transmitting circuit 58.

[0026] The MCU 50 is a unit that performs overall control over each part of the pad attachment 24. The MCU 50 is configured to receive an uplink signal US from the touch device 14 by performing desired reception control on the receiving circuit 56. The MCU 50 is configured to perform desired digital signal processing on the data supplied from the receiving circuit 56, and to transmit a downlink signal DS to the touch device 14 by performing desired transmission control on the transmitting circuit 58.

[0027] The MCU 50 includes a memory 60, which is a non-transitory computer-readable storage medium. In the example of Fig. 3, the memory 60 stores information (hereinafter referred to as "device information 62") about the virtual input device 18 shown in Fig. 1 and Fig. 2. The specific contents of the device information 62 will be described later with reference to Fig. 4.

[0028] The first switch 52 is a switch element configured so that a common terminal is connected to either an R terminal or a T terminal. The common terminal of the first switch 52 is connected to the first electrode 40, the R terminal is connected to the input terminal of the receiving circuit 56, and the T terminal is connected to the output terminal of the transmitting circuit 58. The MCU 50 supplies a first switch control signal SWC1 to the first switch 52 to perform switching control, thereby selectively receiving an uplink signal US and transmitting a downlink signal DS.

[0029] The second switch 54 is a switch element configured so that a common terminal is connected to either an R terminal or a T terminal. The common terminal of the second switch 54 is connected to the second electrode 42, the R terminal is connected to the input terminal of the receiving circuit 56, and the T terminal is connected to the output terminal of the transmitting circuit 58. The MCU 50 supplies a second switch control signal SWC2 to the second switch 54 to perform switching control, thereby selectively receiving an uplink signal US and transmitting a downlink signal DS.

[0030] The receiving circuit 56 is a circuit that demodulates the uplink signal US induced in the first electrode 40 or the second electrode 42, and outputs the demodulated data to the MCU 50. The ground terminal of the receiving circuit 56 is grounded (or connected to ground) to the housing of the pad attachment 24. Specifically, the receiving circuit 56 is configured to include a waveform regenerator 64 and a correlation calculator 66.

[0031] The waveform regenerator 64 binarizes the level of the voltage induced in the first electrode 40 or the second electrode 42 using a clock of a predetermined rate, shapes it into a binary string of positive and negative polarity values ​​(i.e., a chip string), and outputs it. This clock frequency is set, for example, to an integer multiple of the chip rate of the spreading code.

[0032] The correlation calculator 66 stores the chip sequence from the waveform regenerator 64 in a register and performs correlation calculations with the spreading code while sequentially shifting the signal with the clock, thereby decoding the chip sequence contained in the uplink signal US.

[0033] The transmission circuit 58 is a circuit that generates the downlink signal DS under the control of the MCU 50. When the downlink signal DS is a "position signal," the transmission circuit 58 outputs an unmodulated carrier signal, and when the downlink signal DS is a "data signal," the transmission circuit 58 modulates the carrier signal using transmission data and outputs the modulated carrier signal. Specifically, the transmission circuit 58 is configured to include a modulator 68 and a boost circuit 70.

[0034] The modulator 68 generates a carrier signal such as a square wave or a triangular wave, and outputs it modulated or as is under the control of the MCU 50. When transmitting a burst signal, the modulator 68 outputs the carrier signal as is without modulating it, in accordance with instructions from the MCU 50. On the other hand, when transmitting a data signal, the modulator 68 modulates (OOK, PSK, etc.) the carrier signal with data supplied from the MCU 50, and outputs the resulting modulated signal.

[0035] The boost circuit 70 generates a downlink signal DS by boosting the output signal supplied from the modulator 68 to a certain amplitude. The downlink signal DS generated by the boost circuit 70 is sent to the outside from the first electrode 40 via the first switch 52 or from the second electrode 42 via the second switch 54.

[0036] The power storage unit 46 is composed of, for example, a battery or a capacitor, and stores or releases electrical energy. The power storage unit 46 supplies drive power to the electronic components or electronic elements mounted on the main board 44. The PMIC 48 is an integrated circuit that monitors the state of the power storage unit 46 and supplies power to the main board 44.

[0037] <Configuration of Touch Device 14> Fig. 4 is a block diagram showing an example of the configuration of the touch device 14 in Fig. 1. Specifically, the touch device 14 includes the touch panel display 12, a display drive integrated circuit (IC) 84, a touch IC 86, a communication module 88, a host processor 90, and a memory 92.

[0038] The touch panel display 12 includes a display panel 80 that can visibly display content, and a touch sensor 82 that is arranged to overlap the display panel 80 in a planar view. The display panel 80 can display monochrome or color images, and is configured, for example, from a liquid crystal panel, an organic electroluminescence (EL) panel, electronic paper, or a quantum dot panel. The liquid crystal panel may be a backlight type, a mini LED type, or a micro LED type.

[0039] In the example shown in this figure, the touch sensor 82 is an "external type" sensor that is attached to the outside of the display panel 80, but it may alternatively be an "internal type" sensor (further classified as an on-cell type or an in-cell type) that is integrally configured with the display panel 80.

[0040] The touch sensor 82 is a capacitance-type sensor having a plurality of sensor electrodes arranged in a plane. The touch sensor 82 includes, for example, a plurality of X-line electrodes for detecting the position on the X axis of a sensor coordinate system and a plurality of Y-line electrodes for detecting the position on the Y axis. Instead of the mutual capacitance-type sensor described above, the touch sensor 82 may be a self-capacitance-type sensor in which block-shaped electrodes are arranged in a two-dimensional lattice pattern.

[0041] The display driver IC 84 is an integrated circuit that is electrically connected to the display panel 80 and controls the driving of the display panel 80. The display driver IC 80 drives the display panel 80 based on a display signal supplied from the host processor 90. As a result, various images or videos are displayed within the display area of ​​the display panel 80.

[0042] The touch IC 86 is an integrated circuit that is electrically connected to the touch sensor 82 and controls the driving of the touch sensor 82. The touch IC 86 drives the touch sensor 82 based on a control signal supplied from the host processor 90. As a result, the touch IC 86 performs the following functions: [1] a "pen detection function" that detects the electronic pen 16; [2] a "touch detection function" that detects a touch by a user's finger or the like; and [3] a "device detection function" that detects the virtual input device 18.

[0043] The pen detection function includes, for example, a function of scanning the touch sensor 82, a function of receiving and analyzing the downlink signal DS, a function of estimating the state of the electronic pen 16 (e.g., position, posture, writing pressure), and a function of generating and transmitting an uplink signal US including a command to the electronic pen 16. The touch detection function includes, for example, a function of two-dimensionally scanning the touch sensor 82, a function of acquiring a heat map on the touch sensor 82, and a function of classifying areas on the heat map (e.g., classification into fingers, palm, etc.). The device detection function includes, for example, a function of scanning the touch sensor 82, a function of receiving and analyzing the downlink signal DS, a function of identifying the operation pad 22, a function of estimating the arrangement of the operation pad 22 (e.g., position and posture), and a function of accepting operations via the operation pad 22.

[0044] In this way, by combining the input functions of the electronic pen 16, the virtual input device 18, and the touch sensor 82 with the output function of the display panel 80, a graphical user interface (GUI) is constructed.

[0045] The communication module 88 has a communication function for performing wired or wireless communication with an external device, thereby enabling the touch device 14 to exchange various data with the external device.

[0046] The host processor 90 is configured by an arithmetic processing device including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an MPU (Micro-Processing Unit). The host processor 90 functions as a drawing processing unit 94 and an operation processing unit 96 by reading and executing programs and data stored in a memory 92.

[0047] The drawing processing unit 94 receives writing operations using the electronic pen 16 and performs digital ink generation processing and rendering processing. Examples of digital ink data formats, so-called "ink description languages," include WILL (Wacom Ink Layer Language), InkML (Ink Markup Language), and ISF (Ink Serialized Format).

[0048] The operation processing unit 96 processes data related to operations on the virtual input device 18. Specifically, the operation processing unit 96 includes a device detection unit 98, a coordinate value calculation unit 99, and an operation identification unit 100.

[0049] The device detection unit 98 detects the virtual input device 18 placed on the touch device 14. Examples of the determination conditions include (1) detection of communication with the pad accessory 24, and (2) detection of contact of the virtual input device 18 with the touch surface 20. Note that "contact with the touch surface 20" can be detected by the presence or absence of pressure, magnetic attraction, or the presence or absence of a gap between the two.

[0050] When the device detection unit 98 detects the virtual input device 18, the coordinate value calculation unit 99 uses the pad information 62b corresponding to the operation pad 22 to determine whether or not there is a touch on the pad coordinate system, and calculates a coordinate value indicating the touch position (hereinafter also referred to as the "touch coordinate value").

[0051] The operation identification unit 100 identifies a user's operation on the operation pad 22 using the touch coordinate values ​​calculated by the coordinate value calculation unit 99 and the pad information 62b corresponding to the operation pad 22. For example, if the operation element 34 is a character key, the corresponding "character" or the "function" associated with the corresponding key is identified. If the operation element 34 is a trackball, the corresponding "direction" is identified. If the operation element 34 is an electronic keyboard, the corresponding "sound" and "touch strength" are identified. For example, by expressing touch strength in three or more levels, analog-like operation and output can be realized.

[0052] The memory 92 is a non-transitory computer-readable storage medium. In the example shown in the figure, the memory 92 stores drawing information 102 and device information 62.

[0053] The drawing information 102 includes drawing results obtained through operations using an input device including the electronic pen 16. An example of the drawing results is stroke data (or digital ink) including coordinate values, pen pressure values, tilt angles, etc. when drawing a stroke.

[0054] The device information 62 includes various information related to the virtual input device 18. In the example of Fig. 4, the device information 62 includes a device ID (Identifier) ​​62a, pad information 62b, and additional area information 62c.

[0055] The device ID 62a includes identification information that can uniquely identify the individual or type of the virtual input device 18. Examples of the identification information include [1] the serial number, model name, and model number of the virtual input device 18, [2] the serial number, model name, and model number of the operation pad 22, or [3] the serial number, model name, and model number of the pad accessory 24.

[0056] The pad information 62b includes various information related to the operation pad 22. Examples of the pad information 62b include "layout information" related to the layout of the operation pad 22 and "operation setting information" related to the operation settings of the operation pad 22.

[0057] The above-mentioned "layout information" includes the layout relationship of one or more operation elements 34 of the operation pad 22. Examples of the layout relationship include the type, position, shape, and effective range of the operation element 34, or the effect associated with the operation. The position of the operation element 34 may be expressed, for example, as a relative position from a reference point (or a position on the pad coordinate system).

[0058] The above-mentioned "operation setting information" includes setting contents related to the operation of the operation pad 22, which can be set via the GUI of the touch device 14. Examples of the setting contents include touch sensitivity, enabling / disabling of operation, customizing the key layout, turning on / off the display function of the simulated image, the display color of the simulated image, function assignment, audiovisual effects associated with operation, etc. Furthermore, if the pad attachment 24 has a detachable structure, the setting contents may further include a fixing position of the pad attachment 24.

[0059] The additional area information 62c is used when an operation function is to be assigned to a specific area (hereinafter referred to as "additional area") within the sensor area of ​​the touch sensor 82 that is different from the arrangement area of ​​the operation pad 22. The additional area information 62c includes, for example, the position, size or shape of the additional area, the type of operation function, or settings related to the operation function. The additional area may be provided in a position adjacent to the arrangement area, or in a position away from the arrangement area. The number of additional areas may be one, or two or more.

[0060] The above-mentioned "operation function" may be a function related to the operation of the operation pad 22 (hereinafter also referred to as "related function"), or may be a function not related to the operation of the operation pad 22 (hereinafter also referred to as "non-related function").

[0061] An example of a related function is a "complementary function" that complements operations performed by the operation pad 22. For example, if a rectangular additional area extending in the vertical direction along the right side of the operation pad 22 is set, the display screen of the display panel 80 can be scrolled in the vertical direction by a user swiping within the additional area.

[0062] An example of a non-related function is a "disable function" that disables touches around the operation pad 22. For example, if a rectangular disabled area extending in the left-right direction along the bottom edge of the operation pad 22 is set, it is possible to prevent erroneous operations caused by touches with the palm when operating the operation pad 22.

[0063] <Operation of input system 10> The input system 10 in the first embodiment is configured as described above. Next, the operation of the input system 10 will be described with reference to FIGS.

[0064] 5 is a diagram schematically illustrating a method for sharing device information 62. In this sharing method, the pad accessory 24 provides the device information 62 that it holds to the touch device 14. The touch device 14 is illustrated as comprising a touch sensor 82, a touch IC 86, and a host processor 90. The virtual input device 18 is also illustrated as comprising an operation pad 22 and a pad accessory 24.

[0065] First, the touch device 14 attempts to communicate with an external device in the vicinity of the touch device 14. The touch IC 86 of the touch device 14 generates an uplink signal US modulated with data required for connection and transmits the uplink signal US from the sensor electrodes that make up the touch sensor 82. The electronic pen 16 or pad attachment 24 then receives the uplink signal US and transmits a signal (i.e., a downlink signal DS) in response to the uplink signal US.

[0066] For example, when the touch device 14 receives a downlink signal DS from the electronic pen 16 via the touch sensor 82, communication between the touch device 14 and the electronic pen 16 is established and AES communication is initiated. Alternatively, when the touch device 14 receives a downlink signal DS from the pad attachment 24 via the touch sensor 82, communication between the touch device 14 and the pad attachment 24 is established and AES communication is initiated.

[0067] While communication between the touch device 14 and the pad accessory 24 is being performed, the host processor 90 of the touch device 14 acquires the device information 62 corresponding to the operation pad 22 through reception of the downlink signal DS. As a result, the host processor 90 operates to implement the input function using the operation pad 22.

[0068] FIG. 6 is a diagram showing an example of an operation mode of the touch IC 86 of FIGS. 4 and 5. The touch IC 86 operates by switching between a plurality of operation modes including a time-division mode. The time-division mode corresponds to an operation mode in which, for example, one touch scan (TS) and one pen scan (PS) are executed in a time-division manner. Here, "touch scan" refers to a scanning operation for detecting a passive pointer (for example, finger F in FIG. 5) that does not transmit a signal. Also, "pen scan" refers to a scanning operation for detecting an electronic pen 16 that transmits a downlink signal DS.

[0069] The time slot for touch scan is assigned a time length of T1 (unit: ms). The time slot for global scan is assigned a time length of T2 (unit: ms). In other words, [1] touch scan with a time length of T1 and [2] global scan with a time length of T2 constitute one operation unit (cycle: Tc = T1 + T2).

[0070] In the above-described time-division mode, the scanning operation for detecting the virtual input device 18, i.e., the device scan (DS), is omitted, but a time slot for the device scan may be newly added. In this case, the pad accessory 24 has a communication circuit configuration in common with the electronic pen 16, so the touch IC 86 can simultaneously execute the device scan in addition to the execution of the pen scan.

[0071] <Touch scan operation> Fig. 7 is a flowchart relating to the touch scanning operation by the touch device 14 of Fig. 4. Here, the touch IC 86 executes steps SP10 and SP12, and the host processor 90 executes steps S14 to SP30.

[0072] In step SP10, the touch IC 86 sequentially transmits signals to the X-line electrodes that make up the touch sensor 82 and receives signals from the Y-line electrodes that make up the touch sensor 82, thereby generating a signal distribution (i.e., a heat map) that correlates with the amount of change in capacitance.

[0073] In step SP12, the touch IC 86 performs an analysis process on the heat map generated in step SP10 and detects a touch on the touch surface 20. The touch IC 86 outputs the presence or absence of a touch and the position (hereinafter also referred to as "position information") to the host processor 90.

[0074] In step SP14, the operation processing unit 96 (more specifically, the device detection unit 98) of the host processor 90 checks the communication status with the pad attachment 24. Specifically, the device detection unit 98 checks whether or not a downlink signal DS was received from the pad attachment 24 during the most recently executed pen scan.

[0075] In step SP16, if there is no communication with the pad accessory 24 (step SP16: NO), the operation processing section 96 assumes that the virtual input device 18 has not been detected, and proceeds to step SP18.

[0076] In step SP18, the operation processing unit 96 refers to the detection result in step SP12 and supplies data including the position information to the operating system (hereinafter referred to as "OS"). Then, the host processor 90 processes the supplied data and performs an operation according to the touch position on the touch surface 20 (step SP30).

[0077] On the other hand, returning to step SP16, if communication with the pad accessory 24 has been detected (step SP16: YES), the operation processing section 96 determines that the virtual input device 18 has been detected, and proceeds to step SP20.

[0078] In step SP20, the operation processing unit 96 (more specifically, the coordinate value calculation unit 99) sets the operation area R ( FIG. 9 ) of the virtual input device 18. Prior to this setting, the coordinate value calculation unit 99 acquires [1] the detected positions of the first electrode 40 and the second electrode 42, and [2] the device information 62 corresponding to the pad attachment 24.

[0079] 8 is a first diagram showing a method for setting the operation area R on the operation pad 22. Any position on the touch surface 20 is defined within a rectangular sensor area (0≦X≦Xo, 0≦Y≦Yo). For example, the detection position of the first electrode 40 is point P (X1, Y1), and the detection position of the second electrode 42 is point Q (X2, Y2).

[0080] Here, a plane coordinate system (hereinafter referred to as the pad coordinate system X'-Y') is defined, with the characteristic point (ΔX, ΔY) of the pad attachment 24 as the origin O'. For example, the origin O' of the pad coordinate system is set to the midpoint between the two points P and Q. In this case, (ΔX, ΔY) are calculated by the following formulas: ΔX = (X1 + X2) / 2, ΔY = (Y1 + Y2) / 2. The X' axis of the pad coordinate system is set in a direction perpendicular to the line segment PQ. The Y' axis of the pad coordinate system is set in a direction parallel to the line segment PQ.

[0081] FIG. 9 is a second diagram illustrating a method for setting the operation area R of the operation pad 22. Specifically, FIG. 9 illustrates the relative positional relationship between the reference areas 110a, 110b, and 110c and the detection areas 112a, 112b, and 112c. The reference area 110a and the detection area 112a each indicate the range in which the pad accessory 24 exists. The reference area 110b and the detection area 112b each indicate the range in which the operation pad 22 exists. The reference area 110c and the detection area 112c each indicate the range in which an additional area (e.g., a touch-disabled area) exists. Here, it is assumed that the origin O' is shifted by (ΔX, ΔY) from the origin O, and the X'-axis (Y'-axis) is tilted by an angle θ with respect to the X-axis (Y-axis). In this case, the detection area 112b is set as the operation area R of the virtual input device 18.

[0082] 7, the coordinate value calculation unit 99 checks whether or not at least one touch position exists within the operation area R set in step SP20. If no touch position exists within this operation area R (step SP22: NO), the operation processing unit 96 skips step SP24 and proceeds to step SP26. Step SP24 will be described later.

[0083] In step SP26, the host processor 90 processes data including operation information, which will be described later, and performs an operation according to the touch position on the touch surface 20. If step SP24 is omitted, the host processor 90 performs the same operation as in step SP18.

[0084] In step SP28, the host processor 90 performs control to display an image simulating the operation pad 22 on the touch panel display 12. Specifically, the host processor 90 reads out a template (i.e., image data) of the operation pad 22 from the memory 92, performs image processing according to the conversion parameters, and then outputs the processed image data to the display panel 80 as display data.

[0085] 10 is a diagram showing screen transitions on the touch panel display 12 in response to changes in the position of the virtual input device 18. As shown in the upper part of Fig. 10, when a user starts using the virtual input device 18, the user places the virtual input device 18 on the touch surface 20 in a position and orientation suitable for the state of use of the touch device 14. Then, upon the start of communication, the display state transitions from the display state shown in the upper part of Fig. 10 to the display state shown in the lower part.

[0086] As shown in the lower part of Fig. 10, an image simulating a character keyboard (hereinafter referred to as a simulated image 124) is displayed at a position overlapping the operation pad 22 (here, within the operation area R in Fig. 9). This simulated image 124 is formed by arranging characters indicating the types of operation elements 34 on the operation pad 22 two-dimensionally according to a desired key arrangement. In other words, this virtual input device 18 exhibits an input function as a pseudo "character keyboard."

[0087] 7 repeatedly, the touch device 14 maintains the virtual input device 18 in a usable state. If the user wishes to adjust the position of the virtual input device 18, the user moves the virtual input device 18 to a desired position and orientation. Then, the pad attachment 24 is detected in the changed position and orientation, and as a result, the simulated image 124 is displayed following the movement of the virtual input device 18.

[0088] Fig. 11 is a diagram schematically showing an operation state of the virtual input device 18. More specifically, Fig. 11 is a diagram schematically showing a change in the heat map when a user's finger F presses one operation element 34 on the operation pad 22. When the finger F approaches the touch sensor 82 (distance d), the capacitance at the position of the finger F changes to the positive side. When the peak value of this change exceeds a threshold value, this peak is detected as the touch position.

[0089] Then, in step SP22 of FIG. 7, a touch is detected within the operation region R (step SP22: YES), and the operation processing unit 96 proceeds to the next step SP24.

[0090] In step SP24, the operation processing unit 96 (more specifically, the coordinate value calculation unit 99 and the operation identification unit 100) performs a data conversion process to convert the touch position within the operation region R detected in step SP22 into the type of operation element 34 on the operation pad 22. This data conversion method will be described in detail with reference to FIG. 12.

[0091] Fig. 12 is a diagram showing an example of the data structure of the pad information 62b in Fig. 4. This pad information 62b is table data linked to the type of operation pad 22, and describes the correspondence relationship between range information indicating the effective range of operation (i.e., the range of a small area) and the type of operation element 34. For example, if the shape of the effective range is a rectangle, the range information is composed of positions on the pad coordinate system indicating the four vertices (e.g., P001 → P002 → P003 → P004 → P001). Types of operation elements 34 include, for example, alphabetic characters including Q, W, E, R, and T, numbers including 1, 2, and 3, modifier keys including Enter and Shift, and directional keys including ↓ (down) and → (right).

[0092] The coordinate value calculation unit 99 converts the coordinate values ​​indicating the touch position (i.e., touch coordinate values) from the sensor coordinate system XY to the pad coordinate system X'-Y' by affine transformation using the three transformation parameters (ΔX, ΔY, θ) calculated in step SP20. Then, the operation identification unit 100 refers to the pad information 62b to identify the effective range to which the touch coordinate values ​​after the affine transformation belong, and obtains the type of the operation element 34 associated with the corresponding effective range. For example, if the coordinates after the transformation belong to the effective range surrounded by P017 to P020, "T" is obtained as the type of the operation element 34.

[0093] 7, the operation processing unit 96 generates data including information indicating the operation state of the virtual input device 18 (i.e., operation information), and supplies the data to the OS. Here, unlike in step SP18, the operation information is output instead of the position information.

[0094] It is assumed that when the user finishes using the virtual input device 18, the user removes the virtual input device 18 and returns it to a location away from the touch panel display 12. Then, the start of communication triggers the end of the display of the simulated image 124 shown in FIG.

[0095] In this way, a capacitive contactless keyboard can be replaced by a combination of the touch device 14 and the virtual input device 18. This makes it possible to [1] reduce product costs, [2] provide a new UI experience, [3] expand use cases, [4] make effective use of the touch device 14, or [5] simplify customization.

[0096] <Summary of the First Embodiment> As described above, the input system 10 in the first embodiment includes the touch device 14 having the capacitive touch sensor 82, and the virtual input device 18 used together with the touch device 14. The virtual input device 18 includes the operation pad 22 having at least one physical operation element 34, and the pad attachment 24 attached to the operation pad 22.

[0097] The pad attachment 24 includes a memory 60 that stores device information 62 related to the virtual input device 18, one or more communication elements (here, electrodes) for wireless communication with the touch device 14 via the touch sensor 82, and a transmitting circuit 58 that transmits a transmission signal including the device information 62 read from the memory 60 via the one or more electrodes.

[0098] With this configuration, the device information 62 held by the pad accessory 24 can be provided to the touch device 14 without providing a special structure to the operation pad 22. This improves the degree of freedom in designing the operation pad 22 that constitutes the virtual input device 18.

[0099] The device information 62 may also include pad information 62b, which is the type of the operation pad 22, the arrangement of the physical operation elements 34, or settings related to the operation of the operation pad 22. This allows the touch device 14 to perform a desired input function on the operation pad 22 by acquiring the pad information 62b.

[0100] Furthermore, when an operation function is to be assigned to an additional area within the sensor area of ​​the touch sensor 82, which is different from the area where the operation pad 22 is arranged, the device information 62 may include additional area information 62c which is the type of operation function, the position, size or shape of the additional area, or settings related to the operation function. This allows the touch device 14 to assign a desired operation function to an additional area different from the operation pad 22 by acquiring the additional area information 62c.

[0101] The device information 62 may also include identification information of the virtual input device 18, the operation pad 22, or the pad attachment 24. This allows the touch device 14 to identify the virtual input device 18, the operation pad 22, or the pad attachment 24 with a smaller amount of data.

[0102] Furthermore, when the operation pad 22 is placed on the touch device 14 and the user presses the physical operation element 34 from the surface 28 side, the operation pad 22 may be configured to cause a change in capacitance at the portion of the touch sensor 82 corresponding to the position of the physical operation element 34 due to the approach of the user or a conductor other than the user.

[0103] In this case, the processor (the touch IC 86 or the host processor 90) may generate or output data including operation information indicating the operation state of the operation pad 22, which is information different from position information indicating the detection position, in accordance with a change in capacitance within the operation area R determined according to the detection position of one or more electrodes while wireless communication is being performed with the pad accessory 24. This makes it possible to output information corresponding to the user's operation regardless of whether the virtual input device 18 is present or not.

[0104] <Modification> Fig. 13 is a plan view of a virtual input device 120 in a first modified example of the first embodiment. The example of Fig. 13 shows a state in which the virtual input device 120 is placed on the touch surface 20 of the touch device 14. The virtual input device 120 is configured to include an operation pad 122 and a pad accessory 24. The operation pad 122 has a three-dimensional shape that simulates the form (shape and color) of an electronic keyboard. The pad accessory 24 is detachably provided on the side of the operation pad 122 (the front in a plan view in the example of Fig. 13).

[0105] With the virtual input device 120 placed on the touch surface 20 of the touch device 14, the user presses an operation element (a key of an electronic keyboard) on the operation pad 122. In response, the touch device 14 outputs a sound corresponding to the pressed key from the speaker 126. In this way, the virtual input device 18 exhibits an input function as a pseudo "electronic piano."

[0106] In this way, the shape of the operation pad or the effects of operation may be changed in various ways depending on the purpose of the virtual input device. Other examples of virtual input devices include a numeric keypad, a trackpoint, a creative controller, a game controller, a finger drum, a magnet device, or a large-capacity storage device.

[0107] Furthermore, the operation pad 122 is not limited to a transparent or semi-transparent color, and at least a part of the front surface 28 or the back surface 30 may have a non-transparent color. For example, if the entire operation pad 122 is colored to resemble the real thing, there is no need to display a simulated image in the operation region R directly below the operation pad 122.

[0108] 14 is a diagram showing the overall configuration of an input system 130 according to a second modified example of the first embodiment. The input system 130 includes a virtual input device 18, a touch device 132, and a server device 134. The touch device 132 can communicate with the server device 134 via a network NT.

[0109] In the initial state, the virtual input device 18 holds a device ID 62a from the device information 62. The touch device 132 does not hold device information 62 corresponding to the virtual input device 18. The server device 134 holds the device ID 62a, pad information 62b, and additional area information 62c from the device information 62 corresponding to the virtual input device 18. In this case, the device information 62 can be shared as follows.

[0110] First, when communication between the touch device 132 and the virtual input device 18 starts, the touch device 14 receives the downlink signal DS and acquires the device ID 62a corresponding to the virtual input device 18. Then, the touch device 14 transmits a signal including the device ID 62a (i.e., a request signal for the device information 62) to the server device 134.

[0111] The server device 134 receives a request signal from the touch device 132 via the network NT, and transmits a response signal including pad information 62b and additional area information 62c corresponding to the device ID 62a to the touch device 132. By receiving the response signal from the server device 134 via the network NT, the touch device 132 can obtain the pad information 62b and additional area information 62c corresponding to the virtual input device 18.

[0112] In this way, the host processor 90 of the touch device 132 may obtain the identification information (i.e., the device ID 62a) of the operation pad 22 through wireless communication with the pad attachment 24, and may obtain the arrangement of the physical operation elements 34 corresponding to the device ID 62a or the setting contents related to the operation of the operation pad 22 through communication with an external device (here, the server device 134) different from the pad attachment 24. In this way, the device information 62 is distributed and held in a plurality of devices (here, the virtual input device 18 and the server device 134), so the amount of data held by the pad attachment 24 can be reduced.

[0113] 15 is a diagram showing the overall configuration of an input system 140 according to a third modified example of the first embodiment. The input system 140 includes a touch device 142, an electronic pen 144, and a virtual input device 146.

[0114] The electronic pen 144 is an electromagnetic induction (EMR) type stylus that detects an alternating magnetic field emitted from the electronic pen 144 through a plurality of detection coils arranged two-dimensionally. A coil 156 (corresponding to a "communication element") that generates and transmits an induction signal is provided at the pen tip of the electronic pen 144. Hereinafter, the magnetic field generated by the excitation signal is referred to as the "alternating magnetic field AMF," and the signal output from the coil 156 in association with the alternating magnetic field AMF is referred to as the "EMR signal ES."

[0115] The virtual input device 146 includes, in addition to the operation pad 22 (FIG. 2), a pad attachment 24A having a different configuration from that of the first embodiment. The pad attachment 24A has an EMR communication function, similar to the electronic pen 144. The pad attachment 24A is provided with a coil 158 that generates and transmits an induction signal.

[0116] In addition to the touch sensor 82 described above, the touch device 142 includes an EMR sensor 148, a first sensor IC 150, a second sensor IC 152, and a host processor 154. Here, the EMR sensor 148 is a planar sensor separate from the touch sensor 82, and is configured with a plurality of detection coils arranged two-dimensionally.

[0117] The first sensor IC 150 is an integrated circuit that is electrically connected to the touch sensor 82 and controls the driving of the touch sensor 82. The second sensor IC 152 is an integrated circuit that is electrically connected to the EMR sensor 148 and controls the driving of the EMR sensor 148. The host processor 154 exchanges various data with the first sensor IC 150 or the second sensor IC 152.

[0118] In the initial state, the virtual input device 146 holds the device ID 62a from the device information 62. The touch device 142 holds the device ID 62a, pad information 62b, and additional area information 62c from the device information 62 corresponding to the virtual input device 146. In this case, the device information 62 can be shared as follows.

[0119] First, the touch device 142 attempts to communicate with an external device in the vicinity of the touch device 142. The second sensor IC 152 of the touch device 142 generates an alternating magnetic field AMF by providing an excitation signal to a power coil that is part of the EMR sensor 148. The electronic pen 144 or pad attachment 24A then generates and transmits an EMR signal ES via the coils 156, 158 that pass through the alternating magnetic field AMF.

[0120] For example, when the touch device 142 receives an EMR signal ES from the electronic pen 144 via the EMR sensor 148, communication between the touch device 142 and the electronic pen 144 is established, and EMR communication is initiated. Alternatively, when the touch device 142 receives an EMR signal ES from the pad accessory 24A via the EMR sensor 148, communication between the touch device 142 and the pad accessory 24A is established, and EMR communication is initiated.

[0121] While communication between the touch device 142 and the pad accessory 24A is being performed, the host processor 154 of the touch device 142 acquires the device information 62 corresponding to the operation pad 22 from the second sensor IC 152 through reception of the EMR signal ES. As a result, the host processor 154 operates to exercise the input function of the operation pad 22.

[0122] In this way, the touch device 142 and the pad accessory 24A may exchange the device information 62 by EMR communication instead of AES communication. Also, the touch device 142 and the pad accessory 24A may exchange the device information 62 by using not only the AES or EMR communication but also another wireless communication method including Bluetooth (registered trademark).

[0123] In the case of the EMR system, part or all of the EMR sensor 148 may be incorporated into other members constituting the touch device 142. For example, the transmission coil (or power coil) of the EMR sensor 148 may be integrally provided on the light source substrate of the display panel 80, and the detection coil of the EMR sensor 148 may be integrally provided on the touch sensor 82.

[0124] FIG. 16 is a cross-sectional view of an operation pad 180 in a fourth modified example of the first embodiment. The operation pad 180 is transparent or translucent in color and is made of a non-conductive and elastic material (for example, a resin material). The operation pad 180 elastically deforms when an external force is applied, and can restore its original shape when the external force is released. This operation pad 180 has a surface 182 with concaves and convexes formed thereon and a flat back surface 184, similar to the case of the operation pad 22 of the first embodiment (FIGS. 2 and 11). One or more operation elements 186 are provided on the surface 182 of the operation pad 180.

[0125] Each operating element 186 is composed of an operating member 188 and a housing portion 190 that houses the operating member 188. The bell-shaped operating member 188 is made of a conductive and elastic material (e.g., conductive urethane or conductive silicone rubber). When the operating element 186 is not being operated, the housing portion 190 houses the operating member 188 while fixing it in place and spaced apart from the back surface 184.

[0126] 16, similar to the case of FIG. 11, schematically shows a change in the heat map when the user's finger F presses one operation element 186 on the operation pad 180. When the finger F presses the operation element 186, the operation member 188 approaches the touch sensor 82 (not shown), and the heat map changes to the positive side at the position of the operation member 188. After that, when the operation element 186 is pressed further, the lower end of the operation member 188 abuts against the back surface 184, and the height and width of the peak in the heat map increase due to elastic deformation of the operation member 188.

[0127] In this way, the operation pad 180 is not limited to being made of a non-conductive material, and a conductor may be provided on at least a part of the front surface 182 or the back surface 184. In particular, the operation element 186 includes an operation member that is conductive and elastic, which makes it possible to [1] improve the holding force of the operation element 186, [2] improve the sensitivity of touch operation, or [3] detect an operation using a gloved finger F.

[0128] [Second embodiment] Next, an input system 200 according to the second embodiment will be described with reference to FIGS.

[0129] <Overall configuration of input system 200> 17 is a diagram showing the overall configuration of an input system 200 according to the second embodiment of the present invention. The input system 200 basically comprises a touch device 202 equipped with a touch panel display 12, an electronic pen 204 which is a pen-type pointing device, and a virtual input device 206 which can be freely placed on the touch device 202.

[0130] The touch device 202 is configured as a tablet terminal with or without a display function, a smartphone, a laptop, or a personal computer, similar to the touch device 14 of the first embodiment. The electronic pen 204 is configured to be able to communicate with the touch device 202 in one direction or two directions, similar to the electronic pen 16 of the first embodiment.

[0131] Similar to the virtual input device 18 of the first embodiment, the virtual input device 206 is a pseudo physical device that performs input according to a user's operation on the touch device 202. Specifically, the virtual input device 206 includes, in addition to the operation pad 22 (FIG. 2), a pad accessory 24B (FIG. 18) whose configuration is different from that of the first embodiment.

[0132] <Configuration of pad attachment 24B> 18 is a block diagram showing an example of the configuration of the pad attachment 24B in the second embodiment. This pad attachment 24B is configured to further include a photoelectric conversion unit 208 in addition to the same configuration as the pad attachment 24 in the first embodiment (FIG. 3).

[0133] The photoelectric conversion unit 208 converts external light into electrical energy and supplies the electrical energy to the power storage unit 46. The photoelectric conversion unit 208 is configured, for example, by a solar cell. The external light may be any of [1] display light for the touch device 202, [2] illumination light separate from the touch device 202, or [3] an optical signal used for optical wireless communication with the touch device 202. An example of optical wireless communication is Li-Fi (Light Fidelity), which uses ultraviolet, infrared, and visible light.

[0134] <Configuration of Touch Device 202> Fig. 19 is a block diagram showing an example of the configuration of the touch device 202 in Fig. 17. Specifically, this touch device 14 includes a touch panel display 12, a display drive IC 84, a touch IC 86, and a communication module 88, as well as a host processor 210 and a memory 212 whose functions are different from those of the first embodiment.

[0135] The host processor 210 is configured by an arithmetic processing unit including a CPU, a GPU, or an MPU, similar to the host processor 90 (FIG. 4) of the first embodiment. The host processor 210 reads and executes programs and data stored in a memory 212, thereby functioning as a light emission control unit 214 in addition to the drawing processing unit 94 and the operation processing unit 96.

[0136] The light emission control unit 214 controls the light emission of the display panel 80 via the display drive IC 84. Specifically, the light emission control unit 214 includes a condition determination unit 216 and a light emission setting unit 218.

[0137] The condition determination unit 216 determines whether or not a condition (hereinafter also referred to as a "chargeable condition") regarding whether or not the pad accessory 24B can be charged is satisfied. Examples of the chargeable condition include: (1) detection of communication with the pad accessory 24B; (2) detection of contact of the virtual input device 206 with the touch surface 20; and (3) detection of an intention not to use the touch device 202 or the virtual input device 206.

[0138] When the condition determination unit 216 determines that the chargeable conditions are met, the light emission setting unit 218 sets conditions (hereinafter referred to as "light emission conditions") related to the light emission of the display panel 80. The light emission conditions include a "range condition" related to the light emission range, a "color condition" related to the light emission color, or an "intensity condition" related to the light emission intensity.

[0139] The light emission setting unit 218 may set the light emission range according to the detected position of the pad attachment 24B or the photoelectric conversion unit 208. In this case, for example, the light emission range is set so as to encompass the area where the pad attachment 24B is present. Furthermore, if the position of the photoelectric conversion unit 208 is known, the light emission range may be set so as to encompass the area where the photoelectric conversion unit 208 is present.

[0140] The light emission setting unit 218 may set the emitted color according to the type of the pad attachment 24B or the photoelectric conversion unit 208. In this case, for example, the emitted color is set to include many wavelength components at which the photoelectric conversion efficiency of the photoelectric conversion unit 208 is relatively high and few wavelengths at which the photoelectric conversion efficiency of the photoelectric conversion unit 208 is relatively low. Furthermore, when content is displayed within the area where the pad attachment 24B exists, gradually changing the color near the area where the content exists reduces any discomfort felt by the user.

[0141] The light emission setting unit 218 may set the light emission intensity according to the remaining power of the power storage unit 46 or the type of charging mode that has been set. In this case, for example, the light emission intensity is set to be high when the remaining power of the power storage unit 46 is relatively low, and low when the remaining power of the power storage unit 46 is relatively high. Furthermore, when the rapid charging mode is set, the light emission intensity may be set to be higher than that in the normal charging mode.

[0142] The memory 212 is a non-transitory computer-readable storage medium. In the example of the figure, the memory 212 stores drawing information 102 and device information 62, similar to the memory 92 of the first embodiment. In the example of Fig. 18, the device information 62 further includes light emission setting information 62d in addition to a device ID 62a, pad information 62b, and additional area information 62c.

[0143] The light emission setting information 62d includes [1] information for specifying the chargeable conditions used for the determination by the condition determining unit 216, or [2] information for specifying the light emission conditions used for the setting by the light emission setting unit 218.

[0144] <Operation of Touch Device 202> The input system 200 in the second embodiment is configured as described above. Next, the automatic charging operation by the touch device 202 will be described with reference to FIGS.

[0145] 20 is a flowchart of the automatic charging operation of the touch device 202. Each step of the flowchart is executed by the host processor 210 of the touch device 202.

[0146] In step SP50, the host processor 210 (more specifically, the light emission control unit 214) detects the virtual input device 206 on the touch surface 20 of the touch device 202.

[0147] In step SP52, the light emission control unit 214 (more specifically, the condition determination unit 216) checks whether or not the virtual input device 206 was detected in step SP50. If the virtual input device 206 was not detected (step SP52: NO), the light emission control unit 214 ends execution of the flowchart in Fig. 20. On the other hand, if the virtual input device 206 was detected (step SP52: YES), the light emission control unit 214 proceeds to the next step SP54.

[0148] In step SP54, the condition determination unit 216 determines whether or not the chargeable condition is met. If the chargeable condition is not met (step SP54: NO), the light emission control unit 214 ends execution of the flowchart in Fig. 20. On the other hand, if the chargeable condition is met (step SP52: YES), the light emission control unit 214 proceeds to the next step SP56.

[0149] In step SP56, the light emission control section 214 (more specifically, the light emission setting section 218) sets the light emission conditions for the display panel 80.

[0150] FIG. 21 is a diagram showing an example of a method for setting the light-emitting area 232 on the display panel 80. Specifically, FIG. 21 shows the relative positional relationship between the detection area 230 and the light-emitting area 232. The detection area 230 indicates the range in which the pad accessory 24B is present. The light-emitting area 232 indicates the range in which light is emitted for charging. Here, the origin O' is shifted in position by (ΔX, ΔY) from the origin O, and the X'-axis (Y'-axis) is tilted by an angle θ with respect to the X-axis (Y-axis). In this way, the light-emitting area 232 is set to encompass the range in which the pad accessory 24B is present.

[0151] In step SP58 of FIG. 20, the light emission control section 214 controls the light emission of the display panel 80 in accordance with the light emission conditions set in step SP56.

[0152] Fig. 22 is a diagram showing screen transitions on the touch panel display 12 accompanying changes in the position of the virtual input device 206. As shown in the upper part of Fig. 22, it is assumed that the user places the virtual input device 206 on the touch surface 20 in an arbitrary position and orientation in order to charge the pad accessory 24B of the virtual input device 206. Then, upon the start of communication, the display state shown in the upper part of Fig. 22 transitions to the display state shown in the lower part.

[0153] 22, only the portion of the light-emitting area 232 that includes the pad accessory 24B of the touch panel display 12 emits light. This allows the touch device 202 to perform either [1] an "automatic charging function" that automatically charges the pad accessory 24B, or [2] a "communication function" that uses visible light as a transmission medium.

[0154] <Summary of the second embodiment> As described above, the input system 200 in the second embodiment includes a touch device 202 having a capacitive touch sensor 82, and a virtual input device 206 used together with the touch device 202. The pad attachment 24B, which constitutes a part of the virtual input device 206, includes a power storage unit 46 that stores or releases electric energy, and a photoelectric conversion unit 208 that converts external light into electric energy and supplies the electric energy to the power storage unit 46.

[0155] The touch device 202 further includes a display panel 80 that displays an image or video, and a host processor 210 that controls the light emission of the display panel 80. When the pad accessory 24B is placed on the display panel 80, the host processor 210 controls the display panel 80 to emit light in at least a part of the area where the pad accessory 24B is present. This makes it possible to automatically charge the pad accessory 24B while utilizing the light emission function of the display panel 80.

[0156] The host processor 210 may also set light emission conditions related to the light emission range, light emission color, or light emission intensity, and control the light emission of the display panel 80 according to the light emission conditions. This can improve the charging efficiency of the pad accessory 24B in terms of the light emission range, light emission color, or light emission intensity.

[0157] Furthermore, the host processor 210 may set the light emission range according to the detected position of the pad attachment 24B or the photoelectric conversion unit 208. This enables light emission that takes into account the position of the pad attachment 24B or the photoelectric conversion unit 208, thereby further improving the charging efficiency of the pad attachment 24B.

[0158] Furthermore, the host processor 210 may set the emission color according to the type of the pad attachment 24B or the photoelectric conversion unit 208. This enables emission of light that takes into account the wavelength dependency of the photoelectric conversion efficiency in the photoelectric conversion unit 208, thereby further improving the charging efficiency of the pad attachment 24B.

[0159] The host processor 210 may also set the light emission intensity according to the remaining power of the power storage unit 46 or the type of charging mode that has been set. This allows light emission that takes into account the remaining power or the charging mode, and allows charging of the pad accessory 24B according to the power situation. [Explanation of symbols]

[0160] 10,130,140,200...Input system, 14,132,142,202...Touch device, 18,120,146,206...Virtual input device, 20...Touch surface, 22,122,180...Operation pad, 24,24A,24B...Pad accessories, 40...First electrode (communication element), 42...Second electrode (communication element), 46...Power storage unit, 56...Receiving circuit, 58...Transmitting circuit, 60...Memory, 62...Device information, 62a...Device ID (identification information), 62b...Pad information, 62c...Additional area information, 62d...Light emission setting information, 82...Touch sensor, 90,154,210...Host processor (processor), 148...EMR sensor (surface sensor), 156...Coil (communication element), 208...Photoelectric conversion unit

Claims

1. A pad accessory attached to an operation pad having at least one physical operation element and disposed on a touch device having a capacitive touch sensor, a memory for storing device information relating to a virtual input device including the operation pad; one or more communication elements for wirelessly communicating with the touch device via the touch sensor or a planar sensor different from the touch sensor; a transmitting circuit for transmitting a transmission signal including the device information read from the memory via the one or more communication elements; A pad attachment comprising:

2. the device information includes pad information, which is a type of the operation pad, an arrangement of the physical operation elements, or settings related to the operation of the operation pad; The pad attachment according to claim 1.

3. When an operation function is provided to an additional area within the sensor area of ​​the touch sensor, which is different from the area where the operation pad is arranged, The device information includes additional area information, which is the type of the operation function, the position, size or shape of the additional area, or settings related to the operation function. The pad attachment according to claim 1.

4. The device information includes identification information of the virtual input device, the operation pad, or the device itself. The pad attachment according to claim 1.

5. The pad attachment according to claim 1; an operation pad having the physical operation element and having the pad attachment attached thereto; A virtual input device comprising:

6. a virtual input device including the pad attachment according to claim 1 and an operation pad having the physical operation element and to which the pad attachment is attached; a touch device having the touch sensor; An input system comprising:

7. the touch device further comprises a processor; The processor generates or outputs data including operation information indicating an operation state of the operation pad, which is information different from position information indicating the detection position, in accordance with a change in capacitance within an operation area determined according to the detection position of the one or more electrodes while wireless communication is being performed with the pad accessory.

7. The input system according to claim 6.

8. The processor: acquiring identification information of the virtual input device by wireless communication with the pad attachment; acquiring the layout of the physical operation elements corresponding to the identification information of the virtual input device or the setting contents related to the operation of the operation pad by communicating with an external device different from the pad accessory; 8. The input system according to claim 7.

9. The operation pad is configured such that, when the physical operation element is pressed from the surface side by a user while the operation pad is placed on the touch device, a change in capacitance occurs in a portion of the touch sensor corresponding to the position of the physical operation element, in association with the approach of the user or a conductor different from the user.

7. The input system according to claim 6.

10. The physical operating element includes an operating member that is conductive and elastic.

10. The input system according to claim 9.

11. The pad attachment is a power storage unit that stores or releases electrical energy; a photoelectric conversion unit that converts external light into electrical energy and supplies the electrical energy to the power storage unit; Further provided with 7. The input system according to claim 6.

12. The touch device is a display panel for displaying an image or video; a processor for controlling the light emission of the display panel; Furthermore, When the pad accessory is placed on the display panel, the processor controls the display panel to emit light in at least a part of an area where the pad accessory is present. The input system of claim 11.

13. the processor sets a light emission condition related to a light emission range, a light emission color, or a light emission intensity, and controls light emission of the display panel in accordance with the light emission condition.

13. The input system of claim 12.

14. the processor sets the light emission range in accordance with a detected position of the pad attachment or the photoelectric conversion unit.

14. The input system of claim 13.

15. the processor sets the emitted light color depending on the type of the pad attachment or the photoelectric conversion unit.

14. The input system of claim 13.

16. the processor sets the light emission intensity according to the remaining power of the power storage unit or the type of charging mode that is set.

14. The input system of claim 13.

Images