Information processing system, information processing apparatus, program, and control method

Abstract

To improve the feeling of use of an input device which presents vibration in response to contact.SOLUTION: An input device includes a vibrating unit. An information processing apparatus includes a touch sensor and a display unit. The touch sensor and the display unit are superposed on each other. A controller of the information processing apparatus or the input device supplies, to the vibrating unit, a drive signal obtained by synthesizing a vibration component with a sound component in response to contact of the input device with the touch sensor, and determines output characteristics of the sound component based on an acoustic environment of the information processing apparatus. The controller detects a moving direction of a contract position at which the input device comes into contact with the touch sensor, refers to a reference waveform set for each reference azimuth and a synthesis ratio of reference waveforms for each moving direction, and synthesizes a vibration signal indicating the vibration component, based on the reference waveform, at the synthesis ratio corresponding to the detected moving direction. This embodiment can be implemented in any configuration of an information processing system, an information processing apparatus, a program, and a control method.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to the control of an input device including an information processing system, an information processing apparatus, a program, and a control method.

Background Art

[0002] A haptic pen is a digital pen having a haptics feedback function. A digital pen is an input device used for displaying and recording information such as handwritten characters on an information processing apparatus. A digital pen is also called an electronic pen, a smart pen, a stylus, or the like. Haptic feedback is realized by vibrating a vibrator in response to contact with a touch panel that displays characters or the like. By providing a haptic feedback function, an attempt has been made to improve the writing feeling. In a haptic pen, an attempt has also been made to simulate vibrations generated by mechanical contact of a writing instrument with paper.

[0003] For example, Patent Document 1 describes an interactive stylus including a stylus body having a first end, the first end being configured to provide a first function and a second function when used by a user. The interactive stylus further includes a controller configured to drive the first end so as to capacitively couple the first end with an electrode matrix of an interactive display device. Capacitive coupling is related to the first function, and the second function is disabled in response to automatically sensing the intention of a user using the first function without explicit user input. The interactive stylus further includes a haptic actuator included in the stylus body and related to the second function, and disabling the second function includes preventing the operation of the haptic actuator.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] However, it is generally difficult to reproduce the same feeling as a mechanical writing instrument such as a ballpoint pen by only simulating vibration. During writing, in addition to vibration, other types of stimuli such as sound are generated. It is considered that this is because the user perceives the writing feeling by combining these. Therefore, the usability of the haptic pen has not always been improved. [Means for Solving the Problems]

[0006] The present invention has been made to solve the above problems, and an information processing system according to an aspect of the present invention includes an information processing device and an input device. The input device includes a vibration unit, and the information processing device includes a touch sensor and a display unit. The information processing system is an information processing system in which the touch sensor and the display unit overlap. A controller of the information processing device or the input device supplies a drive signal obtained by synthesizing a vibration component and an audio component to the vibration unit in response to contact with the touch sensor of the input device, and determines an output characteristic of the audio component based on an acoustic environment of the information processing device. The controller detects a moving direction of a contact position where the input device contacts the touch sensor, and synthesizes a vibration signal indicating the vibration component based on the reference waveform with a synthesis ratio corresponding to the detected moving direction by referring to a synthesis ratio between a reference waveform set in advance for each reference azimuth and reference waveforms for each moving direction.

[0007] In the information processing system described above, the controller may determine the intensity of the audio component according to a volume setting for the input device.

[0008] In the above information processing system, when the mute mode is set, the controller may stop the output of the voice component.

[0009] In the above information processing system, the information processing apparatus includes a speaker, and when the volume from the speaker exceeds a predetermined reference volume, the controller may stop the output of the voice component.

[0010] In the above information processing system, the information processing apparatus includes a sound collection unit that collects ambient sound, and the controller may determine the intensity of the voice component based on the intensity of the sound collected by the sound collection unit.

[0011] An information processing apparatus according to a second aspect of the present invention includes a controller, a touch sensor, and a display unit, wherein the touch sensor and the display unit overlap, and the controller supplies a drive signal obtained by synthesizing a vibration component and a voice component to a vibration unit of the input device in response to contact with the touch sensor of the input device, and determines output characteristics of the voice component based on an acoustic environment of the own device. The controller detects a moving direction of a contact position where the input device contacts the touch sensor, and synthesizes a vibration signal indicating the vibration component based on the reference waveform according to a synthesis ratio corresponding to the detected moving direction with reference to a synthesis ratio between a reference waveform set in advance for each reference azimuth and reference waveforms for each moving direction.

[0012] A program according to a third aspect of the present invention is a program for causing a computer to function as the above information processing apparatus.

[0013] The control method according to the fourth aspect of the present invention includes an information processing device and an input device. The input device includes a vibration unit, and the information processing device includes a touch sensor and a display unit. The control method in an information processing system in which the touch sensor and the display unit overlap, wherein a controller of the information processing device or the input device supplies a drive signal obtained by synthesizing a vibration component and an audio component to the vibration unit in response to contact with the touch sensor of the input device, and determines output characteristics of the audio component based on an acoustic environment of the information processing device. The controller detects a moving direction of a contact position where the input device contacts the touch sensor, and synthesizes a vibration signal indicating the vibration component based on the reference waveform with a synthesis ratio corresponding to the detected moving direction by referring to a reference waveform set in advance for each reference direction and a synthesis ratio between reference waveforms for each moving direction.

Advantages of the Invention

[0014] According to an embodiment of the present invention, it is possible to improve the usability of the input device by presenting a sound according to the acoustic environment together with vibration.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, a configuration example of an information processing system S1 according to this embodiment will be described. In the following description, the case where the information processing device 1 is a tablet terminal and the input device 30 is a digital pen having a tactile presentation function is mainly described. The digital pen is a pen-shaped operation medium and can be configured as a writing device. In the present application, the digital pen may be simply referred to as a "pen". FIG. 1 is a perspective view showing an external configuration example of the information processing system S1 according to this embodiment. The information processing system S1 includes an information processing device 1 and an input device 30.

[0017] The information processing device 1 has a flat plate shape with a thickness thinner than its width and height. The information processing device 1 has a housing CS1 and has a touch screen 20 on one main surface of the housing CS1 (hereinafter sometimes referred to as the "front surface" in the following description). The touch screen 20 covers the front surface of the housing CS1. The outer periphery of the touch screen 20 is supported by the housing CS1. Other members of the information processing device 1 are housed in the housing CS1.

[0018] The input device 30 is held by the user and moved while approaching or contacting the surface of the touch screen 20, and is used for data input or editing such as characters, symbols, and figures. The touch screen 20 includes a display unit 21 and a touch sensor 22 (described later). The display unit 21 visibly displays various information on a display screen DF installed on its surface. The touch sensor 22 is disposed so as to overlap the surface of the display unit 21. The touch sensor 22 detects an input operation of the input device 30 on its surface. The information processing device 1 monitors the presence or absence of the input operation, identifies the position where the input operation is detected, and executes data input or editing based on the input operation. The information processing device 1, for example, causes a trajectory of a contact position where the touch sensor 22 is contacted to be displayed on the display screen DF.

[0019] The input device 30 includes a vibration unit 55 (described later). The information processing device 1 detects the acoustic environment of its own device and determines the output characteristics of the voice component based on the detected acoustic environment. The information processing device 1, for example, determines the intensity of the voice component according to the volume setting for the input device 30. The information processing device 1 may collectively set the necessity of voice output for all presentable devices that are connected to its own device and can present voice. In that case, the information processing device 1 also applies the necessity of voice output to the input device 30.

[0020] Next, a schematic configuration example of the information processing system S1 according to the present embodiment will be described. FIG. 2 is a block diagram showing a schematic configuration example of the information processing system S1 according to the present embodiment. The information processing system S1 includes an information processing device 1 and an input device 30. The information processing apparatus 1 includes a control unit 10, a touch sensor 22, and a wireless communication unit 29. The touch sensor 22 detects contact with or approach within a certain distance (hovering) of the display screen DF as an input operation of the input device 30, and specifies the position thereof. The touch sensor 22 generates input data including the detected contact or approach and detection information indicating the position, and outputs the generated input data to the control unit 10. The touch sensor 22 may detect the pressure at the contact position, which is the position where the input device 30 is in contact, as the contact pressure, and include information on the detected contact pressure in the input data and output it to the control unit 10. The touch sensor 22 may detect the angle formed by the input device 30 with respect to the display screen DF in the longitudinal direction as the pen angle. The touch sensor 22 may include information on the detected pen angle in the input data and output it to the control unit 10.

[0021] The control unit 10 controls the overall functions of the information processing apparatus 1. The control unit 10 includes a display processing unit 102, an audio setting unit 103, an audio processing unit 104, and a vibration control unit 110. The audio setting unit 103 causes, for example, an audio setting screen (not shown) to be displayed as the display screen DF (FIG. 1) of the display unit 21. The audio setting screen is a screen for setting a presentation device for presenting audio and the volume. The audio setting screen represents a presentable device connected to the own unit and capable of presenting audio, and screen components for instructing a volume adjustment amount are arranged for each presentable device. The volume adjustment amount corresponds to the gain for the audio signal.

[0022] The audio setting unit 103 may set an audio presentation mode indicating the necessity of audio output for each presentable device. Examples of the audio presentation mode include a normal mode, a mute mode, etc. The normal mode is an operation mode that enables audio to be presented based on the acquired audio signal. The mute mode is an operation mode that does not present audio based on the acquired audio signal. The mute mode is also called a manner mode. The audio presentation mode may be set commonly for the presentable devices of the information processing apparatus 1.

[0023] The voice setting unit 103 selects, for example, one of the presentable devices based on the input data from the touch sensor 22 as the presentation device, and sets the volume adjustment amount for the presentation device. The presentable devices include, for example, a speaker 26 (described later) and an input device 30. The voice setting unit 103 outputs voice setting information indicating the selected presentation device and the volume adjustment amount to the voice processing unit 104. The voice setting information may include information on the voice presentation mode. The voice setting unit 103 may apply the mute mode common to the presentable devices of the information processing apparatus 1 with priority over the voice presentation modes for the individual presentable devices. That is, when the mute mode is set as the voice presentation mode common to the presentable devices, the mute mode is notified as the voice setting information for the input device 30 as the presentation device.

[0024] Regardless of whether the input device 30 is selected as the presentation device, the voice setting unit 103 may be able to set the volume adjustment amount for the input device 30. The input device 30 may be allowed to generate a contact sound in response to contact with the touch sensor 22 regardless of whether it is selected as a presentable device. When the volume adjustment amount for the input device 30 is set, the voice setting unit 103 outputs the voice setting information indicating the volume adjustment amount for the input device 30 to the input device 30 as acoustic environment information via the wireless communication unit 29. The voice setting information can also be regarded as information for transmitting the acoustic environment of the information processing apparatus 1.

[0025] The function of the voice setting unit 103 can be realized, for example, by executing a device setting API (Application Programming Interface) function of an operating system (OS). The function of the voice setting unit 103 may also be realized by calling the API function according to an application program (in this application, sometimes referred to as an "app"). In this application, the execution of a program includes the meaning of executing the processes instructed by various instructions described in the program.

[0026] When voice presentation is instructed, the voice processing unit 104 acquires a voice signal. The voice presentation can be instructed by input data indicating, for example, pressing of a voice playback button from the touch sensor 22. The voice processing unit 104 may acquire the voice signal by using any of the following methods: input of a voice signal from outside the information processing apparatus 1, reading of a pre-stored voice signal, synthesis of a voice signal, etc. The voice processing unit 104 may be realized as a part of the function of an application related to the voice setting unit 103. The application may be, for example, any of a voice call application, a recording playback application, a voice synthesis application, a voice distribution application, etc.

[0027] Voice setting information is input to the voice processing unit 104 from the voice setting unit 103. The voice processing unit 104 adjusts the volume of the acquired voice signal by the volume adjustment amount indicated by the voice setting information. The voice processing unit 104 outputs the voice signal after volume adjustment to the presentation device indicated by the acoustic environment information. Note that when a mute mode (manner mode) is indicated in the voice setting information, the voice processing unit 104 stops the output of the acquired voice signal until the cancellation of the mute mode is next instructed.

[0028] Note that the voice setting unit 103 may determine the volume of the voice signal input from the microphone 25 as the ambient volume. The voice setting unit 103 may output information indicating the determined ambient volume to the input device 30 via the wireless communication unit 29 included in the acoustic environment information. Also, when the speaker 26 is selected as the presentation device in the voice setting unit 103, the voice processing unit 104 may output the voice signal after volume adjustment to the voice setting unit 103. The voice setting unit 103 determines the volume of the voice signal after volume adjustment input from the voice processing unit 104 as the speaker volume. The voice setting unit 103 may output information indicating the determined speaker volume to the input device 30 via the wireless communication unit 29 included in the acoustic environment information.

[0029] The display processing unit 102 performs data input or editing according to the input operation indicated by the input data from the touch sensor 22. For example, the display processing unit 102 specifies the contact position indicated by the input data input at regular intervals and forms a time series of the contact positions. The display processing unit 102 generates a movement locus based on the formed time series of the contact positions and causes the display unit 21 to display it.

[0030] The vibration control unit 110 provides information used for controlling the vibration by the vibration unit 55 to the input device 30 based on the input data indicating the input operation from the touch sensor 22. The function of the vibration control unit 110 may be realized, for example, by executing the device driver of the input device 30. The vibration control unit 110 detects one or both of the contact position information and the contact pressure information as vibration control information from the input data indicating the contact with the touch sensor 22. The vibration control unit 110 outputs the detected vibration control information to the input device 30 via the wireless communication unit 29. The vibration control unit 110 may estimate the angle formed by the moving direction of the contact position of the input device 30 on the display screen DF and the longitudinal direction of the input device 30 from the input data as the azimuth angle. For example, the vibration control unit 110 can determine the direction of displacement from the contact position at the immediately preceding time to the contact position at that time as the moving direction of the input device 30. The vibration control unit 110 can determine the pen angle indicated in the input data as the direction of the longitudinal direction of the input device 30 at that time. The vibration control unit 110 may output the information indicating the estimated azimuth angle to the input device 30 included in the vibration control information. The wireless communication unit 29 wirelessly transmits and receives various data with the input device 30 using a predetermined communication method.

[0031] Next, a schematic configuration example of the input device 30 will be described. The input device 30 includes a wireless communication unit 31, a device control unit 50, and a vibration unit 55. The wireless communication unit 31 wirelessly transmits and receives various data with the information processing apparatus 1 using a predetermined communication method. The device control unit 50 comprehensively controls the functions of the input device 30. For example, acoustic environment information and vibration control information are input to the device control unit 50 from the information processing device 1 via the wireless communication unit 31. The device control unit 50 synthesizes a drive signal for vibrating the vibration unit 55 using the acoustic environment information and the vibration control information (drive signal synthesis). The device control unit 50 outputs the synthesized drive signal to the vibration unit 55.

[0032] The device control unit 50 refers to preset reference vibration waveform information and generates a vibration signal indicating a vibration component based on the reference vibration control information. The reference vibration waveform information is information indicating a reference vibration waveform. The characteristics of the vibration component are adjusted by the device control unit 50 using the reference vibration control information. The device control unit 50 refers to preset reference audio waveform information and generates an audio signal indicating an audio component based on the audio setting information. The output characteristics of the audio component are adjusted by the device control unit 50 using the acoustic environment information. An example of generating a drive signal will be described later.

[0033] The vibration unit 55 vibrates according to the drive signal input from the device control unit 50. Among the vibrations of the vibration unit 55, the audio component generates sound waves and is presented as sound. Among the vibrations of the vibration unit 55, the vibration component is presented as vibration.

[0034] Generally, the audible range of humans is about 20 Hz - 20 kHz, and the sensitivity to the 500 Hz - 2 kHz band is higher than that to other frequency bands. In lower frequency bands, the sensitivity decreases. In contrast, the human tactile sense has a higher sensitivity to the 10 - 100 Hz band than to other frequency bands. In higher frequency bands, the sensitivity decreases. At frequencies above 500 Hz, almost no vibration is perceived. The vibration signal mainly consists of low-frequency components of about 200 Hz or less. In contrast, the audio signal mainly consists of high-frequency components of about 200 Hz or more.

[0035] Next, a hardware configuration example of the information processing system S1 according to the present embodiment will be described. FIG. 3 is a block diagram showing a hardware configuration example of the information processing system S1 according to the present embodiment. The information processing system S1 includes an information processing apparatus 1 and an input device 30. The information processing apparatus 1 includes a processor 11, a main memory 12, a flash memory 13, a touch screen 20, an audio system 24, a microphone 25, a speaker 26, a baseband chip 27, a second communication unit 28, and a wireless communication unit 29.

[0036] The processor 11 controls the functions of the entire information processing apparatus 1. As the processor 11, for example, one or more CPUs (Central Processing Units) are applied. The processor 11 executes a predetermined program and cooperates with the main memory 12 and other hardware to function as a control unit 10. The main memory 12 is a writable memory used as a work area of the processor 11, that is, a read area for an execution program and various setting data, and a write area for processing data obtained by executing the program. The main memory 12 is configured to include, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. The execution program includes an OS, various device drivers for controlling peripheral devices, various services / utilities, applications, and the like.

[0037] The flash memory 13 stores an OS, various device drivers, various services / utilities, applications, and various data in advance. The display unit 21 displays various display screens based on display data output from the processor 11. The display unit 21 may be, for example, any of a liquid crystal display, an organic electroluminescence display, and the like.

[0038] The touch sensor 22 includes a contact detection unit 221 and a device detection unit 222. The contact detection unit 221 detects an object that has come into contact with the display screen DF (in this application, mainly the input device 30) and its contact position. The contact detection unit 221 detects the contact pressure of the object that has come into contact with the display screen DF. The contact detection unit 221 is, for example, a capacitance pressure sensor. The contact detection unit 221 may detect the inclination of the contacting object, that is, the pen angle. When the contact detection unit 221 is a three-axis pressure sensor, the pen angle can be specified using the direction cosines for the pressures in the individual axial directions.

[0039] The device detection unit 222 detects an input device 30 that has not come into contact but has approached its own part and its position as the approach position. The device detection unit 222 includes, for example, an electromagnetic induction sensor. The electromagnetic induction sensor detects the alternating induction magnetic field generated in the resonance circuit LC1 (described later) due to the approach. The electromagnetic induction sensor can detect the presence or absence of the approach of the input device 30 based on whether there is a position where the magnitude of the detected magnetic field at its resonance frequency exceeds a certain value. The electromagnetic induction sensor can specify the position where the magnitude of the detected magnetic field exceeds a certain value as the approach position of the input device 30.

[0040] The audio system 24 performs processes such as processing, input, output, recording, playback, encoding, and decoding of audio signals. The audio system 24 includes, for example, an audio IC (Integrated Circuit). A microphone 25 and a speaker 26 are connected to the audio system 24. An audio signal can be input to the audio system 24 from the processor 11, the microphone 25, or the baseband chip 27. The audio system 24 reads out the audio signal recorded in its own part. The audio system 24 can output the acquired audio signal to the speaker 26 or the processor 11. The audio signal output to the processor 11 can be output to the input device 30 via the second communication unit 28 or the wireless communication unit 29 via the baseband chip 27.

[0041] The microphone 25 picks up the sound arriving at its own part and outputs the audio signal of the picked-up sound to the audio system 24. The speaker 26 presents sound based on an audio signal input from the audio system 24.

[0042] The baseband chip 27 is an application-specific integrated circuit (ASIC) for controlling communication using the second communication unit 28. The baseband chip 27 realizes communication using, for example, a public wireless communication system such as 4G (the fourth-generation wireless communication system), 5G (the fifth-generation wireless communication system), or a wired local area network defined in IEEE 802.11. The baseband chip 27 is connected to other devices via a communication network using the second communication unit 28 in accordance with control from the processor 11, and is capable of transmitting and receiving various types of data.

[0043] The second communication unit 28 is a wireless communication module for connecting to a wireless communication network. The second communication unit 28 includes an antenna for transmitting and receiving radio waves. The wireless communication unit 29 is a wireless communication module for transmitting and receiving wireless data to and from the input device 30. As a communication method, the wireless communication unit 29 can use, for example, the wireless PAN (Personal Area Network) method defined in IEEE 802.15.1.

[0044] The input device 30 is an operating medium having an elongated shape with a length greater than its diameter. The input device 30 includes a wireless communication unit 31, an MCU 32, a vibration unit 55, and a resonance circuit LC1. The wireless communication unit 31 is a wireless communication module for transmitting and receiving wireless data to and from the wireless communication unit 29 of the information processing device 1.

[0045] The MCU (Micro Controller Unit) 32 comprehensively controls the functions of the input device 30. The MCU 32 includes a processor, a memory such as a ROM and a RAM, and various input / output interfaces. The MCU 32 operates independently of the information processing device 1. The MCU 32 executes a predetermined program to function as the device control unit 50 described above. The MCU 32 receives voice setting information and vibration control information from the information processing device 1 via the wireless communication unit 31. The MCU 32 synthesizes a drive signal using one or both of the voice setting information and the vibration control information. The MCU 32 outputs the synthesized drive signal to the vibration unit 55.

[0046] The vibration unit 55 includes a DAC 33, an amplifier 34, and a vibration generator 35. The DAC (Digital-to-Analog Converter) 33 converts the digital drive signal input from the MCU 32 into an analog drive signal. The DAC 33 outputs the converted analog drive signal to the amplifier 34. The amplifier 34 adjusts the amplitude of the drive signal input from the DAC 33 and outputs the drive signal with the adjusted amplitude to the vibration generator 35. The vibration generator 35 is an actuator that generates vibration according to the drive signal input from the amplifier 34. The vibration generator 35 includes, for example, a piezoelectric vibrator such as a piezo element.

[0047] The resonance circuit LC1 is an electric circuit that generates a current resonating at a certain resonance frequency. The resonance circuit LC1 is configured, for example, by connecting a coil and a capacitor in series. The resonance circuit LC1 generates a magnetic field whose polarity fluctuates at its resonance frequency by the alternating current it generates. The approach of the input device 30 is detected by the device detection unit 222 of the touch sensor 22 by the generated magnetic field.

[0048] Next, a functional configuration example of the information processing system S1 according to the present embodiment will be described. FIG. 4 is a block diagram showing a functional configuration example of the information processing system S1 according to the present embodiment. In the example of FIG. 4, differences from FIGS. 2 and 3 will be mainly described. For common points with FIGS. 2 and 3, the same reference numerals are given, and the above description will be incorporated by reference unless otherwise specified.

[0049] The information processing system S1 includes an information processing device 1 and an input device 30. The information processing apparatus 1 includes a control unit 10, a touch screen 20, a wireless communication unit 29, and a storage unit 40. The storage unit 40 is realized by, for example, a main memory 12 and a flash memory 13. The control unit 10 includes an input processing unit 101, a display processing unit 102, an audio setting unit 103, an audio processing unit 104, and a vibration control unit 110. The input device 30 includes a wireless communication unit 31, a device control unit 50, and a vibration unit 55.

[0050] The input processing unit 101 controls the input from the touch sensor 22. The function of the input processing unit 101 is realized, for example, by executing a device driver of the touch sensor 22 by the processor 11. The input processing unit 101 outputs the input data input from the touch sensor 22 to the vibration control unit 110 according to the instructions of the OS.

[0051] The input device 30 includes a vibration control unit 510 in the device control unit 50. The vibration control unit 510 controls the vibration of the vibration unit 55 based on the acoustic environment information and the vibration control information input using the wireless communication unit 31 as described above. When the vibration control information notifies the contact with the touch sensor 22, the vibration control unit 510 generates a drive signal including an audio component and a vibration component. The vibration control unit 510 outputs the generated drive signal to the vibration unit 55.

[0052] The vibration control unit 510 controls the output characteristics of the entire drive signal based on the contact state with the touch sensor 22 transmitted by the vibration control information. For example, when the information on the contact position is included in the contact state, the higher the moving speed of the contact position, the larger the amplitude of the vibration component or the entire drive signal. When the information on the contact pressure is included in the contact state, the vibration control unit 510 may increase the amplitude of the vibration component or the entire drive signal as the contact pressure increases. When the information on the volume adjustment amount (pen volume) for the input device 30 is included in the acoustic environment information, the vibration control unit 510 may adjust the volume of the voice component based on the volume adjustment amount. Also, when the information indicating the mute mode as the voice presentation mode is included in the acoustic environment information, the vibration control unit 510 stops the output of the voice component. When stopping the output of the voice component, before outputting the drive signal, the vibration control unit 510 includes the vibration component in the drive signal without including the voice component.

[0053] When the information indicating the speaker volume is included in the acoustic environment information, the vibration control unit 510 may control the output characteristics of the voice component based on the speaker volume. For example, when the speaker volume is larger than the reference volume of a predetermined speaker volume, the vibration control unit 510 stops the output of the voice component. When the information indicating the ambient volume is included in the acoustic environment information, the vibration control unit 510 may control the output characteristics of the voice component based on the ambient volume. For example, when the ambient volume is larger than the reference volume of a predetermined ambient volume, the vibration control unit 510 stops the output of the voice component.

[0054] Next, an example of the output characteristics of the oscillator 35 according to the present embodiment is shown. FIG. 5 is a diagram showing an example of the output characteristics of the oscillator 35 according to the present embodiment. The horizontal axis and the vertical axis in FIG. 5 indicate frequency (Hz) and amplitude (Gmrs: effective acceleration), respectively. As the amplitude, the amplitude of the effective acceleration generated in the oscillator 35 with respect to a constant driving power for each frequency is shown. The larger the amplitude value, the more efficiently the vibrating unit 55 vibrates. The amplitude generally reaches a maximum at 100 Hz. In the low frequency range where the frequency is lower than 100 Hz, the lower the frequency, the smaller the amplitude, and in the high frequency range where the frequency is higher, the higher the frequency, the smaller the amplitude. However, the decrease in amplitude in the high frequency range is gentler than that in the low frequency range. This indicates that it is possible to present sound in the high frequency range that is mainly perceived as sound and not as vibration. The vibration control unit 510 can mainly control the output characteristics of the sound component by adjusting the amplitude of the output characteristics in the high frequency range.

[0055] Next, an example of a method for controlling the sound component will be described. FIG. 6 is a diagram showing a first example of a method for controlling the sound component according to the present embodiment. In the first example, a reference waveform signal and a control table (not shown) are set in advance in the vibration control unit 510. The reference waveform signal is a time-series signal including a vibration component and a sound component. The vibration control unit 510 includes a variable filter and a variable amplifier. Information indicating the correspondence between the filter coefficient of the variable filter and the output characteristics of the sound component, and information indicating the correspondence between the set of moving speed and contact pressure and the gain are set in advance in the control table.

[0056] When generating a drive signal, the vibration control unit 510 reads the set reference waveform signal and supplies it to the variable filter. The vibration control unit 510 refers to the control table, determines the filter coefficient corresponding to the output characteristics notified by the acoustic environment information, and sets the determined filter coefficient in the variable filter. A sound adjustment signal indicating a waveform in which the sound component is adjusted to have the set output characteristics is output from the variable filter to the variable amplifier.

[0057] The vibration control unit 510 identifies the moving speed and contact pressure of the contact position based on the contact state notified by the vibration control information. The vibration control unit 510 refers to the control table, determines the gain corresponding to the set of the identified moving speed and contact pressure, and sets the determined gain in the variable amplifier. From the variable filter, a drive signal whose amplitude is adjusted by the set gain is output to the vibration unit 55.

[0058] Next, an example of a method for controlling the audio component will be described. FIG. 7 is a diagram showing a second example of the method for controlling the audio component according to the present embodiment. In the second example, a control table (not shown) is set in the vibration control unit 510, and an audio signal indicating an audio waveform and a vibration signal indicating a vibration waveform are acquired. Instead of the variable filter for the reference waveform, the vibration control unit 510 includes a variable resistor element for the audio component (hereinafter referred to as "audio variable resistor element") and a variable resistor element for the vibration component (hereinafter referred to as "vibration variable resistor element") and an adder for each of the audio waveform and the vibration waveform. The control table includes information indicating the correspondence between the resistance value of the audio variable resistor element and the output characteristics of the audio component, information indicating the correspondence between the resistance value of the vibration variable resistor element and the output characteristics of the audio component, and information indicating the correspondence between the set of the moving speed and the contact pressure and the gain. The information indicating the correspondence between the resistance value of the vibration variable resistor element and the output characteristics of the audio component is not essential, but may be set for the purpose of adjusting the relative ratio of the audio component in the drive signal.

[0059] The vibration control unit 510 refers to the control table, identifies the resistance values for the audio variable resistor element and the vibration variable resistor element corresponding to the output characteristics notified by the acoustic environment information, and sets the respectively identified resistance values in the audio variable resistor element and the vibration variable resistor element. The vibration control unit 510 outputs the acquired audio signal and vibration signal to the audio variable resistor element and the vibration variable resistor element, respectively. The vibration control unit 510 outputs the audio signal and the amplitude signal whose amplitudes are adjusted from the audio variable resistor element and the vibration variable resistor element to the adder, respectively.

[0060] The adder adds the input voice signal and the amplitude signal to generate a drive signal, and outputs the generated drive signal to the variable amplifier. Similar to the first example, the vibration control unit 510 refers to the control table to set the gain for the variable amplifier, and adjusts the gain of the drive signal using the variable amplifier. Note that in the second example, instead of the voice variable resistor element and the vibration variable resistor element, an attenuator or amplifier for the voice component and an attenuator or amplifier for the vibration component may be used.

[0061] In the second example, the reference voice waveform signal and the reference vibration waveform signal set in the vibration control unit 510 in advance may be used as the voice signal and the vibration signal to be processed. The reference voice waveform is not necessarily limited to the voice waveform of the frictional sound generated by the friction between the writing instrument and the paper during writing, and a warning sound or other notification sound may be used. Also, in this configuration, the output characteristics of the voice signal and the vibration signal can be controlled independently. Therefore, the vibration control unit 510 may use the voice signal input from the information processing device 1 as the voice signal to be processed, or may use the voice signal synthesized independently by itself as the voice signal to be processed. The input or synthesized voice signal can be specified by the voice setting unit 103 with the input device 30 as the output destination for the presentation device.

[0062] The vibration control unit 510 may synthesize the vibration signal independently. For example, when the vibration control information has azimuth angle information, the vibration control unit 510 may synthesize the vibration signal corresponding to the azimuth angle as the vibration signal to be processed. In that case, the reference waveform information and the synthesis table are stored in the vibration control unit 510 in advance. The reference waveform information is information indicating the reference waveform for each of a plurality of predetermined reference azimuth angles. The reference azimuth angles may be set to three or more between 0° and 180°. The synthesis table is a data table indicating the synthesis ratio for the azimuth angle with respect to the reference waveform for each reference azimuth angle. The vibration control unit 510 refers to the synthesis table to determine the synthesis ratio for the notified azimuth angle for each reference azimuth angle. The vibration control unit 510 can generate the weighted sum of the vibration waveforms between the reference azimuth angles with the determined synthesis ratio as the weight coefficient as the vibration signal.

[0063] Next, an example of a method for controlling the audio component will be described. FIG. 8 is a diagram showing a third example of the method for controlling the audio component according to the present embodiment. In the third example, a control table (not shown) is set in the vibration control unit 510, and a plurality of waveform patterns of drive signals are set in advance. Each waveform pattern includes at least a vibration component, and the output characteristics of the audio component are different from each other. The control table includes information indicating the correspondence relationship between the set of moving speed and contact pressure and the gain. The control table may not include information regarding the output characteristics of the audio signal.

[0064] The vibration control unit 510 reads out the waveform pattern corresponding to the output characteristics notified by the acoustic environment information from among the plurality of waveform patterns, and outputs the drive signal related to the read waveform pattern to the variable amplifier. Similar to the first example, the vibration control unit 510 refers to the control table, sets the gain for the variable amplifier, and adjusts the gain of the drive signal using the variable amplifier. Note that in the vibration control unit 510, the above reference waveform signal, reference audio waveform signal, reference vibration waveform signal, and drive signal (waveform pattern) are stored and stored in a data file having a predetermined file format (for example, WAVE format).

[0065] Next, an example of the audio output control method according to the present embodiment will be described. FIG. 9 is a flowchart showing an example of the audio output control method according to the present embodiment. (Step S102) The vibration control unit 510 of the input device 30 waits for the acoustic environment information input from the information processing device 1. The vibration control unit 510 determines whether the pen volume has been set and changed based on whether the acoustic environment information includes information on the volume adjustment amount. When it is determined that the setting has been changed (Step S102 YES), the process proceeds to the process of Step S104. When it is determined that the setting has not been changed (Step S102 NO), the process proceeds to the process of Step S106. (Step S104) The vibration control unit 510 adjusts the volume of the audio component based on the volume adjustment amount notified by the acoustic environment information (pen volume change).

[0066] (Step S106) The vibration control unit 510 determines whether the voice presentation mode is set to the mute mode based on whether the acoustic environment information includes information indicating the mute mode (manner mode). When it is determined that the mute mode is set (Step S106 YES), the process proceeds to the process of Step S114. When it is determined that the mute mode is not set (Step S106 NO), the process proceeds to the process of Step S108.

[0067] (Step S108) When the acoustic environment information includes information indicating the speaker volume, the vibration control unit 510 determines whether the speaker volume is greater than a predetermined reference volume. When it is determined to be greater (Step S108 YES), the process proceeds to the process of Step S114. When it is determined not to be greater (Step S108 NO), the process proceeds to the process of Step S110.

[0068] (Step S110) When the acoustic environment information includes information indicating the ambient volume, the vibration control unit 510 determines whether the ambient volume is greater than a predetermined reference volume. When it is determined to be greater (Step S110 YES), the process proceeds to the process of Step S114. When it is determined not to be greater (Step S110 NO), the process proceeds to the process of Step S112.

[0069] (Step S112) The vibration control unit 510 outputs a drive signal including an audio component to the vibration unit 55. The vibration unit 55 presents audio based on the audio component included in the drive signal (pen audio output). Then, the process returns to the process of Step S102. (Step S114) The vibration control unit 510 outputs a drive signal that does not include an audio component but includes a vibration component to the vibration unit 55. The vibration unit 55 vibrates according to the drive signal, and the presentation of audio is stopped (pen audio output stop). Then, the process returns to the process of Step S102.

[0070] In the above example, as shown in FIG. 4, the case is obtained where the input device 30 includes the vibration control unit 510 and generates a drive signal based on the acoustic environment information and the vibration control information input from the information processing device 1, but it is not limited thereto. As illustrated in FIG. 10, the vibration control unit 110 of the information processing apparatus 1 may generate a drive signal based on the acoustic environment information and the vibration signal information without outputting the acoustic environment information and the vibration control information to the input device 30, in the same manner as the above-described vibration control unit 510. Parameters and other data used for the processing are set in advance in the vibration control unit 110. The vibration control unit 110 outputs the generated drive signal to the input device 30 using the wireless communication unit 29. In the input device 30, the device control unit 50 outputs the drive signal input from the information processing apparatus 1 using the wireless communication unit 31 to the vibration unit 55. In this example, the vibration control unit 510 may be omitted in the input device 30.

[0071] Also, in the above example, the case where the information processing apparatus 1 is a tablet terminal device is mainly described, but it is not limited thereto. The information processing apparatus 1 may be an information communication device using a touch screen, such as a multifunctional mobile phone (including a so-called smartphone), a personal computer, or the like.

[0072] As described above, the information processing system S1 according to the present embodiment includes the information processing apparatus 1 and the input device 30. The input device 30 includes a vibration unit 55. The information processing apparatus 1 includes a touch sensor 22 and a display unit 21, and the touch sensor 22 and the display unit 21 overlap. A controller (for example, either one or both of the processor 11 and the MCU 32) of the information processing apparatus 1 or the input device 30 supplies a drive signal obtained by synthesizing a vibration component and an audio component to the vibration unit in response to contact with the touch sensor 22 of the input device 30, and determines the output characteristics of the audio component based on the acoustic environment of the information processing apparatus 1. With this configuration, the output characteristics of the audio component are determined based on the acoustic environment of the information processing apparatus 1. In response to contact with the input device 30, the vibration unit vibrates according to the vibration component of the drive signal, and audio is presented according to the audio component. The output characteristics of the presented audio are determined based on the acoustic environment. Therefore, the usability of the input device 30 that vibrates in response to contact with the touch sensor 22 is improved.

[0073] The controller may determine the intensity (e.g., amplitude) of the voice component according to the volume setting (e.g., volume adjustment amount) for the input device 30. With this configuration, according to the volume setting for the input device 30, the input device 30 can present voice with a desired intensity.

[0074] When the muting mode (i.e., the manner mode) is set, the controller may stop the output of the voice component. With this configuration, when the muting mode is set, the presentation of voice from the input device 30 is stopped.

[0075] The information processing apparatus 1 includes a speaker 26, and when the volume from the speaker 26 exceeds a predetermined reference volume, the controller may stop the output of the voice component. With this configuration, the presentation of voice of the input device 30 with a volume of the speaker 26 larger than the reference volume is stopped. When the sound from the speaker 26 makes it inaudible or difficult to listen to, the presentation of voice from the input device 30 is avoided.

[0076] The information processing apparatus 1 includes a sound collection unit (e.g., microphone 25) that collects ambient sound, and the controller may determine the intensity of the voice component based on the intensity of the sound collected by the sound collection unit (e.g., when exceeding the reference volume, stop the output of the voice component). With this configuration, voice is presented from the input device 30 with an intensity determined based on the intensity of the ambient sound collected by the sound collection unit. Therefore, the user can listen to the voice presented from the input device 30 at a volume corresponding to the volume of the ambient sound without performing any special operations.

[0077] As described above, the embodiments of the present invention have been described in detail with reference to the drawings, but the specific configuration is not limited to the above-described embodiments, and designs and the like within the scope not departing from the gist of the present invention are also included. Each configuration described in the above embodiments can be arbitrarily combined.

Description of Reference Numerals

[0078] S1… Information processing system, 1… Information processing device, 10… Control unit, 11… Processor, 12… Main memory, 13… Flash memory, 20… Touch screen, 21… Display unit, 22… Touch sensor, 24… Audio system, 25… Microphone, 26… Speaker, 27… Baseband chip, 28… Second communication unit, 29… Wireless communication unit, 30… Input device, 31… Wireless communication unit, 32… MCU, 33… DAC, 34… Amplifier, 35… Vibration generator, 40… Storage unit, 50… Device control unit, 55… Vibration unit, 101… Input processing unit, 102… Display processing unit, 103… Audio setting unit, 104… Audio processing unit, 110… Vibration control unit, 221… Contact detection unit, 222… Device detection unit, 510… Vibration control unit

Claims

1. Equipped with an information processing device and an input device, the input device is a writing device having a vibration unit, The information processing device includes a touch sensor, a display unit, and a sound collection unit that collects surrounding sounds, An information processing system in which the touch sensor and the display unit are superimposed, A controller of the information processing device or the input device, supplying to the vibration unit a drive signal in which a vibration component and a sound component are combined in response to a touch of the input device on the touch sensor; determining an output characteristic of the audio component based on an acoustic environment of the information processing device; When the volume of the sound picked up by the sound pickup unit is greater than a predetermined reference volume, output of the sound component is stopped. Information processing system.

2. Equipped with an information processing device and an input device, The input device includes a vibration unit, the information processing device includes a touch sensor, a display unit, and a speaker; An information processing system in which the touch sensor and the display unit are superimposed, A controller of the information processing device or the input device, supplying to the vibration unit a drive signal in which a vibration component and a sound component are combined in response to a touch of the input device on the touch sensor; determining an output characteristic of the audio component based on an acoustic environment of the information processing device; If the volume from the speaker is greater than a predetermined reference volume, output of the audio component is stopped. Information processing system.

3. The controller: determining an intensity of the audio component according to a volume setting for the input device; 3. The information processing system according to claim 1 or 2.

4. The controller: When the mute mode is set, the output of the audio component is stopped.

3. The information processing system according to claim 1 or 2.

5. The device is equipped with a controller, a touch sensor, a display unit, and a sound pickup unit that picks up surrounding sounds. The touch sensor and the display unit are overlapped with each other, The controller: In response to a touch of the writing device on the touch sensor, a drive signal in which a vibration component and a sound component are combined is supplied to a vibration unit of the writing device; determining an output characteristic of the voice component based on an acoustic environment of the device; When the volume of the sound picked up by the sound pickup unit is greater than a predetermined reference volume, output of the sound component is stopped. Information processing device.

6. The device includes a controller, a touch sensor, a display, and a speaker. The touch sensor and the display unit are overlapped with each other, The controller: supplying a drive signal, which is a combination of a vibration component and a sound component, to a vibration unit of the input device in response to a touch of the input device on the touch sensor; determining an output characteristic of the voice component based on an acoustic environment of the device; If the volume from the speaker is greater than a predetermined reference volume, output of the audio component is stopped. Information processing device.

7. To your computer A program for causing the information processing device according to claim 5 to function as the information processing device.

8. Equipped with an information processing device and an input device, the input device is a writing device having a vibration unit, The information processing device includes a touch sensor, a display unit, and a sound collection unit that collects surrounding sounds, A control method for an information processing system in which the touch sensor and the display unit are superimposed, comprising: A controller of the information processing device or the input device, supplying to the vibration unit a drive signal in which a vibration component and a sound component are combined in response to a touch of the input device on the touch sensor; determining an output characteristic of the audio component based on an acoustic environment of the information processing device, The controller: When the volume of the sound picked up by the sound pickup unit is greater than a predetermined reference volume, output of the sound component is stopped. Control methods.

9. Equipped with an information processing device and an input device, The input device includes a vibration unit, the information processing device includes a touch sensor, a display unit, and a speaker; A control method for an information processing system in which the touch sensor and the display unit are superimposed, comprising: A controller of the information processing device or the input device, supplying to the vibration unit a drive signal in which a vibration component and a sound component are combined in response to a touch of the input device on the touch sensor; determining an output characteristic of the audio component based on an acoustic environment of the information processing device, The controller: If the volume from the speaker is greater than a predetermined reference volume, output of the audio component is stopped. Control methods.

Images