Information processing system, method for information processing, and program

Abstract

To allow a person to have a sensation close to the sensation that the user has when actually writing on a physical medium.SOLUTION: An information processing system includes: a contact parameter detection unit for detecting at least one contact parameter corresponding to the contact state of a pen-type input device with respect to an operation target surface; and a feedback control unit for deriving, on the basis of the detected contact parameters, a feedback level corresponding to a writing tool type set in association with the pen-type input device, and for driving a haptic reproduction unit in the pen-type input device according to the derived feedback level.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] It is known that when inputting data to an interactive display system, a pen-shaped stylus (pen-type input device) that enables tactile feedback by generating vibrations that drive a tactile actuator is used as an input device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-537395 Summary of the Invention [Problem to be solved by the invention]

[0004] The vibrations emitted by the pen-type input device simulate the tactile sensation of sliding the pen tip across a writing medium such as paper, allowing users using the pen-type input device to get a sensation similar to that of writing on a writing medium.

[0005] When inputting data using a pen-type input device, it is desirable to have a feeling as close as possible to that of writing on an actual writing medium. In consideration of the above-mentioned problems, the present invention aims to provide an input using a pen-type input device that provides a feeling similar to that of actually writing on a writing medium. [Means for solving the problem]

[0006] One aspect of the present invention that solves the above-mentioned problems is an information processing system that includes a contact parameter detection unit that detects one or more contact parameters corresponding to the contact state of a pen-type input device with respect to an operation surface, and a feedback control unit that derives a feedback level corresponding to a writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device based on the derived feedback level.

[0007] One aspect of the present invention is an information processing method in an information processing system, comprising: a contact parameter detection step in which a contact parameter detection unit detects one or more contact parameters corresponding to the contact state of a pen-type input device with respect to an operation surface; and a feedback control step in which a feedback control unit derives a feedback level corresponding to a writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device using the derived feedback level.

[0008] One aspect of the present invention is a program for causing a computer in an information processing system to function as a contact parameter detection unit that detects one or more contact parameters corresponding to the contact state of a pen-type input device with respect to an operation surface, and a feedback control unit that derives a feedback level corresponding to a writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device based on the derived feedback level. [Effects of the Invention]

[0009] According to the present invention, when inputting using a pen-type input device, it is possible to obtain an effect that the feeling of actually writing on a writing medium can be made to be closer to that of the user. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a diagram illustrating an example of the external configuration of an information processing system according to an embodiment of the present invention. [Figure 2] 10 is a diagram showing writing response level characteristics for each writing implement type in this embodiment. FIG. [Figure 3] FIG. 1 is a diagram illustrating an example of the hardware configuration of an information processing apparatus and a pen-type input device according to an embodiment of the present invention. [Figure 4] 1 is a diagram illustrating an example of the functional configuration of an information processing apparatus and a pen-type input device according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating an example of a feedback table according to the present embodiment. [Figure 6] 5 is a diagram showing an example of vibration waveform data stored in a vibration waveform data storage unit in the present embodiment. FIG. [Figure 7] 3 is a diagram showing an example of writing sound data stored in a writing sound data storage unit in the present embodiment. FIG. [Figure 8] FIG. 2 is a diagram illustrating an example of a functional configuration related to driving a vibration unit and a sound output unit in the information processing system according to the present embodiment. [Figure 9] 10A and 10B are diagrams illustrating an example of a processing procedure executed by the information processing device and the pen-type input device in relation to haptic feedback according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] 1 shows an example of the external configuration of an information processing system according to this embodiment. As shown in the figure, the information processing system according to this embodiment includes an information processing device 100 and a pen-type input device 200.

[0012] The information processing device 100 is capable of executing information processing in response to an input operation using a pen-type input device 200. The information processing device 100 in the figure is an example of a tablet terminal or a notebook-type personal computer.

[0013] The information processing device 100 includes a touch panel display unit 30. The touch panel display unit 30 is a component that combines a touch panel and a display unit. The touch panel display unit 30 displays an image on a panel surface (display surface: an example of a surface to be operated), and allows operation by contacting the panel surface with an operating object such as a pen-type input device or a finger.

[0014] The pen-type input device 200 is a pen-type input device that is used by a user to perform operations on the touch panel of the touch panel display unit 30 of the information processing device 100. The user holds the pen-type input device 200 and moves the pen tip so that it comes into contact with the panel surface of the touch panel display unit 30, thereby performing handwritten input operations for characters, pictures, figures, etc.

[0015] Note that, as an operation using the pen-type input device 200, a pointing operation on a user interface image displayed on the touch panel display unit 30 may also be possible.

[0016] The detection method of the pen-type input device 200 by the touch panel in the touch panel display unit 30 of this embodiment is not particularly limited, but examples thereof include a capacitance method and an electromagnetic induction method. In the following description, a case where the capacitance method is used will be taken as an example.

[0017] An application compatible with input operations using the pen-type input device 200 (pen operation compatible application) is installed in the information processing device 100. The pen operation compatible application may be capable of, for example, inputting characters by touching the tip of the pen-type input device 200 to the touch panel display unit 30, displaying characters or pictures drawn in response to handwriting input operations such as drawing, on the touch panel display unit 30, or performing processes such as converting characters or pictures drawn by the operation into data.

[0018] Furthermore, the pen operation compatible application of this embodiment is capable of setting the type of writing implement when performing a writing operation using the pen type input device 200. Specifically, examples of the writing implement type include pencils, ballpoint pens, crayons, felt-tip pens, and writing brushes. In response to a writing operation using the pen type input device 200, the pen operation compatible application performs drawing using handwriting corresponding to the set type of writing implement.

[0019] Furthermore, in the information processing system of this embodiment, the pen-type input device 200 emits vibrations and sounds (writing sounds) in response to an operation (writing operation) corresponding to writing. The vibrations and writing sounds emitted by the pen-type input device 200 allow the user to feel a tactile sensation similar to that felt when writing on a writing medium such as paper, and thus the user can obtain a sensation similar to that felt when writing on a writing medium with an actual writing implement.

[0020] In the following description, the generation of vibrations and sounds when a writing operation is performed with the pen-type input device 200 is also referred to as "tactile feedback." Furthermore, the control executed in response to the "tactile feedback" is also referred to as "feedback control."

[0021] Next, an outline of the feedback control in this embodiment will be described. Line L1 in FIG. 2(A) shows an example of an algorithm for feedback control in which the feedback level (vibration level, writing sound level) is changed linearly with an increase in the movement speed, which is one of the contact parameters. Generally, in actual writing, the vibration and writing sound generated by the writing tool increase as the moving speed of the writing tool on the writing medium increases. Therefore, by performing feedback control using the algorithm of Fig. 2(A), the vibration and writing sound of the pen-type input device 200 moving on the panel surface due to the writing operation change according to the moving speed, and a tactile sensation that is somewhat close to reality can be reproduced.

[0022] The pen operation compatible application of this embodiment allows the type of writing implement to be set as described above. In actual writing, the vibration of the writing implement and the change in the volume of the writing sound caused by changes in the moving speed differ depending on the type of writing implement used.

[0023] Specifically, line L11 in Figure 2(B) shows the actual writing response level for a certain writing implement type (first type). That is, line L11 shows the writing response level (the volume of vibrations and writing sounds generated by writing) relative to the movement speed of the first type of writing implement. In addition, line L1 is also shown in the figure for comparison with line L11. The first type of writing instrument in the figure produces a writing response level greater than line L1 when the tip begins to move, but once the movement speed reaches a certain level or higher, a writing response level that is linear with the movement speed, similar to line L1, is obtained.

[0024] 2(C) shows the relationship between the movement speed and the writing response level for a certain writing implement type (second type) in reality. In the same figure, line L1 is also shown for comparison with line L12. According to the same figure, the second type of writing instrument produces a writing response level greater than line L1 up to a certain range of movement speed after the tip starts moving, but the increase in writing response level becomes more gradual as the movement speed increases thereafter.

[0025] Furthermore, line L13 in Figure 2(D) shows the relationship between the movement speed and the writing response level for a certain writing implement type (third type) in reality. Also shown in the figure is line L1 for comparison with line L13. This relationship between the movement speed and the writing response level may be obtained by measurement, or may be based on previous knowledge. According to the same figure, for the third type of writing instrument, the change in writing response level relative to the movement speed is linear for a short period of time immediately after the tip begins to move, but as the movement speed increases further, the writing response level rises sharply due to the friction of the tip against the writing medium, and as the movement speed increases further, the writing response level drops sharply and then remains approximately constant at a certain writing response level regardless of the movement speed.

[0026] The relationship between the movement speed and the writing response level shown by the lines L11, L12, L13, etc. may be obtained by measurement or based on previous knowledge. When obtained by measurement, for example, a method may be used in which a writing implement is actually moved over a writing medium so that the movement speed increases from a state where the movement speed is zero, and the level of vibrations generated by the writing implement and the level of writing sound are measured during the movement.

[0027] As can be seen from Figures 2(B), 2(C), and 2(D), the writing response level produced by an actual writing instrument may deviate from that shown in Figure 2(A). Therefore, the information processing system of this embodiment is configured to perform feedback control so as to cancel such deviations. By performing feedback control in this manner, tactile feedback according to the set writing instrument type is provided when a writing operation is performed with the pen-type input device 200. As a result, it is possible to achieve a tactile sensation similar to that experienced when actually writing with a writing instrument of the same writing instrument type as that set in the pen operation-compatible application.

[0028] An example of the hardware configuration of the information processing device 100 and the pen-type input device 200 will be described with reference to FIG. First, we will explain an example of the hardware configuration of an information processing device 100. The information processing device 100 in the figure includes a processor 11, a main memory 12, a flash memory 13, a peripheral device 14, a short-range communication unit 15, a baseband chip 21, a communication unit 22, an audio system 23, a microphone 24, a speaker 25, and a touch panel display unit 30.

[0029] The processor 11 is, for example, an application processor including a CPU (Central Processing Unit). The processor 11 controls the information processing device 100 as a whole.

[0030] Main memory 12 is a writable memory used as a read area for the executable program of processor 11 or as a work area for writing processing data for the executable program. Main memory 12 is composed of, for example, multiple DRAM (Dynamic Random Access Memory) chips. This executable program includes an OS (Operating System), various device drivers for operating peripheral hardware, various services / utilities, application programs (application software), etc.

[0031] The flash memory 13 is, for example, a flash EEPROM (Electrically Erasable Programmable Read Only Memory), and stores an OS, various drivers, various services / utilities, application programs (hereinafter sometimes referred to as applications), and various data.

[0032] The touch panel display unit 30 is a part that displays an image and allows an operation to be performed on the panel surface on which the image is displayed using the pen-type input device 200. The touch panel display unit 30 may also be operable using an operating object other than the pen-type input device 200, such as a finger, but here, an example will be described in which the operating object is the pen-type input device 200. The touch panel display unit 30 includes a display unit 31 and a panel sensor 32 . The display unit 31 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays an image based on drawing data (display data) output from the processor 11. The panel sensor 32 detects the contact state of the pen tip of the pen-type input device 200 with the panel surface of the display unit 31. Specifically, the panel sensor 32 may be capable of detecting the position on the panel surface where the pen tip of the pen-type input device 200 is in contact, the pressure of the pen tip in contact with the panel surface, the distance from the panel surface of the pen-type input device 200, etc. The detection method of such a panel sensor 32 is not particularly limited, but examples thereof include a capacitance method and an electromagnetic induction method. Furthermore, the panel sensor 32 may be configured to use a combination of such multiple detection methods.

[0033] The peripheral device 14 is, for example, a wireless local area network (WLAN) module, a global positioning system (GPS) module, and sensors such as an acceleration sensor.

[0034] The audio system 23 is, for example, an audio IC (Integrated Circuit), and inputs, records, plays back, and outputs sound data. For example, a microphone 24 and a speaker 25 are connected to the audio system 23. For example, the audio system 23 outputs sound data picked up by the microphone 24 to the processor 11 or the baseband chip 21. Furthermore, the audio system 23 converts sound data acquired from the processor 11 or the baseband chip 21 into a sound signal and outputs it to the speaker 25, for example.

[0035] The microphone 24 picks up sounds around the information processing device 100. For example, the microphone 24 picks up sounds such as the user's voice when audio integration with another terminal is performed. The speaker 25 outputs various sounds to the outside of the information processing device 100. For example, when audio integration with another terminal is performed, the speaker 25 outputs (emits) sounds received from the other terminal.

[0036] The baseband chip 21 is a dedicated IC that controls wireless communications such as 4G (fourth generation mobile communication system) and 5G (fourth generation mobile communication system). The baseband chip 21, for example, causes the audio system 23 to output sound data received by the communication unit 22 from the speaker 25 as sound. Furthermore, the baseband chip 21 acquires, for example, sound data collected by a microphone 24 via an audio system 23, and outputs the sound data via a mobile communication system using a communication unit 22. The baseband chip 21 may also be configured to exchange input and output data for data communication via the mobile communication system with the processor 11.

[0037] The communication unit 22 is a wireless communication device including an antenna for performing wireless communication via a mobile communication system.

[0038] The short-distance communication unit 15 is, for example, a Bluetooth (registered trademark) module, and performs short-distance wireless communication with the pen-type input device 200.

[0039] Next, with reference to the same FIG. 3, an example of the hardware configuration of the pen-type input device 200 will be described. The pen-type input device 200 includes an MCU 41, a short-range communication unit 42, a vibration unit 43 (an example of a tactile reproduction unit), a sound output unit 44 (an example of a tactile reproduction unit), and a flash memory 45.

[0040] The short-range communication unit 42 is, for example, a Bluetooth (registered trademark) module, and performs short-range wireless communication with the information processing device 100.

[0041] The MCU (Micro Controller Unit) 41 includes a CPU, memories such as ROM and RAM, and I / O related components, and executes control of the pen-type input device 200. The MCU 41 exchanges information via a short-range communication unit 42. The MCU 41 also controls the generation of vibrations by a vibration unit 43 and the output of writing sounds by a sound output unit 44 so that haptic feedback is performed based on feedback control information transmitted from the information processing device 100.

[0042] The vibration unit 43 is a part that includes, for example, an actuator and vibrates in response to the control of the MCU 41. The sound output unit 44 includes, for example, a speaker, and outputs writing sounds under the control of the MCU 41.

[0043] The flash memory 45 stores various data corresponding to the pen-type input device 200, such as programs.

[0044] An example of the functional configuration of the information processing device 100 and the pen-type input device 200 will be described with reference to Fig. 4. In the figure, functional units that can be considered equivalent to the parts in the hardware configuration of Fig. 3 are assigned the same reference numerals as in Fig. 3, and descriptions thereof will be omitted as appropriate. The functions of the information processing device 100 shown in the figure may be realized by the processor 11 (FIG. 3) executing a program.

[0045] First, a description will be given of an example of the functional configuration of the information processing device 100. The information processing device 100 includes a communication unit 22, a touch panel display unit 30, a touch parameter detection unit 101, a control unit 102, and a storage unit 103 as functional units.

[0046] The contact parameter detection unit 101 detects the contact state of the pen tip of the pen-type input device 200 with the panel surface of the touch panel display unit 30 . The contact parameter detection unit 101 outputs parameters (contact parameters) detected in accordance with the contact state based on the detection signal output by the panel sensor 32. In the following description, the contact parameters will be exemplified by the aforementioned movement speed, writing pressure, movement direction, and tilt angle.

[0047] The control unit 102 executes various controls in the information processing device 100. The control unit 102 includes an application support processing unit 121 and a feedback control unit 122.

[0048] The application processing unit 121 executes processing corresponding to a pen operation compatible application. When a writing operation is performed with the pen-type input device 200 as an operation for a pen operation compatible application, pen operation information is input from the panel sensor 32 to the application compatible processing unit 121. In response to the input of the pen operation information, the application compatible processing unit 121 performs processing such as drawing.

[0049] As described above, the application of this embodiment is capable of setting the type of writing implement corresponding to the writing operation. The setting of the writing implement type may be performed, for example, by the user specifying the type of writing implement, or may be set by the application-compatible processing unit 121 in response to the selection of the writing medium.

[0050] The feedback control unit 122 executes feedback control, which is a control for causing the pen-type input device 200 to generate vibrations and writing sounds as tactile feedback.

[0051] The feedback control unit 122 of this embodiment executes feedback control in real time based on the contact parameters detected by the contact parameter detection unit 101 . Furthermore, the feedback control unit 122 performs feedback control according to the set writing instrument type. That is, the feedback control unit 122 generates feedback control information including a set feedback level (level of vibration and writing sound) by referring to a feedback table (an example of table information) according to the set writing instrument type, and signal source data (vibration waveform data, writing sound data) according to the set writing instrument type. The feedback control unit 122 outputs (transmits) the generated feedback control information to the pen-type input device 200 via the communication unit 22.

[0052] The storage unit 103 stores various types of information corresponding to the information processing device 100. The storage unit 103 includes a writing setting data storage unit 131, a feedback table storage unit 132, a vibration waveform data storage unit 133, and a writing sound data storage unit .

[0053] The writing setting data storage unit 131 stores writing setting data. The writing setting data is setting information including information (writing instrument type data) indicating the type of writing instrument set by a pen operation compatible application in response to a writing operation using the pen-type input device 200. Note that the writing setting data may also include information (writing medium data) indicating the set writing medium.

[0054] The feedback table storage unit 132 stores a feedback table, which indicates the correspondence between the movement speed and the feedback level for each type of writing implement.

[0055] 5 shows an example of a feedback table stored in the feedback table storage unit 132. In this figure, the feedback table storage unit 132 stores two feedback tables corresponding to two feedback types, vibration and writing sound, for each of the three writing instrument types, first to third. In this embodiment, the number of writing instrument types that can be set is not particularly limited, but here, an example will be described in which any one selected from the three writing instrument types, first to third, can be set. It is also possible to store one feedback table that is used in common for vibration and writing sound, corresponding to one writing implement type.

[0056] Specifically, the feedback table 1-A corresponding to the vibration of the first type of writing instrument may be a table created to associate the movement speed with the vibration-corresponding feedback level based on the relationship between the movement speed and the vibration writing response level illustrated as line L11 in Figure 2(B). Furthermore, the feedback table 1-B corresponding to the writing sound of the first type of writing instrument may be a table created to associate a feedback level with a movement speed, for example, based on the relationship between a pre-measured movement speed and a writing response level as a writing sound.

[0057] Furthermore, the feedback table 2-A corresponding to the vibration of the second type of writing instrument may be a table created to associate the vibration-corresponding feedback level with the movement speed based on the relationship between the movement speed and the vibration writing response level illustrated as line L12 in Figure 2(C). Furthermore, the feedback table 2-B corresponding to the writing sound of the second type of writing instrument may be a table created to associate a feedback level with the movement speed, for example, based on the relationship between a pre-measured movement speed and the writing response level as the writing sound.

[0058] Furthermore, the feedback table 3-A corresponding to the vibration of the third type of writing instrument may be a table created to associate the vibration-corresponding feedback level with the movement speed based on the relationship between the movement speed and the vibration writing response level illustrated as line L13 in Figure 2(D). Furthermore, the feedback table 3-B corresponding to the writing sound of the third type of writing instrument may be a table created to associate a feedback level with the movement speed, for example, based on the relationship between a pre-measured movement speed and the writing response level as the writing sound.

[0059] Returning to the explanation of Fig. 4, the vibration waveform data storage unit 133 stores the vibration waveform data. 6 shows an example of vibration waveform data stored in the vibration waveform data storage unit 133. In the figure, an example is shown in which the vibration waveform data storage unit 133 stores vibration waveform data in association with each of three writing implement types, namely, first type to third type. The vibration waveform data corresponding to one writing implement type is waveform data created so as to reproduce the vibrations that occur when actually writing with a writing implement of the corresponding writing implement type.

[0060] Returning to the explanation of Fig. 4, the writing sound data storage unit 134 stores writing sound data. 7 shows an example of writing sound data stored in the writing sound data storage unit 134. In the figure, an example is shown in which the writing sound data storage unit 134 stores writing sound data in association with three writing implement types, namely, first type to third type. The writing sound data corresponding to one writing implement type may be audio data created to reproduce the writing sound that is produced when actually writing with a writing implement of the corresponding writing implement type.

[0061] Next, an example of the functional configuration of the pen-type input device 200 will be described with reference to Fig. 4. The functions of the pen-type input device 200 shown in the figure may be realized by the MCU 41 (Fig. 5) executing a program. The pen-type input device 200 includes a short-range communication unit 42, a vibration unit 43, a sound output unit 44, and a control unit 201 as functional units.

[0062] The control unit 201 executes various controls in the pen-type input device 200. The control unit 201 includes a drive control unit 211. The drive control unit 211 controls the vibration unit 43 to generate vibrations as haptic feedback, and also controls the sound output unit 44 to generate writing sounds as haptic feedback. The drive control unit 211 uses the vibration waveform data, writing sound data, and feedback level included in the feedback control information transmitted from the information processing device 100 to control the vibration unit 43 and the sound output unit 44 .

[0063] An example of a functional configuration related to driving of the vibration unit 43 and the sound output unit 44 based on feedback control information will be described with reference to FIG. In the information processing device 100, the contact parameter detection unit 101 includes a movement speed detection unit 111, a writing pressure detection unit 112, a movement direction detection unit 113, and an inclination angle detection unit 114.

[0064] The movement speed detection unit 111 detects the movement speed of the pen tip that moves while touching the panel surface. The movement speed detection unit 111 may detect the movement speed based on the amount of movement per unit time of the contact position of the pen tip detected by the panel sensor 32. The movement speed detection unit 111 outputs the detected movement speed parameter Dt1 to the feedback control unit 122.

[0065] The writing pressure detection unit 112 detects the pressure (writing pressure) of the pen tip in contact with the panel surface. The writing pressure detection unit 112 may detect the writing pressure based on the capacitance value at the contact position of the pen tip detected by the panel sensor 32. The writing pressure detection unit 112 outputs a writing pressure parameter Dt2 indicating the detected writing pressure to the feedback control unit 122.

[0066] The movement direction detection unit 113 detects the movement direction of the pen tip that moves while in contact with the panel surface. The movement speed detection unit 111 may detect the movement direction based on the movement direction of the contact position of the pen tip detected by the panel sensor 32. The movement direction detection unit 113 outputs a movement direction parameter Dt3 indicating the detected movement direction to the feedback control unit 122.

[0067] The tilt angle detection unit 114 detects the tilt angle of the main body of the pen-type input device 200 relative to the panel surface when the pen tip is in contact with the panel surface. The tilt angle detection unit 114 may detect the tilt angle based on the distribution of capacitance corresponding to the main body of the pen-type input device 200 detected around the contact position of the pen tip by the panel sensor 32. The tilt angle detection unit 114 outputs the detected tilt angle parameter Dt4 to the feedback control unit 122.

[0068] In the information processing device 100, the feedback control unit 122 generates feedback control information FB. The feedback control unit 122 recognizes the writing instrument type indicated by the writing setting data stored in the writing setting data storage unit 131. The feedback control unit 122 derives (acquires) a feedback level (vibration level and writing sound level) corresponding to the movement speed indicated by the movement speed parameter Dt1 by referring to the feedback table stored in the feedback table storage unit 132 and associated with the recognized writing instrument type. Then, the feedback control unit 122 may correct the feedback level corresponding to the movement speed by using all or some of the other contact parameters (writing pressure parameter Dt2, movement direction parameter Dt3, and tilt angle parameter Dt4). The feedback level may be common to both the vibration level and the writing sound level. Furthermore, the feedback control unit 122 acquires vibration waveform data associated with the recognized writing implement type from the vibration waveform data storage unit 133 . Furthermore, the feedback control unit 122 acquires writing sound data associated with the recognized writing implement type from the writing sound data storage unit 134 . The feedback control unit 122 generates feedback control information FB including the feedback levels (vibration levels and writing sound levels) obtained as described above, vibration waveform data, and writing sound data. The feedback control unit 122 outputs (transmits) the generated feedback control information FB to the pen-type input device 200.

[0069] Next, a description will be given of an example of the functional configuration of the pen-type input device 200. The functions of the pen-type input device 200 shown in the figure may be realized by the MCU 41 (FIG. 5) executing a program. The pen-type input device 200 includes a short-range communication unit 42, a vibration unit 43, a sound output unit 44, a control unit 201, and a storage unit 203 as functional units.

[0070] The control unit 201 executes various controls in the pen-type input device 200. The control unit 201 includes a drive control unit 211. The drive control unit 211 drives the vibration unit 43 and the sound output unit 44 based on the input feedback control information FB.

[0071] In the figure, an example of the configuration of a vibration unit 43 and a sound output unit 44 is shown. The vibration unit 43 includes an amplifier 431 and an actuator 432 . The drive control unit 211 inputs the vibration waveform data included as signal source data in the received feedback control information FB to the amplifier 431 as the signal source S1. The drive control unit 211 also instructs the amplifier 431 on the vibration level included as a feedback level in the received feedback control information FB. The amplifier 431 amplifies the input signal source S1 to the instructed vibration level and outputs it to the actuator 432. The actuator 432 vibrates according to the waveform of the input signal source S1 and the instructed vibration level.

[0072] The sound output unit 44 includes, for example, an amplifier 441 and a speaker 442 . The drive control unit 211 inputs the writing sound data included as signal source data in the received feedback control information FB to the amplifier 441 as the signal source S2. The drive control unit 211 also instructs the amplifier 441 on the writing sound level included as a feedback level in the received feedback control information FB. The amplifier 441 amplifies the input signal source S2 to the instructed writing sound level and outputs it to the speaker 442. The input signal source S2 is output from the speaker 442 as the writing sound at the level instructed by the drive control unit 211.

[0073] An example of a processing procedure executed by the information processing device 100 and the pen-type input device 200 in relation to haptic feedback will be described with reference to the flowchart of FIG. First, an example of a processing procedure of the information processing device 100 will be described. When the processing in the figure is being executed, the feedback control unit 122 is configured to capture, for example, at each predetermined capture cycle (capture period), the contact parameters (movement speed parameter Dt1, writing pressure parameter Dt2, movement direction parameter Dt3, and tilt angle parameter Dt4) detected by the contact parameter detection unit 101. At this time, the feedback control unit 122 may capture one sample of contact parameters at each capture cycle (period), or may capture contact parameters corresponding to each of a plurality of sample cycles.

[0074] First, an example of a processing procedure executed by the information processing device 100 will be described. Step S100: In the information processing device 100, the feedback control unit 122 waits for the pen tip of the pen-type input device 200 to change from a state in which it is in contact with the panel surface and is stopped to a state in which it starts moving (movement start state) based on the contact parameters acquired at the current sample timing. In this case, the feedback control unit 122 may determine that the pen tip has started moving when, for example, the movement speed parameter Dt1 in the contact parameters has changed from zero to a value greater than zero. Alternatively, the feedback control unit 122 may determine that the pen tip has started moving when the movement direction parameter Dt3 has changed from a value that does not indicate a movement direction to a value that indicates a certain movement direction, due to the pen tip not moving (stopping). Furthermore, the feedback control unit 122 may determine whether the pen tip has started moving by using the movement speed parameter Dt1 and the movement direction parameter Dt3 in combination.

[0075] Step S102: When the feedback control unit 122 determines in step S100 that the pen tip has started to move, it refers to the writing setting data stored in the writing setting data storage unit 131 and determines the type of writing implement currently set by the pen operation compatible application.

[0076] Step S104: The feedback control unit 122 sets the feedback table associated with the set writing implement type determined in step S102 from among the feedback tables stored in the feedback table storage unit 132 as the reference object corresponding to the current movement of the pen tip.

[0077] Step S106: The feedback control unit 122 derives a feedback level based on the contact parameters captured in the capture cycle corresponding to the current time point. Here, the feedback control unit 122 may derive two feedback levels corresponding to the feedback types of vibration and writing sound. Specifically, for example, the feedback control unit 122 acquires feedback levels corresponding to the vibration and the writing sound associated with the movement speed parameter Dt1 in the currently acquired contact parameters from the feedback table of vibration and writing sound set as the reference target in step S104. The feedback levels acquired in this manner may be used as the derived feedback levels. Furthermore, the feedback control unit 122 may correct the feedback level obtained from the reference feedback table using all or some of the contact parameters other than the movement speed (pen pressure parameter Dt2, movement direction parameter Dt3, and tilt angle parameter Dt4). The corrected feedback level reflects changes in vibration and writing sound according to, for example, the pen pressure, movement direction, and tilt angle of the current writing operation.

[0078] Step S108: Furthermore, the feedback control unit 122 acquires signal source data. Specifically, the feedback control unit 122 acquires, as signal source data, vibration waveform data associated with the writing implement type determined in step S102 from the vibration waveform data storage unit 133. The feedback control unit 122 also acquires, as signal source data, writing sound data associated with the writing implement type determined in step S102 from the writing sound data storage unit 134.

[0079] Step S110: The feedback control unit 122 generates feedback control information FB including feedback levels corresponding to the vibration and writing sound derived in step S106 and the signal source data (vibration waveform data, writing sound data) acquired in step S108. The feedback control unit 122 transmits the generated feedback control information FB to the pen-type input device 200.

[0080] Step S112: The feedback control unit 122 determines whether the movement of the pen tip that started in response to step S100 has stopped. The feedback control unit 122 may determine that the movement of the pen tip has stopped in response to the movement speed parameter Dt1, which had been greater than 0 until then, becoming zero. If it is determined that the pen tip is in a moving state (moving state), that is, that the movement has not stopped, the process returns to step S106. In this case, the feedback control unit 122 thereafter repeatedly executes the processes of steps S106 to S110 until the movement of the pen tip is stopped. In other words, the feedback control unit 122 executes a loop of generating feedback control information FB based on the contact parameters at the current time point and transmitting the generated feedback control information FB to the pen-type input device 200. On the other hand, if it is determined that the movement of the pen tip has stopped, the process returns to step S100. By returning the process to step S100 in this way, the transmission of the feedback control information FB to the pen-type input device 200 is stopped until the movement of the pen tip starts again.

[0081] Next, an example of a processing procedure executed by the pen-type input device 200 will be described. Step S200: In the pen-type input device 200, the drive control unit 211 waits to receive the feedback control information FB transmitted from the information processing device 100 in step S110.

[0082] Step S202: When feedback control information FB is received, drive control unit 211 inputs the vibration waveform data included as signal source data in the received feedback control information FB as signal source S1 to amplifier 431. Also, drive control unit 211 inputs the vibration waveform data included as signal source data in the received feedback control information FB as signal source S2 to amplifier 431.

[0083] Step S204: Furthermore, the feedback control information FB sets two feedback levels corresponding to the vibration and the writing sound included in the received feedback control information FB in the amplifiers 431 and 441, respectively.

[0084] A modification of this embodiment will now be described. In the above embodiment, a feedback table that associates feedback levels with movement speeds is used to derive the feedback levels. However, feedback tables corresponding to contact parameters other than movement speed, such as pen pressure, movement direction, and tilt angle, may be provided. In addition, the feedback control unit 122 may derive the feedback levels by referring to one or more feedback tables corresponding to any one or more contact parameters among movement speed, pen pressure, movement direction, and tilt angle.

[0085] Furthermore, instead of referring to the feedback table, the feedback control unit 122 may derive the feedback level by calculation using predetermined contact parameters such as the moving speed.

[0086] The haptic feedback may be provided by either vibration or sound. In this case, the information processing device 100 and the pen-type input device 200 may have a configuration corresponding to one of the vibration and sound haptic feedback, but not the other. Alternatively, whether vibration or sound is used for the haptic feedback may be set according to, for example, a selection operation by the user or the type of writing implement or medium set in a pen operation-compatible application.

[0087] The predetermined functional units of the information processing device 100 in the above embodiment may be provided in the pen type input device 200. Alternatively, the predetermined functional units of the pen type input device 200 in the above embodiment may be provided in the information processing device 100. As a specific example, the feedback control unit 122 may be provided in the pen-type input device 200. In this case, the information processing device 100 may be configured to transmit the contact parameters detected by the contact parameter detection unit 101 to the pen-type input device 200. Also, for example, at least one of the vibration waveform data storage unit 133 and the writing sound data storage unit 134 may be stored in the pen-type input device 200. In this case, the feedback control information FB transmitted from the information processing device 100 may include information specifying the type of writing implement corresponding to the vibration waveform data and writing sound data used to output vibration and writing sound in the pen-type input device 200, and feedback levels corresponding to each of the vibration and writing sound.

[0088] Note that a program for implementing the functions of the information processing device 100, pen-type input device 200, etc. described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform processing as the information processing device 100, pen-type input device 200, etc. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program on a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including communication lines such as the Internet, WAN, LAN, and dedicated lines. The term "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, a recording medium storing a program may be a non-transitory recording medium such as a CD-ROM. The recording medium may also include internal or external recording media accessible from a distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code executable on a terminal device. In other words, the format in which the program is stored on the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that is executable on the terminal device. The program may be divided into multiple parts, each of which may be downloaded at different times and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a storage medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be one that realizes part of the functions described above.Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program). [Explanation of symbols]

[0089] 30 touch panel display unit, 31 display unit, 32 panel sensor, 43 vibration unit, 44 sound output unit, 100 information processing device, 101 contact parameter detection unit, 102 control unit, 103 memory unit, 111 movement speed detection unit, 112 writing pressure detection unit, 113 movement direction detection unit, 114 angle detection unit, 121 application corresponding processing unit, 122 feedback control unit, 131 writing setting data storage unit, 132 feedback table storage unit, 133 vibration waveform data storage unit, 134 writing sound data storage unit, 200 pen type input device, 201 control unit, 203 memory unit, 211 drive control unit

Claims

1. a contact parameter detection unit that detects one or more contact parameters corresponding to a contact state of the pen-type input device with respect to the operation surface; a feedback control unit that derives a feedback level corresponding to the writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device using the derived feedback level; An information processing system comprising:

2. The feedback control unit derives a feedback level by acquiring a feedback level corresponding to the detected contact parameter from table information indicating a correspondence relationship between a contact parameter value and a feedback level corresponding to the set writing implement type. The information processing system according to claim 1 .

3. The contact parameters are all or some of the following: a speed at which the pen tip of the pen-type input device moves on the operation surface; a pressure of the pen tip on the operation surface; a direction in which the pen tip moves on the operation surface; and a tilt angle of the pen-type input device with respect to the operation surface.

3. The information processing system according to claim 1.

4. The tactile reproduction unit includes a vibration unit that generates vibrations in response to an input from a signal source.

3. The information processing system according to claim 1.

5. The haptic reproduction unit includes a sound output unit that outputs sound in response to an input from a signal source.

3. The information processing system according to claim 1.

6. An information processing method in an information processing system, comprising: a contact parameter detection step in which a contact parameter detection unit detects one or more contact parameters corresponding to a contact state of the pen-type input device with respect to an operation target surface; a feedback control step in which a feedback control unit derives a feedback level corresponding to the writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device using the derived feedback level; An information processing method comprising:

7. The computer in the information processing system a contact parameter detection unit that detects one or more contact parameters corresponding to a contact state of the pen-type input device with respect to the operation surface; a feedback control unit that derives a feedback level corresponding to the writing implement type set for the pen-type input device based on the detected contact parameters, and drives a tactile reproduction unit in the pen-type input device using the derived feedback level; A program to function as a

Images