Information processing system, information processing method, and program

Abstract

The present invention configures an information processing system that can approach a feeling when writing on a writing medium in reality. The information processing system includes: a contact parameter detection unit that detects one or more contact parameters corresponding to a contact state of the pen-shaped input device with respect to an operation target surface; and a feedback control unit that derives, on the basis of the detected contact parameter, a feedback level corresponding to a type of writing tool set corresponding to the pen shape input device, and drives a tactile sense reproduction unit in the pen shape input device by means of the derived feedback level.

Description

TECHNICAL FIELD

[0001] The present application relates to an information processing system, an information processing method, and a program. BACKGROUND

[0002] It is known that, at the time of input to an interactive display system, a stylus (pen-shaped input device) capable of performing haptic feedback by generating vibration that drives a haptic actuator is used as an input device (for example, refer to Patent Literature 1).

[0003] Patent Literature 1: Japanese Patent Application Laid-Open No. 2017-537395

[0004] By causing the pen-shaped input device to vibrate, a feeling such as when a pen nib is slid on a writing medium such as paper is approximately obtained, and thus a user who performs input using the pen-shaped input device can obtain a feeling close to when writing on a writing medium.

[0005] At the time of input using the pen-shaped input device, there is an aspect in which it is preferable to approximate a feeling when writing on a real writing medium as much as possible. SUMMARY

[0006] Therefore, the present application considers the above-described problem, and aims to approximate a feeling when writing on a writing medium in reality at the time of input using a pen-shaped input device.

[0007] One aspect of the present application that solves the above-described problem is an information processing system including: a contact parameter detection unit that detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object; and a feedback control unit that derives a feedback level corresponding to a writing tool category based on the detected contact parameters, and drives a haptic reproduction unit in the pen-shaped input device by the derived feedback level, the writing tool category being set in correspondence with the pen-shaped input device.

[0008] One aspect of the present application is an information processing method, which is an information processing method in an information processing system, including: a contact parameter detection step in which a contact parameter detection unit detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object; and a feedback control step in which a feedback control unit derives a feedback level corresponding to a writing tool category based on the detected contact parameters, and drives a haptic reproduction unit in the pen-shaped input device by the derived feedback level, the writing tool category being set in correspondence with the pen-shaped input device.

[0009] One embodiment of the present application is a program for causing a computer in an information processing system to function as a contact parameter detection section that detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object, and a feedback control section that derives a feedback level corresponding to a writing tool type based on the detected contact parameters, and drives a haptic reproduction section in the pen-shaped input device by the derived feedback level, the writing tool type being set in correspondence with the pen-shaped input device.

[0010] According to the present application, an effect of being able to approach a feeling at the time of writing on a writing medium in reality at the time of input using a pen-shaped input device is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a diagram showing an example of an external structure of an information processing system in the present embodiment.

[0012] Figure 2 is a diagram showing a writing reaction level characteristic in the present embodiment for each writing tool type.

[0013] Figure 3 is a diagram showing an example of a hardware structure of an information processing apparatus and a pen-shaped input device in the present embodiment.

[0014] Figure 4 is a diagram showing an example of a functional structure of an information processing apparatus and a pen-shaped input device in the present embodiment.

[0015] Figure 5 is a diagram showing an example of a feedback table in the present embodiment.

[0016] Figure 6 is a diagram showing an example of vibration waveform data stored in a vibration waveform data storage section in the present embodiment.

[0017] Figure 7 is a diagram showing an example of writing sound data stored in a writing sound data storage section in the present embodiment.

[0018] Figure 8 is a diagram showing an example of a functional structure related to driving of a vibration section and a sound output section in the information processing system of the present embodiment.

[0019] Figure 9 is a diagram showing an example of processing steps performed by an information processing apparatus and a pen-shaped input device in relation to haptic feedback in the present embodiment.

[0020] BRIEF DESCRIPTION OF DRAWINGS

[0021] 30 … touch panel display section; 31 … display section; 32 … panel sensor; 43 … vibration section; 44 … sound output section; 100 … information processing apparatus; 101 … contact parameter detection section; 102 … control section; 103 … storage section; 111 … movement speed detection section; 112 … pen pressure detection section; 113 … movement direction detection section; 114 … angle detection section; 121 … application corresponding processing section; 122 … feedback control section; 131 … writing setting data storage section; 132 … feedback table storage section; 133 … vibration waveform data storage section; 134 … writing sound data storage section; 200 … pen-shaped input device; 201 … control section; 203 … storage section; 211 … drive control section. DETAILED DESCRIPTION

[0022] Figure 1 An example of an external appearance structure of an information processing system according to the present embodiment is shown. As shown in the figure, the information processing system according to the present embodiment has an information processing apparatus 100 and a pen-shaped input device 200.

[0023] The information processing apparatus 100 is capable of performing information processing in accordance with an input operation of the pen-shaped input device 200. The information processing apparatus 100 of the figure shows an example of a tablet terminal or a notebook computer or the like.

[0024] The information processing apparatus 100 is provided with a touch panel display section 30. The touch panel display section 30 is a section in which a touch panel and a display section are combined. The touch panel display section 30 displays an image on a panel face (display face: an example of an operation target face), and is capable of being operated by bringing a pen-shaped input device, a finger or the like into contact with the panel face.

[0025] The pen-shaped input device 200 is a pen-shaped input device used by a user to operate a touch panel in the touch panel display section 30 of the information processing apparatus 100.

[0026] The user holds the pen-shaped input device 200, brings the nib into contact with the panel face of the touch panel display section 30, and moves it, thereby performing an operation of handwriting inputting characters, drawings, figures or the like.

[0027] Further, as an operation using the pen-shaped input device 200, there can be a pointing operation or the like on a user interface image displayed in the touch panel display section 30.

[0028] Further, the detection method of the pen-shaped input device 200 based on the touch panel in the touch panel display section 30 according to the present embodiment is not particularly limited, and for example, an electrostatic capacity method, an electromagnetic induction method or the like can be cited. In the following description, a case using the electrostatic capacity method is cited.

[0029] An application corresponding to an input operation of the pen-shaped input device 200 (a pen operation corresponding application) is installed in the information processing apparatus 100.

[0030] The pen operation corresponding application can display, for example, characters or drawings drawn by a handwriting input operation of inputting characters or drawing by bringing the pen nib of the pen-shaped input device 200 into contact with the touch panel display section 30, on the touch panel display section 30, and can execute processing of dataizing the characters or drawings drawn by the operation.

[0031] Further, the pen operation corresponding application of the present embodiment can set a writing tool kind when based on a writing operation of the pen-shaped input device 200. Specifically, as the writing tool kind, there can be cited a pencil, a ballpoint pen, a crayon, a fountain pen, a brush, and the like. The pen operation corresponding application draws a trace corresponding to the set writing tool kind based on the writing operation of the pen-shaped input device 200.

[0032] Further, in the information processing system of the present embodiment, the pen-shaped input device 200 emits vibration and sound (writing sound) in correspondence with a writing operation (writing operation) of performing an operation corresponding to writing. By the vibration and writing sound emitted by the pen-shaped input device 200, the user feels a tactile sensation close to that when writing on a writing medium such as paper, and can obtain a feeling close to that when writing on a writing medium by a writing tool of a real object.

[0033] In the following description, the generation of vibration and sound when performing a writing operation by the pen-shaped input device 200 is also referred to as "tactile feedback". Further, the control performed in correspondence with the "tactile feedback" is also referred to as "feedback control".

[0034] Next, an outline of the feedback control in the present embodiment is described.

[0035] As the algorithm in the feedback control, Figure 2 The line L1 in (A) shows an example of changing the feedback level (vibration level, writing sound level) linearly with respect to an increase in the moving speed, which is one of the contact parameters.

[0036] As a general case, in actual writing, the vibration and writing sound generated on the writing tool also become larger as the moving speed of the writing tool on the writing medium becomes faster. Therefore, if the feedback control is performed by the algorithm of (A), Figure 2 If the feedback control is performed by the algorithm of (A), the vibration and writing sound of the pen-shaped input device 200 moving on the panel surface by the writing operation change according to the moving speed, and a tactile sensation close to a certain degree of reality can be reproduced.

[0037] The pen operation corresponding application of the present embodiment can set the writing tool kind as described above.

[0038] In actual writing, the vibration of the writing tool and the change in the size of the writing sound due to the change in the moving speed differ depending on the type of the writing tool used for writing.

[0039] Specifically, Figure 2 The line L11 in (B) indicates the writing reaction level of the writing tool with respect to a certain writing tool type (first type) in reality. That is, the line L11 indicates the writing reaction level (vibration generated by writing, size of the writing sound) with respect to the moving speed of the first type of writing tool. In addition, in the figure, the line L1 is also shown as a comparison with the line L11.

[0040] The first type of writing tool in the figure generates a writing reaction level larger than the line L1 when the leading end portion starts to move, but obtains a writing reaction level that becomes linear with respect to the moving speed as with the line L1 in a state where the moving speed becomes a certain speed or more thereafter.

[0041] In addition, Figure 2 The line L12 in (C) indicates the relationship between the moving speed and the writing reaction level of the writing tool with respect to a certain writing tool type (second type) in reality. In the figure, the line L1 is also shown as a comparison with the line L12.

[0042] According to the figure, the second type of writing tool generates a writing reaction level larger than the line L1 from the range where the leading end portion starts to move to a certain moving speed, but changes to a slow increase in the writing reaction level with respect to the increase in the moving speed thereafter.

[0043] In addition, Figure 2 The line L13 in (D) indicates the relationship between the moving speed and the writing reaction level of the writing tool with respect to a certain writing tool type (third type) in reality. In addition, in the figure, the line L1 is also shown as a comparison with the line L13. Such a relationship between the moving speed and the writing reaction level can also be obtained by measurement, or can be based on the knowledge insight so far.

[0044] According to the figure, the third type of writing tool has a linear change in the writing reaction level with respect to the moving speed in a short period after the leading end portion starts to move, but if the moving speed further increases, the writing reaction level sharply rises in a standing manner due to the friction of the leading end portion with respect to the writing medium, and after a sharp decrease in the writing reaction level, becomes substantially constant regardless of the moving speed at a certain writing reaction level as the moving speed further increases.

[0045] Moreover, the relationship between the moving speed and the writing reaction level indicated by the lines L11, L12, L13, and the like described above can be obtained by measurement or can be based on insights thus far. In the case of being obtained by measurement, the following method can be employed: for example, in reality, the writing instrument is caused to move on a certain writing medium in a manner in which the moving speed is increased from a state in which the moving speed is zero, and the level of vibration and the level of writing sound generated on the writing instrument are measured during the movement.

[0046] As understood from Figure 2 (B) of Figure 2 (C) of Figure 2 (D) of Figure 3 (A) deviates. Therefore, in the information processing system of the present embodiment, feedback control is performed so as to eliminate the deviation as described above. In this way, by performing the feedback control, when the writing operation is performed by the pen-shaped input device 200, the haptic feedback corresponding to the writing instrument category set is provided. As a result, it is possible to approach the haptics when writing in reality by the writing instrument of the same writing instrument category as the writing instrument category set in correspondence with the pen operation.

[0047] With reference to Figure 3 , a hardware configuration example of the information processing apparatus 100 and the pen-shaped input device 200 will be described.

[0048] First, a hardware configuration example of the information processing apparatus 100 will be described. The information processing apparatus 100 of the drawing is provided with a processor 11, a main memory 12, a flash memory 13, a peripheral device 14, a close proximity communication section 15, a baseband chip 21, a communication section 22, an audio system 23, a microphone 24, a speaker 25, and a touch panel display section 30.

[0049] The processor 11 is, for example, an application processor including a CPU (Central Processing Unit). The processor 11 controls the entire information processing apparatus 100.

[0050] The main memory 12 is a writable memory utilized as a read-in area of an execution program of the processor 11, or as a job area in which processing data is written in writing an execution program. The main memory 12 is constituted by, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. The execution program includes an OS (Operating System), various device drivers for hardware operation of the peripheral device class, various services / utilities, an application program (application software), and the like.

[0051] The flash memory 13 is, for example, a flash EEPROM (Electrically Erasable Programmable Read Only Memory), and stores an OS, various drivers, various services / utilities, an application (hereinafter, sometimes referred to as an application), and various data.

[0052] The touch panel display section 30 is a section that displays an image and that enables operation of a panel surface on which the image is displayed by the pen-shaped input device 200. The touch panel display section 30 can also be operated by an operating body other than the pen-shaped input device 200, such as a finger, for example, and a case in which the operating body is the pen-shaped input device 200 will be described here.

[0053] The touch panel display section 30 is provided with a display section 31 and a panel sensor 32.

[0054] The display section 31 is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, and displays an image based on drawing data (display data) output from the processor 11.

[0055] The panel sensor 32 detects a contact state of a pen tip of the pen-shaped input device 200 with respect to a panel surface of the display section 31. Specifically, the panel sensor 32 can be able to detect a position at which the pen tip of the pen-shaped input device 200 contacts on the panel surface, a pressure based on the pen tip contacting the panel surface, a distance of the pen-shaped input device 200 from the panel surface, and the like. As a detection method of such a panel sensor 32, there is no particular limitation, and an electrostatic capacity type or an electromagnetic induction type or the like can be cited. In addition, the panel sensor 32 can be configured to use a plurality of detection methods in combination.

[0056] The peripheral device 14 is, for example, a WLAN (Wireless Local Area Network) module, a GPS (Global Positioning System) module, and a sensor such as an acceleration sensor or the like.

[0057] The audio system 23 is, for example, an audio IC (Integrated Circuit), and performs input, recording, reproduction, and output of sound data. The audio system 23 is connected to, for example, a microphone 24 and a speaker 25. The audio system 23 outputs, for example, sound data of a sound picked up by the microphone 24 to the processor 11 or the baseband chip 21.

[0058] In addition, the audio system 23 converts, for example, sound data acquired from the processor 11 or the baseband chip 21 into a sound signal, and outputs the sound signal to the speaker 25.

[0059] The microphone 24 collects sounds around the information processing device 100. For example, the microphone 24 collects sounds such as the user's voice during a voice call with another terminal.

[0060] The speaker 25 outputs various sounds to the outside of the information processing device 100. For example, during a voice call with another terminal, the speaker 25 outputs (plays) the sound received from the other terminal.

[0061] The baseband chip 21 is, for example, a dedicated IC that controls wireless communications such as 4G (fourth generation mobile communication system) and 5G (fifth generation mobile communication system).

[0062] The baseband chip 21 outputs, for example, audio data received by the communication unit 22 as audio from the speaker 25 via the audio system 23 .

[0063] The baseband chip 21 acquires audio data collected by the microphone 24 via the audio system 23 and outputs it via the mobile communication system via the communication unit 22. The baseband chip 21 can also transmit and receive input and output data to and from the processor 11 via data communication via the mobile communication system.

[0064] The communication unit 22 is a wireless communication device including an antenna for performing wireless communication based on a mobile communication system.

[0065] The short-range communication unit 15 is, for example, a Bluetooth (registered trademark) module, and performs short-range wireless communication with the pen-shaped input device 200 .

[0066] Next, refer to the same Figure 4 , an example of the hardware structure of the pen-shaped input device 200 is described.

[0067] The pen-type input device 200 includes an MCU 41 , a short-distance communication unit 42 , a vibration unit 43 (an example of a tactile sense reproduction unit), a sound output unit 44 (an example of a tactile sense reproduction unit), and a flash memory 45 .

[0068] The short-range communication unit 42 is, for example, a Bluetooth (registered trademark) module, and performs short-range wireless communication with the information processing device 100 .

[0069] The MCU (Micro Controller Unit) 41 includes a CPU, memory such as ROM and RAM, and I / O components, and executes control of the pen-shaped input device 200. The MCU 41 transmits and receives information to and from the short-range communication unit 42. Furthermore, the MCU 41 controls the generation of vibrations by the vibration unit 43 and the output of writing sounds by the sound output unit 44, thereby providing tactile feedback based on feedback control information transmitted from the information processing device 100.

[0070] The vibration unit 43 includes, for example, an actuator, and vibrates according to the control of the MCU 41 .

[0071] The sound output unit 44 includes, for example, a speaker, and outputs writing sound according to the control of the MCU 41 .

[0072] The flash memory 45 stores various data corresponding to the pen-shaped input device 200 , such as programs.

[0073] Reference Figure 3 , the functional configuration example of the information processing apparatus 100 and the pen-shaped input device 200 is explained. Figure 3 The same functional parts in the hardware structure are marked with Figure 3 The same reference numerals are used and descriptions thereof are appropriately omitted.

[0074] The functions of the information processing device 100 shown in the figure can be performed by the processor 11 ( Figure 5 ) to execute the program.

[0075] First, a functional configuration example of the information processing device 100 will be described. The information processing device 100 includes a communication unit 22, a touch panel display unit 30, a contact parameter detection unit 101, a control unit 102, and a storage unit 103 as functional units.

[0076] The contact parameter detection unit 101 detects the contact state of the pen tip of the pen-shaped input device 200 with respect to the panel surface of the touch panel display unit 30 .

[0077] The contact parameter detection unit 101 outputs a parameter (contact parameter) detected according to the contact state based on the detection signal output by the panel sensor 32. In the following description, the contact parameter is exemplified by the aforementioned moving speed, pen pressure, moving direction, and tilt angle.

[0078] The control unit 102 executes various controls in the information processing apparatus 100. The control unit 102 includes an application-compatible processing unit 121 and a feedback control unit 122.

[0079] The application-compatible processing unit 121 executes processing corresponding to the pen operation-compatible application.

[0080] As the operation corresponding to the application with respect to the pen operation, when the writing operation based on the pen-shaped input device 200 is performed, the pen operation information is input from the panel sensor 32 to the application corresponding processing section 121. The application corresponding processing section 121 performs, for example, a drawing or the like in response to the input of the input pen operation information.

[0081] In addition, the application of the present embodiment can also set the writing tool kind corresponding to the writing operation as described above. The setting of the writing tool kind can be performed, for example, by the operation of the user designating the writing tool kind, or can be set by the application corresponding processing section 121 according to the selection of the writing medium.

[0082] The feedback control section 122 performs feedback control. The feedback control in the present embodiment is control to generate vibration and writing sound as haptic feedback in the pen-shaped input device 200.

[0083] The feedback control section 122 of the present embodiment performs feedback control in real time based on the contact parameter detected by the contact parameter detection section 101.

[0084] In addition, the feedback control section 122 performs feedback control corresponding to the set writing tool kind. That is, the feedback control section 122 generates feedback control information containing the feedback level (the level of vibration, writing sound) set with reference to the feedback table (one example of table information) corresponding to the set writing tool kind and the signal source data (vibration waveform data, writing sound data) corresponding to the set writing tool kind. The feedback control section 122 outputs (transmits) the generated feedback control information to the pen-shaped input device 200 via the communication section 22.

[0085] The storage section 103 stores various information corresponding to the information processing apparatus 100. The storage section 103 is provided with a writing setting data storage section 131, a feedback table storage section 132, a vibration waveform data storage section 133, and a writing sound data storage section 134.

[0086] The writing setting data storage section 131 stores writing setting data. The writing setting data is setting information containing information (writing tool kind data) indicating the writing tool kind set by the pen operation corresponding application corresponding to the writing operation based on the pen-shaped input device 200. In addition, the writing setting data can also contain information (writing medium data) indicating the set writing medium.

[0087] The feedback table storage section 132 stores a feedback table. The feedback table is a table indicating the correspondence relationship of the moving speed and the feedback level for each writing tool kind.

[0088] Figure 2An example of the feedback table stored in the feedback table storage section 132 is shown in this figure. In this figure, an example is shown in which the feedback table storage section 132 stores two feedback tables corresponding to the two feedback categories of vibration and writing sound for each of the three writing instrument categories of the first to third categories. In the present embodiment, the number of writing instrument categories that can be set is not particularly limited, and here, a case in which any one of the three writing instrument categories of the first to third categories can be set is described as an example.

[0089] Further, one feedback table that is used in common for vibration and writing sound can also be stored for one writing instrument category.

[0090] Specifically, the feedback table 1-A corresponding to vibration of the first writing instrument can be a table in which the feedback level corresponding to vibration is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of vibration exemplified by the line Ll l of (B). Figure 2

[0091] Further, the feedback table 1-B corresponding to writing sound of the first writing instrument can be, for example, a table in which the feedback level is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of writing sound that is measured in advance.

[0092] Further, the feedback table 2-A corresponding to vibration of the second writing instrument can be a table in which the feedback level corresponding to vibration is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of vibration exemplified by the line L12 of (C). Figure 2

[0093] Further, the feedback table 2-B corresponding to writing sound of the second writing instrument can be, for example, a table in which the feedback level is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of writing sound that is measured in advance.

[0094] Further, the feedback table 3-A corresponding to vibration of the third writing instrument can be a table in which the feedback level corresponding to vibration is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of vibration exemplified by the line L13 of (D). Figure 4

[0095] Further, the feedback table 3-B corresponding to writing sound of the third writing instrument can be, for example, a table in which the feedback level is made to correspond to the moving speed based on the relationship between the moving speed and the writing reaction level of writing sound that is measured in advance.

[0096] The description will return to Figure 6 The vibration waveform data storage section 133 stores vibration waveform data. ​​​

[0097] Figure 4 An example of the vibration waveform data stored in the vibration waveform data storage section 133 is shown in this figure. In this figure, an example is shown in which the vibration waveform data storage section 133 stores vibration waveform data in correspondence with each of the three writing instrument categories of the first category to the third category.

[0098] The vibration waveform data corresponding to one writing instrument category is waveform data made in such a way as to reproduce the vibration generated when a writing instrument of the corresponding writing instrument category is used to write in reality.

[0099] The explanation will return to Figure 7 . The writing sound data storage section 134 stores writing sound data.

[0100] Figure 4 An example of the writing sound data stored in the writing sound data storage section 134 is shown in this figure. In this figure, an example is shown in which the writing sound data storage section 134 stores writing sound data in correspondence with each of the three writing instrument categories of the first category to the third category.

[0101] The writing sound data corresponding to one writing instrument category can be audio data made in such a way as to reproduce the writing sound generated when a writing instrument of the corresponding writing instrument category is used to write in reality.

[0102] Next, the functional configuration example of the pen-shaped input device 200 will be explained with reference to Figure 3 . The functions of the pen-shaped input device 200 shown in this figure can be realized by the MCU 41 Figure 8 ) executing a program.

[0103] The pen-shaped input device 200 is provided with the near distance communication section 42, the vibration section 43, the sound output section 44, and the control section 201 as functional sections.

[0104] The control section 201 performs various controls in the pen-shaped input device 200. The control section 201 is provided with the drive control section 211.

[0105] The drive control section 211 controls the vibration section 43 so that vibration as haptic feedback is generated. In addition, the drive control section 211 controls the sound output section 44 so that writing sound as haptic feedback is generated.

[0106] The drive control section 211 uses the vibration waveform data, the writing sound data, and the feedback level included in the feedback control information transmitted from the information processing apparatus 100 when controlling the vibration section 43 and the sound output section 44.

[0107] With reference to Figure 3An example of a functional configuration related to driving of the vibration section 43 and the sound output section 44 based on feedback control information will be described.

[0108] In the information processing apparatus 100, the contact parameter detection section 101 includes a movement speed detection section 111, a pen pressure detection section 112, a movement direction detection section 113, and a tilt angle detection section 114.

[0109] The movement speed detection section 111 detects the movement speed of the pen nib moving while contacting the panel surface. The movement speed detection section 111 can detect the movement speed based on the movement amount per unit time of the contact position of the pen nib detected by the panel sensor 32. The movement speed detection section 111 outputs the detected movement speed parameter Dt1 to the feedback control section 122.

[0110] The pen pressure detection section 112 detects the pressure (pen pressure) of the pen nib contacting the panel surface. The pen pressure detection section 112 can detect the pen pressure based on the value of the electrostatic capacity at the contact position of the pen nib detected by the panel sensor 32. The pen pressure detection section 112 outputs the pen pressure parameter Dt2 indicating the detected pen pressure to the feedback control section 122.

[0111] The movement direction detection section 113 detects the movement direction of the pen nib moving while contacting the panel surface. The movement speed detection section 111 can detect the movement direction based on the direction of the movement of the contact position of the pen nib detected by the panel sensor 32. The movement direction detection section 113 outputs the movement direction parameter Dt3 indicating the detected movement direction to the feedback control section 122.

[0112] The tilt angle detection section 114 detects the tilt angle of the pen-shaped input apparatus 200 main body with respect to the panel surface in the state where the pen nib is contacting the panel surface. The tilt angle detection section 114 can detect the tilt angle based on the distribution of the electrostatic capacity corresponding to the pen-shaped input apparatus 200 main body detected by the panel sensor 32 around the contact position of the pen nib. The tilt angle detection section 114 outputs the detected tilt angle parameter Dt4 to the feedback control section 122.

[0113] In the information processing apparatus 100, the feedback control section 122 generates feedback control information FB.

[0114] The feedback control section 122 identifies the writing tool type indicated by the writing setting data stored in the writing setting data storage section 131. The feedback control section 122 refers to the feedback table stored in the feedback table storage section 132 that corresponds to the identified writing tool type, and derives (obtains) the feedback level (vibration level and writing sound level) that corresponds to the movement speed indicated by the movement speed parameter Dt1. On this basis, the feedback control section 122 can correct the feedback level that corresponds to the movement speed using all or some of the other contact parameters (pen pressure parameter Dt2, movement direction parameter Dt3, tilt angle parameter Dt4).

[0115] Further, the feedback level can be common in the vibration level and the writing sound level.

[0116] In addition, the feedback control section 122 obtains the vibration waveform data that corresponds to the identified writing tool type from the vibration waveform data storage section 133.

[0117] In addition, the feedback control section 122 obtains the writing sound data that corresponds to the identified writing tool type from the writing sound data storage section 134.

[0118] The feedback control section 122 generates feedback control information FB that contains the feedback level (vibration level and writing sound level) derived as described above, the vibration waveform data, and the writing sound data.

[0119] The feedback control section 122 outputs (sends) the generated feedback control information FB to the pen-shaped input device 200.

[0120] Next, an example of the functional structure of the pen-shaped input device 200 will be described. The functions of the pen-shaped input device 200 shown in the figure can be realized by the MCU 41 Figure 9 ) executing a program.

[0121] The pen-shaped input device 200 is provided with a close proximity communication section 42, a vibration section 43, a sound output section 44, a control section 201, and a storage section 203 as functional sections.

[0122] The control section 201 performs various controls in the pen-shaped input device 200. The control section 201 is provided with a drive control section 211.

[0123] The drive control section 211 drives the vibration section 43 and the sound output section 44 based on the input feedback control information FB.

[0124] An example of the structure of the vibration section 43 and the sound output section 44 is shown in the figure.

[0125] The vibration section 43 is provided with an amplifier 431 and an actuator 432.

[0126] The drive control section 211 inputs the vibration waveform data included as the signal source data in the received feedback control information FB to the amplifier 431 as the signal source S1. In addition, the drive control section 211 instructs the vibration level included as the feedback level in the received feedback control information FB to the amplifier 431. The amplifier 431 amplifies the input signal source S1 so as to become the instructed vibration level and outputs to the actuator 432. The actuator 432 vibrates by the waveform of the input signal source S1 and the instructed vibration level.

[0127] The sound output section 44 has, for example, an amplifier 441 and a speaker 442.

[0128] The drive control section 211 inputs the writing sound data included as the signal source data in the received feedback control information FB to the amplifier 441 as the signal source S2. In addition, the drive control section 211 instructs the writing sound level included as the feedback level in the received feedback control information FB to the amplifier 441. The amplifier 441 amplifies the input signal source S2 so as to become the instructed writing sound level and outputs to the speaker 442. The input signal source S2 is output as the writing sound from the speaker 442 at the level instructed by the drive control section 211.

[0129] Referring to the flowchart of ​ , the processing steps example of the information processing apparatus 100 and the pen-shaped input device 200 executed in relation to the tactile feedback will be described.

[0130] First, the processing steps example of the information processing apparatus 100 will be described. In executing the processing of this drawing, the feedback control section 122 acquires the contact parameters (the moving speed parameter Dt1, the pen pressure parameter Dt2, the moving direction parameter Dt3, the tilt angle parameter Dt4) detected by the contact parameter detection section 101, for example, every prescribed acquisition period (acquisition period). At this time, the feedback control section 122 can acquire one sample of the contact parameters every acquisition period (period), or can acquire the contact parameters corresponding to each of a plurality of sampling periods.

[0131] First, the processing steps example executed by the information processing apparatus 100 will be described.

[0132] Step S100: In the information processing apparatus 100, the feedback control section 122 waits for the case where the state in which the nib of the pen-shaped input device 200 is in contact with the panel surface and stopped becomes a state in which it starts to move (moving start state) based on the contact parameters acquired at this sampling timing.

[0133] In this case, the feedback control section 122 can determine that the pen tip starts to move, for example, by the moving speed parameter Dt1 in the contact parameter changing from zero to a value larger than zero. Alternatively, the feedback control section 122 can determine that the pen tip starts to move from the fact that the moving direction parameter Dt3 changes from a value indicating no moving direction to a value indicating a certain moving direction, in a case where the pen tip is not moving (stopped). In addition, the feedback control section 122 can determine whether the pen tip starts to move by using the moving speed parameter Dt1 and the moving direction parameter Dt3 in combination.

[0134] Step S102: If it is determined in step S100 that the pen tip starts to move, the feedback control section 122 refers to the writing setting data stored in the writing setting data storage section 131, and determines the writing tool type set by the pen operation corresponding to the application at present.

[0135] Step S104: The feedback control section 122 sets, as a reference object corresponding to the movement of the pen tip this time, the feedback table corresponding to the set writing tool type determined in step S102, from the feedback tables stored in the feedback table storage section 132.

[0136] Step S106: The feedback control section 122 derives the feedback level based on the contact parameter acquired by the acquisition period corresponding to the present time. Here, the feedback control section 122 can derive two feedback levels corresponding to the respective feedback types of vibration and writing sound.

[0137] Specifically, for example, the feedback control section 122 acquires, from the feedback table of vibration and writing sound set as the reference object by step S104, the feedback levels corresponding to the respective vibration and writing sound, corresponding to the moving speed parameter Dt1 in the contact parameter acquired this time. The feedback levels thus acquired can be set as the derivation result of the feedback level.

[0138] Also, the feedback control section 122 can correct the feedback level acquired from the feedback table of the reference object using all or a part of the contact parameters other than the moving speed (the pen pressure parameter Dt2, the moving direction parameter Dt3, the tilt angle parameter Dt4). The feedback level thus corrected reflects, for example, the changes in vibration and writing sound corresponding to the pen pressure, the moving direction, the tilt angle, and the like in the current writing operation.

[0139] Step S108: In addition, the feedback control section 122 acquires the signal source data.

[0140] Specifically, the feedback control section 122 acquires, as the signal source data, vibration waveform data corresponding to the writing instrument category determined by the step S102 from the vibration waveform data storage section 133. In addition, the feedback control section 122 acquires, as the signal source data, writing sound data corresponding to the writing instrument category determined by the step S102 from the writing sound data storage section 134.

[0141] Step S110: The feedback control section 122 generates feedback control information FB including the feedback level corresponding to the vibration and the writing sound derived by the step S106 and the signal source data (vibration waveform data, writing sound data) acquired by the step S108.

[0142] The feedback control section 122 transmits the generated feedback control information FB to the pen-shaped input device 200.

[0143] Step S112: The feedback control section 122 determines whether or not the movement of the pen tip started in correspondence with the step S100 is stopped. The feedback control section 122 can determine that the movement of the pen tip is stopped in accordance with the movement speed parameter Dt1 becoming zero as a value larger than 0 up to this point.

[0144] In a case where it is determined that the pen tip is in a state of being moved (moving state), that is, the movement is not stopped, the process returns to the step S106. In this case, the feedback control section 122 repeatedly executes the processes of the steps S106 to S110 until the movement of the pen tip is stopped. That is, the feedback control section 122 cyclically executes the process of generating the feedback control information FB based on the contact parameter at the current time and transmitting the generated feedback control information FB to the pen-shaped input device 200.

[0145] On the other hand, in a case where it is determined that the movement of the pen tip is stopped, the process returns to the step S100. In this way, since the process returns to the step S100, the transmission of the feedback control information FB to the pen-shaped input device 200 is stopped until the movement of the pen tip is started again.

[0146] Next, a processing step example executed by the pen-shaped input device 200 will be described.

[0147] Step S200: In the pen-shaped input device 200, the drive control section 211 waits to receive the feedback control information FB transmitted from the information processing apparatus 100 by the step S110.

[0148] Step S202: If the feedback control information FB is received, the drive control section 211 inputs the vibration waveform data included in the signal source data in the received feedback control information FB as the signal source S1 to the amplifier 431. In addition, the drive control section 211 inputs the writing sound data included in the signal source data in the received feedback control information FB as the signal source S2 to the amplifier 441.

[0149] Step S204: In addition, the feedback control information FB sets two feedback levels corresponding to the vibration and the writing sound included in the received feedback control information FB in the amplifiers 431, 441, respectively.

[0150] Hereinafter, a modification of the present embodiment will be described.

[0151] In the above-described embodiment, when deriving the feedback level, the feedback table that establishes a correspondence between the feedback level and the moving speed is used. However, feedback tables corresponding to each of the contact parameters other than the moving speed, i.e., the pen pressure, the moving direction, and the tilt angle can be provided. On this basis, the feedback control section 122 can derive the feedback level by referring to one or more feedback tables corresponding to one or more of the arbitrary contact parameters among the moving speed, the pen pressure, the moving direction, and the tilt angle.

[0152] In addition, the feedback control section 122 can derive the feedback level by using an operation using the prescribed contact parameter such as the moving speed instead of referring to the feedback table.

[0153] The tactile feedback can be performed by either of the vibration and the sound. In this case, the information processing apparatus 100 and the pen-shaped input device 200 can have a structure corresponding to the tactile feedback of one of the vibration and the sound and do not have a structure corresponding to the tactile feedback of the other. Alternatively, for example, which of the vibration and the sound is used for the tactile feedback can be set in accordance with a selection operation of the user, the kind of the writing tool set in the pen operation corresponding application, the kind of the medium of the writing object.

[0154] The prescribed functional section that the information processing apparatus 100 in the above-described embodiment has can be provided to the pen-shaped input device 200. Alternatively, the prescribed functional section that the pen-shaped input device 200 in the above-described embodiment has can be provided to the information processing apparatus 100.

[0155] As a specific example, the feedback control section 122 can be provided in the pen-shaped input device 200. In this case, the information processing apparatus 100 can transmit the contact parameter detected by the contact parameter detection section 101 to the pen-shaped input device 200.

[0156] Also, at least either one of the vibration waveform data storage section 133 and the writing sound data storage section 134 can be stored in the pen-shaped input device 200, for example.

[0157] In this case, the feedback control information FB transmitted from the information processing apparatus 100 can include information that specifies the writing tool type corresponding to the vibration waveform data and the writing sound data used for the output of the vibration and the writing sound in the pen-shaped input device 200, and the feedback levels corresponding to the vibration and the writing sound, respectively.

[0158] Further, a program for realizing the functions of the information processing apparatus 100, the pen-shaped input device 200, and the like described above can be recorded in a recording medium that is readable by a computer, and the processing as the information processing apparatus 100, the pen-shaped input device 200, and the like described above can be performed by causing a computer system to read in and execute the program recorded in the recording medium. Here, the "causing a computer system to read in and execute the program recorded in the recording medium" includes installing the program in the computer system. The "computer system" here includes an OS, hardware such as peripheral devices, and the like. Also, the "computer system" can include a plurality of computer apparatuses connected via a network including a communication line such as the Internet, a WAN, a LAN, a dedicated line, and the like. Also, the "recording medium that is readable by a computer" refers to a portable medium such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, and the like, a storage device such as a hard disk built in a computer system, and the like. Thus, the recording medium in which the program is stored can be a non-transitory recording medium such as a CD-ROM. Also, the recording medium includes a recording medium provided inside or outside that is accessible from a distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server can be different from the code of the program that can be executed in the terminal apparatus. That is, as long as it is code that can be installed in a form that can be downloaded from the distribution server and executed in the terminal apparatus, it does not matter what form it is stored in the distribution server. Further, the program can be divided into a plurality of pieces, and the terminal apparatus can be configured to be assembled after downloading each piece at different times, and the distribution server that distributes each of the divided pieces can be different. Also, the "recording medium that is readable by a computer" includes a structure that holds the program for a certain period of time, such as a server that transmits the program via a network, and a volatile memory (RAM) inside a computer system that becomes a client. Also, the program described above can be a structure that realizes a part of the functions described above. Further, it can be a so-called differential file (differential program) that can realize the functions described above in combination with a program already recorded in the computer system.

Claims

1. An information processing system comprising: a contact parameter detection section that detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object; and a feedback control section that derives a feedback level corresponding to a writing tool category set in correspondence with the pen-shaped input device, based on the detected contact parameters, and drives a haptic reproduction section in the pen-shaped input device by the derived feedback level.

2. The information processing system according to claim 1, wherein the feedback control section derives the feedback level by acquiring a feedback level corresponding to the detected contact parameters from table information indicating a correspondence relationship between values of contact parameters and feedback levels corresponding to the set writing tool category.

3. The information processing system according to claim 1 or 2, wherein the contact parameters are all or a part of a speed of a pen tip of the pen-shaped input device moving on the surface of the operation object, a pressure of the pen tip with respect to the surface of the operation object, a direction of the pen tip moving on the surface of the operation object, and an inclination angle of the pen-shaped input device with respect to the surface of the operation object.

4. The information processing system according to claim 1 or 2, wherein the haptic reproduction section includes a vibration section that generates vibration in accordance with an input from a signal source.

5. The information processing system according to claim 1 or 2, wherein the haptic reproduction section includes a sound output section that outputs sound in accordance with an input from a signal source.

6. An information processing method in an information processing system, comprising: a contact parameter detection step in which a contact parameter detection section detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object; and a feedback control step in which a feedback control section derives a feedback level corresponding to a writing tool category set in correspondence with the pen-shaped input device, based on the detected contact parameters, and drives a haptic reproduction section in the pen-shaped input device by the derived feedback level.

7. A program for causing a computer in an information processing system to function as a contact parameter detection section and a feedback control section, wherein the contact parameter detection section detects one or more contact parameters corresponding to a contact state of a pen-shaped input device with respect to a surface of an operation object, the feedback control section derives a feedback level corresponding to a writing tool category set in correspondence with the pen-shaped input device, based on the detected contact parameters, and drives a haptic reproduction section in the pen-shaped input device by the derived feedback level. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

Images