Information processing system, pen-type input device, and program
Through the sensor and vibration control unit of the information processing system, the contact situation of the pen input device is detected and feedback parameters are output, which solves the problem that the pen input device in the prior art cannot fully simulate the touch and sound of the real writing medium, and improves the user experience .
Patent Information
- Application Number
- CN202410688268.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-01
- Filing Date
- 2024-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
Existing pen input devices have difficulty in fully simulating the haptic and acoustic feedback on the realistic writing medium when inputting, resulting in poor user experience.
An information processing system is designed, combining sensors and vibration control units to detect the contact situation of the pen input device, and send feedback parameters to control vibration and sound output, simulating the touch and sound during writing.
It realizes tactile and sound feedback on the writing medium closer to real-life writing medium when using pen input devices, improving the user experience.
Smart Images

Figure CN119065493A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing system, a pen-type input device, and a program. Background Art
[0002] It is known that when inputting to an interactive display system, a pen-shaped stylus (pen-type input device) capable of providing tactile feedback by generating vibrations to drive a tactile actuator is used as an input device.
[0003] Patent Document 1: Japanese Patent Application No. 2017-537395
[0004] By vibrating the pen-type input device, a user can obtain a tactile sensation similar to that of sliding the pen tip on a writing medium such as paper, thereby being able to obtain a sensation close to that of writing on a writing medium using the pen-type input device. Summary of the invention
[0005] When inputting using a pen-type input device, it is desirable to achieve a feeling as close as possible to that of writing on a real writing medium.
[0006] Therefore, the present invention takes the above-mentioned problems into consideration, and aims to make the input using a pen-type input device close to the feeling of writing on a writing medium in reality.
[0007] One method of the present invention for solving the above-mentioned problems is an information processing system, comprising an information processing device and a pen-type input device, wherein the information processing device comprises: a sensor unit for detecting contact of the pen-type input device; and a parameter output unit for sending feedback parameters to the pen-type input device based on the detection of the contact by the sensor unit, wherein the feedback parameters at least include parameters for controlling the vibration of the pen-type input device, and the pen-type input device comprises: a vibration unit; and a vibration control unit for vibrating the vibration unit based on the received feedback parameters, and at least one of the pen-type input device and the information processing device comprises a sound output control unit, which outputs sound from the sound output unit at a timing corresponding to the vibration of the vibration unit based on the feedback parameters output by the parameter output unit.
[0008] One embodiment of the present invention is a pen-type input device, comprising: a communication unit, which communicates with an information processing device, and the information processing device sends feedback parameters to the pen-type input device based on the detection of contact with the pen-type input device, wherein the feedback parameters at least include parameters for controlling the vibration of the pen-type input device; a vibration unit; a vibration control unit, which vibrates the vibration unit based on the received feedback parameters; and a sound output control unit, which outputs sound from the sound output unit at a timing corresponding to the vibration of the vibration unit based on the received feedback parameters.
[0009] One embodiment of the present invention is a program for causing a computer serving as a pen-type input device to function as a communication unit, a vibration control unit, and a sound output control unit for communicating with an information processing device, the information processing device sending feedback parameters to the pen-type input device based on detection of contact by a sensor unit for detecting contact of the pen-type input device, wherein the feedback parameters include at least parameters for controlling the vibration of the pen-type input device, the vibration control unit vibrates the vibration unit based on the received feedback parameters, and the sound output control unit outputs sound from the sound output unit at a timing corresponding to the vibration of the vibration unit based on the received feedback parameters.
[0010] According to the present invention, when inputting using a pen-type input device, an effect is achieved in which the feeling of writing on a writing medium in reality can be approximated. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 It is a diagram showing an example of the external configuration of the information processing system in the first embodiment.
[0012] Figure 2 This is a diagram showing an example of a functional configuration corresponding to tactile / writing sound feedback of the information processing apparatus and the pen-type input device in the first embodiment.
[0013] Figure 3 It is a diagram showing an example of parameter data stored in the parameter storage unit in the first embodiment.
[0014] Figure 4 This is a diagram showing an example of handwriting sound data stored in the handwriting sound data storage unit in the first embodiment.
[0015] Figure 5 This is a flowchart showing an example of processing steps executed by the information processing apparatus and the pen-type input device in accordance with the first embodiment in response to tactile and writing sound feedback.
[0016] Figure 6 It is a diagram showing an example of a functional configuration corresponding to tactile / writing sound feedback of the information processing apparatus and the pen-type input device in the second embodiment.
[0017] Figure 7 This is a flowchart showing an example of processing steps executed by the information processing apparatus and the pen-type input device in accordance with the tactile / writing sound feedback in the second embodiment.
[0018] Figure 8 It is a diagram showing an example of a functional configuration corresponding to tactile / writing sound feedback of the information processing apparatus and the pen-type input device in the third embodiment.
[0019] Fig. 9 This is a flowchart showing an example of processing steps executed by the information processing apparatus and the pen-type input device in accordance with the tactile / writing sound feedback in the third embodiment.
[0020] Description of Reference Numerals
[0021] 100…information processing device; 101…communication unit; 102…touch panel display unit; 103…control unit; 104…storage unit; 105…sound output unit; 121…display unit; 122…touch panel; 131…application corresponding processing unit; 132…sound output control unit; 132A…sound output control unit; 141…parameter storage unit; 142…writing setting data storage unit; 143…handwriting sound data storage unit; 200…pen-type input device; 201…communication unit; 202…vibration unit; 203…sound output unit; 204…control unit; 205…storage unit; 241…vibration control unit; 242…sound output control unit; 243…sound output indication unit; 251…handwriting sound data storage unit; 1311…parameter output unit. DETAILED DESCRIPTION
[0022] <First embodiment>
[0023] Figure 1 FIG. 1 is a diagram showing an example of an external configuration of an information processing system according to the present embodiment. As shown in the diagram, the information processing system according to the present embodiment includes an information processing apparatus 100 and a pen-type input device 200 .
[0024] The information processing apparatus 100 can execute information processing according to an input operation using the pen-type input device 200. The information processing apparatus 100 shown in the figure is an example of a tablet terminal, a notebook computer, or the like.
[0025] The information processing device 100 includes a touch panel display unit 102. The touch panel display unit 102 is a combination of a touch panel and a display unit. The touch panel display unit 102 displays an image on a display surface and can be operated by bringing an operating body such as a pen-type input device or a finger into contact with the display surface.
[0026] The pen-shaped input device 200 is a pen-shaped input device used by a user to operate the touch panel in the touch panel display unit 102 of the information processing apparatus 100 .
[0027] The user holds the pen-type input device 200 and moves the pen tip in contact with the display surface of the touch panel display unit 102 to perform handwriting input operations such as characters, pictures, and graphics.
[0028] In addition, the "contact" between the pen tip and the display surface of the touch panel display unit 102 when operating the pen-type input device 200 includes, in addition to the physical contact, a state in which there is no physical contact with the display surface but the touch panel of the touch panel display unit 102 is close enough to be able to detect the state of the pen tip.
[0029] Furthermore, as an operation using the pen-shaped input device 200 , an instruction operation on a user interface image displayed on the touch-panel display portion 102 or the like may be performed.
[0030] In addition, the detection method of the pen-type input device 200 based on the touch panel in the touch panel display unit 102 of the present embodiment is not particularly limited, and examples thereof include an electrostatic capacitance method and a pressure-sensitive method.
[0031] An application corresponding to input operation by the pen-type input device 200 (pen operation corresponding application) is installed in the information processing apparatus 100 .
[0032] The pen operation corresponding application can, for example, display text and pictures on the touch panel display unit 102 based on handwriting input operations such as text input and painting performed by bringing the pen tip of the pen-type input device 200 into contact with the touch panel display unit 102, or can perform processing to digitize the text and pictures drawn by the operation.
[0033] In addition, in the information processing system of this embodiment, the pen-type input device 200 vibrates in response to an operation corresponding to writing (writing operation). Through the vibration of the pen-type input device 200, the user feels a tactile sensation similar to that of writing on a writing medium such as paper, and can obtain a sensation similar to that of writing on a writing medium using a real writing tool.
[0034] Furthermore, in the information processing system of the present embodiment, the pen-type input device 200 outputs a sound (writing sound) produced by writing on a writing medium using a real writing tool in response to an operation corresponding to writing.
[0035] In this way, when the pen-type input device 200 is used for writing, vibration is generated and writing sound is outputted at the same time, so that the feeling of writing on a writing medium using a real writing tool can be closer.
[0036] In the following description, the generation of vibration and the output of writing sound when the pen-type input device 200 is used for writing operation are also referred to as "tactile / writing sound feedback".
[0037] Figure 2An example of a functional configuration corresponding to tactile / writing sound feedback of the information processing apparatus 100 and the pen-type input device 200 according to the present embodiment is shown.
[0038] First, a functional configuration example of the information processing device 100 is described. The functions of the information processing device 100 shown in the figure are realized by executing a program by a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) etc. provided as hardware of the information processing device 100.
[0039] The information processing device 100 in the figure includes a communication unit 101 , a touch panel display unit 102 , a control unit 103 , and a storage unit 104 .
[0040] The communication unit 101 performs wireless communication with the pen-shaped input device 200. There is no particular limitation on the method of wireless communication supported by the communication unit 101. In addition, the communication unit 101 may also support communication with the wired pen-shaped input device 200.
[0041] The touch panel display unit 102 is a device that combines a display unit 121 and a touch panel 122 (an example of a sensor unit). The display unit 121 displays an image under the control of the control unit 103. The touch panel 122 is provided corresponding to the display surface of the display unit 121, detects the position where the pen tip of the pen-type input device 200 touches, and outputs the coordinates of the detected position as pen operation information.
[0042] The control unit 103 executes various controls in the information processing device 100. The control unit 103 includes an application-supporting processing unit 131 that executes a process corresponding to the pen operation-supporting application.
[0043] The application corresponding processing unit 131 includes a parameter output unit 1311. When the pen-type input device 200 is operated to write as an operation for the pen operation corresponding application, pen operation information is input from the touch panel 122 to the application corresponding processing unit 131. In response to the input of the pen operation information, the application corresponding processing unit 131 outputs a feedback parameter for causing the pen-type input device 200 to perform a tactile / writing sound feedback. The communication unit 101 sends the output feedback parameter D1 to the pen-type input device 200.
[0044] The storage unit 104 stores various information corresponding to the information processing device 100. The storage unit 104 includes a parameter storage unit 141 and a writing setting data storage unit 142.
[0045] The parameter storage unit 141 stores predetermined parameter data included in the feedback parameters.
[0046] Figure 3FIG. 2 shows an example of parameter data stored in parameter storage unit 141. In this figure, an example is shown in which parameter storage unit 141 stores vibration waveform data and writing sound data as parameter data for each writing tool type.
[0047] The vibration waveform data is data indicating a vibration waveform generated in order to cause the pen-type input device 200 to generate a desired vibration state.
[0048] The pen operation corresponding application of this embodiment can set the writing tool category when the pen-type input device 200 is used for writing operation. Specifically, as the writing tool category, pencils, ballpoint pens, crayons, signature pens, brushes, etc. can be listed. The pen operation corresponding application draws with handwriting corresponding to the set writing tool category according to the writing operation of the pen-type input device 200.
[0049] In reality, the tactile sensation felt when writing on a writing medium differs depending on the type of writing tool. Each of the vibration waveform data stored in parameter storage unit 141 is created to reproduce the tactile sensation of each corresponding type of writing tool.
[0050] In addition, the tactile sensation during writing also varies according to the type of writing medium, such as Western paper, Japanese paper, resin, etc. In other words, the tactile sensation during writing may vary according to each combination of the type of writing tool and the type of writing medium. Therefore, the parameter storage unit 141 may also store parameter data (vibration waveform data) that varies according to each combination of the type of writing tool and the type of writing medium, for example.
[0051] Return instructions to Figure 2 .
[0052] The writing setting data storage unit 142 is setting data including writing tool type data (writing tool type data) set in the pen operation corresponding application in accordance with the writing operation by the pen-shaped input device 200. The writing setting data may also include data indicating the set writing medium (writing medium data).
[0053] Next, a functional configuration example of the pen-type input device 200 will be described. The functions of the pen-type input device 200 shown in the figure can be realized by causing a microprocessor such as an MCU (Micro Controller Unit) included in the pen-type input device 200 as hardware to execute a program.
[0054] The pen-shaped input device 200 in the figure includes a communication unit 201 , a vibration unit 202 , a sound output unit 203 , a control unit 204 , and a storage unit 205 .
[0055] The communication unit 201 is connected to the communication unit 101 of the information processing device 100 so as to be communicable therewith.
[0056] The vibration unit 202 (actuator) is a part that vibrates in order to provide a tactile sense during a writing operation to a user holding the pen-type input device 200. The vibration of the vibration unit 202 is controlled by the vibration control unit 241.
[0057] The sound output unit 203 includes, for example, a speaker, and outputs handwriting sound according to the control of the sound output control unit 242. The sound output unit 203 can also output sounds such as electronic sounds and voices.
[0058] The control unit 204 executes various controls in the pen-type input device 200. The control unit 204 includes a vibration control unit 241 and a sound output control unit 242.
[0059] The vibration control unit 241 vibrates the vibration unit 202 based on the vibration waveform data included in the received feedback parameter.
[0060] The sound output control unit 242 outputs the audio data as the handwriting sound data stored in the handwriting sound data storage unit 251 to the sound output unit 203 , thereby outputting the handwriting sound from the sound output unit 203 .
[0061] The storage unit 205 stores various information corresponding to the pen-type input device 200. The storage unit 205 includes a handwriting sound data storage unit 251.
[0062] The handwriting sound data storage unit 251 stores handwriting sound data. Figure 4 FIG. 2 shows an example of the handwriting sound data stored in the handwriting sound data storage unit 251. As shown in the figure, the handwriting sound data storage unit 251 stores the writing sound data in association with each writing tool type.
[0063] In reality, the handwriting sound produced when writing on a writing medium differs depending on the type of writing tool. Each piece of handwriting sound data stored in the handwriting sound data storage unit 251 is created to reproduce the handwriting sound produced corresponding to the type of writing tool.
[0064] Reference Figure 5 Flowchart based on Figure 2 An example of processing steps executed by the information processing apparatus 100 and the pen-type input device 200 in accordance with tactile / writing sound feedback will be described.
[0065] First, an example of a process procedure executed by the information processing device 100 will be described.
[0066] Step S100 : In the information processing apparatus 100 , the touch panel 122 detects coordinates specified by a writing operation using the pen-type input device 200 and outputs pen operation information indicating the detected coordinates. The application-compatible processing unit 131 acquires the pen operation information output from the touch panel 122 .
[0067] The application corresponding processing unit 131 performs processing such as drawing the handwriting corresponding to the acquired pen operation information on the display unit 121. At this time, when the parameter output unit 1311 receives the pen pressure data indicating the pen pressure detected by the pen-type input device 200, for example, it can draw the handwriting in a manner that changes the thickness of the handwriting according to the pen pressure indicated by the received pen pressure data.
[0068] Step S102 : In addition, according to the pen operation information obtained in step S100 , the parameter output unit 1311 obtains the parameter data included in the feedback parameter.
[0069] For example, the currently set writing tool type is determined by referring to the writing setting data stored in the writing setting data storage unit 142. The parameter output unit 1311 acquires data indicating the determined writing tool type (writing tool type data) as one of the parameter data.
[0070] Furthermore, the parameter output unit 1311 acquires the vibration waveform data stored in the parameter storage unit 141 in correspondence with the determined writing tool type as a feedback parameter.
[0071] In addition, when the writing setting data includes writing volume data for setting the volume of the writing sound output from the pen-shaped input device 200, the parameter output unit 1311 may obtain the volume indicated by the writing volume data as one of the parameter data. Alternatively, when the parameter output unit 1311 receives pen pressure data sent from the pen-shaped input device 200, it may obtain the writing volume set according to the pen pressure indicated by the received pen pressure data as the parameter data.
[0072] Step S104 : the parameter output unit 1311 transmits the feedback parameter D1 including the parameter data acquired in step S102 to the pen-type input device 200 by using the communication unit 101 .
[0073] The process of acquiring parameter data in steps S102 and S104 and sending the parameter data as the feedback parameter D1 to the pen-type input device 200 can be realized by an API (Application Programming Interface) constructed corresponding to the tactile / writing sound feedback.
[0074] Next, an example of a processing procedure executed by the pen-type input device 200 will be described.
[0075] Step S200 : In the pen-type input device 200 , the communication unit 201 receives the feedback parameter D1 transmitted from the information processing apparatus 100 in step S104 .
[0076] Step S202 : The vibration control unit 241 drives the vibration unit 202 using the vibration waveform data included in the feedback parameter received in step S200 , so that the vibration unit 202 vibrates.
[0077] Step S204 : In addition, the sound output control unit 242 acquires the handwriting sound data stored in association with the writing tool type indicated by the writing tool type data included in the feedback parameter received in step S200 from the handwriting sound data storage unit 251 .
[0078] Step S206: The sound output control unit 242 outputs the writing sound from the sound output unit 203 using the writing sound data acquired in step S204. At this time, if the feedback parameter includes the writing volume data, the sound output control unit 242 can adjust the volume of the writing sound output from the sound output unit 203 based on the writing volume data.
[0079] For example, there are known drawing applications that output sound based on writing using a pen-type input device, but there are also drawing applications that do not output sound corresponding to writing. In this case, whether to output sound corresponding to writing depends on whether the application has a function corresponding to the output of sound corresponding to writing.
[0080] In contrast, according to the above structure of this embodiment, even if the drawing application does not have a function of outputting sound corresponding to writing, feedback parameters can be sent from the pen operation corresponding application to the pen-type input device 200 through the API, thereby enabling the pen-type input device 200 to output writing sound. Figure 4 The processing of the vibration of the vibration unit 202 in step S202 and the output of the writing sound in step S204 are performed in accordance with the same pen operation information acquired in step S100, and are therefore performed at a synchronized timing. Specifically, the writing sound is output at the same timing as the vibration generated in the pen-type input device 200.
[0081] In this case, the information processing device 100 only needs to apply the above-mentioned API, and no major changes in hardware or software are required, for example.
[0082] In addition, in this embodiment, the handwriting sound data of each writing tool category is stored in the pen-type input device 200, and the pen data representing the writing tool category set in the pen operation corresponding application is included in the feedback parameter. With such a structure, it is possible to output the handwriting sound suitable for the writing tool category set in the pen operation corresponding application. In addition, even if a plurality of different pen operation corresponding applications are installed in the information processing device 100, each pen operation corresponding application has the above structure, so that the handwriting sound suitable for the set writing tool category can be output corresponding to any one of the pen operation corresponding applications.
[0083] In addition, in a drawing application that outputs sound based on writing by a pen-type input device, the information processing device outputs sound corresponding to the writing. In contrast, in this embodiment, since the writing sound is output from the pen-type input device 200, it can be closer to the feeling of writing with a pen in reality.
[0084] In addition, in this embodiment, the information processing device 100 may store handwriting sound data corresponding to each writing tool category (or a combination of a writing tool category and a writing medium) as data corresponding to the application corresponding to the pen operation, instead of the pen-type input device 200. In this case, the handwriting sound data corresponding to the set writing tool category (or a combination of a writing tool category and a writing medium) is included in the feedback parameter.
[0085] The pen-type input device 200 can output writing sound using the handwriting sound data included in the received feedback parameters.
[0086] <Second embodiment>
[0087] Next, a second embodiment will be described. The information processing system of this embodiment is configured such that the information processing device 100 outputs writing sound.
[0088] Figure 6 FIG. 2 shows an example of a functional configuration corresponding to the tactile / writing sound feedback of the information processing apparatus 100 and the pen-type input device 200 of the present embodiment. Figure 2 The same parts are marked with the same reference numerals. Figure 2 The differences are explained below.
[0089] In the present embodiment, the control unit 103 of the information processing device 100 includes a sound output control unit 132 .
[0090] The storage unit 104 of the information processing device 100 includes a handwriting sound data storage unit 143 in addition to the parameter storage unit 141 and the writing setting data storage unit 142 .
[0091] Furthermore, the information processing device 100 includes a sound output unit 105 .
[0092] The sound output control unit 132 is a functional unit corresponding to the writing sound output software resident in the information processing device 100 , and the handwriting sound data storage unit 143 is a storage unit provided corresponding to the writing sound output software.
[0093] In the pen-type input device 200, the Figure 2 The control unit 204 includes a sound output unit 203, a handwriting sound data storage unit 251, and a sound output control unit 242. In the control unit 204, instead of the sound output control unit 242, a sound output instruction unit 243 is provided.
[0094] The sound output instruction unit 243 instructs the sound output control unit 132 of the information processing device 100 to output the handwriting sound.
[0095] Reference Figure 7 Flowchart based on Figure 6 An example of processing steps executed by the information processing apparatus 100 and the pen-type input device 200 in accordance with tactile / writing sound feedback will be described.
[0096] First, an example of a process procedure executed by the information processing device 100 will be described.
[0097] The processing of steps S300 to S304 can be Figure 5 Steps S100 to S104 are the same.
[0098] Step S306: According to the transmission of the feedback parameter D1 based on step S304, the pen-type input device 200 transmits a sound output instruction signal D2. The sound output instruction signal D2 is a signal that instructs the sound output control unit 132 to output the writing sound. In the information processing device 100, the communication unit 101 receives the transmitted sound output instruction signal D2. The sound output instruction signal includes writing tool category data, writing volume data, etc. as parameter data included in the feedback parameter transmitted by step S204.
[0099] Step S306 : Based on the received sound output instruction signal D2 , the writing sound data corresponding to the writing tool type indicated by the writing tool type data included in the received sound output instruction signal D2 is acquired from the writing setting data storage unit 142 .
[0100] Step S308: The sound output control unit 132 outputs the writing sound from the sound output unit 105 using the writing sound data acquired in step S306. At this time, the volume of the writing sound output from the sound output unit 105 can be adjusted based on the writing volume data included in the received sound output instruction signal D2.
[0101] Next, an example of a processing procedure executed by the pen-type input device 200 will be described.
[0102] The processing of steps S400 and S402 is Figure 5 Steps S200 and S202 are the same.
[0103] Step S404 : The audio output instruction unit 243 generates an audio output instruction signal D2 including the writing tool type data (and writing volume data) included in the feedback parameter received in step S400 . The audio output instruction unit 243 transmits the generated audio output instruction signal D2 from the communication unit 201 to the information processing device 100 .
[0104] According to the configuration of the information processing system of the present embodiment described above, when a writing operation is performed on the touch panel display unit 102 of the information processing device 100 using the pen-type input device 200, the information processing device 100 can output a writing sound while vibrating the tactile sensation corresponding to the writing in the pen-type input device 200. Figure 7 The processing based on the vibration of the vibration unit 202 in step S402 and the output of the writing sound in step S404 are performed corresponding to the same pen operation information obtained in step S300, and are therefore performed at a synchronized timing.
[0105] In addition, the pen-type input device 200 in this embodiment includes the writing tool category data (and writing volume data) included in the feedback parameters sent from the information processing device 100 in the sound output instruction signal D2, and returns it as data that can be used for controlling the sound output by the sound output control unit 132 of the information processing device 100. With such a structure, for example, even in an environment where a plurality of different pen operation corresponding applications are introduced by the information processing device 100, an appropriate writing sound corresponding to the set writing tool category can be output corresponding to any of the pen operation corresponding applications.
[0106] <Third Embodiment>
[0107] Next, a third embodiment will be described. The information processing system of this embodiment is configured such that the information processing device 100 outputs writing sound.
[0108] Figure 8 FIG. 2 shows an example of a functional configuration corresponding to the tactile / writing sound feedback of the information processing apparatus 100 and the pen-type input device 200 of the present embodiment. Figure 6 The same parts are marked with the same reference numerals. Figure 6 The differences are explained below.
[0109] In the present embodiment, the control unit 103 of the information processing apparatus 100 includes a sound output control unit 132A. In addition, the sound output instruction unit 243 is omitted in the pen-type input device 200 .
[0110] The sound output control unit 132A is a functional unit corresponding to the writing sound output software resident in the information processing device 100. Figure 6 The same as the sound output control unit 132. However, Figure 6 The sound output control unit 132 outputs writing sound based on the sound output instruction signal D2 sent from the pen-type input device 200 , whereas the sound output control unit 132A of this embodiment obtains the feedback parameter output by the parameter output unit 1311 for transmission from the communication unit 101 .
[0111] Reference Fig. 9 Flowchart based on Figure 8 An example of processing steps executed by the information processing apparatus 100 and the pen-type input device 200 in accordance with tactile / writing sound feedback will be described.
[0112] First, an example of a process procedure executed by the information processing device 100 will be described.
[0113] The processing of steps S500 to S504 can be Figure 7 However, the feedback parameter sent in step S504 only needs to include the vibration waveform data, and does not need to include the writing volume data and other data related to the output control of the writing sound.
[0114] Step S506 : The sound output control unit 132A obtains the feedback parameter output by the parameter output unit 1311 to the communication unit 101 in order to transmit it to the pen-type input device 200 in step S504 .
[0115] Step S508 : The sound output control unit 132A acquires, from the writing setting data storage unit 142 , writing sound data associated with the writing tool type indicated by the writing tool type data included in the feedback parameter acquired in step S506 .
[0116] Step S510: The sound output control unit 132A outputs the writing sound from the sound output unit 105 using the writing sound data acquired in step S508. At this time, if the feedback parameter includes the writing volume data, the volume of the writing sound output from the sound output unit 105 can be adjusted based on the writing volume data.
[0117] The processing of steps S600 and S602 performed by the pen-type input device 200 may be the same as that of Figure 7 Steps S400 and S402 are the same.
[0118] According to the configuration of the information processing system of the present embodiment described above, in response to a writing operation using the pen-type input device 200 on the touch panel display unit 102 of the information processing device 100, a writing sound can be outputted from the information processing device 100 in synchronization with the vibration of the tactile sensation corresponding to the writing in the pen-type input device 200. In addition, the pen-type input device 200 vibrates the vibration unit 202 according to the vibration waveform data included in the feedback parameter D1 sent from the information processing device 100. With such a configuration, in the present embodiment, in response to the user writing on the touch panel display unit 102 using the pen-type input device 200, the pen-type input device 200 vibrates, and a writing sound synchronized with the vibration of the pen-type input device 200 can be outputted from the information processing device 100. In this case, in the present embodiment, the vibration waveform can include a vibration waveform corresponding to the set writing tool type in the feedback parameter D1 sent from the information processing device 100. Thus, the vibration corresponding to the set writing tool type can be provided to the pen-type input device 200. In addition, through the above-mentioned structure, in this embodiment, even in an environment where multiple different pen operation corresponding applications are imported into the information processing device 100, it is possible to output appropriate writing sounds corresponding to the set writing tool category in accordance with any pen operation corresponding application.
[0119] Furthermore, the structure of the second embodiment or the third embodiment may be combined with the first embodiment so that the writing sound can be output from both the pen-type input device 200 and the information processing device 100. In this case, for example, it is possible to switch which of the pen-type input device 200 and the information processing device 100 outputs the writing sound, depending on the user's operation, application conditions, etc.
[0120] In addition, the program for realizing the functions of the above-mentioned information processing device 100, pen-type input device 200, etc. can also be recorded in a computer-readable recording medium, and the processing of the above-mentioned information processing device 100, pen-type input device 200, etc. can be performed by making the computer system read the program recorded in the recording medium and execute it. Here, the so-called "making the computer system read the program recorded in the recording medium and execute it" includes installing the program to the computer system. The so-called "computer system" here includes hardware such as OS and peripheral devices. In addition, the "computer system" may also include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. In addition, the so-called "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, CD-ROMs, and storage devices such as HDDs and SSDs built into the computer system. In this way, the recording medium storing the program can be a non-temporary recording medium such as a CD-ROM. In addition, the recording medium also includes a recording medium set inside or outside that can be accessed from a distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may also be a code different from the code of the program in a form that can be executed in the terminal device. That is, as long as it can be downloaded from the distribution server and installed in a form that can be executed in the terminal device, the form stored in the distribution server does not matter. In addition, the program may be divided into multiple parts, and the structure formed by combining the terminal device after downloading at different time intervals, and the distribution server for each of the divided programs may be different. Moreover, the so-called "computer-readable recording medium" also includes a structure that keeps the program for a certain period of time, such as a server in the case of sending a program via a network, and a volatile memory (RAM) inside a computer system that becomes a client. In addition, the above-mentioned program may also be a structure for realizing a part of the above-mentioned functions. Furthermore, it may also be a so-called differential file (differential program) that can realize the above-mentioned functions by combining them with the programs already recorded in the computer system.
Claims
1. An information processing system, An information processing device and a pen-type input device are provided, The above-mentioned information processing device comprises: a sensor unit for detecting contact with the pen-type input device; and The parameter output unit sends the feedback parameter to the pen-type input device according to the detection of the contact by the sensor unit, wherein: The feedback parameters at least include parameters for controlling the vibration of the pen-type input device. The pen-type input device has: a vibration part; and a vibration control unit that causes the vibration unit to vibrate based on the received feedback parameter, At least one of the pen-type input device and the information processing apparatus includes a sound output control unit that outputs sound from the sound output unit at a timing corresponding to vibration of the vibration unit based on the feedback parameter output by the parameter output unit.
2. The information processing system according to claim 1, wherein: The pen-type input device in the pen-type input device and the information processing apparatus includes the sound output unit and the sound output control unit.
3. The information processing system according to claim 1 or 2, wherein: The feedback parameter includes writing tool type data indicating the writing tool type set corresponding to the pen-type input device. The sound output unit outputs a handwriting sound corresponding to the writing tool type indicated by the writing tool type data.
4. The information processing system according to claim 1, wherein: The information processing device in the pen-type input device and the information processing device comprises the sound output unit and the sound output control unit. The pen-type input device includes a sound output instruction unit that transmits a sound output instruction signal including a predetermined parameter included in the feedback parameter in response to reception of the feedback parameter to instruct the sound output control unit to output a sound.
5. A pen-type input device comprising: The communication unit communicates with the information processing device, and the information processing device sends a feedback parameter to the pen-type input device according to the situation that the pen-type input device is touched, wherein: The feedback parameters at least include parameters for controlling the vibration of the pen-type input device; Vibration Department; a vibration control unit that causes the vibration unit to vibrate based on the received feedback parameter; as well as The sound output control unit outputs sound from the sound output unit at a timing corresponding to the vibration of the vibration unit based on the received feedback parameter.
6. A program for causing a computer as a pen-type input device to function as a communication unit, a vibration control unit, and a sound output control unit for communicating with an information processing device, The information processing device sends a feedback parameter to the pen-type input device based on the detection of the contact by the sensor unit for detecting the contact of the pen-type input device, wherein: The feedback parameters at least include parameters for controlling the vibration of the pen-type input device. The vibration control unit vibrates the vibration unit based on the received feedback parameter. The sound output control unit outputs sound from the sound output unit at a timing corresponding to the vibration of the vibration unit based on the received feedback parameter.
