Information processing system, information processing method, and program

Abstract

The goal is to make the experience of using a pen-type input device as close as possible to the feeling of actually writing on a writing medium. [Solution] The information processing system detects the speed and direction of movement of the pen tip of a pen-type input device on the target surface as contact parameters. If the direction has not reversed between a predetermined time before the current time and the current time, it calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time. If the direction has reversed between a predetermined time before the current time and the current time, it performs the calculation based on a formula in which the speed at the current time is substituted as zero. The information processing system is configured to drive the tactile reproduction unit of the pen-type input device based on the calculated feedback level.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] When inputting to an interactive display system, it is known to use a pen-shaped stylus (pen-type input device) capable of providing tactile feedback by generating vibrations that drive a tactile actuator as an input device (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When vibrations are generated by a pen-type input device, for example, when sliding the pen tip on a writing medium such as paper, a pseudo tactile sensation can be obtained. Therefore, a user inputting using a pen-type input device can obtain a feeling similar to when writing on a writing medium.

[0005] In the case of input using a pen-type input device, there is an aspect that it is preferable to be as close as possible to the feeling when actually writing on a writing medium. Therefore, in consideration of the above problems, an object of the present invention is to make the input using a pen-type input device closer to the feeling when actually writing on a writing medium.

Means for Solving the Problems

[0006] One aspect of the present invention that solves the above-mentioned problems is an information processing system comprising: a contact parameter detection unit that detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on a surface to be operated on; a feedback level derivation unit that acquires the speed at predetermined time intervals and derives a feedback level at predetermined time intervals based on the acquired speed, and calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time when the direction is in a non-reversal state where it has not reversed between a predetermined time before the current time and the current time, and calculates the feedback level based on the formula in which the speed at the current time is substituted as zero when the direction is in a reversed state where it has reversed between a predetermined time before the current time and the current time; and a drive control unit that drives a tactile reproduction unit in the pen-type input device based on the feedback level calculated by the feedback level derivation unit.

[0007] One aspect of the present invention is an information processing method in an information processing system, comprising: a contact parameter detection step in which a contact parameter detection unit detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on a surface to be operated on; a feedback level derivation step in which a feedback level derivation unit acquires the speed at predetermined time intervals and derives a feedback level at predetermined time intervals based on the acquired speed, wherein if the direction is in a non-reversal state where it has not reversed between a predetermined time before the current time and the current time, it calculates a feedback level corresponding to the current time based on a predetermined formula using the speed at the current time, and if the direction is in a reversed state where it has reversed between a predetermined time before the current time and the current time, it calculates based on the formula in which the speed at the current time is substituted as zero; and a drive control step in which a drive control unit drives a tactile reproduction unit in the pen-type input device based on the feedback level calculated in the feedback level derivation step.

[0008] One aspect of the present invention is a program for causing a computer, which is a pen-type input device in an information processing system comprising an information processing device that detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on an operating surface, and the pen-type input device that receives the contact parameters, to function as a feedback level derivation unit that acquires the speed at predetermined time intervals and derives a feedback level at predetermined time intervals based on the acquired speed, and when the direction is in a non-reversal state where it has not reversed between a predetermined time before the current time and the current time, it calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time, and when the direction is in a reversed state where it has reversed between a predetermined time before the current time and the current time, it calculates the feedback level based on the formula in which the speed corresponding to the current time is substituted as zero the program for causing a computer, which is a pen-type input device, to function as a feedback level derivation unit that drives a tactile reproduction unit in the pen-type input device based on the feedback level calculated by the feedback level derivation unit. [Effects of the Invention]

[0009] According to the present invention, when using a pen-type input device, it is possible to achieve an effect that closely resembles the feeling of actually writing on a writing medium. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the external configuration of the information processing system in the first embodiment. [Figure 2] This figure shows a specific example of handwriting using a pen-type input device in the first embodiment. [Figure 3] This figure shows an example of the speed detected in relation to the handwriting in Figure 2, and the vibration intensity of the pen-type input device calculated according to that speed. [Figure 4] This figure shows an example of vibration intensity correction in the first embodiment. [Figure 5] This figure shows an example of the hardware configuration of the information processing device and pen-type input device in the first embodiment. [Figure 6]This figure shows an example of the functional configuration of the information processing device and the pen-type input device in the first embodiment. [Figure 7] This figure shows an example of the functional configuration related to the drive of the vibration unit in the information processing system of the first embodiment. [Figure 8] This figure shows an example of a processing procedure performed by the information processing device and the pen-type input device in the first embodiment in relation to haptic feedback. [Figure 9] This figure shows an example of vibration intensity correction in the second embodiment. [Figure 10] This figure shows an example of a processing procedure performed by the information processing device and the pen-type input device in the second embodiment in relation to haptic feedback. [Modes for carrying out the invention]

[0011] <First Embodiment> Figure 1 shows an example of the external configuration of the information processing system of this embodiment. As shown in the figure, the information processing system of this embodiment includes an information processing device 100 and a pen-type input device 200.

[0012] The information processing device 100 is capable of performing information processing in response to input operations from the pen-type input device 200. The information processing device 100 in the figure is shown as an example of a tablet terminal or a notebook personal computer.

[0013] The information processing device 100 includes a touch panel display unit 30. The touch panel display unit 30 is a component that combines a touch panel and a display unit. The touch panel display unit 30 displays an image on the panel surface (display surface: an example of an operating surface) and allows operation by touching an operating object such as a pen-type input device or a finger to the panel surface.

[0014] The pen-type input device 200 is a pen-type input device used by the user to operate the touch panel on the touch panel display unit 30 of the information processing device 100. The user is made to perform handwriting input operations such as characters, pictures, graphics, etc. by gripping the pen-type input device 200 and moving it while touching the pen tip to the panel surface of the touch panel display unit 30.

[0015] In addition, as an operation using the pen-type input device 200, pointing operations on the user interface image displayed on the touch panel display unit 30 may also be enabled.

[0016] Note that the detection method of the pen-type input device 200 by the touch panel in the touch panel display unit 30 of the present embodiment is not particularly limited, and examples thereof include a capacitance method, an electromagnetic induction method, etc. In the following description, the case where the capacitance method is used will be taken as an example.

[0017] An application (pen operation corresponding application) corresponding to the input operation by the pen-type input device 200 is installed in the information processing apparatus 100. The pen operation corresponding application can, for example, display the characters and pictures drawn according to the handwriting input operations such as character input and picture drawing performed by touching the pen tip of the pen-type input device 200 to the touch panel display unit 30, or execute processing for converting the characters and pictures drawn by the operation into data.

[0018] Also, in the information processing system of the present embodiment, the pen-type input device 200 vibrates in response when a writing operation (writing operation) corresponding to writing is being performed. Due to the vibration generated by the pen-type input device 200, the user can feel a similar tactile sensation when writing on a writing medium such as paper, etc., and can obtain a sensation similar to when writing on the writing medium with a real writing tool.

[0019] In the following explanation, the vibration generated when writing is performed using the pen-type input device 200 will also be referred to as "tactile feedback." Furthermore, the control performed in response to the "tactile feedback" will also be referred to as "feedback control."

[0020] Next, the setting of vibration intensity in the feedback control in this embodiment will be described. Figure 2 shows an example of a handwriting HR produced by a user's writing operation. In the handwriting HR shown in the figure, the pen tip moves diagonally upward to the right from the starting point P0, reverses direction to the left and downward to the left when it reaches the turning point p1, reverses direction again to the right and upward to the right when it reaches the turning point p2, reverses direction again to the left and downward to the left when it reaches the turning point p3, and completes writing when it reaches the ending point p4.

[0021] In the case of this type of handwriting (HR), at points p1, p2, and p3 where the direction of pen tip movement reverses in the left-right direction, the pen tip temporarily stops moving and its speed becomes zero. When providing haptic feedback through vibration in response to writing operations that produce this type of handwriting (HR), increasing the vibration intensity as the speed of pen tip movement increases makes it easier to obtain a sensation similar to writing on a writing medium with a real writing instrument.

[0022] Figure 3 shows the relationship between the pen tip speed detected by the touch panel display unit 30, corresponding to the handwriting HR in Figure 2, and the vibration intensity (calculated vibration intensity) calculated using the detected pen tip speed. Specifically, in the figure, the velocity detected at time t(n) at predetermined time intervals is shown as the X-axis velocity Vx and Y-axis velocity Vy on a two-dimensional coordinate system corresponding to the panel sensor in the touch panel display unit 30. The X-axis velocity Vx is a vector quantity corresponding to the X-axis direction (horizontal direction), where movement to the left is positive and movement to the right is negative. The X-axis velocity Vx is a vector quantity corresponding to the Y-axis direction (vertical direction), where movement upward is positive and movement downward is negative. Furthermore, the calculated vibration intensity Ve shown in the figure is calculated by a predetermined calculation that utilizes the velocity Vx in the X-axis direction and the velocity Vy in the Y-axis direction. For example, the calculated vibration intensity Ve[n] at time t(n) may be calculated using the following equation 1. Ve[n]=sqrt(x[n]^2+ y[n]^2) (Formula 1) In equation 1 above, "x[n]" is the velocity in the X-axis direction Vx at time t(n), and "y[n]" is the velocity in the Y-axis direction Vy at time t(n). Such equation 1 corresponds to an operation using the speeds in the X-axis and Y-axis directions as scalar quantities.

[0023] As shown in Figure 3, in the case of the handwriting HR in Figure 2, the Y-axis velocity Vy shows little change near zero, but the X-axis velocity Vx shows a relatively large change, with its absolute value increasing and decreasing in response to the state of the pen tip's movement, and becoming zero at the turning point. In this case, the calculated vibration intensity Ve for each time t(n) calculated by Equation 1 approximates the absolute value of the X-axis velocity Vx, as shown in the figure. The pen-type input device 200 vibrates due to this calculated vibration intensity Ve, resulting in relatively large vibrations when the left and right pen tips move along the left-right direction during the writing operation of the handwriting HR in Figure 2, and smaller vibrations near the turning point. These changes in vibration allow the user to obtain a sensation similar to writing on a writing medium with a real writing instrument.

[0024] In this embodiment, the speed detected by the touch panel display unit 30 is transmitted from the information processing device 100 to the pen-type input device 200 at predetermined time intervals t(n). The pen-type input device 200 substitutes the speed (X-axis velocity Vx, Y-axis velocity Vy) received at each time t(n) into Equation 1 to calculate the calculated vibration intensity Ve at each time t(n). In this case, as can be seen from Figure 3, for example, in actual writing, the X-axis velocity Vx reverses from positive to negative at time t(5-1) between time t(5) and time t(6). That is, at time t(5-1), the pen tip reaches the turning point and the velocity temporarily becomes zero. However, at both time t(5) and time t(6), the X-axis velocity Vx is not zero but has a relatively large absolute value, so the calculated oscillation intensity Ve is also a relatively large value. Furthermore, in actual writing, the X-axis velocity Vx reverses from negative to positive at time t(11-1) between time t(11) and time t(12). In other words, at time t(11-1), the pen tip reaches the turning point, and the velocity temporarily becomes zero. However, since the X-axis velocity Vx and Y-axis velocity Vy are not zero at both time t(11) and time t(12), the calculated oscillation intensity Ve is also not zero. Furthermore, during actual writing, the X-axis velocity Vx reverses from positive to negative at time t(16-1) between time t(16) and time t(17), causing the pen tip to reach a turning point and the velocity to temporarily become zero. However, at both time t(16) and time t(17), the X-axis velocity Vx is not zero but has a relatively large absolute value, resulting in a relatively large calculated oscillation intensity Ve. For this reason, during the periods from time t(5) to time t(6), from time t(11) to time t(12), and from time t(16) to time t(17), even though the pen tip has reached the turning point during actual writing and has temporarily stopped moving, a certain degree of vibration is still occurring, which may cause the user to feel uncomfortable. Therefore, in this embodiment, the above-mentioned discomfort is mitigated by correcting the calculated vibration intensity Ve as described below.

[0025] Figure 4 shows an example of a correction method for the calculated vibration intensity Ve in this embodiment. In this figure, the X-axis velocity Vx, Y-axis velocity Vy, and calculated vibration intensity Ve are shown as in Figure 3. Furthermore, Figure 4 shows the corrected vibration intensity Veec, which is obtained by correcting the calculated vibration intensity Ve. In this embodiment, the pen-type input device 200 calculates the vibration intensity at the current time t(n) by determining whether a reversal state has occurred between the previous time t(n-1) and time t(n), where the positive and negative signs of the X-axis velocity Vx and the Y-axis velocity Vy are reversed. If a reversal state has occurred for the X-axis velocity Vx, it corresponds to the direction of movement of the pen tip in the X-axis direction being reversed between time t(n-1) and time t(n). If a reversal state has occurred for the Y-axis velocity Vy, it corresponds to the direction of movement of the pen tip in the Y-axis direction being reversed between time t(n-1) and time t(n). If it is determined that no inversion has occurred (i.e., it is in a non-inversion state), the pen-type input device 200 calculates the corrected vibration intensity Vec by directly substituting the values ​​of the X-axis velocity Vx and Y-axis velocity Vy received at time t(n) into Equation 1. On the other hand, if it is determined that an inversion state has occurred, the pen-type input device 200 converts the velocity in the X-axis direction Vx and the Y-axis direction Vy received at time t(n) that caused the inversion state to zero, and then substitutes this into Equation 1 to calculate the corrected vibration intensity Vec.

[0026] Specifically, for example, if time t(4) in Figure 4 is the current time, then during the period from the previous time t(3) to time t(4), both the X-axis velocity Vx and the Y-axis velocity Vy are in a non-reversed state. In this case, the pen-type input device 200 calculates the corrected vibration intensity Vec by substituting the values ​​of the X-axis velocity Vx and the Y-axis velocity Vy received at time t(4) into Equation 1. In this case, the calculated value will be the same as the calculated vibration intensity Ve.

[0027] On the other hand, if time t(6) in Figure 4 is the current time, then in the period from the previous time t(5) to time t(6), both the X-axis velocity Vx and the Y-axis velocity Vy have reversed at time t(5-1). In this case, the pen-type input device 200 does not use the values ​​of the X-axis velocity Vx and Y-axis velocity Vy received at time t(6), but instead substitutes zero for the values ​​of the X-axis velocity Vx and Y-axis velocity Vy in Equation 1 to calculate the corrected vibration intensity Vec. In this case, the value calculated as the corrected vibration intensity Vec will be closer to zero than the calculated vibration intensity Ve calculated at the same time t(6), as shown in Figure 4.

[0028] Furthermore, if time t(12) in Figure 4 is the current time, then in the period from the previous time t(11) to time t(12), both the X-axis velocity Vx and the Y-axis velocity Vy are inverted at time t(11-1). For this reason, the pen-type input device 200 does not use the values ​​of the X-axis velocity Vx and Y-axis velocity Vy received at time t(12), but instead substitutes zero for the values ​​of the X-axis velocity Vx and Y-axis velocity Vy in Equation 1 to calculate the corrected vibration intensity Vec. In this case as well, the value calculated as the corrected vibration intensity Vec will be closer to zero than the calculated vibration intensity Ve calculated at the same time t(12).

[0029] Furthermore, if time t(17) in Figure 4 is the current time, then in the period from the previous time t(16) to time t(17), both the X-axis velocity Vx and the Y-axis velocity Vy are inverted at time t(16-1). For this reason, the pen-type input device 200 does not use the values ​​of the X-axis velocity Vx and Y-axis velocity Vy received at time t(17), but instead substitutes zero for the values ​​of the X-axis velocity Vx and Y-axis velocity Vy in Equation 1 to calculate the corrected vibration intensity Vec. In this case as well, the value calculated as the corrected vibration intensity Vec will be closer to zero than the calculated vibration intensity Ve calculated at the same time t(17).

[0030] The pen-type input device 200 of this embodiment performs feedback control to vibrate itself according to the corrected vibration intensity Vec calculated as described above. This feedback control makes it possible to reduce the vibration of the pen-type input device 200 to zero or significantly at a timing close to the turning point in the direction of movement of the pen tip during actual writing. As a result of obtaining such vibration, the vibration of the pen-type input device is suppressed in response to the temporary cessation of pen tip movement upon reaching the turning point during actual writing, making it less likely for the user to feel discomfort.

[0031] Furthermore, when a user is drawing a circle as part of a writing operation, the pen tip is constantly moving at a speed above a certain level. In this situation, the sign of the X-axis velocity and the Y-axis velocity reverse at different timings. In this case, at the timing when the X-axis velocity reverses and the timing when the sign of the Y-axis velocity reverses, the pen tip is moving at a speed above a certain level, but a value of zero for the X-axis velocity or Y-axis velocity is substituted into Equation 1, resulting in an error where the value of the corrected vibration intensity Vec becomes small. However, when drawing a circle, even if either the velocity in the X-axis direction or the velocity in the Y-axis direction reverses, the other velocity can maintain a relatively large value. Therefore, the value of the corrected vibration intensity Vec does not become extremely small, and the user is unlikely to notice any discomfort.

[0032] Referring to Figure 5, an example of the hardware configuration of the information processing device 100 and the pen-type input device 200 will be described. First, an example of the hardware configuration of the information processing device 100 will be described. The information processing device 100 in the figure comprises a processor 11, main memory 12, flash memory 13, peripheral devices 14, a short-range communication unit 15, a baseband chip 21, a communication unit 22, an audio system 23, a microphone 24, a speaker 25, and a touch panel display unit 30.

[0033] The processor 11 is, for example, an application processor including a CPU (Central Processing Unit). The processor 11 controls the entire information processing device 100.

[0034] Main memory 12 is writable memory used as a reading area for the executable program of the processor 11, or as a work area for writing processing data of the executable program. Main memory 12 is composed of, for example, multiple DRAM (Dynamic Random Access Memory) chips. This executable program includes the OS (Operating System), various device drivers for hardware operation of peripheral devices, various services / utilities, application programs (application software), etc.

[0035] The flash memory 13 is, for example, a flash EEPROM (Electrically Erasable Programmable Read Only Memory) and stores the OS, various drivers, various services / utilities, application programs (hereinafter sometimes referred to as applications), and various data.

[0036] The touch panel display unit 30 displays images and allows operation of the panel surface on which the image is displayed using a pen-type input device 200. The touch panel display unit 30 may also be operable with other operating bodies such as a finger, but here we will explain using the case where the operating body is a pen-type input device 200 as an example. The touch panel display unit 30 comprises a display unit 31 and a panel sensor 32. The display unit 31 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays an image based on drawing data (display data) output from the processor 11. The panel sensor 32 detects the contact state of the pen tip of the pen-type input device 200 with the panel surface of the display unit 31. Specifically, the panel sensor 32 may be capable of detecting the position where the pen tip of the pen-type input device 200 is in contact with the panel surface, the pressure exerted by the pen tip in contact with the panel surface, and the distance of the pen-type input device 200 from the panel surface. The detection method of such a panel sensor 32 is not particularly limited, but examples include a capacitive method or an electromagnetic induction method. Furthermore, the panel sensor 32 may be configured to use a combination of such detection methods.

[0037] Peripheral devices 14 include, for example, a WLAN (Wireless Local Area Network) module, a GPS (Global Positioning System) module, and sensors such as an accelerometer.

[0038] The audio system 23 is, for example, an audio IC (Integrated Circuit) that performs input, recording, playback, and output of sound data. The audio system 23 is connected to, for example, a microphone 24 and a speaker 25. The audio system 23 outputs the sound data picked up by, for example, the microphone 24 to the processor 11 or the baseband chip 21. Furthermore, the audio system 23 converts sound data acquired from, for example, the processor 11 or the baseband chip 21 into an audio signal and outputs it to the speaker 25.

[0039] The microphone 24 picks up sounds from the vicinity of the information processing device 100. For example, when synchronizing voices with other terminals, the microphone 24 picks up sounds such as the user's voice. Speaker 25 outputs various sounds to the outside of the information processing device 100. For example, when synchronizing voices with other terminals, speaker 25 outputs (breathes out) sounds received from other terminals.

[0040] The baseband chip 21 is a dedicated IC that controls wireless communication such as 4G (fourth-generation mobile communication system) and 5G (fourth-generation mobile communication system). The baseband chip 21, for example, receives sound data from the communication unit 22 and outputs it as sound from the speaker 25 via the audio system 23. Furthermore, the baseband chip 21 acquires sound data picked up by the microphone 24, for example, via the audio system 23, and outputs it via the mobile communication system via the communication unit 22. The baseband chip 21 may also be configured to exchange input and output data for data communication by the mobile communication system with the processor 11.

[0041] The communication unit 22 is a wireless communication device, including an antenna, for performing wireless communication using a mobile communication system.

[0042] The short-range communication unit 15 is, for example, a Bluetooth® module and performs short-range wireless communication with the pen-type input device 200.

[0043] Next, referring to the same Figure 3, we will describe an example of the hardware configuration of the pen-type input device 200. The pen-type input device 200 includes an MCU 41, a short-range communication unit 42, a vibration unit 43 (an example of a tactile feedback unit), and a flash memory 45.

[0044] The short-range communication unit 42 is, for example, a Bluetooth® module and performs short-range wireless communication with the information processing device 100.

[0045] The MCU (Micro Controller Unit) 41 includes a CPU, memory such as ROM and RAM, I / O-related components, and performs control of the pen-type input device 200. The MCU 41 handles the exchange of information transmitted and received by the short-range communication unit 42. The MCU 41 also controls the output of writing sounds caused by vibrations generated in the vibration unit 43, so that tactile feedback is provided based on contact parameters detected by the information processing device 100.

[0046] The vibrating part 43 is, for example, equipped with an actuator and vibrates in accordance with the control of the MCU 41.

[0047] The flash memory 45 stores various types of data compatible with the pen-type input device 200, such as programs.

[0048] Referring to Figure 6, an example of the functional configuration of the information processing device 100 and the pen-type input device 200 will be described. In this figure, functional parts that can be considered equivalent to the parts in the hardware configuration of Figure 5 are given the same reference numerals as in Figure 5, and their descriptions are omitted as appropriate. The functions of the information processing device 100 shown in the figure may be realized by the processor 11 (Figure 5) executing a program.

[0049] First, an example of the functional configuration of the information processing device 100 will be described. The information processing device 100 includes, as functional units, a communication unit 22, a touch panel display unit 30, a contact parameter detection unit 101, a control unit 102, and a storage unit 103.

[0050] The contact parameter detection unit 101 detects the contact state of the pen tip of the pen-type input device 200 with the panel surface of the touch panel display unit 30. The contact parameter detection unit 101 outputs parameters (contact parameters) that correspond to the contact state based on the detection signal output by the panel sensor 32. In the following description, the contact parameters are given as an example, for example, velocity (velocity in the X-axis direction, velocity in the Y-axis direction) and tilt angle.

[0051] The control unit 102 performs various controls on the information processing device 100. The control unit 102 includes an application-responsive processing unit 121 and a feedback-responsive processing unit 122.

[0052] The application-compatible processing unit 121 executes processing corresponding to pen-operated applications. When a writing operation is performed using the pen-type input device 200 as an operation for a pen-operated application, pen operation information is input from the panel sensor 32 to the application-compatible processing unit 121. The application-compatible processing unit 121 performs processing, such as drawing, in response to the input of pen operation information.

[0053] The feedback-responding processing unit 122 executes the processing that the information processing device 100 is responsible for in response to feedback control. In this embodiment, the feedback-responding processing may be the process of transmitting the contact parameters detected by the contact parameter detection unit 101 from the communication unit 22 to the pen-type input device 200.

[0054] In this embodiment, for example, the communication period from the information processing device 100 to the pen-type input device 200 may be different from the communication period from the pen-type input device 200 to the information processing device 100. As a specific example, the communication period from the information processing device 100 to the pen-type input device 200 may be 60 Hz, and the communication period from the pen-type input device 200 to the information processing device 100 may be 300 Hz.

[0055] The memory unit 103 stores various types of information corresponding to the information processing device 100.

[0056] Next, an example of the functional configuration of the pen-type input device 200 will be described. The functions of the pen-type input device 200 shown in the figure may be realized by the MCU 41 (Figure 5) executing a program. The pen-type input device 200 includes, as functional units, a short-range communication unit 42, a vibration unit 43, a control unit 201, and a storage unit 202.

[0057] The control unit 201 performs various controls on the pen-type input device 200. The control unit 201 includes a feedback level derivation unit 211 and a drive control unit 212.

[0058] The feedback level derivation unit 211 derives the feedback level. The feedback level derivation unit 211 calculates the corrected vibration intensity Vec as the feedback level.

[0059] The drive control unit 212 controls the vibration unit 43 so that vibrations are generated as tactile feedback. The drive control unit 212 drives the vibration unit 43 using the corrected vibration intensity Vec calculated by the feedback level derivation unit 211.

[0060] The memory unit 202 stores information corresponding to the pen-type input device 200. The memory unit 202 in the figure includes a vibration waveform data storage unit 221. The vibration waveform data storage unit 221 stores vibration waveform data. The vibration waveform data is waveform data created to reproduce the vibrations that actually occur when writing with a writing instrument using the pen-type input device 200.

[0061] Referring to Figure 7, an example of a functional configuration related to the driving of the vibration unit 43 based on the corrected vibration intensity Vec will be described. In this figure, the same reference numerals are used for parts that are the same as in Figure 6, and explanations are omitted as appropriate. In the information processing device 100, the contact parameter detection unit 101 includes a speed detection unit 111. The speed detection unit 111 detects the speed of the pen tip as it moves while in contact with the panel surface. The speed detected by the speed detection unit 111 is a vector quantity that indicates the speed and direction of movement of the pen tip in the X-axis direction and the Y-axis direction, respectively. The speed detection unit 111 may detect the speed based on the amount of movement and direction of movement of the contact position of the pen tip per unit time detected by the panel sensor 32. The speed detection unit 111 outputs the detected speed to the feedback processing unit 122.

[0062] In the information processing device 100, the feedback processing unit 122 transmits the contact parameter (velocity) detected by the contact parameter detection unit 101 to the pen-type input device 200 at time intervals t(n).

[0063] The contact parameters detected by the contact parameter detection unit 101 may be used by the application-responsive processing unit 121 (Figure 6) for drawing according to the writing.

[0064] Next, an example of the functional configuration of the pen-type input device 200 will be described. The functions of the pen-type input device 200 shown in the figure may be realized by the MCU 41 (Figure 5) executing a program. The pen-type input device 200 includes, as functional units, a feedback level derivation unit 211, a drive control unit 212, a vibration waveform data storage unit 221, and a vibration unit 43.

[0065] The feedback level derivation unit 211 calculates the corrected vibration intensity Vec at each time t(n) using the speed transmitted from the information processing device 100 at each time t(n). The drive control unit 212 drives the vibration unit 43 using the corrected vibration intensity Vec calculated by the feedback level derivation unit 211.

[0066] The figure shows an example of the configuration of the vibrating section 43. The vibrating unit 43 includes an amplifier 431 and an actuator 432. The drive control unit 212 inputs the vibration waveform data stored in the vibration waveform data storage unit 221 as a signal source to the amplifier 431. The drive control unit 212 sets the drive level of the amplifier 431 according to the corrected vibration intensity Vec calculated by the feedback level derivation unit 211. The amplifier 431 amplifies the input signal source at the set drive level and outputs it to the actuator 432. The actuator 432 vibrates according to the waveform of the input signal source and the set drive level.

[0067] Referring to the flowchart in Figure 9, an example of a processing procedure performed by the information processing device 100 and the pen-type input device 200 in relation to haptic feedback will be described. First, an example of the processing procedure of the information processing device 100 will be described. When the processing shown in the figure is being executed, the speed detection unit 111 may continuously detect the speed.

[0068] First, we will explain an example of a processing procedure executed by the information processing device 100. Step S100: In the information processing device 100, the feedback-responsive processing unit 122 waits for the tip of the pen-type input device 200 to come into contact with the panel surface and begin to move (movement start state) based on the contact parameters acquired at the current sample timing. In this case, the feedback processing unit 122 may determine that the pen tip has started moving, for example, when the speed detected by the speed detection unit 111 changes from zero to a value greater than zero.

[0069] Step S102: When the feedback processing unit 122 determines in step S100 that the pen tip has started to move, it acquires the speed (X-axis speed Vx, Y-axis speed Vx) detected by the speed detection unit 111 at the timing t(n) corresponding to the current time.

[0070] Step S104; The feedback processing unit 122 transmits the speed obtained in step S102 to the pen-type input device 200.

[0071] Step S106: The feedback response processing unit 122 determines whether the detection of the pen tip, which was started in accordance with step S100, has stopped. If it is determined that the detection of the pen tip has stopped, the process returns to step S102. In this case, the feedback-responsive processing unit 122 repeatedly executes the processes in steps S102 to S106 until the movement of the pen tip stops. On the other hand, if it is determined that the movement of the pen tip has stopped, the process returns to step S100. By returning to step S100 in this way, the transmission of speed to the pen-type input device 200 is stopped until the movement of the pen tip starts again.

[0072] Next, we will describe an example of a processing procedure performed by the pen-type input device 200. Step S200: In the pen-type input device 200, the drive control unit 212 receives the speed transmitted from the information processing device 100 in step S110.

[0073] Step S202: When the velocity is received, the feedback level derivation unit 211 in the pen-type input device 200 determines whether the X-axis velocity Vx(n) at the velocity received corresponding to time t(n) is in a reversed state. That is, it determines whether the X-axis velocity Vx(n) at time t(n) has a reversed sign from the X-axis velocity Vx(n-1) at the previous time t(n).

[0074] Step S204: If it is determined in step S202 that the X-axis velocity Vx(n) is in a reversed state, the feedback level derivation unit 211 sets the X-axis velocity Vx(n) at time t(n) to "0".

[0075] Step S206: If it is determined in step S202 that the X-axis velocity Vx(n) is in a non-reversed state, or after processing in step S204, the feedback level derivation unit 211 determines whether the Y-axis velocity Vy(n) at time t(n) is in a reversed state at the received velocity. In other words, it determines whether the Y-axis velocity Vy(n) at time t(n) has reversed in sign from the Y-axis velocity Vy(n-1) at the previous time t(n).

[0076] Step S208: If it is determined in step S206 that the Y-axis velocity Vy(n) is in a reversed state, the feedback level derivation unit 211 sets the Y-axis velocity Vy(n) at time t(n) to "0".

[0077] Step S210: If it is determined in step S206 that the Y-axis velocity Vy(n) is in a non-reversal state, or after the processing in step S208, the feedback level derivation unit 211 substitutes the values ​​of the X-axis velocity Vx(n) and the Y-axis velocity Vy(n) obtained in the previous processing into Equation 1 to calculate the corrected vibration intensity Vec at time t(n).

[0078] Step S212: The drive control unit 212 sets a drive level for the amplifier 431, which has input vibration waveform data, according to the corrected vibration intensity Vec calculated in step S210. After the processing in step S212, the process returns to step S200.

[0079] <Second Embodiment> Next, a second embodiment will be described. In the example shown in Figure 4, the corrected vibration intensity Vec (and calculated vibration intensity Ve) calculated at time t(11) is relatively close to zero. At the next time t(12), the velocity is reversed during the period from time t(11) to time t(12), so the calculated corrected vibration intensity Vec is also zero. In this case, the vibration of the pen-type input device 200 becomes smaller during the period from time t(11) to time t(12), which may cause the user to feel uncomfortable. Therefore, the pen-type input device 200 of this embodiment is configured to calculate the corrected vibration intensity Vec as follows.

[0080] Figure 10 shows the velocity (velocity in the X-axis direction Vx, velocity in the Y-axis direction Vy), calculated vibration intensity Ve, and corrected vibration intensity Vec corresponding to the calculation method for corrected vibration intensity Vec in this embodiment. In this figure, the velocity (velocity in the X-axis direction Vx, velocity in the Y-axis direction Vy) and calculated vibration intensity Ve are the same as in Figure 4. As shown in the figure, in this embodiment, a threshold range BD is set for the velocity (velocity Vx in the X-axis direction, velocity Vy in the Y-axis direction). The threshold range BD is defined as a predetermined numerical range with zero velocity as the reference, and a positive region and a negative region. In the figure, the threshold range BD is defined with the same numerical range for the positive region and the same numerical range for the negative region, but it is acceptable to set the numerical range for the positive region and the numerical range for the negative region to be different. In the figure, an example is shown where a common threshold range BD is set for the X-axis velocity Vx and the Y-axis velocity Vy; however, threshold ranges may be defined individually for the X-axis velocity Vx and the Y-axis velocity Vy.

[0081] In this embodiment, when the feedback level derivation unit 211 determines that a reversal state has occurred for either the X-axis velocity Vx(n) or the Y-axis velocity Vy(n) at the current time t(n), it determines whether both the X-axis velocity Vx(n-1) and the Y-axis velocity Vy(n-1) at the previous time t(n-1) are within the threshold range BD. The state in which both the X-axis velocity Vx(n-1) and the Y-axis velocity Vy(n-1) are within the threshold range BD indicates that the pen tip speed is considerably low (low speed state). In other words, in this case, the feedback level derivation unit 211 determines whether or not the pen is in a low speed state. If the feedback level derivation unit 211 determines that time t(n-1) is a low-speed state, it calculates the corrected vibration intensity Vec by substituting the values ​​of the X-axis velocity Vx(n) and Y-axis velocity Vy(n) detected by the velocity detection unit 111 into Equation 1, even though an inversion state has occurred at the current time t(n). On the other hand, if the feedback level derivation unit 211 determines that time t(n-1) is not a low-speed state, it calculates the corrected vibration intensity Vec by substituting the velocity at which an inversion state has occurred at the current time t(n) as zero into Equation 1.

[0082] Specifically, in the example shown in Figure 10, if time t(6) is the current time, the X-axis velocity Vx and Y-axis velocity Vy are in a reversed state during the period from the previous time t(5) to the current time t(6). At time t(5), the Y-axis velocity Vy is within the threshold range BD, but the X-axis velocity Vx is outside the threshold range BD. In other words, neither the X-axis velocity Vx nor the Y-axis velocity Vy are within the threshold range BD. Therefore, in this case, the feedback level derivation unit 211 calculates the corrected vibration intensity Vec at time t(6) by substituting the X-axis velocity Vx and Y-axis velocity Vy as zero into Equation 1, similar to the first embodiment.

[0083] Furthermore, if time t(12) is the current time, the X-axis velocity Vx and Y-axis velocity Vy are in a reversed state during the period from the previous time t(11) to the current time t(12). At time t(11), both the X-axis velocity Vx and Y-axis velocity Vy are within the threshold range BD. Therefore, in this case, the feedback level derivation unit 211 does not set the X-axis velocity Vx and Y-axis velocity Vy to zero, but substitutes the values ​​detected by the velocity detection unit 111 into equation 1 to calculate the corrected vibration intensity Vec at time t(12).

[0084] By calculating the corrected vibration intensity Vec in this way, the corrected vibration intensity Vec will no longer remain at a low value from time t11 to time t12, thereby mitigating the discomfort experienced by the user.

[0085] Referring to the flowchart in Figure 10, an example of a processing procedure performed by the information processing device 100 and the pen-type input device 200 in relation to the haptic feedback of this embodiment will be described. The processing of the information processing device 100 in steps S300 to S306 is the same as in steps S100 to S106 in Figure 8.

[0086] Next, we will describe an example of a processing procedure performed by the pen-type input device 200. Steps S400 and S402 are the same as steps S200 and S202 in Figure 8.

[0087] Step S404: If it is determined in step S402 that the X-axis velocity Vx(n) is in a reversed state, the feedback level derivation unit 211 further determines whether the previous time t(n-1) was in a low-speed state. In other words, the feedback level derivation unit 211 determines whether both the X-axis velocity Vx(n-1) and the Y-axis velocity Vy(n-1) at time t(n-1) are within the threshold range BD.

[0088] Step S406: If it is determined in step S404 that the low-speed state was not present, the feedback level derivation unit 211 sets the X-axis velocity Vx(n) at time t(n) to "0".

[0089] Step S408: If it is determined in step S402 that the X-axis velocity Vx(n) is not in a reversed state, if it is determined in step S404 that it is in a low-speed state, or after the processing in step S406, the feedback level derivation unit 211 determines whether the Y-axis velocity Vy(n) at the velocity received corresponding to time t(n) is in a reversed state.

[0090] Step S410: If it is determined in step S408 that the Y-axis velocity Vy(n) is in a reversed state, the feedback level derivation unit 211 further determines whether the previous time t(n-1) was in a low-speed state.

[0091] Step S412: If it is determined in step S410 that the speed was low, the feedback level derivation unit 211 sets the Y-axis velocity Vy(n) at time t(n) to "0".

[0092] If it is determined in step S408 that the Y-axis velocity Vy(n) is not in a reversed state, if it is determined in step S410 that it is in a low-speed state, or after the processing in step S412, the feedback level derivation unit 211 executes the processes in steps S414 and S416. The processes in steps S414 and S416 are the same as the processes in steps S210 and S212 in Figure 8.

[0093] The feedback level derivation unit 211 may be provided in the information processing device 100. In this case, the information processing device 100 may configure the feedback level derivation unit 211 to calculate the corrected vibration intensity Vec and transmit the calculated corrected vibration intensity Vec from the information processing device 100 to the drive control unit 212 of the pen-type input device 200. However, in such a configuration, there may be a large discrepancy between the timing of drawing the trajectory corresponding to the writing operation and the timing of the vibration of the pen-type input device 200. To mitigate such a timing discrepancy, it is preferable to provide the function of the feedback level derivation unit 211 in the pen-type input device 200, as in the above embodiments.

[0094] In each of the above embodiments, the velocity detection unit 111 used the velocity as a vector quantity to determine the occurrence of an inversion state in which the detected velocity value reversed in sign. In this embodiment, for example, the speed and direction of movement as scalar quantities may be detected in the X-axis direction and the Y-axis direction, respectively, in response to the movement of the pen tip. In this case, for example, the corrected vibration intensity Vec may be calculated by using an equation that finds the square root of the sum of the squared value of the speed in the X-axis direction and the squared value of the speed in the Y-axis direction, and substituting the speed in the corresponding axis direction as zero in response to the occurrence of an inversion state in which the detected direction of movement reverses.

[0095] Furthermore, tactile feedback may be provided not only by vibration but also by sound. The sound may be output from the pen-type input device 200 or from the information processing device 100. In this case, the sound level may also be corrected by the same control as the corrected vibration intensity Vec.

[0096] Furthermore, the information processing device 100 and the pen-type input device 200 described above may be performed by recording a program for realizing the functions of the information processing device 100 and the pen-type input device 200 on a computer-readable recording medium, loading the program recorded on this recording medium into a computer system, and executing it. Here, "loading the program recorded on the recording medium into a computer system and executing it" includes installing the program into the computer system. Here, "computer system" includes hardware such as the OS and peripheral devices. Also, "computer system" may include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Also, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Thus, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. Also, the recording medium includes internal or external recording media that can be accessed from the distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code in a format executable by the terminal device. In other words, the format in which the program is stored on the distribution server is irrelevant, as long as it can be downloaded from the distribution server and installed in a form that can be executed on the terminal device. Furthermore, the program may be divided into multiple parts, each downloaded at a different time and then combined on the terminal device, and each divided program may be distributed by a different distribution server. In addition, "computer-readable recording medium" includes volatile memory (RAM) within computer systems that act as servers or clients when a program is transmitted over a network, which retains the program for a certain period of time. Moreover, the above program may be intended to implement only a part of the functions described above.Furthermore, the above-mentioned functions may be implemented in combination with programs already recorded in the computer system, such as so-called differential files (differential programs). [Explanation of Symbols]

[0097] 11 Processor, 12 Main memory, 13 Flash memory, 14 Peripheral devices, 15 Short-range communication unit, 21 Baseband chip, 22 Communication unit, 23 Audio system, 24 Microphone, 25 Speaker, 30 Touch panel display unit, 31 Display unit, 32 Panel sensor, 42 Short-range communication unit, 43 Vibration unit, 45 Flash memory, 100 Information processing unit, 101 Contact parameter detection unit, 102 Control unit, 103 Storage unit, 111 Speed ​​detection unit, 121 Application-compatible processing unit, 122 Feedback-compatible processing unit, 200 Pen-type input device, 201 Control unit, 202 Storage unit, 211 Feedback level derivation unit, 212 Drive control unit, 221 Vibration waveform data storage unit, 431 Amplifier, 432 Actuator

Claims

1. A contact parameter detection unit detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on the surface to be operated on, A feedback level derivation unit that acquires the speed at predetermined time intervals and derives a feedback level at predetermined time intervals based on the acquired speed, calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time if the direction is in a non-reversal state where it has not reversed between a predetermined time interval prior to the current time and the current time, and calculates the feedback level based on the formula obtained by substituting the speed at the current time as zero if the direction is in a reversed state where it has reversed between a predetermined time interval prior to the current time and the current time, A drive control unit drives the haptic reproduction unit in the pen-type input device based on the feedback level calculated by the feedback level derivation unit. Equipped with, The above formula represents the vibration intensity, and it becomes zero when the speed at the current time is zero. Information processing system.

2. The tactile reproduction unit is a vibrating unit in the pen-type input device that vibrates in accordance with the vibration intensity, which is the feedback level. The information processing system according to claim 1.

3. The contact parameter detection unit detects the speed and direction as the speed and direction of the pen tip in the X-axis direction and the Y-axis direction in a two-dimensional coordinate system corresponding to the surface being operated on. The feedback level derivation unit calculates the feedback level using the formula which takes the square root of the sum of the squared value of the velocity in the X-axis direction and the squared value of the velocity in the Y-axis direction. The feedback level is calculated based on the formula obtained by determining whether the velocity in the X-axis direction and the velocity in the Y-axis direction are in an inverted or non-inverted state, substituting the detected value for the non-inverted velocity, and substituting zero for the detected value for the velocity determined to be in an inverted state. The information processing system according to claim 1 or 2.

4. The feedback level derivation unit calculates the feedback level corresponding to the current time based on the formula obtained by substituting the speed at the current time if, in the case of the inverted state, the speed at a predetermined time prior to the current time was within a predetermined threshold range. The information processing system according to claim 1 or 2.

5. An information processing method in an information processing system, The contact parameter detection unit performs a contact parameter detection step in which it detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on the surface to be operated, The feedback level derivation unit acquires the speed at predetermined time intervals and derives the feedback level at predetermined time intervals based on the acquired speed. In the case of a non-reversal state where the direction has not reversed between a predetermined time before the current time and the current time, it calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time. In the case of a reversed state where the direction has reversed between a predetermined time before the current time and the current time, it calculates the feedback level based on the formula in which the speed corresponding to the current time is substituted as zero. The drive control unit performs a drive control step in which it drives the haptic reproduction unit in the pen-type input device based on the feedback level calculated in the feedback level derivation step. Equipped with, The above formula represents the vibration intensity, and it becomes zero when the speed at the current time is zero. Information processing methods.

6. An information processing system comprising an information processing device that detects the speed and direction of movement of the pen tip of a pen-type input device as contact parameters on a target surface, and the pen-type input device that receives the contact parameters, wherein the computer is used as the pen-type input device. A feedback level derivation unit that acquires the speed at predetermined time intervals and derives a feedback level at predetermined time intervals based on the acquired speed, calculates the feedback level corresponding to the current time based on a predetermined formula using the speed at the current time if the direction is in a non-reversal state where it has not reversed between a predetermined time interval prior to the current time and the current time, and calculates the feedback level based on the formula obtained by substituting the speed at the current time as zero. The drive control unit drives the haptic reproduction unit in the pen-type input device based on the feedback level calculated by the feedback level derivation unit. It is a program designed to function as such. The above formula represents the vibration intensity, and it becomes zero when the speed at the current time is zero. program.

Images