System for providing interaction

Abstract

The present invention relates to an interaction providing system and may include: a spatiotemporal tactile output device comprising a button to which a user's finger contacts, generating state information of the user based on a physical force applied to the button, and providing a spatiotemporal tactile environment based on state change information corresponding to the state information; and a computing device that receives the state information, generates the state change information based on the state information, and provides the state change information to the spatiotemporal tactile output device so that the spatiotemporal tactile environment is provided in a feedforward form or a feedback form.

Description

Technology Field

[0001] The present invention relates to a technology that provides information necessary for interaction. Background Technology

[0003] Buttons are primarily used to operate various electronic devices, and a single button can become an interface that performs different functions depending on the context of use or the way it interacts. For example, in the context of use where a phone is ringing, a single press of a smartphone's screen lock button performs the function of changing the phone to silent mode, while a double press performs the function of hanging up the phone.

[0004] In addition, while products or research adding visual displays to each button of such keyboards are being discussed, there are limitations in providing accurate information because users must constantly check the button to receive information via the visual display, and fingers may obscure the button display while pressing the button.

[0005] Meanwhile, technologies that provide haptic feedback to the finger pressing the button, such as those adding vibration actuators or force actuators to the button, have the advantage of being able to provide information even without visually checking the button and while the button is being pressed. However, in the case of vibration actuators or force actuators, there are limitations in information transmission capabilities because it is ambiguous what information a specific force pattern or vibration pattern represents.

[0006] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as publicly known technology disclosed to the general public prior to the filing of the present invention. Prior art literature

[0008] Published Patent Application No. 10-2013-0024420 (March 8, 2013) The problem to be solved

[0009] In an embodiment of the present invention, we propose a technology capable of providing feedforward and feedback-type interactions through a button of an electronic device.

[0010] In an embodiment of the present invention, we propose a button structure for an electronic device that can provide feedforward and feedback through spatiotemporal tactile output.

[0011] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems to be solved will be clearly understood by those skilled in the art to which the present invention pertains from the descriptions below. means of solving the problem

[0013] According to an embodiment of the present invention, a spatiotemporal tactile output device comprising a button contacted by a user's finger, generating state information of the user based on a physical force applied to the button, and comprising a plurality of tactile actuators disposed on the button; and a computing device that receives the state information, generates state change information based on the state information, and provides the state change information to the spatiotemporal tactile output device so that the spatiotemporal tactile environment is provided in a feedforward form or a feedback form; wherein the spatiotemporal tactile output device provides spatial tactile stimulation by activating one or more selected tactile actuators among the plurality of tactile actuators based on the state change information corresponding to the operation state of the button, and provides the spatiotemporal tactile environment in which the spatial tactile stimulation changes over time to the user may be provided with an interaction providing system.

[0014] Here, the spatiotemporal tactile output device includes a sensor that detects the input time and pressing depth of the button and provides the state information according to the input time and pressing depth to the computing device, and the plurality of tactile actuators can transmit the spatiotemporal tactile environment to the user in the form of the feedforward or the feedback.

[0015] In addition, the spatiotemporal tactile output device may include a Braille display device.

[0016] Additionally, the computing device may include: an acquisition unit for acquiring the state information; a state processing unit for generating the state change information based on the state information; and an interaction part for transmitting the state information to the state processing unit and transmitting the state change information to the plurality of tactile actuators of the spatiotemporal tactile output device.

[0017] In addition, the plurality of tactile actuators can transmit the spatiotemporal tactile environment related to the operation of the button to the user in the form of feedback while the user is pressing the button.

[0018] Additionally, the button includes a power button of a notebook computer or a desktop computer, and the plurality of tactile actuators activate a pin of the braille display device according to the input time of the power button, wherein the activation can be applied gradually from the center point of the power button to the edge.

[0019] Additionally, the button includes a control button of a vehicle seat heating control assembly, and the plurality of tactile actuators activate a pin of the braille display device according to the heating level of the vehicle seat heating control assembly, wherein the activation can be selectively applied to any one of the lower point, the central point, and the upper point of the control button according to the heating level.

[0020] In addition, the plurality of tactile actuators can transmit the spatiotemporal tactile environment related to the operation of the button to the user in the form of the feedforward before the user presses the button.

[0021] Additionally, the button includes an input button of a game controller, and the plurality of tactile actuators activate a pin of the Braille display device according to the function of the input button when the user's finger comes into contact with the input button, and the activation can be selectively applied to a vertical line or a horizontal line of the input button according to the movement of a character on an application executed through the interaction providing system.

[0022] Additionally, the above button includes a volume control button of a portable terminal, and the computing device may switch the volume control button to a shooting function button when the user's finger touches the volume control button and the camera application of the portable terminal is activated, switch the volume control button to an alarm end function button when the user's finger touches the volume control button and the alarm application of the portable terminal is activated, and switch the volume control button to a call end function button when the user's finger touches the volume control button and the call application of the portable terminal is activated.

[0023] In addition, the plurality of tactile actuators can convey the spatiotemporal tactile environment related to the state of the interaction providing system to the user in the form of feedback before the user presses the button.

[0024] Additionally, the button includes a power button of a vacuum cleaner, and the plurality of tactile actuators activate a pin of the Braille display device according to the remaining battery level of the interaction providing system when the user's finger comes into contact with the power button, and the activation may be selectively applied to the entire area or a part area of ​​the power button according to the remaining battery level.

[0025] In addition, the plurality of tactile actuators can transmit the spatiotemporal tactile environment regarding the result of operating the button to the user in the form of feedback after the user presses the button.

[0026] Additionally, the button includes the left button of the mouse, and the plurality of tactile actuators activate the pins of the Braille display device according to the processing of the interaction providing system, wherein the activation may be applied in the order of the lower left, upper left, upper right, and lower right of the left button. Effects of the invention

[0028] According to an embodiment of the present invention, by proposing a system that provides spatiotemporal tactile output and information necessary for interaction to a finger pressing a button of an electronic device, and an interaction method utilizing the same, a user can receive necessary information without additional visual or auditory channels. In the process of a user using a button interface, the act of touching the surface of the button is presupposed; the present invention has the advantage of being able to immediately and actively provide information related to interaction during the natural process of touching the surface of the button by providing spatiotemporal tactile output to the surface of the button. Brief explanation of the drawing

[0030] FIG. 1 is a block diagram illustrating the function of an interaction providing system according to an embodiment of the present invention. FIG. 2 is a perspective view illustrating one embodiment of a spatiotemporal tactile output device in the interaction providing system of FIG. 1. FIG. 3 is a diagram illustrating an exemplary embodiment of a case in which feedback is provided while a button of an interaction providing system according to an embodiment of the present invention is being pressed. FIG. 4 is a diagram illustrating another embodiment of providing feedback while a button of an interaction providing system according to an embodiment of the present invention is being pressed. FIG. 5 is a diagram illustrating an exemplary embodiment of a case in which feedforward is provided before pressing a button of an interaction providing system according to an embodiment of the present invention. FIG. 6 is a diagram illustrating another embodiment of the case in which feedforward is provided before pressing a button of an interaction providing system according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an exemplary embodiment of a case in which feedback is provided before a button is pressed in an interaction providing system according to an embodiment of the present invention. FIG. 8 is a diagram illustrating an exemplary embodiment of the case in which feedback is provided after a button is pressed in an interaction providing system according to an embodiment of the present invention. Specific details for implementing the invention

[0031] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the scope of the present invention is defined only by the claims.

[0032] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted unless actually necessary for describing the embodiments of the present invention. Furthermore, the terms described below are defined in consideration of the functions in the embodiments of the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0033] Among haptic actuators, spatiotemporal tactile output actuators (e.g., products that add spatiotemporal tactile output actuators to the button positions of a mouse) have the advantage of being able to express shapes, directions, specific locations, etc., thereby intuitively conveying information. However, existing products and research have limitations in that spatiotemporal tactile output actuators are independent of button interaction. For instance, while research is being conducted on various forms of interaction provision systems such as haptic actuators and spatiotemporal tactile output actuators, there is a lack of exploration into what information can be provided in terms of button interaction when the actuator operates according to the state of the button.

[0034] Accordingly, in an embodiment of the present invention, we propose an interaction technology that utilizes a spatiotemporal tactile output actuator that operates according to the state of a button to provide feedforward information and feedback information necessary for button interaction.

[0035] A system and method of interaction that enable spatiotemporal tactile output on the surface of a button and provide feedback and feedforward during the button interaction process should be emphasized as features of the invention. Regardless of what device or what shape the button is, or what spatiotemporal output actuator is used, if spatiotemporal tactile feedback is provided on the surface of a button and information necessary for button interaction is provided through that tactile feedback, it can be considered to fall within the scope of the present invention.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0037] FIG. 1 is a block diagram for explaining the function of an interaction providing system (10) according to an embodiment of the present invention.

[0038] As illustrated in FIG. 1, the interaction providing system (10) may include a spatiotemporal tactile output device (100) and a computing device (200), and may provide feedforward and feedback to the user through spatiotemporal tactile output.

[0039] The interaction providing system (10) is an electronic device carried by a user, and in an embodiment of the present invention, the electronic device may include at least one of, for example, a smartphone, an e-book reader, a tablet personal computer, a laptop personal computer, a netbook computer, a desktop computer, a server computer, and a wearable device. In another embodiment, the electronic device may include at least one of network security equipment, navigation equipment, marine electronic equipment (e.g., a marine navigation device, a gyrocompass, etc.), avionics, a vehicle head unit, a point of sales (POS) of a store, and an Internet of Things device. Additionally, the electronic device may be equipped with an application for executing various operations of the electronic device (e.g., on / off commands, operation commands, screen output, etc.). Applications may include, for example, smartphone applications, PC (personal computer) applications, set-top box (STB) applications, web applications, instant applications, etc., and are not limited to specific applications.

[0040] The electronic device that can be applied to the interaction providing system (10) in the embodiment of the present invention described below may include, for example, at least one of a laptop computer, a desktop computer, a vehicle seat heating control assembly, a game controller, a smartphone (portable terminal), a vacuum cleaner (home appliance), and a mouse (UI device of a computing device).

[0041] In FIG. 1, the spatiotemporal tactile output device (100) includes a button that is contacted by a user's finger, generates user state information based on the physical force applied to the button, and can provide a spatiotemporal tactile environment to the user based on state change information corresponding to the generated state information.

[0042] Such spatiotemporal tactile output device (100) may include a sensor (110) and an actuator (120).

[0043] The sensor (110) can detect the input time and press depth of a button and provide state information based on the detected input time and press depth to a computing device (200). The sensor (110) may include, for example, a button depth sensor, and the button depth sensor may detect the input time of a button contacted by a user or the press depth of a button pressed by a user.

[0044] The actuator (120) can transmit a spatiotemporal tactile environment to the user in a feedforward form or a feedback form. This actuator (120) can provide physical feedforward or feedback through a button to provide a spatiotemporal tactile environment to the user. A spatiotemporal tactile output device (100) having this actuator (120) will be described in more detail in FIG. 2 below.

[0045] In FIG. 1, the computing device (200) receives state information from the spatiotemporal tactile output device (100), generates state change information based on the received state information, and provides the generated state change information to the spatiotemporal tactile output device (100) so that the spatiotemporal tactile environment is provided to the user in a feedforward form or a feedback form.

[0046] This computing device (200) may include an acquisition unit (210), an interaction unit (220), and a state processing unit (230).

[0047] The acquisition unit (210) receives state information provided by the spatiotemporal tactile output device (100), namely state information regarding the input time and pressing depth of the button, and can update the received state information. This acquisition unit (210) may include, for example, an input / output interface, a communication device, etc.

[0048] The interaction unit (220) performs the function of transmitting updated state information from the acquisition unit (210) to the state processing unit (230). Additionally, the interaction unit (220) can transmit state change information generated by the state processing unit (230) to the actuator (120) of the spatiotemporal tactile output device (100).

[0049] The state processing unit (230) can generate state change information based on updated state information of a button transmitted through the interaction unit (220). Depending on the state change information of the state processing unit (230), various spatiotemporal tactile environments can be transmitted to the user from the spatiotemporal tactile output device (100), and embodiments of transmitting various spatiotemporal tactile environments through the actuator (120) of the spatiotemporal tactile output device (100) will be described in detail in FIGS. 3 to 8 below. Here, the state processing unit (230) may include, for example, a microprocessor-based processing device.

[0050] FIG. 2 is a perspective view illustrating one embodiment of a spatiotemporal tactile output device (100) in the interaction providing system (10) of FIG. 1.

[0051] In the case of spatiotemporal tactile output button hardware, the surface of the button is made capable of spatiotemporal tactile output to provide spatiotemporal haptic feedback to the finger touching the button. As a representative example of this, as shown in FIG. 2, a button capable of spatiotemporal tactile output can be created by configuring the surface of the button with a Braille display device.

[0052] A Braille display device is an example of an actuator for spatiotemporal tactile output. Spatiotemporal tactile output refers to activating or deactivating tactile output at specific locations in space over time. Since a Braille display device can activate (raise the Braille pin) or deactivate (lower the Braille pin) the Braille pins at each location over time, it belongs to the category of actuators capable of spatiotemporal tactile output. By utilizing a spatiotemporal tactile actuator, such as a Braille display device, on a button surface, spatiotemporal tactile animations can be provided to the surface of the finger interacting with the user while the user touches or presses the button. For example, if the Braille pins in the row above are actuated to rise sequentially starting from the Braille pin in the bottom row of the Braille display device, a tactile animation that appears to gradually fill up over time can be provided.

[0053] In the embodiment of the present invention, the spatiotemporal tactile output device (100) is exemplified as a Braille display device, but it is necessary to understand that in addition to such a Braille display device, various actuators capable of activating or deactivating tactile output at specific locations in space over time can be applied to the embodiment of the present invention.

[0054] In addition, in an embodiment of the present invention, the braille display device for the spatiotemporal tactile output device (100) can be implemented with a set of multiple actuators rather than a single actuator. For example, the spatiotemporal tactile output required for the present invention can be implemented by placing multiple micro-vibration actuators on a button and activating or deactivating each vibration actuator.

[0055] Various information can be provided to a user during the process of pressing a button by utilizing a spatiotemporal tactile output device (100) including a braille display device according to an embodiment of the present invention. What information is provided at which stage during the pressing process is an interaction method using an interaction providing system (10) according to an embodiment of the present invention.

[0056] The process of a user pressing a button can be broadly divided into three stages: while the button is being pressed with a finger, before pressing, and after pressing. The interaction providing system (10) according to the present invention can provide feedback on the button operation while the button is being pressed with a finger, feedforward feedback on the function to be performed by operating the button and the method of operation, and feedback on the current system state before pressing, and feedback on the result of the button operation after pressing.

[0057] Hereinafter, these matters will be explained in detail with reference to FIGS. 3 to FIGS. 8.

[0058] FIG. 3 is a diagram illustrating an exemplary embodiment of providing feedback while a button of an interaction providing system (10) according to an embodiment of the present invention is being pressed.

[0059] In FIG. 3, the actuator (120) can provide the user with a spatiotemporal tactile environment related to the operation of the button in the form of feedback while the user is pressing the button.

[0060] Here, the button may include, for example, a power button of a laptop computer or a desktop computer, and the actuator (120) activates the pins of the braille display device according to the input time of the power button, wherein the activation may be applied gradually from the center point of the power button to the edge.

[0061] The button's status and the system's status change as a result of the user's operation of pressing the button with a finger. The button's status includes the time or depth of the button being pressed, and the system's status refers to the state of the system that has changed as a result of operating the button.

[0062] To describe the state of the button first, the user can adjust the duration for which the button is held and the depth of the press when operating the button. Both the duration for which the user holds the button and the depth of the press can serve as examples of the button's state. In the spatiotemporal tactile output button device utilizing a Braille display presented as an example in the present invention, it is possible to increase the number of activated pins according to the duration for which the button is pressed. More specifically, as the pressing time increases by one second, the pins surrounding it can be activated sequentially, starting from the pin in the center. The present invention can be used to provide feedback on the duration for which the user pressed the button in a scenario where a forced shutdown or forced boot occurs when the button is pressed for more than four seconds, such as with a laptop power button as shown in FIG. 3.

[0063] FIG. 4 is a diagram illustrating another embodiment of providing feedback while a button of the interaction providing system (10) according to an embodiment of the present invention is being pressed.

[0064] In FIG. 4, the actuator (120) can provide the user with a spatiotemporal tactile environment related to the operation of the button in the form of feedback while the user is pressing the button.

[0065] Here, the button may include, for example, a control button of a vehicle seat heating control assembly, and the actuator (120) activates a pin of a braille display device according to the heating level of the vehicle seat heating control assembly, wherein the activation may be selectively applied to any one of the lower point, the central point, and the upper point of the control button according to the heating level. That is, the current heating level of the vehicle seat heating button may serve as an example of the system state.

[0066] It is difficult for the driver to visually check the current heating level while driving. In this case, it is possible to activate pins located in specific rows on the surface of the heating button according to the current heating level. For example, as shown in Fig. 4, it is possible to activate the pins located in the bottom row when the heating level is level 1, and activate the pins located in the middle row when the heating level is level 2.

[0067] FIG. 5 is a diagram illustrating an exemplary embodiment of a case in which feedforward is provided before pressing a button of an interaction providing system (10) according to an embodiment of the present invention.

[0068] In FIG. 5, the actuator (120) can provide the user with a spatiotemporal tactile environment related to the operation of the button in a feedforward form before the user presses the button.

[0069] Here, the button may include, for example, an input button of a game controller, and the actuator (120) activates a pin of the Braille display device according to the function of the input button when the user's finger contacts the input button, wherein the activation may be selectively applied to a vertical line or a horizontal line of the input button according to the movement of a character on an application executed through the interaction providing system (10).

[0070] Depending on the usage context or the executed application, the function performed by operating the button may vary.

[0071] For example, in the case of game controller buttons, the function performed varies depending on the game currently running. At this time, as shown in FIG. 5, a feedforward regarding the function to be performed by the button can be provided during the process of touching it with a finger before operating the button using spatiotemporal haptic output. For example, a button that performs the character's running function can provide a vertical spatiotemporal haptic output by activating the pins located in the middle column, and a button that performs the character's prone function can provide a horizontal spatiotemporal haptic output by activating the pins located in the bottom row, thereby providing a hint regarding the function that each button will perform in the current game.

[0072] FIG. 6 is a diagram illustrating another embodiment of the case in which feedforward is provided before pressing a button of the interaction providing system (10) according to an embodiment of the present invention.

[0073] In FIG. 6, the actuator (120) can provide the user with a spatiotemporal tactile environment related to the operation of the button in a feedforward form before the user presses the button.

[0074] Here, the button may include, for example, a volume control button of a portable terminal, and the computing device (200) may switch the volume control button to a shooting function button when the user's finger touches the volume control button and the camera application of the portable terminal is activated, switch the volume control button to an alarm end function button when the user's finger touches the volume control button and the alarm application of the portable terminal is activated, and switch the volume control button to a call end function button when the user's finger touches the volume control button and the call application of the portable terminal is activated.

[0075] In other words, it can provide button information that can be used in the current context and the timing for pressing the button.

[0076] First, as an example of providing information on usable buttons, the present invention can be applied to a smartphone volume control button as shown in FIG. 6. The smartphone volume control button is used as a photo-taking button when the camera app is turned on, as a button to turn off the alarm when the alarm is ringing, and as a button to mute the phone sound when a call comes in. It is possible to activate all the pins on the volume control button for each context in which the volume control button is used for a special purpose. Through this, the user can be informed that the volume control button can be used for a special purpose in the current context.

[0077] FIG. 7 is a diagram illustrating an exemplary embodiment of a case in which feedback is provided before pressing a button of an interaction providing system (10) according to an embodiment of the present invention.

[0078] In FIG. 7, the actuator (120) can provide the user with a spatiotemporal tactile environment related to the state of the interaction providing system (10) in the form of feedback before the user presses the button.

[0079] Here, the button may include, for example, a power button of a vacuum cleaner, and the actuator (120) activates a pin of a Braille display device according to the remaining battery level of the interaction providing system when a user's finger comes into contact with the power button, wherein the activation may be selectively applied to the entire area or a part of the power button depending on the remaining battery level.

[0080] That is, as shown in FIG. 7, it is possible to provide information regarding the remaining battery level of a vacuum cleaner by utilizing the present invention on the power button of the vacuum cleaner. The number of activated pins on the power button can be varied depending on the remaining battery level of the vacuum cleaner. For example, when the remaining battery level is 80%, it is possible to activate the pins from the bottom row up to the row located at the 8 / 10 position from the bottom. Through this, the user can obtain information about the current remaining battery level of the vacuum cleaner simply by touching the power button.

[0081] FIG. 8 is a diagram illustrating an exemplary embodiment of providing feedback after pressing a button of an interaction providing system (10) according to an embodiment of the present invention.

[0082] In FIG. 8, the actuator (120) can provide the user with a spatiotemporal tactile environment regarding the result of the button operation in the form of feedback after the user presses the button.

[0083] Here, the button may include, for example, the left button of a mouse of a computing device, and the actuator (120) activates the pins of the Braille display device according to the processing of the interaction providing system (10), wherein the activation may be applied in the order of the lower left, upper left, upper right, and lower right of the left button of the mouse.

[0084] In other words, by using the present invention, it is possible to provide tactile feedback on the system state that has changed when a user presses a button. In particular, for time-consuming tasks such as system updates or video conversion, it is possible to provide the progress status of the task to a finger placed on the button. By repeatedly activating the pins at the bottom left, top left, top right, and bottom right, a clockwise rotating tactile pattern can be provided. As shown in FIG. 8, by continuously providing a clockwise rotating tactile pattern to the mouse button, it is possible to convey to the user that a desktop video conversion task is in progress.

[0085] According to an embodiment of the present invention as described above, by proposing a system that provides spatiotemporal tactile output and information necessary for interaction to a finger pressing a button of an electronic device, and an interaction method utilizing the same, a user can receive necessary information without additional visual or auditory channels. In the process of a user using a button interface, the act of touching the surface of the button is presupposed; therefore, the present invention is expected to provide information related to interaction immediately and actively during the natural process of touching the surface of the button by providing spatiotemporal tactile output to the surface of the button.

[0086] In addition, the advantage of being able to provide necessary information to the user without additional visual or auditory channels, as in the present invention, can be particularly helpful in environments where visual and auditory channels are difficult to use. For example, considering that it is difficult to visually check buttons other than those in front while driving, the present invention can be utilized for things such as a car seat heating button. The car seat heating button repeatedly changes the heating state from 'Level 1 heating - Level 2 heating - Level 3 heating - Off' each time it is pressed. At this time, the current heating state can be tactilely indicated on the surface of the button through spatiotemporal tactile output. Assuming the button is created using a Braille display consisting of three rows, it can be designed so that only the Braille pins in the bottom row rise when in the Level 1 heating state, only the Braille pins in the second row from the bottom rise when in the Level 2 heating state, only the Braille pins in the third row from the bottom rise when in the Level 3 heating state, and all Braille pins go down when in the Off state. In this case, the user can perceive the current heating status simply by touching the car seat heating button with their finger. If the current status of the system can be provided solely through tactile feedback in situations where it is difficult to visually check the heating status, such as while driving, it can eliminate the inconvenience of having to visually check the button.

[0087] Meanwhile, combinations of each block of the attached block diagram and each step of the flowchart may be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a specialized computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create a means to perform the functions described in each block of the block diagram.

[0088] Since these computer program instructions may be stored in a computer-available or computer-readable recording medium (or memory), etc., which can be directed toward a computer or other programmable data processing equipment to implement a function in a specific way, the instructions stored in the computer-available or computer-readable recording medium (or memory) may also be used to produce a manufactured item containing instruction means that perform the function described in each block of the block diagram.

[0089] And, since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on a computer or other programmable data processing equipment to create a process executed by a computer and perform the computer or other programmable data processing equipment may also provide steps for executing the functions described in each block of the block diagram.

[0090] Additionally, each block may represent a module, segment, or part of code containing at least one executable instruction for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to the corresponding function. Explanation of the symbols

[0092] 10: Interaction-providing system 100: Space-Time Tactile Output Device 110: Sensor 120: Actuator 200: Computing device 210: Acquisition Section 220: Interaction Department 230: State processing unit

Claims

Claim 1 An interaction providing system comprising: a spatiotemporal tactile output device including a button contacted by a user's finger, generating state information of the user based on a physical force applied to the button, and including a plurality of tactile actuators disposed on the button; and a computing device that receives the state information, generates state change information based on the state information, and provides the state change information to the spatiotemporal tactile output device so that a spatiotemporal tactile environment is provided in a feedforward form or a feedback form; wherein the spatiotemporal tactile output device provides spatial tactile stimulation by activating one or more selected tactile actuators among the plurality of tactile actuators based on the state change information corresponding to the operation state of the button, and provides a spatiotemporal tactile environment to the user in which the spatial tactile stimulation changes over time. Claim 2 In claim 1, the spatiotemporal tactile output device includes a sensor that detects the input time and pressing depth of the button and provides the state information according to the input time and pressing depth to the computing device, and the plurality of tactile actuators are an interaction providing system that transmits the spatiotemporal tactile environment to the user in the form of the feedforward or the feedback. Claim 3 In claim 2, the spatiotemporal tactile output device is an interaction providing system including a Braille display device. Claim 4 In claim 2, the computing device comprises: an acquisition unit for acquiring state information; a state processing unit for generating state change information based on state information; and an interaction part for transmitting state information to the state processing unit and transmitting state change information to the plurality of tactile actuators of the spatiotemporal tactile output device; an interaction providing system. Claim 5 In claim 3, the plurality of tactile actuators are an interaction providing system that transmits the spatiotemporal tactile environment related to the operation of the button to the user in the form of feedback while the user is pressing the button. Claim 6 In claim 5, the button includes a power button of a notebook computer or a desktop computer, and the plurality of tactile actuators activate a pin of the braille display device according to the input time of the power button, wherein the activation is applied gradually from the center point of the power button to the edge, an interaction providing system. Claim 7 In claim 5, the button includes a control button of a vehicle seat heating control assembly, and the plurality of tactile actuators activate a pin of the braille display device according to the heating level of the vehicle seat heating control assembly, wherein the activation is selectively applied to any one of the lower point, the central point, and the upper point of the control button according to the heating level, an interaction providing system. Claim 8 In claim 3, the plurality of tactile actuators are an interaction providing system that transmits the spatiotemporal tactile environment related to the operation of the button to the user in the form of the feedforward before the user presses the button. Claim 9 In claim 8, the button includes an input button of a game controller, and the plurality of tactile actuators activate a pin of the Braille display device according to the function of the input button when the user's finger contacts the input button, wherein the activation is selectively applied to a vertical line or a horizontal line of the input button according to the movement of a character on an application executed through the interaction providing system. Claim 10 In claim 8, the plurality of tactile actuators are an interaction providing system that transmits the spatiotemporal tactile environment related to the state of the interaction providing system to the user in the form of feedback before the user presses the button. Claim 11 In claim 10, the button includes a power button of a vacuum cleaner, and the plurality of tactile actuators activate a pin of the Braille display device according to the remaining battery level of the interaction providing system when the user's finger contacts the power button, wherein the activation is selectively applied to the entire area or a part area of ​​the power button according to the remaining battery level. Claim 12 In claim 3, the plurality of tactile actuators are an interaction providing system that transmits the spatiotemporal tactile environment regarding the result of operating the button to the user in the form of feedback after the user presses the button. Claim 13 In claim 12, the button includes the left button of a mouse, and the plurality of tactile actuators activate the pins of the Braille display device according to the processing of the interaction providing system, wherein the activation is applied in the order of the lower left, upper left, upper right, and lower right of the left button.

Images