Pseudo-tactile presentation device, pseudo-tactile presentation method, and program

Abstract

To control pseudo tactile perception by auditory information regardless of the frequency of the auditory information.SOLUTION: When operation information based on action is inputted, a presentation signal for presenting auditory information based on auditory adjustment information is generated. The auditory adjustment information is information for controlling an auditory adjustment parameter being at least any one among an occurrence delay amount corresponding to a time until the presentation of the auditory information is started after the operation is inputted, a stop delay amount corresponding to a time until the presentation of the auditory information ends after the operation information is inputted, a continuation amount corresponding to a time till an end after the presentation of the auditory information is started, or sound volume of the auditory information. However, the auditory adjustment parameter corresponds to a desired pseudo tactile perception.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for presenting a simulated tactile sensation. [Background technology]

[0002] It is known that when a button on a car audio device is operated, the weight (sense of weight) of the button operation felt by the operator varies depending on the frequency of the operation sound (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Koji Sakamoto, Shunsuke Ishimitsu, Takayuki Arai, Toshikazu Yoshimi, Yuichi Fujimoto, Kenichi Kawasaki, A Study on Evaluation of Button Pressing Sound of Car Audio Main Units, Journal of the Japan Society of Kansei Engineering, Vol. 10, No. 3, 2010, p. 375-385, 2011. Summary of the Invention [Problem to be solved by the invention]

[0004] In this conventional method, it was necessary to present operation sounds of different frequencies in order to make the user perceive different weights. However, in a method in which the presented weight depends only on the frequency of the operation sound, this difference in frequency cannot be used for other purposes (such as distinguishing between operation buttons). This problem is not only present when the weight of button operation is perceived through the operation sound at the time of button operation, but is common to all cases in which pseudo-tactile sensations are perceived through auditory information.

[0005] The present invention has been made in consideration of these points, and has an object to provide a technique that can control pseudo-tactile sensations using auditory information regardless of the frequency of the auditory information. [Means for solving the problem]

[0006] The pseudo-tactile presentation device of the first embodiment generates a presentation signal for presenting auditory information based on auditory adjustment information when operation information based on an action is input. Here, the auditory adjustment information is information for controlling an auditory adjustment parameter, which is at least one of an onset delay amount corresponding to the time from input of the operation information to start of presentation of the auditory information, a stop delay amount corresponding to the time from input of the operation information to end of presentation of the auditory information, a duration amount corresponding to the time from start to end of presentation of the auditory information, or a volume of the auditory information. Here, the auditory adjustment parameter corresponds to a desired pseudo-tactile sensation.

[0007] The pseudo-tactile presentation device of the second embodiment generates a presentation signal for presenting auditory information based on auditory adjustment information when operation information for operating the operation device is output. Here, the auditory adjustment information is information for controlling an auditory adjustment parameter which is at least one of an onset delay amount corresponding to the time from when the operation information is output to when the presentation of the auditory information starts, a stop delay amount corresponding to the time from when the operation information is output to when the presentation of the auditory information ends, a duration amount corresponding to the time from when the presentation of the auditory information starts to when it ends, or a volume of the auditory information. Effect of the Invention

[0008] This makes it possible to control the pseudo-tactile sensation by the auditory information regardless of the frequency of the auditory information. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating a functional configuration of a pseudo-tactile presentation device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating experimental results on which the embodiment is based. [Diagram 3] FIG. 3 is a diagram illustrating experimental results on which the embodiment is based. [Figure 4] FIG. 4 is a diagram illustrating experimental results on which the embodiment is based. [Diagram 5] FIG. 5 is a flow diagram for explaining the processing of the embodiment. [Figure 6] FIG. 6 is a flow diagram for explaining the processing of the embodiment. [Figure 7] FIG. 7 is a block diagram illustrating a functional configuration of the pseudo-tactile presentation device according to the embodiment. [Figure 8] FIG. 8 is a block diagram illustrating the hardware configuration of the pseudo-tactile presentation device of this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [principle] The inventor conducted the following experiment and discovered the underlying natural law (physiological law).

[0011] <Experiment 1> Testing Procedure: A clickable test button (square) is displayed on the screen. When the subject clicks the test button, a pure tone (test tone) with a frequency of 200 Hz or 400 Hz is presented (sound output) for 0.5 seconds after a delay time (either 0, 100, 200, 300, or 500 ms) has elapsed since the click. A reference button (square) separate from the test button is also displayed on the screen. When the subject clicks the reference button, a pure tone (reference tone) with a frequency of 300 Hz is presented after a delay time of 200 ms has elapsed since the click. The subject compares the sense of weight felt when clicking the test button with the sense of weight felt when clicking the reference button, and answers on a 5-point scale from 1 to 5 below. 1: The standard button is much heavier 2: The reference button is heavier 3: The reference button and the test button have the same weight 4: The test button is heavier 5: The test button is much heavier

[0012] Test Results: FIG. 2 shows the experimental results of Experiment 1. The horizontal axis of FIG. 2 represents the delay time (delay), and the vertical axis represents the average answer result (ansAve). The solid line graph represents the results when a test sound with a frequency of 200 Hz was presented, and the dashed line graph represents the results when a test sound with a frequency of 400 Hz was presented. As shown in FIG. 2, regardless of the frequency of the test sound presented, the longer the delay time, the more significantly the subject's sense of heaviness increases. It can also be seen that the sense of heaviness increases more when a test sound with a frequency of 200 Hz is presented than when a test sound with a frequency of 400 Hz is presented. From this result, it can be seen that the longer the delay time from the click operation to the start of the presentation of the test sound and the lower the frequency of the test sound, the greater the sense of heaviness perceived by the subject.

[0013] <Experiment 2> Experimental procedure: In Experiment 1, when the subject clicked the test button, a test sound was presented when a delay time had elapsed from this click. In contrast, in Experiment 2, a pure tone (test sound) with a frequency of 200 Hz or 400 Hz was presented from the beginning, and when the subject clicked the test button, the presentation of the test sound was stopped (ended) when a delay time (either 0, 100, 200, 300, or 500 ms) had elapsed from this click. The rest of the experiment was the same as in Experiment 1.

[0014] Test Results: FIG. 3 shows the experimental results of Experiment 2. The horizontal axis of FIG. 3 represents the delay time (delay), and the vertical axis represents the average answer result (ansAve). The solid line graph represents the results when a test sound with a frequency of 200 Hz was presented, and the dashed line graph represents the results when a test sound with a frequency of 400 Hz was presented. As shown in FIG. 3, regardless of the frequency of the test sound presented, the longer the delay time, the more significantly the subject's sense of heaviness increases. It can also be seen that the heaviness increases more when a test sound with a frequency of 200 Hz is presented than when a test sound with a frequency of 400 Hz is presented. From this result, it can be seen that the longer the delay time from when the click operation is performed until the test sound stops and the lower the frequency of the test sound, the greater the sense of heaviness perceived by the subject.

[0015] <Experiment 3> In Experiment 1, a test tone of 200Hz or 400Hz was presented, and a reference tone of 300Hz was presented. In contrast, in Experiment 3, a pure tone (test tone) with high sound pressure (high volume) was presented instead of the test tone of 200Hz, a pure tone (test tone) with low sound pressure (low volume) was presented instead of the test tone of 400Hz, and a pure tone (reference tone) with medium sound pressure (medium volume) was presented instead of the reference tone of 300Hz. Note that the volume of medium sound pressure is larger than low sound pressure and smaller than high sound pressure. The rest of the experiment was the same as in Experiment 1.

[0016] Test Results: FIG. 4 shows the experimental results of Experiment 3. The horizontal axis of FIG. 4 represents the delay time (delay), and the vertical axis represents the average answer result (ansAve). The solid line graph represents the results when a test sound with a large sound pressure was presented, and the dashed line graph represents the results when a test sound with a small sound pressure was presented. As shown in FIG. 4, regardless of the magnitude of the test sound presented, the longer the delay time, the more significantly the subject's sense of weight increases. It can also be seen that the greater the sense of weight when a test sound with a large sound pressure was presented than when a test sound with a small sound pressure was presented. From this result, it can be seen that the longer the delay time from when the click operation was performed until the test sound stopped and the greater the sound pressure of the test sound, the greater the sense of weight perceived by the subject.

[0017] <Consideration> As described above, regardless of the frequency of the test sound, the longer the delay time from when the click operation is performed until the start of the presentation of the test sound, the greater the sense of weight perceived by the subject. Furthermore, regardless of the frequency of the test sound, the longer the delay time from when the click operation is performed until the test sound stops, the greater the sense of weight perceived by the subject. Furthermore, regardless of the frequency of the test sound, the greater the sound pressure of the test sound, the greater the sense of weight perceived by the subject. Such a natural law (physiological law) has not been known until now. In this experiment, the presentation of the test sound was started or stopped after a delay time had elapsed since the square test button was clicked. However, this natural law generally holds when the presentation of the test sound is started or stopped after a delay time has elapsed since an operation based on some subject's movement is performed. The operation based on the subject's movement may be an operation based on the movement of the subject's body part, and may be an operation performed with the subject's body part in contact with an object, or may be an operation performed without contacting the object. However, a clearer effect can be expected when the operation is performed with the subject's body part in contact with an object. This natural law also holds when the presentation of the test sound is started at a time when a first delay time has elapsed since the subject's motion-based operation, and when the presentation of the test sound is stopped at a time when a second delay time (where the second delay time is longer than the first delay time) has elapsed since the subject's motion-based operation, and the longer the first delay time and the second delay time are, the greater the sense of weight perceived by the subject. Furthermore, it is possible to control the sense of weight perceived by the subject depending on the time from when the presentation of the test sound starts to when it ends, and it is expected that the longer the time from when the presentation of the test sound starts to when it ends, the greater the sense of weight perceived by the subject. Furthermore, such a natural law also generally holds when the test sound is auditory information (acoustic information) other than a pure tone, for example, a sound other than a pure tone such as a natural sound, music, a voice, an alarm sound, a noise sound, or a machine sound. Furthermore, the sense of weight perceived by the subject may be perceived as a sense of resistance or a sense of hardness. More generally, the sensation perceived by the subject as described above can be expressed as a pseudo-tactile sensation.That is, by controlling at least one of the onset delay amount corresponding to the time from input of operation information to start of presentation of auditory information, the stop delay amount corresponding to the time from input of operation information to end of presentation of auditory information, the duration corresponding to the time from start to end of presentation of auditory information, or the volume of auditory information, it is possible to make the subject perceive a desired pseudo-haptic sensation. That is, the larger the onset delay amount, stop delay amount, duration, or volume, the greater the pseudo-haptic sensation such as a sense of weight perceived by the subject.

[0018] Moreover, such a law of nature also holds true when a test sound is presented to a subject while the subject is watching an operation based on the movement of a body part of a human or animal other than the subject. That is, by controlling at least one of the onset delay amount, the stop delay amount, the duration, or the volume, the subject can be made to perceive a desired pseudo-tactile sensation. That is, the larger the onset delay amount, the stop delay amount, the duration, or the volume, the greater the pseudo-tactile sensation such as a sense of weight that the subject perceives.

[0019] Furthermore, such a law of nature also holds when a test sound is presented to a subject while the subject is watching an operation based on the operation of a motion device such as a robot other than the subject. In this case, such a law of nature also holds when the trigger is the output of motion information for operating the motion device, instead of the operation based on the motion of the motion device. However, in this case, the amount of delay that occurs corresponds to the time from the output of motion information to the start of presentation of auditory information, and the amount of stop delay corresponds to the time from the output of motion information to the start of presentation of auditory information.

[0020] Furthermore, such a natural law also holds when controlling a combination of any two or more of the onset delay amount, the stop delay amount, the duration, and the volume. That is, by controlling a combination of any two or more of the onset delay amount, the stop delay amount, the duration, and the volume, the magnitude of the pseudo-haptic sensation perceived by the subject can be controlled. Furthermore, as described above, such a natural law also holds when controlling the frequency of the test sound presented in addition to at least one of the onset delay amount, the stop delay amount, the duration, and the volume. That is, by controlling a combination of at least one of the onset delay amount, the stop delay amount, the duration, and the volume and the frequency of the test sound, the magnitude of the pseudo-haptic sensation perceived by the subject can also be controlled.

[0021] <Utilizing the Laws of Nature> In each embodiment, when operation information based on an operation is input, the pseudo-tactile presentation device generates a presentation signal for presenting auditory information based on auditory adjustment information. However, the auditory adjustment information is information for controlling an auditory adjustment parameter that is at least one of an occurrence delay amount (hereinafter, sometimes simply referred to as "occurrence delay amount") corresponding to the time from input of operation information to start of presentation of auditory information, a stop delay amount (hereinafter, sometimes simply referred to as "stop delay amount") corresponding to the time from input of operation information to end of presentation of auditory information, a duration (hereinafter, sometimes simply referred to as "duration amount") corresponding to the time from start of presentation of auditory information to end, or a volume of auditory information (hereinafter, sometimes simply referred to as "volume"). Alternatively, when operation information for operating an operating device is output, the pseudo-tactile presentation device generates a presentation signal for presenting auditory information based on auditory adjustment information. However, the hearing adjustment information is information for controlling a hearing adjustment parameter which is at least one of an occurrence delay amount (hereinafter, sometimes simply referred to as "occurrence delay amount") corresponding to the time from when the operation information is output to when the presentation of the hearing information is started, a stop delay amount (hereinafter, sometimes simply referred to as "stop delay amount") corresponding to the time from when the operation information is output to when the presentation of the hearing information is ended, a duration corresponding to the time from when the presentation of the hearing information is started to when it is ended, or a volume of the hearing information. However, the hearing adjustment parameter corresponds to a desired pseudo-tactile sensation. This is based on the above-mentioned newly discovered natural law, and is information that would not be used for presentation to a pseudo-tactile sensation if such a discovery had not been made. By presenting hearing information based on such a presentation signal to a user, it is possible to control the pseudo-tactile sensation perceived by the sound signal regardless of the frequency of the hearing information. Each embodiment will be described below in detail.

[0022] [First embodiment] A first embodiment of the present invention will be described. <Configuration> 1, the pseudo-tactile presentation device 1 of this embodiment has an auditory adjustment information generation unit 11, a presentation signal generation unit 12, an output unit 13, an input unit 14, an operation detection unit 15, an auditory adjustment information generation unit 17, a control unit 18, and a storage unit 19, and executes each process under the control of the control unit 18. Input information to the pseudo-tactile presentation device 1 and information obtained by each unit are stored in the storage unit 19, and are read out and used as necessary. The hardware configuration of the pseudo-tactile presentation device 1 will be described later.

[0023] An operation device 101 is connected to the input unit 14 of the pseudo-tactile presentation device 1 by wire or wirelessly. The operation device 101 is a device that detects an operation based on the movement of a body part of the user 100. For example, a device that detects an operation performed in a state where a body part of the user 100 is in contact with an object can be used as the operation device 101. In this embodiment, for the sake of simplicity, a case will be described in which the operation device 101 is a physical key (e.g., a key of a keyboard) that is pressed by the finger of the user 100. However, this is only one example, and any device may be used as the operation device 101 as long as it detects an operation based on the movement of a body part of the user 100 (e.g., a hand, a finger, a foot, a head, a face, an eye, etc.).

[0024] In addition, an auditory information presentation device 102 is connected to the output unit 13 of the pseudo-tactile presentation device 1 via a wired or wireless connection. The auditory information presentation device 102 is a device that presents auditory information. The "auditory information" may be any acoustic information that can be captured by the human hearing. In this embodiment, for the sake of simplicity of explanation, a case is exemplified in which the auditory information is a pure tone and the auditory information presentation device 102 is a speaker that emits (presents) the pure tone. However, this is merely an example, and any device that presents auditory information may be used as the auditory information presentation device 102.

[0025] <Processing> Next, the process of the first embodiment will be described with reference to FIG. 1 and FIG. The auditory adjustment information generating unit 11 of the pseudo-tactile presentation device 1 generates information V 0Any initial auditory information may be generated. For example, a pure tone of a predetermined frequency and volume belonging to the audible range is used as the initial auditory information. 0 is sent to the presentation signal generating unit 12 (step S11).

[0026] The user 100 performs an operation on the operating device 101 (an operation based on a movement of a body part) at any timing. As described above, the operating device 101 illustrated in this embodiment is a physical key, and the user 100 can perform an operation on the operating device 101 with his / her finger (a pressing operation in this embodiment). When an operation is performed on the operating device 101, the operating device 101 outputs operation information (operation information based on a movement of a body part) O indicating that the operation has been performed. in The operation information O output from the operation device 101 is in is input to the input unit 14 of the pseudo-tactile presentation device 1 and sent to the operation detection unit 15. The operation detection unit 15 detects the operation information O in The input of operation information O is monitored. in If it is determined that no operation information has been input, the operation detection unit 15 detects the operation information O in On the other hand, the operation information O in When it is determined that the operation information O in The hearing adjustment information generating unit 17 then sends a trigger signal T indicating that the hearing adjustment information has been input (step S15).

[0027] The auditory adjustment information generating unit 17 receives the trigger signal T and generates auditory adjustment information A. The auditory adjustment information A is based on a previously unknown natural law, and includes: (1) operation information O in (2) the amount of delay corresponding to the time from when the operation information O is input to when the presentation of the auditory information begins inInformation for controlling at least one of the following auditory adjustment parameters: the stop delay amount corresponding to the time from when the operation information O is input until the presentation of the auditory information ends, or (3) the duration amount corresponding to the time from when the presentation of the auditory information starts until it ends, or (4) the volume (sound pressure) of the auditory information, provided that the auditory adjustment parameter corresponds to a desired pseudo-tactile sensation.

[0028] Specific examples of the generation delay amount are as follows. · Operation information O in The time Ds1 from when the operation information O is input to any of the input unit 14, the operation detection unit 15, and the auditory adjustment information generation unit 17 until the presentation of the auditory information starts · Information capable of specifying the time Ds1 There is no limit to the magnitude of the generation delay amount. For example, the time Ds1 can be any time between 0 - 2000 [ms], or information capable of specifying the time Ds1 can be used as the generation delay amount. If Ds1 = α and Ds1 = β, and α > β, the magnitude of the pseudo-tactile sensation (e.g., sense of weight) perceived when Ds1 = α is greater than the magnitude of the pseudo-tactile sensation perceived when Ds1 = β. For example, the longer the time Ds1, the greater the pseudo-tactile sensation that can be presented.

[0029] Specific examples of the stop delay amount are as follows. · Operation information O in The time De1 from when the operation information O is input to any of the input unit 14, the operation detection unit 15, and the auditory adjustment information generation unit 17 until the presentation of the auditory information ends · Information capable of specifying the time De1 There is no limit to the magnitude of the stop delay amount. For example, the time De1 can be any time between 0 - 2000 [ms], or information capable of specifying the time De1 can be used as the stop delay amount. However, the relationship Ds1 < De1 must be satisfied. If De1 = α and De1 = β, and α > β, the magnitude of the pseudo-tactile sensation perceived when De1 = α is greater than the magnitude of the pseudo-tactile sensation perceived when De1 = β. For example, the longer the time De1, the greater the pseudo-tactile sensation that can be presented.

[0030] Specific examples of the duration amount are as follows. - Time from the start to the end of the presentation of auditory information Du1 Information that can identify time Du1 There is no limit to the magnitude of the duration, but for example, any time Du1 between 10-2000 [ms], or information that can identify the time Du1, may be used as the duration. When Du1=α, Du1=β, and α>β, the magnitude of the pseudo-tactile sensation (e.g., weight) perceived when Du1=α is greater than the magnitude of the pseudo-tactile sensation perceived when Du1=β. For example, the longer the time Du1, the greater the pseudo-tactile sensation that can be presented.

[0031] (4) Specific examples of the volume of auditory information are as follows: Constant volume Am1 Am2, the volume that gradually increases over time from the start to the end of the presentation of auditory information Am3: A volume that gradually decreases over time from the beginning to the end of the presentation of auditory information Am4: A volume that gradually increases and then decreases over time from the beginning to the end of the presentation of auditory information Am5: A volume that gradually decreases and then increases over time from the beginning to the end of the presentation of auditory information -The volume changes at a specified rate between the start and end of the presentation of auditory information (for example, it becomes louder at a specified rate, quieter at a specified rate, louder at a specified rate and then quieter, or quieter at a specified rate and then louder) Am6 -The volume changes at a selected rate between the beginning and end of the presentation of auditory information (for example, it gets louder at a selected rate, it gets quieter at a selected rate, it gets louder at a selected rate and then quieter, or it gets quieter at a selected rate and then louder) Volume Am7 - Volume that changes discontinuously between the start and end of the presentation of auditory information (for example, discontinuously increasing, decreasing, increasing and then decreasing, or decreasing and then increasing) Am8 A volume in which only the volume of a specific frequency component changes over time from the start to the end of the presentation of auditory information (the way in which the volume of the specific frequency component changes is, for example, one of the specific examples described above) Am9 When α>β, the magnitude of the pseudo-haptic sensation perceived when the volume is α is larger than the magnitude of the pseudo-haptic sensation perceived when the volume is β. For example, the longer the volume is, the larger the pseudo-haptic sensation can be presented.

[0032] The hearing adjustment information A is information that allows the presentation signal generating unit 12 to uniquely identify at least one of the hearing adjustment parameters (1) occurrence delay amount, (2) stop delay amount, (3) duration amount, or (4) volume of the hearing information. Any information may be used as the hearing adjustment information A as long as it is such information. For example, if the presentation signal generating unit 12 does not use a predetermined constant value as the (1) occurrence delay amount, the hearing adjustment information A may include at least the (1) occurrence delay amount. In any case, the hearing adjustment information A may include information indicating how the volume of the hearing information changes over time, or information indicating which frequency component of the hearing information changes in volume over time. The hearing adjustment information generating unit 17 sends the hearing adjustment information A to the presentation signal generating unit 12 (step S172).

[0033] The presentation signal generation unit 12 to which the auditory adjustment information A is input generates information V 0 A presentation signal V for presenting (outputting) auditory information specified by the auditory adjustment information A (auditory information based on the auditory adjustment information) using t For example, the presentation signal generation unit 12 generates information V based on the hearing adjustment information A. 0 A presentation signal V for presenting auditory information in which at least one of the presentation start time, stop time, duration, or volume of the initial auditory information represented by tThe presentation start time, stop time, duration, and volume of the auditory information are uniquely determined based on the auditory adjustment information A. The presentation start time, stop time, duration, and volume of the auditory information that are not determined based on the auditory adjustment information A may be determined in advance, may be determined based on an input, or may be determined randomly. The generated presentation signal V t is sent from the output unit 13 to the auditory information presentation device 102, and the auditory information presentation device 102 converts this presentation signal V t According to the above, the auditory information at each time t is presented to the user 100 (the auditory information is made to be heard by the user 100) (step S122).

[0034] <Features of the first embodiment> The pseudo-tactile presentation device 1 of this embodiment receives operation information O based on a motion. in A presentation signal V for presenting auditory information based on auditory adjustment information A when t Here, the hearing adjustment information A is generated and output based on the operation information O in the amount of delay corresponding to the time from when the operation information O is input to when the presentation of the auditory information begins, in The information is for controlling an auditory adjustment parameter, which is at least one of a stop delay amount corresponding to the time from input of the presentation signal V to the end of presentation of the auditory information, a duration amount corresponding to the time from the start to the end of presentation of the auditory information, and a volume of the auditory information. Note that the auditory adjustment parameter corresponds to a desired pseudo-haptic sensation. t By presenting the auditory information at each time t to the user 100 according to the above, it is possible to make the user 100 (human) perceive a desired pseudo-tactile sensation such as a sense of weight in a body part (e.g., a body part that has been operated with a finger or the like). These principles are based on the discovery of the above-mentioned laws of nature. As described above, the magnitude of such pseudo-tactile sensation varies depending on the onset delay amount, the stop delay amount, the duration amount, and the volume of the auditory information. Therefore, by controlling the auditory adjustment parameter, it is possible to change the magnitude of the pseudo-tactile sensation (e.g., the magnitude of the sense of weight or the magnitude of the sense of hardness, etc.). In this way, the pseudo-tactile sensation can be controlled by the auditory information, regardless of the frequency of the auditory information.

[0035] The auditory adjustment parameters include at least one of the onset delay amount, the stop delay amount, or the duration amount, and the volume of the auditory information, and by controlling both the onset delay amount, the stop delay amount, or the duration amount, and the volume of the auditory information, the clarity of the perceived pseudo-tactile sensation can be improved and the degree of freedom in controlling the presented pseudo-force sensation can be increased.

[0036] In addition, when an operation is performed while a body part is in contact with an object, the pseudo-tactile sensation can be more clearly perceived. Therefore, it is preferable that the operation device 101 is a device that accepts an operation performed while a body part is in contact with an object and outputs operation information based on the operation.

[0037] [Modification 1 of the first embodiment] In the first embodiment, auditory information was presented after an operation was performed on the operation device 101 (step S122). However, initial auditory information may be presented before an operation is performed on the operation device 101, and auditory information based on the auditory adjustment information A may be presented after an operation is performed on the operation device 101. Hereinafter, differences from the matters described so far will be mainly described, and the same reference numbers will be used to simplify the description of matters that have already been described.

[0038] <Configuration> This is the same as the first embodiment.

[0039] <Processing> The process of this modified example will be described with reference to FIG. After the process of step S11 described above, the presentation signal generation unit 12 generates information V 0 Using the information V 0 The initial presentation signal V for presenting (outputting) the initial auditory information specified by 0 Generate the initial presentation signal V 0 is sent to the auditory information presentation device 102, which then converts this initial presentation signal V 0In step S122, the presentation signal V t The presentation of the initial auditory information by the auditory information presenting device 102 is continued until the presentation of auditory information based on the auditory adjustment information (auditory information based on the auditory adjustment information A) is started (step S121).

[0040] Then, the process of step S15 is performed, and the operation information O in When it is determined that the operation information O in The hearing adjustment information generating unit 17 then sends a trigger signal T indicating that the hearing adjustment information has been input (step S15).

[0041] The auditory adjustment information generation unit 17, to which the trigger signal T is input, generates auditory adjustment information A. The auditory adjustment information A is sent to the presentation signal generation unit 12 (step S172).

[0042] The presentation signal generation unit 12 to which the auditory adjustment information A is input generates information V 0 A presentation signal V for presenting (outputting) auditory information specified by the auditory adjustment information A (auditory information based on the auditory adjustment information) using t The presentation signal V t is sent to the auditory information presentation device 102, which then converts the presentation signal V t According to the above, auditory information at each time t is presented to the user 100 (step S122).

[0043] <Features of Modification 1 of First Embodiment> Even in this case, the same effects as in the first embodiment can be obtained.

[0044] [Modification 2 of the First Embodiment] In the first embodiment, after operation information is input to the pseudo-tactile presentation device, auditory adjustment information is generated (step S172), and a presentation signal for presenting auditory information based on the auditory adjustment information is generated and output (step S122). However, this does not limit the present invention, and the present invention may be implemented in other processing orders. For example, before operation information is input to the pseudo-tactile presentation device, auditory adjustment information may be generated, and a presentation signal for presenting auditory information based on the auditory adjustment information may be generated.

[0045] <Configuration> This is the same as the first embodiment.

[0046] <Processing> The process of this modification will be described with reference to FIG. 6. The auditory adjustment information generating unit 11 of the pseudo-tactile presentation device 1 generates information V 0 Generate information V 0 is sent to the presentation signal generating unit 12 (step S11).

[0047] The auditory adjustment information generation unit 17 generates the auditory adjustment information A. The auditory adjustment information generation unit 17 sends the auditory adjustment information A to the presentation signal generation unit 12 (step S172').

[0048] The presentation signal generator 12 receives information V representing the input initial auditory information. 0 and a presentation signal V for presenting (outputting) auditory information specified by the auditory adjustment information A (auditory information based on the auditory adjustment information) using the auditory adjustment information A. t is generated (step S12).

[0049] The user 100 performs an operation (operation based on the movement of a body part) on the operation device 101 at any timing. When an operation is performed on the operation device 101, the operation device 101 outputs operation information (operation information based on the movement of a body part) O indicating that the operation has been performed. in The operation detection unit 15 outputs the operation information O in The input of operation information O is monitored. inIf it is determined that no operation information has been input, the operation detection unit 15 detects the operation information O in On the other hand, the operation information O in When it is determined that the operation information O in The trigger signal T, which indicates that the input has been made, is sent to the presentation signal generating unit 12 (step S15').

[0050] The presentation signal generating unit 12 receives the trigger signal T and generates a presentation signal V t to the output unit 13, which then transmits it to the auditory information presentation device 102. The auditory information presentation device 102 receives this presentation signal V t According to the above, auditory information is presented at each time t (step S122').

[0051] <Features of Modification 2 of First Embodiment> Even in this case, the same effects as in the first embodiment can be obtained.

[0052] [Third Modification of the First Embodiment] In the second modification of the first embodiment, auditory information is presented after an operation is performed on the operation device 101 (step S122'). However, auditory adjustment information A may be generated and initial auditory information may be presented before an operation is performed on the operation device 101, and auditory information based on the auditory adjustment information A may be presented after the operation is performed on the operation device 101. <Configuration> This is the same as the first embodiment.

[0053] <Processing> The process of this modification will be described with reference to FIG. 6. After the processes of steps S11, S172′, and S12 described in the modification 2 of the first embodiment, the process of step S121 described in the modification 1 of the first embodiment is executed. Then, the process of step S15′ is executed, and the operation information O in When it is determined that the operation information O inThe presentation signal generating unit 12 receives the trigger signal T and sends the trigger signal T, which indicates that the audio information has been input, to the presentation signal generating unit 12 (step S15'). t to the output unit 13, which then transmits it to the auditory information presentation device 102. The auditory information presentation device 102 receives this presentation signal V t According to the above, auditory information is presented at each time t (step S122').

[0054] <Features of Modification 3 of First Embodiment> Even in this case, the same effects as in the first embodiment can be obtained.

[0055] [Second embodiment] This embodiment is a modification of the first embodiment, and differs from the first embodiment in that hearing information is generated using hearing adjustment information based on the operation content.

[0056] <Configuration> As illustrated in Fig. 1, the pseudo-tactile presentation device 2 of this embodiment has an auditory adjustment information generating unit 11, a presentation signal generating unit 12, an output unit 13, an input unit 14, an operation detecting unit 15, an auditory adjustment information generating unit 27, a control unit 18, and a storage unit 19, and executes each process under the control of the control unit 18. The hardware configuration of the pseudo-tactile presentation device 2 will be described later. An operation device 101 is connected to the input unit 14 of the pseudo-tactile presentation device 2 by wire or wirelessly. An auditory information presentation device 102 is connected to the output unit 13 of the pseudo-tactile presentation device 2 by wire or wirelessly. As described above, the operation device 101 and the auditory information presentation device 102 may be of any type.

[0057] <Processing> In this embodiment, as a pre-processing, association information (e.g., a table) that associates a plurality of operation contents with hearing adjustment information corresponding to each of the operation contents is stored in the storage unit 19. The operation contents stored in the association information may be operation contents based on a motion of a body part performed on the operation device 101, or may be operation contents based on a motion of a body part performed on an operation device (a second operation device) other than the operation device 101 (not shown). In this embodiment, for the sake of simplicity of explanation, an example will be described in which operation contents based on a motion of a body part performed on the operation device 101 are stored in the storage unit 19. However, this does not limit the present invention.

[0058] The multiple operation contents stored in the storage unit 19 may be of any type. For example, multiple operation contents with different lengths of operation time and / or different numbers of operations on the operation device 101 may be stored in the storage unit 19. For example, if the operation device 101 is a physical key, operation contents such as "one short press operation," "two consecutive short press operations," and "long press operation" are stored. However, this does not limit the present invention.

[0059] The hearing adjustment information associated with each of the multiple operation contents may be of any type. However, the hearing adjustment information associated with each of the multiple operation contents is not all the same, and at least some of the hearing adjustment information is different from each other.

[0060] For example, different values ​​of the same hearing adjustment parameter (e.g., the amount of generated delay) may be associated with different operation contents. For example, for the same hearing adjustment parameter (e.g., the amount of generated delay), the value of the hearing adjustment parameter specified by the hearing adjustment information associated with a certain operation content (a first operation content) may be different from the value of the hearing adjustment parameter specified by the hearing adjustment information associated with another operation content (a second operation content different from the first operation content).

[0061] Also, for example, different types of hearing adjustment parameters (e.g., an occurrence delay amount and a stop delay amount) may be associated with different operation contents, respectively. For example, the type of hearing adjustment parameter (e.g., an occurrence delay amount) specified by hearing adjustment information associated with a certain operation content (first operation content) may be different from the type of hearing adjustment parameter (e.g., a stop delay amount) specified by hearing adjustment information associated with another operation content (a second operation content different from the first operation content).

[0062] There may also be a correlation between the operation contents and the pseudo-tactile sensation presented by the auditory information based on the auditory adjustment information associated with the operation contents. For example, the longer the operation time on the operation device 101, the greater the auditory adjustment information corresponding to the pseudo-tactile sensation (the greater, heavy feeling) may be associated with the operation contents. For example, the longer the operation time on the operation device 101, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated with the operation contents. In addition, when a plurality of operation contents with different operation counts are stored in the storage unit 19, the greater the number of operations on the operation device 101, the greater the auditory adjustment information corresponding to the pseudo-tactile sensation may be associated with the operation contents. For example, the greater the number of operations on the operation device 101, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated with the operation contents.

[0063] In addition, when an operation based on the movement of a body part is performed on the operation device 101, operation information O representing the operation content is output from the operation device 101. Among such operation information O, only operation information representing a specific operation content (for example, a pressing operation performed for a long time) is output as the above-mentioned operation information O. in (operation information based on the movement of a body part). This specific operation content may be any one of a plurality of operation contents stored in the correspondence information of the storage unit 19, or may be other than that.

[0064] Under the above assumptions, the process of the second embodiment will be described with reference to FIGS. When the user 100 operates the operating device 101 using a body part, operation information O representing the operation content is sent to the input unit 14 and then to the operation detection unit 15. The operation detection unit 15 sends the input operation information O to the hearing adjustment information generation unit 27 (step S261).

[0065] Thereafter, the hearing adjustment information generator 11 executes the process of step S11 described in the first embodiment.

[0066] Operation information O is input to the auditory adjustment information generation unit 27. The auditory adjustment information generation unit 27 refers to the association information in the storage unit 19, extracts auditory adjustment information associated with the operation content represented by the operation information O (for example, a press-down operation performed only once for a short period of time), and sends this as auditory adjustment information A to the presentation signal generation unit 12 (step S272').

[0067] The subsequent processes in steps S12, S15', and S122' are as described in the second modification of the first embodiment.

[0068] <Features of the second embodiment> This also provides the same effect as in the first embodiment. Furthermore, by generating auditory information using auditory adjustment information based on the operation content, it is possible to present a pseudo-tactile sensation according to the operation content.

[0069] For example, if different values ​​of the same hearing adjustment parameter are associated with different operation contents in the association information in the storage unit 19, the value of the hearing adjustment parameter when the operation content is a first operation content can be made different from the value of the hearing adjustment parameter when the operation content is a second operation content different from the first operation content. This makes it possible to present different pseudo haptics depending on whether the operation content is the first operation content or the second operation content.

[0070] For example, if different types of hearing adjustment parameters are associated with different operation contents in the association information in the storage unit 19, the type of hearing adjustment parameter when the operation content is a first operation content can be made different from the type of hearing adjustment parameter when the operation content is a second operation content different from the first operation content. This makes it possible to present different pseudo haptics depending on whether the operation content is the first operation content or the second operation content.

[0071] In addition, if there is a correlation between the operation content in the correspondence information in the memory unit 19 and the pseudo-tactile sensation presented by auditory information based on auditory adjustment information corresponding to the operation content, a pseudo-tactile sensation correlated to the operation content can be presented.

[0072] [Modification 1 of the second embodiment] In the case where the operation contents stored in the correspondence information of the storage unit 19 are operation contents based on the motion of a body part performed on a second operation device other than the operation device 101 (not shown), when an operation based on the motion of a body part is performed on the second operation device, operation information O representing the operation contents is output from the second operation device. Also, when an operation based on the motion of a body part is performed on the operation device 101, operation information O representing the operation contents is output from the second operation device. in is output from the operation device 101. Operation information O and operation information O in are sent to the input unit 14, and further to the operation detection unit 15. The other processes are the same as those in the second embodiment.

[0073] [Modification 2 of the second embodiment] 6, in the second embodiment or the first modified example of the second embodiment, auditory adjustment information A may be generated (step S272') and initial auditory information may be presented (step S121) before an operation is performed on the operation device 101, and auditory information based on the auditory adjustment information A may be presented (step S122') after an operation is performed on the operation device 101. Note that the presentation of the initial auditory information continues until auditory information based on the auditory adjustment information A is presented.

[0074] [Third embodiment] This embodiment is a modification of the first embodiment, and differs from the first embodiment in that hearing information is generated using hearing adjustment information based on an operation state.

[0075] <Configuration> As illustrated in FIG. 1, the pseudo-tactile presentation device 3 of this embodiment has an auditory adjustment information generating unit 11, a presentation signal generating unit 12, an output unit 13, an input unit 14, an operation detecting unit 15, a state detecting unit 36, an auditory adjustment information generating unit 37, a control unit 18, and a storage unit 19, and executes each process under the control of the control unit 18. The hardware configuration of the pseudo-tactile presentation device 3 will be described later. An operation device 101 is connected to the input unit 14 of the pseudo-tactile presentation device 3 by wire or wirelessly. An auditory information presenting device 102 is connected to the output unit 13 of the pseudo-tactile presentation device 3 by wire or wirelessly. As described above, the operation device 101 and the auditory information presenting device 102 may be of any type, but in this embodiment, a case where the operation device 101 is a touch panel is illustrated.

[0076] <Processing> In this embodiment, as a preprocessing, association information (e.g., a table) that associates a plurality of operation states with hearing adjustment information corresponding to each of the operation states is stored in the storage unit 19. The operation states stored in the association information may be operation states based on a motion of a body part performed on the operation device 101, or may be operation states based on a motion of a body part performed on an operation device (a second operation device) other than the operation device 101 (not shown). In this embodiment, for the sake of simplicity of explanation, an example will be described in which an operation state based on a motion of a body part performed on the operation device 101 is stored in the storage unit 19. However, this does not limit the present invention.

[0077] The multiple operation states stored in the storage unit 19 may be of any type. For example, multiple operation states with different operation speeds and / or strengths on the operation device 101 may be stored in the storage unit 19. For example, if the operation device 101 is a touch panel, operation states such as "slow operation state," "fast operation state," "strong force operation state," and "weak force operation state" are stored. However, this does not limit the present invention.

[0078] The hearing adjustment information associated with each of the multiple operation contents may be of any type. However, the hearing adjustment information associated with each of the multiple operation contents is not all the same, and at least some of the hearing adjustment information is different from each other.

[0079] For example, different values ​​of the same hearing adjustment parameter (e.g., the amount of generated delay) may be associated with each of the different operation states. For example, for the same hearing adjustment parameter (e.g., the amount of generated delay), the value of the hearing adjustment parameter specified by the hearing adjustment information associated with a certain operation state (a first operation state) may be different from the value of the hearing adjustment parameter specified by the hearing adjustment information associated with another operation state (a second operation state different from the first operation state).

[0080] Also, for example, different hearing adjustment parameters (e.g., an onset delay amount and a stop delay amount) may be associated with each of the different operation states. For example, the type of hearing adjustment parameter (e.g., an onset delay amount) specified by hearing adjustment information associated with a certain operation state (a first operation state) may be different from the type of hearing adjustment parameter (e.g., a stop delay amount) specified by hearing adjustment information associated with another operation state (a second operation state different from the first operation state).

[0081] There may also be a correlation between an operation state and a pseudo-tactile sensation presented by auditory information based on auditory adjustment information associated with the operation state. For example, the slower the operation state, the greater the auditory adjustment information corresponding to the pseudo-tactile sensation (large, heavy feeling) may be associated. For example, the slower the operation state, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated. In addition, the stronger the operation state of force, the greater the auditory adjustment information corresponding to the pseudo-tactile sensation. For example, the stronger the operation state of force, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated.

[0082] In addition, when an operation based on the movement of a body part is performed on the operation device 101, operation information O representing the operation content and operation state is output from the operation device 101. Of such operation information O, only operation information representing a specific operation content (e.g., long press) is output as the above-mentioned operation information O. in (Operation information based on the movement of body parts).

[0083] Under the above assumptions, the process of the third embodiment will be described with reference to FIG. 1 and FIG. First, after the process of step S11 described in the first embodiment, the process of step S15 is performed. However, the operation device 101 exemplified in this embodiment is a physical key, and the user 100 can operate the operation device 101 with his / her finger (touch operation in the example of this embodiment). When an operation based on the movement of a body part is performed on the operation device 101, operation information O indicating the operation content and operation state is output from the operation device 101. The operation information O output from the operation device 101 is input to the input unit 14 of the pseudo-tactile presentation device 1, and sent to the operation detection unit 15. The operation information O in In step S15, the input of the operation information O (operation information O representing a specific operation) is monitored. in If it is determined that no operation information has been input, the operation detection unit 15 detects the operation information O inOn the other hand, the operation information O in When it is determined that the operation information O in The control unit 34 sends a trigger signal T, which indicates that the input has been made, to the hearing adjustment information generating unit 37, and sends the operation information O to the state detecting unit .

[0084] The state detection unit 36 ​​to which the operation information O has been input extracts the operation state represented by the operation information O, and sends information C representing the operation state to the hearing adjustment information generation unit 37 (step S361).

[0085] The auditory adjustment information generation unit 37 refers to the correspondence information in the memory unit 19, extracts auditory adjustment information associated with the operation state represented by the input information C (e.g., a slow operation state), and sends this to the presentation signal generation unit 12 as auditory adjustment information A (step S372).

[0086] The subsequent process in step S122 is as described in the first embodiment.

[0087] <Features of the third embodiment> This also provides the same effect as in the first embodiment. Furthermore, by generating auditory information using auditory adjustment information based on the operation state, it is possible to present a pseudo-tactile sensation according to the operation state.

[0088] For example, if different values ​​of the same hearing adjustment parameter are associated with different operation states in the association information in storage unit 19, the value of the hearing adjustment parameter when the operation state is a first operation state can be made different from the value of the hearing adjustment parameter when the operation state is a second operation state different from the first operation state. This makes it possible to present different pseudo haptics depending on whether the operation state is the first operation state or the second operation state.

[0089] For example, if different types of hearing adjustment parameters are associated with different operation states in the association information in the storage unit 19, the type of hearing adjustment parameter when the operation state is a first operation state can be made different from the type of hearing adjustment parameter when the operation state is a second operation state different from the first operation state. This makes it possible to present different pseudo haptics depending on whether the operation state is the first operation state or the second operation state.

[0090] In addition, if there is a correlation between an operation state in the correspondence information of the memory unit 19 and a pseudo-tactile sensation presented by auditory information based on auditory adjustment information corresponding to the operation state, a pseudo-tactile sensation correlated to the operation state can be presented.

[0091] [Modification 1 of the third embodiment] In the case where the operation state stored in the correspondence information of the storage unit 19 is an operation state based on a body part motion performed on a second operation device other than the operation device 101 (not shown), when an operation based on a body part motion is performed on the second operation device, operation information O representing the operation state is output from the second operation device. Also, when an operation based on a body part motion is performed on the operation device 101, operation information O representing the operation state is output from the second operation device. in is output from the operation device 101. Operation information O and operation information O in are sent to the input unit 14, and further to the operation detection unit 15. The other processes are the same as those in the third embodiment.

[0092] [Modification 2 of the third embodiment] 5, in the third embodiment or the first modified example of the third embodiment, initial auditory information may be presented before an operation is performed on the operation device 101 (step S121), and auditory information based on the auditory adjustment information A may be presented after an operation is performed on the operation device 101 (step S122). Note that the presentation of the initial auditory information continues until the auditory information based on the auditory adjustment information A is presented.

[0093] [Fourth embodiment] This embodiment is a modification of the first embodiment, and differs from the first embodiment in that hearing information is generated using hearing adjustment information based on the state of the target device (device state).

[0094] <Configuration> As illustrated in FIG. 1, the pseudo-tactile presentation device 4 of this embodiment has an auditory adjustment information generating unit 11, a presentation signal generating unit 12, an output unit 13, an input unit 14, an operation detecting unit 15, a state detecting unit 46, an auditory adjustment information generating unit 47, a control unit 18, and a storage unit 19, and executes each process under the control of the control unit 18. The hardware configuration of the pseudo-tactile presentation device 4 will be described later. The input unit 14 of the pseudo-tactile presentation device 4 is connected to an operation device 101 by wire or wirelessly. The output unit 13 of the pseudo-tactile presentation device 4 is connected to an auditory information presenting device 102 by wire or wirelessly. As described above, the operation device 101 and the auditory information presenting device 102 may be of any type, but in this embodiment, a case will be described in which the operation device 101 is a physical key (for example, a button of a remote controller) that is pressed and operated by a finger of a user 100.

[0095] <Processing> In this embodiment, as a preprocessing, association information (e.g., a table) that associates a plurality of device states (states of the target device) with hearing adjustment information corresponding to each of the device states is stored in the storage unit 19. The target device may be the pseudo-tactile presentation device 4, a device including the pseudo-tactile presentation device 4 (e.g., a remote controller having physical keys that is the operation device 101), or a device external to the pseudo-tactile presentation device 4. The target device may be a device having a function that is correlated with the function of the pseudo-tactile presentation device 4, or a device having a function that is not correlated with the function of the pseudo-tactile presentation device 4.

[0096] The multiple device states stored in the storage unit 19 may be of any type. For example, multiple device states with different functional states and / or environments may be stored in the storage unit 19. For example, if the device is a remote controller with a function of increasing the volume each time a button is pressed, device states such as "volume level" and "temperature" are stored. However, this does not limit the present invention.

[0097] The hearing adjustment information associated with each of the multiple device states may be of any type, but the hearing adjustment information associated with each of the multiple device states is not all the same, and at least some of the hearing adjustment information is different from each other.

[0098] For example, different values ​​of the same hearing adjustment parameter (e.g., the amount of generated delay) may be associated with each of the different device states. For example, for the same hearing adjustment parameter (e.g., the amount of generated delay), the value of the hearing adjustment parameter specified by the hearing adjustment information associated with a certain device state (first state) may be different from the value of the hearing adjustment parameter specified by the hearing adjustment information associated with another device state (second state different from the first state).

[0099] Also, for example, different hearing adjustment parameters (e.g., an occurrence delay amount and a stop delay amount) may be associated with each of the different device states. For example, the type of hearing adjustment parameter (e.g., an occurrence delay amount) specified by hearing adjustment information associated with a certain device state (first state) may be different from the type of hearing adjustment parameter (e.g., a stop delay amount) specified by hearing adjustment information associated with another device state (a second state different from the first state).

[0100] Also, there may be a correlation between the device state and the pseudo-tactile sensation presented by the auditory information based on the auditory adjustment information associated with the device state. For example, the more transitional the device state (for example, a functional state in which a button on the remote controller is pressed many times and the volume is increased), the greater the auditory adjustment information corresponding to the pseudo-tactile sensation (large, heavy feeling) may be associated. For example, the more transitional the device state, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated, or the auditory adjustment information corresponding to the slower change rate may be associated. In addition, the higher the device state is at a higher temperature, the greater the auditory adjustment information corresponding to the pseudo-tactile sensation may be associated. For example, the higher the device state is at a higher temperature, the greater the auditory adjustment information corresponding to the occurrence delay amount and / or the stop delay amount and / or the duration and / or the volume of the auditory information may be associated.

[0101] Under the above assumptions, the process of the fourth embodiment will be described with reference to FIG. 1 and FIG. First, after the process of step S11 described in the first embodiment, the process of step S15 is performed. However, in step S15, the operation information O in If it is determined that no operation information has been input, the operation detection unit 15 detects the operation information O in On the other hand, the operation information O in When it is determined that the operation information O in The hearing adjustment information generating unit 17 and the state detecting unit 46 receive a trigger signal T indicating that the hearing adjustment information generating unit 17 and the state detecting unit 46 have been input.

[0102] The state detection unit 46, to which the trigger signal T is input, detects the device state of the target device, and sends information C representing the device state to the hearing adjustment information generation unit 47 (step S461).

[0103] The auditory adjustment information generation unit 47 refers to the correspondence information in the memory unit 19, extracts auditory adjustment information associated with the device state (e.g., volume level) represented by the input information C, and sends this to the presentation signal generation unit 12 as auditory adjustment information A (step S472).

[0104] The subsequent process in step S122 is as described in the first embodiment.

[0105] <Features of the fourth embodiment> This also provides the same effect as in the first embodiment. Furthermore, by generating auditory information using auditory adjustment information based on the device state, it is possible to present a pseudo-tactile sensation according to the device state.

[0106] For example, if different values ​​of the same hearing adjustment parameter are associated with different device states in the association information in storage unit 19, the value of the hearing adjustment parameter when the device state is a first state can be made different from the value of the hearing adjustment parameter when the device state is a second state different from state 1. This makes it possible to present different pseudo haptics depending on whether the device state is the first state or the second state.

[0107] For example, if different types of hearing adjustment parameters are associated with different device states in the association information in storage unit 19, the type of hearing adjustment parameter when the device state is a first state can be made different from the type of hearing adjustment parameter when the device state is a second state different from state 1. This makes it possible to present different pseudo haptics depending on whether the device state is the first state or the second state.

[0108] In addition, if there is a correlation between the device state in the correspondence information in the memory unit 19 and the pseudo-tactile sensation presented by auditory information based on auditory adjustment information corresponding to the device state, a pseudo-tactile sensation correlated to the device state can be presented.

[0109] [Modification 1 of the fourth embodiment] 5, initial auditory information may be presented before an operation is performed on the operation device 101 (step S121), and auditory information based on the auditory adjustment information A may be presented after an operation is performed on the operation device 101 (step S122). Note that presentation of the initial auditory information continues until auditory information based on the auditory adjustment information A is presented.

[0110] [Modification 2 of the fourth embodiment] Before operation information is input to the pseudo-tactile presentation device, the device state may be detected and auditory adjustment information A may be generated, and after an operation is performed on the operation device 101, auditory information based on the auditory adjustment information A may be presented.

[0111] <Configuration> This is the same as the fourth embodiment.

[0112] <Processing> The process of this modified example will be described with reference to FIG. The state detection unit 46 detects the device state of the target device, and sends information C representing the device state to the hearing adjustment information generation unit 47 (step S461).

[0113] Next, after the processing of step 11 described in the first embodiment is executed, the auditory adjustment information generation unit 47 refers to the correspondence information in the memory unit 19, extracts auditory adjustment information associated with the device state represented by the input information C, and sends this to the presentation signal generation unit 12 as auditory adjustment information A (step S472').

[0114] The subsequent processes in steps S12, S15', and S122' are as described in the second modification of the first embodiment.

[0115] [Modification 3 of the fourth embodiment] 6, in the second modification of the fourth embodiment, initial auditory information may be presented before an operation is performed on the operation device 101 (step S121), and auditory information based on the auditory adjustment information A may be presented after an operation is performed on the operation device 101 (step S122'). Note that the presentation of the initial auditory information continues until the auditory information based on the auditory adjustment information A is presented.

[0116] [Fifth embodiment] In the first to fourth embodiments and their modified examples, when operation information based on the movement of a body part of the user 100 is input, the auditory information based on the auditory adjustment information is presented to the user 100. However, when operation information based on the movement of a human being, an animal, or an operating device (e.g., a robot, etc.) other than the user 100 is input, the auditory information based on the auditory adjustment information may be presented to the user 100. Even in this case, if the user 100 is watching the state of the operation, the user 100 perceives an indirect pseudo-tactile sensation (e.g., a sensation such as "it seems heavy") from the presented auditory information. In addition, such pseudo-tactile sensation can be changed by controlling at least one of the amount of start delay, the amount of stop delay, the amount of continuation, or the volume of the auditory information.

[0117] The fifth embodiment differs from the first to fourth embodiments and their modified examples only in that, instead of operation information based on the movement of a body part of the user 100 being input to the pseudo-tactile presentation device, operation information based on the movement of a body part of a human or animal other than the user 100, or the movement of a part of a motion device, is input to the pseudo-tactile presentation device, and the user 100 watches this. The rest is the same as the first to fourth embodiments and their modified examples.

[0118] <Features of the fifth embodiment> Even in this case, it is possible to obtain the same effects as those of the first to fourth embodiments and their modified examples, however, the pseudo-tactile sensation perceived by the user 100 is an indirect pseudo-tactile sensation such as "it feels heavy."

[0119] [Sixth embodiment] In the fifth embodiment, an example was shown in which auditory information based on auditory adjustment information is presented to the user 100 when operation information based on the operation of an operating device is input. As a further modification, auditory information based on auditory adjustment information may be presented to the user 100 when operation information for operating an operating device (for example, a crane in a crane game) is output. In such a case, the user 100 also perceives an indirect pseudo-tactile sensation such as "it looks heavy." In addition, such pseudo-tactile sensation can be changed by controlling at least one of the generation delay amount, the stop delay amount, the duration, or the volume of the auditory information. For example, when the operating device is a crane in a crane game, at least one of the generation delay amount, the stop delay amount, the duration, or the volume of the auditory information presented to the user 100 when the crane grabs an object can be controlled to make the user perceive the weight or hardness of the grabbed object as being different. However, in this case, the generation delay amount corresponds to the time from when the operation information is output to when the presentation of the auditory information starts, and the stop delay amount corresponds to the time from when the operation information is output to when the presentation of the auditory information ends.

[0120] <Configuration> 7, the pseudo-tactile sense presentation device 6 of this embodiment includes an output unit 64 and an operation control unit 65 instead of the input unit 14 and the operation detection unit 15. Other than this, the configuration of the pseudo-tactile sense presentation device 6 is the same as any one of the pseudo-tactile sense presentation devices 1 to 4 of the first to fourth embodiments.

[0121] The output unit 64 of the pseudo-tactile presentation device 6 is connected to an operating device 601 by wire or wirelessly. The operating device 601 may be any device that performs an operation. For example, the operating device 601 is a device that performs a movement to perform a mechanical action on an object. The operating device 601 may perform a mechanical action in a state where it is in contact with the object, or may perform a mechanical action in a state where it is not in contact with the object. Examples of mechanical actions performed on an object include grabbing, pinching, lifting, pushing, pulling, compressing, stretching, moving, separating, and combining the object. Specific examples of the operating device 601 include a robot and a crane in a crane game, but these do not limit the present invention.

[0122] <Processing> The processing of this embodiment will be described below, focusing on the differences from the processing of the first to fourth embodiments and their modified examples. <Differences from the process in the first embodiment> The processing of the sixth embodiment based on the processing of the first embodiment will be described with reference to Fig. 5. When based on the processing of the first embodiment, the processing of the sixth embodiment is as follows.

[0123] First, the process of step S11 described in the first embodiment is executed.

[0124] Next, instead of step S15, the following step S65 is executed. The operation control unit 65 controls the operation of the operation device 601. The operation control unit 65 receives operation information O for operating the operation device 601. out When the control unit 65 is not outputting the operation information O, the control unit 65 continues the control. On the other hand, the operation control unit 65 outputs the operation information O for operating the operating device 601 at an arbitrary timing, a predetermined timing, or a specified timing. out The output operation information O out The input motion information O is sent to the output unit 64 and then to the motion device 601. out At this time, it is assumed that the user 100 is watching the operation of the operation device 601. The operation control unit 65 performs an operation based on the operation information Oout The hearing adjustment information generating unit 17 then sends a trigger signal T indicating that the hearing adjustment information has been output (step S65).

[0125] When the trigger signal T is sent to the hearing adjustment information generating unit 17, the hearing adjustment information generating unit 17 generates hearing adjustment information A for controlling the hearing information (step S172). out (2) the amount of stop delay corresponds to the time from when the operation information O in This point corresponds to the time from when the user inputs the audio information until the presentation of the audio information is terminated (the same applies to the rest of this embodiment).

[0126] A specific example of the (1) amount of delay that occurs in this embodiment is as follows. Operation information out is output from either the output unit 64 or the operation control unit 65 until the presentation of auditory information starts. Information that can identify the time Ds1' There is no limit to the magnitude of the generated delay amount, but for example, any time Ds1' between 0-2000 [ms], or information that can identify the time Ds1', may be used as the generated delay amount. When Ds1'=α, Ds1'=β, and α>β, the magnitude of the pseudo-haptic sensation (e.g., a sense of weight) perceived when Ds1'=α is greater than the magnitude of the pseudo-haptic sensation perceived when Ds1'=β. For example, the longer the time Ds1', the greater the pseudo-haptic sensation that can be presented.

[0127] A specific example of the (2) stop delay amount in this embodiment is as follows. Operation information out is output from either the output unit 64 or the operation control unit 65 until the presentation of the auditory information is terminated. Information that can identify the time De1' There is no limit to the magnitude of the stop delay amount. For example, any time De1' between 0 - 2000 [ms], or information that can identify the time De1' may be used as the stop delay amount. However, the relationship Ds1' < De1' is satisfied. When De1' = α and De1' = β, and α > β, the magnitude of the pseudo-tactile sensation perceived when De1' = α is greater than the magnitude of the pseudo-tactile sensation perceived when De1' = β. For example, the longer the time De1', the greater the pseudo-tactile sensation that can be presented.

[0128] Note that the continuation amount in (3) and the volume of the auditory information in (4) of this embodiment are the same as those in the first embodiment.

[0129] The processing of the subsequent step S122 is as described in the first embodiment.

[0130] ≪Differences from the processing of Modification Example 1 of the First Embodiment≫ Using FIG. 5, the processing of the sixth embodiment based on the processing of Modification Example 1 of the first embodiment will be described. When based on the processing of Modification Example 1 of the first embodiment, the processing of the sixth embodiment is as follows. After the processing of step S11 described above, the processing of step S121 is executed. Thereafter, the processing of steps S65, S172, and S122 is executed.

[0131] ≪Differences from the processing of Modification Example 2 of the First Embodiment≫ Using FIG. 6, the processing of the sixth embodiment based on the processing of Modification Example 2 of the first embodiment will be described. When based on the processing of Modification Example 2 of the first embodiment, the processing of the sixth embodiment is as follows.

[0132] First, the processing of step S11, step S172', and step S12 described in Modification Example 2 of the first embodiment is executed.

[0133] Next, instead of step S15' described in Modification Example 2 of the first embodiment, the following step S65' is executed. The operation control unit 65 controls the operation of the operating device 601. The operation information O for the operation control unit 65 to operate the operating device 601 outWhen the control unit 18 is not outputting the operation information O, the control unit 18 continues the control. On the other hand, the operation control unit 65 outputs the operation information O for operating the operating device 601 at an arbitrary timing, a predetermined timing, or a specified timing. out The output operation information O out The input motion information O is sent to the output unit 64 and then to the motion device 601. out At this time, it is assumed that the user 100 is watching the operation of the operation device 601. The operation control unit 65 performs an operation based on the operation information O out The trigger signal T, which indicates that the has been output, is sent to the presentation signal generating unit 12 (step S65').

[0134] When the trigger signal T is sent to the presentation signal generating unit 12, the process of step S122' described in the second modification of the first embodiment is executed.

[0135] <Differences from the process of the third modified example of the first embodiment> The process of the sixth embodiment based on the process of the modified example 3 of the first embodiment will be described with reference to FIG. 6. When based on the process of the modified example 3 of the first embodiment, the process of the sixth embodiment is as follows. After the processes of steps S11, S172', and S12 described above, the process of step S121 described in the modified example 1 of the first embodiment is executed. Then, the processes of steps S65' and S122' are executed.

[0136] <Differences from the process of Modification 1 of the second embodiment> The processing of the sixth embodiment based on the processing of the modified example 1 of the second embodiment will be described with reference to Fig. 6. When based on the processing of the modified example 1 of the second embodiment, the processing of the sixth embodiment is as follows.

[0137] First, the pre-processing described in the second embodiment is executed. On this premise, when an operation based on a motion based on a body part is performed on a second operation device (not shown), operation information O representing the operation content is output from the second operation device. The operation information O is sent to the hearing adjustment information generation unit 27. Note that the second operation device may be operated based on a body part of the user 100, or may be operated based on a motion based on a body part of a human being or animal other than the user 100. An example of the second operation device is an operation device for controlling the operation device 601 (for example, an operation device for operating a crane in a crane game). In this case, the operation information O is also sent to the operation control unit 65, and the operation control unit 65 generates the operation information O based on the operation information O. out may be generated and output (step S261).

[0138] Thereafter, the process of step S11 described in the first embodiment, the process of step S272' described in the second embodiment, and the process of step S12 described in the second modification of the first embodiment are executed. Then, the process of step S65' is executed. When the trigger signal T is sent to the presentation signal generation unit 12, the process of step S122' described in the second modification of the first embodiment is executed.

[0139] <Differences from the process of Modification 2 of the second embodiment> The process of the sixth embodiment based on the process of the modified example 2 of the second embodiment will be described with reference to FIG. 6. When based on the process of the modified example 2 of the second embodiment, the process of the sixth embodiment is as follows. As shown in FIG. 6, the processes of steps S261 and S11 are executed, and the operation information O for operating the operating device 601 is generated. out Before the output of the hearing adjustment information A, the initial hearing information is generated (step S272′), the operation information O for operating the operation device 601 is presented (step S121), and the operation information A is generated (step S273′). out After the auditory information based on the auditory adjustment information A is output (step S65'), the auditory information based on the auditory adjustment information A may be presented (step S122'). Note that the presentation of the initial auditory information continues until the auditory information based on the auditory adjustment information A is presented.

[0140] <Differences from the process of Modification 1 of the third embodiment> The process of the sixth embodiment based on the process of the modified example 1 of the third embodiment will be described with reference to Fig. 5. When based on the process of the modified example 1 of the third embodiment, the process of the sixth embodiment is as follows.

[0141] The pre-processing described in the third embodiment is executed. On this premise, after the processing of step S11 described in the first embodiment, the processing of step S65 is executed. However, the trigger signal T output from the operation control unit 65 is sent to the hearing adjustment information generation unit 37.

[0142] As described above, when an operation based on the movement of a body part is performed on the second operation device (not shown), operation information O indicating the operation content and operation state is output from the second operation device. Note that the second operation device may be operated based on a body part of the user 100, or may be operated based on a movement based on a body part of a human being or animal other than the user 100. An example of the second operation device is an operation device for controlling the operation device 601 (for example, an operation device for operating a crane in a crane game). In this case, the operation information O is also sent to the operation control unit 65, and the operation control unit 65 outputs the operation information O based on the operation information O. out You can also generate and output

[0143] The operation information O output from the second operating device is input to the state detection unit 36, and the processes of steps S361 and S372 described in the third embodiment and the process of step S122 described in the first embodiment are executed.

[0144] <Differences from the process of the second modification of the third embodiment> The process of the sixth embodiment based on the process of the first modified example of the first embodiment will be described with reference to FIG. 5. When based on the process of the first modified example of the first embodiment, the process of the sixth embodiment is as follows. After the process of step S11 described above, the process of step S121 is executed. Then, the processes of steps S65, S172, and S122 are executed.

[0145] <Differences from the processing in the fourth embodiment> The processing of the sixth embodiment based on the processing of the fourth embodiment will be described with reference to Fig. 5. When based on the processing of the fourth embodiment, the processing of the sixth embodiment is as follows.

[0146] The pre-processing described in the fourth embodiment is executed. On this premise, after the processing of step S11 described in the first embodiment, the processing of step S65 is executed. However, the trigger signal T output from the operation control unit 65 is sent to the hearing adjustment information generation unit 47 and the state detection unit 46.

[0147] When trigger signal T is input, state detection unit 46 detects the device state of the target device and sends information C representing the device state to hearing adjustment information generation unit 47. As described above, the target device may be any type, but one example of the target device is operating device 601 (step S461).

[0148] Thereafter, the process of step S472 described in the fourth embodiment and the process of step S122 described in the first embodiment are executed.

[0149] <Differences from the process of the first modified example of the fourth embodiment> As shown in FIG. 5, the operation information O for operating the operating device 601 is out Before the input of the input signal, the initial auditory information is presented (step S121), and the motion information O for operating the motion device 601 is output. out After the auditory information based on the auditory adjustment information A is output (step S65'), the auditory information based on the auditory adjustment information A may be presented (step S122). Note that the presentation of the initial auditory information continues until the auditory information based on the auditory adjustment information A is presented.

[0150] <Differences from the process of the second modification of the fourth embodiment> The processing of the sixth embodiment based on the processing of the modified example 2 of the fourth embodiment will be described with reference to Fig. 6. When based on the processing of the modified example 2 of the fourth embodiment, the processing of the sixth embodiment is as follows.

[0151] The pre-processing described in the fourth embodiment is executed. On this premise, the state detection unit 46 detects the device state of the target device, and sends information C representing the device state to the hearing adjustment information generation unit 47. As described above, the target device may be any type, but one example of the target device is the operating device 601 (step S461).

[0152] Next, the process of step 11 described in the first embodiment, the process of step S472' described in the first modification of the fourth embodiment, and the process of step S12 described in the second modification of the first embodiment are executed. After that, when the process of step S65' described above is executed and the trigger signal T is sent to the presentation signal generation unit 12, the process of step S122' described in the second modification of the first embodiment is executed.

[0153] <Differences from the process of the third modified example of the fourth embodiment> The process of the sixth embodiment based on the process of the modified example 3 of the fourth embodiment will be described with reference to FIG. 6. When based on the process of the modified example 3 of the fourth embodiment, the process of the sixth embodiment is as follows. As shown in FIG. 6, the processes of steps S461 and S11 are executed, and the operation information O for operating the operating device 601 is generated. out Before the output of the operation device 601, the auditory adjustment information A is generated (step S472′), a presentation instruction signal is generated (step S12), initial auditory information is presented (step S121), and operation information O for operating the operation device 601 is output. out After the auditory information based on the auditory adjustment information A is output (step S65'), the auditory information based on the auditory adjustment information A is presented (step S122'). Note that the presentation of the initial auditory information continues until the auditory information based on the auditory adjustment information A is presented.

[0154] <Features of the sixth embodiment> Even in this case, it is possible to obtain the same effects as those of the first to fourth embodiments and their modified examples, however, the pseudo-tactile sensation perceived by the user 100 is an indirect pseudo-tactile sensation such as "it feels heavy."

[0155] [Hardware configuration] The pseudo-tactile presentation devices 1-4 and 6 in each embodiment are devices configured by a general-purpose or dedicated computer having a processor (hardware processor) such as a CPU (central processing unit) and memories such as a RAM (random-access memory) and a ROM (read-only memory) executing a predetermined program. That is, the pseudo-tactile presentation devices 1-4 and 6 in each embodiment have, for example, a processing circuit configured to implement each unit that each device has. This computer may have one processor and memory, or may have multiple processors and memories. This program may be installed in the computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units may be configured using electronic circuits that realize processing functions independently, rather than electronic circuits that realize functional configurations by reading programs like a CPU. In addition, the electronic circuits that configure one device may include multiple CPUs.

[0156] FIG. 8 is a block diagram illustrating the hardware configuration of the pseudo-tactile presentation device 1-4, 6 in each embodiment. As illustrated in FIG. 8, the pseudo-tactile presentation device 1-4, 6 in this example has a CPU (Central Processing Unit) 10a, an input unit 10b, an output unit 10c, a RAM (Random Access Memory) 10d, a ROM (Read Only Memory) 10e, an auxiliary storage device 10f, and a bus 10g. The CPU 10a in this example has a control unit 10aa, a calculation unit 10ab, and a register 10ac, and executes various calculation processes according to various programs loaded into the register 10ac. The input unit 10b is an input terminal to which data is input, a keyboard, a mouse, a touch panel, etc. The output unit 10c is an output terminal to which data is output, a display, a LAN card controlled by the CPU 10a that has loaded a predetermined program, etc. The RAM 10d is a static random access memory (SRAM), a dynamic random access memory (DRAM), or the like, and has a program area 10da in which a predetermined program is stored, and a data area 10db in which various data is stored. The auxiliary storage device 10f is, for example, a hard disk, a magneto-optical disc (MO), a semiconductor memory, or the like, and has a program area 10fa in which a predetermined program is stored, and a data area 10fb in which various data is stored. The bus 10g connects the CPU 10a, the input unit 10b, the output unit 10c, the RAM 10d, the ROM 10e, and the auxiliary storage device 10f so that information can be exchanged. The CPU 10a writes the program stored in the program area 10fa of the auxiliary storage device 10f to the program area 10da of the RAM 10d according to the loaded OS (Operating System) program. Similarly, the CPU 10a writes various data stored in the data area 10fb of the auxiliary storage device 10f to the data area 10db of the RAM 10d. The addresses on the RAM 10d where these programs and data are written are stored in the register 10ac of the CPU 10a.The control unit 10aa of the CPU 10a sequentially reads out these addresses stored in the register 10ac, reads out programs and data from the areas on the RAM 10d indicated by the read addresses, sequentially causes the calculation unit 10ab to execute the calculations indicated by the programs, and stores the calculation results in the register 10ac. With this configuration, the functional configuration of the pseudo-tactile presentation devices 1-4 and 6 is realized.

[0157] The above-mentioned program can be recorded in a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

[0158] The distribution of this program is performed, for example, by selling, transferring, lending, etc., a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network. As described above, a computer that executes such a program, for example, first temporarily stores in its own storage device the program recorded in the portable recording medium or the program transferred from the server computer. Then, when executing the process, the computer reads the program stored in its own storage device and executes the process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute the process according to the program, and further, each time the program is transferred from the server computer to the computer, the computer may execute the process according to the received program. Furthermore, the server computer may not transfer the program to the computer, and the above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by issuing an execution instruction and obtaining the result. In addition, the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (such as data that is not a direct instruction to a computer but has the nature of defining computer processing).

[0159] In each embodiment, the present device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0160] [Other modifications, etc.] The present invention is not limited to the above-mentioned embodiment. For example, in the above-mentioned embodiment, the operation device 101 and the auditory information presentation device 102 are provided outside the pseudo-tactile presentation device 1-4. However, the operation device 101 may be provided in the pseudo-tactile presentation device 1-4, and the auditory information presentation device 102 may be provided in the pseudo-tactile presentation device 1-4. Similarly, the operation device 601 may be provided in the pseudo-tactile presentation device 6, and the auditory information presentation device 102 may be provided in the pseudo-tactile presentation device 6.

[0161] In the above embodiment, the operation device 101 is described as a physical key or a touch panel. However, this is only an example, and any device may be used as the operation device 101 as long as it detects an operation based on the body part (e.g., hand, finger, foot, head, face, eye, etc.) of the user 100 or a human or animal other than the user or the operation device and outputs operation information (operation information based on the operation) indicating that the operation has been performed. For example, in addition to physical keys such as a keyboard or a touch panel, a pull tab, a knob, a fader, a mouse, a pedal, etc. may be used as the operation device 101. Examples of operations based on these operations include pressing down, pulling in, rotating, sliding, dragging, clicking, swiping, pinching, and stepping. In addition, a device that detects contact with a physical surface such as a table or floor by a touch detection technology using a camera or an electrostatic sensor, and detects virtual pressing down, pulling in, dragging, clicking, swiping, pinching, stepping, etc. may be used as the operation device 101. In the above embodiment, the case where the operation device 101 that detects an operation performed in a state where the body part of the user 100 is in contact with an object is exemplified, but this does not limit the present invention. That is, an operation device 101 that detects an operation based on the movement of the body part or the part of the operation device without the body part or the part of the operation device being in contact with an object and outputs operation information (operation information based on the movement) indicating that the operation has been performed may be used. For example, a device that detects an operation by the movement of a body part such as waving, nodding, moving the eyeballs, or blinking, or an operation by the movement of a part of the operation device such as grabbing, moving, or rotating may be used as the operation device 101. Examples of such an operation device 101 include a device that tracks the movement of a body part using a motion sensor such as an acceleration sensor or a gyro sensor, and an eye tracker that detects eye movement using known eye tracking techniques such as a corneal reflex method, a dark pupil method, and a stereo image method.For example, any device may be used as the operation device 101 as long as it detects an operation based on a movement of a body part that expresses the intention of the user 100 (for example, an intention to turn some kind of ON / OFF) and outputs operation information (operation information based on the movement of a body part) indicating that the operation has occurred.

[0162] In the above embodiment, the auditory information is a pure tone. However, this is only an example and does not limit the present invention. That is, the auditory information may be any acoustic information that can be captured by human hearing. For example, in addition to a pure tone, sounds other than a pure tone (natural sounds, music, voice, alarm sounds, noise sounds, mechanical sounds, etc.) may be the auditory information. In addition, the auditory information may be a sound of a single frequency, or may be a sound including sounds of multiple frequencies.

[0163] In the above embodiment, the auditory information presentation device 102 is a speaker. However, this is merely an example and does not limit the present invention. In other words, any device that presents (outputs) auditory information may be used as the auditory information presentation device 102. For example, in addition to a speaker, a bone conduction speaker, an alarm device, a musical instrument that can play automatically based on a presentation signal, a bell that can automatically ring based on a presentation signal, a device that produces a mechanical sound based on a presentation signal, or the like may be used as the auditory information presentation device 102.

[0164] There is no restriction on the spatial positional relationship between the auditory information presentation device 102 and the operation device 101. For example, the auditory information presentation device 102 may be a speaker or the like provided on the operation device 101 such as a physical key, a knob, a fader, or the like, may be configured integrally with the operation device 101 such as a touch panel, may be a speaker or the like physically independent from the operation device 101, or may be a device that produces sound by vibrating a surface such as a table.

[0165] The auditory information presentation device 102 may present auditory information at a position corresponding to the operation position of the operation device 101, or may present auditory information at a position uncorrelated to the operation position. For example, if the operation device 101 and the auditory information presentation device 102 are touch panels, the auditory information may be presented at a contact position of the touch panel by a finger or a position on the touch panel corresponding to that position or operation, or may present auditory information at a position uncorrelated to the contact position. In addition, for example, if the operation device 101 is a physical key, the auditory information presentation device 102 may be a speaker provided on the physical key, or may be a speaker independent of the physical key.

[0166] Furthermore, with respect to functions other than the above-mentioned perception of the pseudo-tactile sensation, the operation device 101 and the auditory information presentation device 102 may or may not be linked together. For example, when the operation device 101 is an operation button (physical key) of a remote controller, the auditory information presentation device 102 may be a speaker that emits a sound associated with the operation content of the remote controller, or the auditory information presentation device 102 may be a speaker that is unrelated to the remote controller.

[0167] Also, the pseudo-haptic sensation may be presented by controlling the auditory information to be presented based on at least one of the onset delay amount, the stop delay amount, the duration amount, or the volume of the auditory information, and controlling the frequency of the auditory information. As described above, other conditions being the same, the lower the frequency of the pseudo-haptic sensation, the greater the pseudo-haptic sensation such as a sense of weight, and the higher the frequency of the pseudo-haptic sensation, the smaller the pseudo-haptic sensation such as a sense of weight.

[0168] In addition, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device that executes the processes or on the need of the device. Needless to say, other appropriate modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0169] 1-4,6 Pseudo-tactile display device 12 Presentation signal generation section 36,46 Status detection unit 17, 27, 37, 47 Hearing adjustment information generator 101 Operating device 601 Operating device 102 Auditory information presentation device

Claims

1. a presentation signal generating unit that generates a presentation signal for presenting auditory information based on auditory adjustment information when operation information based on a movement is input, The hearing adjustment information is An occurrence delay amount corresponding to the time from when the operation information is input to when the presentation of the auditory information starts, or a stop delay amount corresponding to a time from when the operation information is input to when the presentation of the auditory information is terminated, the hearing tuning parameters correspond to a desired simulated haptic sensation; At least the occurrence delay amount is the hearing adjustment parameter, the first time is greater than the second time, and the pseudo-haptic sensation presented when the time from inputting the operation information to starting presentation of the hearing information is the first time is greater than the pseudo-haptic sensation presented when the time from inputting the operation information to starting presentation of the hearing information is the second time, or A pseudo-tactile presentation device, wherein at least the stop delay amount is the hearing adjustment parameter.

2. a presentation signal generating unit that generates a presentation signal for presenting auditory information based on the auditory adjustment information when operation information for operating the operating device is output, The hearing adjustment information is An occurrence delay amount corresponding to the time from when the operation information is output to when the presentation of the auditory information starts, or a stop delay amount corresponding to a time from when the operation information is output to when the presentation of the auditory information is terminated, the hearing tuning parameters correspond to a desired simulated haptic sensation; At least the occurrence delay amount is the hearing adjustment parameter, the first time is greater than the second time, and the pseudo-haptic sensation presented when the time from when the operation information is output to when the presentation of the hearing information starts is the first time is greater than the pseudo-haptic sensation presented when the time from when the operation information is output to when the presentation of the hearing information starts is the second time, or A pseudo-tactile presentation device, wherein at least the stop delay amount is the hearing adjustment parameter.

3. The pseudo-tactile presentation device according to claim 1 or 2, The hearing adjustment information is information based on operation content, A pseudo-tactile presentation device, wherein the value of the auditory adjustment parameter when the operation content is a first operation content is different from the value of the auditory adjustment parameter when the operation content is a second operation content different from the first operation content, and / or the type of the auditory adjustment parameter when the operation content is the first operation content is different from the type of the auditory adjustment parameter when the operation content is the second operation content.

4. The pseudo-tactile presentation device according to any one of claims 1 to 3, The hearing adjustment information is information based on an operation state, A pseudo-tactile presentation device, wherein the value of the auditory adjustment parameter when the operation state is a first operation state is different from the value of the auditory adjustment parameter when the operation state is a second operation state different from the first operation state, and / or the type of the auditory adjustment parameter when the operation state is the first operation state is different from the type of the auditory adjustment parameter when the operation state is the second operation state.

5. The pseudo-tactile presentation device according to any one of claims 1 to 4, the hearing adjustment information is information based on a device state, A pseudo-tactile presentation device, in which the value of the auditory adjustment parameter when the device state is a first state is different from the value of the auditory adjustment parameter when the device state is a second state different from the first state, and / or the type of the auditory adjustment parameter when the device state is the first state is different from the type of the auditory adjustment parameter when the device state is the second state.

6. A pseudo-tactile presentation method comprising a processing step executed by a presentation signal generating unit of the pseudo-tactile presentation device according to any one of claims 1 to 5.

7. A program for causing a computer to function as the pseudo-tactile presentation device according to any one of claims 1 to 5.

Images