Sensation control device, sensation control method, and sensation control system

Abstract

To provide a technology for presenting a sense corresponding to the physical characteristics of an operation part.SOLUTION: According to an aspect of the present invention, there is provided an information processing device including an operation unit, an operation detection unit that detects an operation of the operation unit and generates an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, wherein the information processing device includes an adjustment unit that adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation based on a mass and a size of the operation unit as physical characteristics, and the operation unit is a rotary operation unit. The adjustment unit adjusts at least one of the operation signal, the sense presentation signal, and the sense presentation when the operation of rotating the rotary operation unit is performed, based on the mass and the radius, the diameter, or the length of the maximum portion, using the mass and the radius, the diameter, or the length of the maximum portion of the operation unit as the physical characteristics.SELECTED DRAWING: Figure 45

Description

[Technical Field]

[0001] The present invention relates to a sensory control device, a sensory control method, and a sensory control system. [Background technology]

[0002] Conventionally, operating units that provide a sense of sensation by providing some kind of stimulus to a person have been known. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating units.

[0003] A game controller is known in which buttons and the like incorporating vibration devices are interchangeable (see, for example, Patent Document 3). Patent Document 3 discloses a technique for exchanging the vibration device itself to achieve different vibration intensities. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-220168 [Patent Document 2] Patent No. 5662425 [Patent Document 3] Special Publication No. 2020-523068 [Patent Document 4] Special Publication No. 2013-519961 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional technologies have a problem in that they do not adequately present sensations according to the physical characteristics of the operation unit. For example, in the case of a rotary operation unit, the sensation conveyed to the user operating the operation unit varies depending on the size and mass of the operation unit, even if the actuator is driven in the same way.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a technology for presenting sensations according to the physical characteristics of an operation unit. [Means for solving the problem]

[0007] In view of the above problems, the present invention provides a sensory control device having an operation unit, an operation detection unit that detects operation of the operation unit and generates an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, and is equipped with an adjustment unit that, using the mass and size of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating the operation unit based on the mass and size, wherein the operation unit is a rotary operation unit, and using the mass and radius, diameter, or length of the maximum part of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating to rotate the rotary operation unit based on the mass, radius, diameter, or length of the maximum part.

[0008] In view of the above problems, the present invention provides a sensory control device having an operation unit, an operation detection unit that detects operation of the operation unit and generates an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, and is equipped with an adjustment unit that, taking the mass and size of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating the operation unit based on the mass and size, wherein the operation unit is a pressure-type operation unit, and, taking the mass and length in the pressing direction of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when pressing the pressure-type operation unit based on the mass and length in the pressing direction.

[0009] In consideration of the above problems, the present invention provides a sensory control device having an operation unit, an operation detection unit that detects operation of the operation unit and generates an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, and is equipped with an adjustment unit that, using the mass and size of the operation unit as physical properties, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating the operation unit based on the mass and size, wherein the operation unit is a slide operation unit, and using the mass and any one of the slide amount, height, width, and thickness of the operation unit as physical properties, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when sliding the slide operation unit based on the mass and any one of the slide amount, height, width, and thickness.

[0010] In view of the above problems, the present invention provides a sensory control device having an operation unit, an operation detection unit that detects operation of the operation unit and generates an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, and is equipped with an adjustment unit that, using the mass and size of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating the operation unit based on the mass and size, wherein the operation unit is a pivot operation unit, and using the mass and length of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when tilting the pivot operation unit based on the mass and length of the operation unit. [Effects of the Invention]

[0011] It is possible to provide a technology that presents sensations according to the physical characteristics of the operating unit. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing a basic configuration of a sensation control system according to an embodiment of the present disclosure. [Figure 2] 1 is a block diagram illustrating a haptic control system as a first embodiment of a sensation control system according to the present disclosure. [Figure 3] 3 is an explanatory diagram illustrating an example of the configuration of a tactile presentation unit included in the tactile control system of FIG. 2 using an equivalent circuit using a Laplace transform operator. FIG. [Figure 4] 3 is an explanatory diagram showing an equivalent model of an example of a tactile presentation unit included in the tactile control system of FIG. 2. FIG. [Figure 5] 5 is an explanatory diagram illustrating an equivalent circuit and an internal structure of an example of the actuator shown in FIG. 4. FIG. [Figure 6] 1 is a flowchart illustrating a conversion model generation method using the conversion model generation system of the present disclosure. [Figure 7] 10 is a flowchart illustrating a specific example of a transformation model generation method and a tactile presentation method according to the present disclosure. [Figure 8] 10 is a flowchart illustrating a haptic control method using the haptic control system of the present disclosure. [Figure 9] FIG. 10 is an explanatory diagram of the physical characteristics of a pressure-type operating tool. [Figure 10] FIG. 10 is an explanatory diagram showing examples of physical parameters of a pressure-type operation tool. [Figure 11] 10A and 10B are explanatory diagrams illustrating the operation of a pressure-type operating tool. [Figure 12] FIG. 10 is an explanatory diagram showing the relationship between the sensory parameters and the physical parameters in a pressure-type operation tool. [Figure 13] FIG. 10 is an explanatory diagram showing the relationship between the sensory parameters and the physical parameters in a pressure-type operation tool. [Figure 14] FIG. 10 is an explanatory diagram showing the relationship between the sensory parameters and the physical parameters in a pressure-type operation tool. [Figure 15] FIG. 10 is an explanatory diagram showing the relationship between the sensory parameters and the physical parameters in a pressure-type operation tool. [Figure 16] FIG. 10 is an explanatory diagram of the physical characteristics of a pressure-type operating tool. [Figure 17] FIG. 10 is an explanatory diagram showing the relationship between the sensory parameters and the physical parameters in the rotary operating tool. [Figure 18] FIG. 2 is an explanatory diagram of the physical characteristics of a rotary manipulator. [Figure 19] FIG. 1 is a block diagram showing the configuration of a rotary operation mold tool. [Figure 20] FIG. 10 is a block diagram illustrating a haptic control system as a second embodiment of a sensation control system according to an embodiment of the present disclosure. [Figure 21] FIG. 21 is a sequence diagram showing the operation of the haptic control system shown in FIG. 20. [Figure 22] FIG. 2 is a diagram illustrating a first relational expression according to an example of a conversion model generation method of the present disclosure. [Figure 23] FIG. 10 is a diagram illustrating a second relational expression according to an example of the conversion model generation method of the present disclosure. [Figure 24] 10A and 10B are diagrams showing examples of temporal changes in the intensity of a drive signal supplied to a weight based on a tactile sensation signal. [Figure 25] FIG. 1 is a perspective view of a haptic control device. [Figure 26] This is an example of a client-server type haptic control system. [Figure 27] 1 is a diagram showing an example of an outline of an operation in which a user adjusts an operation feel using a haptic control device; [Figure 28] 1 is a diagram showing an example of an outline of an operation in which a user adjusts an operation feel using a haptic control device; [Figure 29] FIG. 2 is an example of a functional block diagram illustrating functions of a haptic control device. [Figure 30] FIG. 10 is a flowchart illustrating an example of a learning flow in generating a classification unit. [Figure 31] FIG. 10 is a flowchart showing the flow of learning the correspondence between the expression frequency of the affective parameter and the physical parameter. [Figure 32] 10 is an example of a flowchart illustrating a flow in which the tactile control device presents a user's preferred operation feel using the classification unit and the first to third conversion models. FIG. [Figure 33] 1A and 1B are diagrams showing an outline of the process by which a user adjusts the operational feel using a haptic control device. [Figure 34]FIG. 2 is an example of a functional block diagram illustrating functions of a haptic control device. [Figure 35] FIG. 10 is a flowchart illustrating an example of a flow of learning physical parameters (load-displacement curves) corresponding to expression frequencies. [Figure 36] FIG. 10 is a flowchart illustrating an example of a flow of curve fitting a load-displacement curve of a reference operating tool. [Figure 37] 10 is a flowchart illustrating an example of a process in which the haptic control device presents a user's preferred operation feel using a physical parameter conversion unit and a comparison unit. FIG. [Figure 38] FIG. 10 is a diagram illustrating an example of a neural network in the case where a classification unit is realized by a neural network. [Figure 39] FIG. 10 is a diagram illustrating an example of a decision tree when a classification unit is realized by a decision tree. [Figure 40] FIG. 10 is a diagram illustrating a first input screen for expression frequency. [Figure 41] 1 is an example of a functional block diagram of a haptic control system in which the haptic control device of the first embodiment is applied to a client-server system. [Figure 42] FIG. 10 is an example of a sequence diagram illustrating the operation of the haptic control system. [Figure 43] FIG. 10 is an example of a functional block diagram of a haptic control system in which the haptic control device of the second embodiment is applied to a client-server system. [Figure 44] FIG. 10 is an example of a sequence diagram illustrating the operation of the haptic control system of the second embodiment. [Figure 45] FIG. 10 is a diagram showing the configuration of a tactile control system of a sensation control system (Example 2). [Figure 46] FIG. 10 is a diagram illustrating an example of operation unit parameters. [Figure 47] 10A and 10B are diagrams illustrating differences in physical characteristics of the operation unit. [Figure 48] 1A to 1C are diagrams illustrating several methods for detecting the size and mass of an operation unit using an operation unit sensor. [Figure 49] 10A and 10B are diagrams illustrating a method for estimating the mass of an operation unit by calibration. [Figure 50] 10A and 10B are diagrams illustrating correction of the mass of the operation unit. [Figure 51] FIG. 10 is a flowchart illustrating a process for adjusting a haptic presentation signal in accordance with physical parameters of an operation unit to which the haptic control system is attached. [Figure 52] FIG. 10 is a flowchart illustrating a process of adjusting a haptic presentation signal in accordance with physical parameters of an operation unit to which the haptic control system is attached (modification). [Figure 53] FIG. 46 is a diagram showing the configuration of a tactile control system as a second embodiment of the sensation control system shown in FIG. 45, together with the flow of signals. [Figure 54] FIG. 10 is a sequence diagram illustrating how the communication device and the terminal device communicate with each other to estimate the affective parameters of the attached operation unit. [Figure 55] 10A and 10B are diagrams illustrating static characteristics obtained by a rigid pressing tool and dynamic characteristics obtained by a finger model pressing tool in which a rigid body and an elastic body are integrated. [Figure 56] 10A and 10B are diagrams illustrating the relative positions of a finger and an operating tool when the finger is deformed. [Figure 57] FIG. 10 is a diagram illustrating a finger model pressing tool. [Figure 58] 10A and 10B are diagrams illustrating generation of a sensation providing signal that gives a clicking sensation. [Figure 59] 1A and 1B are diagrams illustrating an example of a functional configuration and a block diagram of a pressure-type operating tool. [Figure 60] 10A and 10B are diagrams illustrating dynamic characteristics when the operating tool is pressed by the finger model pressing tool. [Figure 61] 10A and 10B are diagrams illustrating the transition over time of the relative positions of the finger model pressing tool and the operating tool. [Figure 62] FIG. 10 is a diagram for explaining dynamic characteristics in more detail along with periods A to C. [Figure 63] 10 is a diagram showing an example of dynamic characteristics when a plurality of operating tools with different dynamic characteristics are pressed by a finger model pressing tool; [Figure 64] FIG. 10 is a flowchart illustrating an example of a flow of determining a physical parameter correlated with an affective parameter. [Figure 65]FIG. 10 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of recovery (no feeling of recovery)", acquired by the processor in step ST153. [Figure 66] FIG. 10 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of being sucked in (not felt)," acquired by the processor in step ST153. [Figure 67] FIG. 10 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of recovery (no feeling of recovery)", acquired by the processor in step ST153. [Figure 68] FIG. 10 is a diagram showing a list of correlation coefficients between each sensory parameter and each dynamic characteristic. [Figure 69] 10 is an example of a sequence diagram in which a communication device and a terminal device communicate with each other to estimate affective parameters of a worn operating tool. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description will be omitted. [Aspect 1] (sensory control system) FIG. 1 shows a basic configuration of a sensation control system 100 according to aspect 1 of the present disclosure. The sensation control system 100 shown in FIG. 1 includes a sensory database 16, a storage unit 11, an input unit 4, a processor 101, and a sensation presentation unit 102. The storage unit 11 stores a sensory parameter-physical parameter conversion model (hereinafter simply referred to as a "conversion model 15"). The sensation presentation unit 102 is a component that presents sensations to a person, and can be configured, for example, as a tactile presentation unit (e.g., a tactile presentation unit 30 described below) that presents a tactile sensation, an auditory presentation unit such as a speaker that presents an auditory sensation, a visual presentation unit such as a display device that presents a visual sensation, or any combination thereof.

[0014] The conversion model 15 is a conversion model capable of converting a sensory parameter into a physical parameter correlated with the sensory parameter. Here, the sensory parameter is a parameter indicating the degree of sensory expression for a sensory presentation. Specifically, in the case of a sensory evaluation using the Semantic Differential Method (SD), for example, the sensory parameter may be a multi-level evaluation indicating which of two sensory expressions (e.g., adjectives, onomatopoeia, or sound symbolic words) the presented sensation is closest to. Specifically, combinations of two sensory expressions may be "comfortable-uncomfortable" or "light-heavy." The multi-level evaluation using the SD method may be expressed, for example, with the expression frequency of the sensory parameter for "most comfortable" being "1," with "2," "3," "4," and so on increasing toward "uncomfortable," and "7" being "least comfortable." The sensory parameter is not limited to a combination of two sensory expressions and may be the intensity of a single sensory expression. Alternatively, the sensory parameter may be a multi-dimensional parameter expressed by combining multiple axes of sensory expression. Physical parameters are included in physical characteristics related to sensory presentation, and there are multiple types of physical parameters. Physical characteristics related to sensory presentation are physical characteristics that can affect the entire sensory transmission system, including sensory presentation means such as the sensory presentation unit 102 and the body parts of the person, when presenting a sensory presentation to the person. In other words, physical characteristics related to sensory presentation are not limited to physical characteristics of the sensory presentation means, but can also include physical characteristics of the body parts of the person to whom the sensory presentation is presented.

[0015] Here, the description will be given assuming that the sensibility database 16 is stored in a storage unit (not shown) other than the storage unit 11, but the sensibility database 16 may also be stored in the storage unit 11. The processor 101 controls the overall operation of the sense control system 100. The processor 101 is a general term for one or more processors. For example, the components of the sense control system 100 may be shared and controlled by multiple processors, or one processor may control all the components. Furthermore, the components of the sense control system 100 may be connected to each other as long as they can communicate information with each other so as to be able to execute the transformation model generation method and the sense control method described below. There are no particular limitations on the connection method between them. For example, the components of the sense control system 100 may be connected via a wired connection or a wireless connection including a network connection. The sense control system 100 may be composed of multiple devices or a single device.

[0016] The conversion model 15 included in the sense control system 100 is obtained by the following conversion model generation method. In the conversion model generation method, first, the sensitivity database 16 stores, for one or more types of sensory presentation, correspondence information in which physical characteristics related to a predetermined sensory presentation are associated with sensitivity parameters indicating the degree of sensory expression for the sensory presentation (storage step). The processor 101 extracts physical parameters correlated with the sensitivity parameters from among multiple types of physical parameters included in the physical characteristics related to the sensory presentation based on the correspondence information for each of the one or more types of sensory presentation in the sensitivity database 16 (extraction step). Thereafter, the processor 101 generates the conversion model 15 based on the sensitivity parameters and the extracted physical parameters (generation step). The conversion model 15 generated in this manner is a conversion model that can convert newly accepted sensitivity parameters into physical parameters correlated with the sensitivity parameters. When executing the above-mentioned conversion model generation method, the sense control system 100 functions as a conversion model generation system. In addition, in the extraction step, in order to derive multiple types of physical parameters included in the physical characteristics related to sensory presentation, extraction may be performed from the physical characteristics related to the sensory presentation means, or from the physical characteristics of a system including human body parts.

[0017] The conversion model generation method may be executed by a conversion model generation system separate from the sense control system 100. In this case, the conversion model generation system includes at least a sensibility database 16 and a processor 101. The sense control system 100 may acquire a conversion model 15 obtained by the execution of the conversion model generation method by the separate conversion model generation system, and store the conversion model 15 in the storage unit 11. In this case, the sense control system 100 does not need to include the sensibility database 16.

[0018] Furthermore, the above-mentioned correspondence information stored in the affective database 16 may be updatable, and the transformation model 15 may also be updated based on the updated correspondence information. In particular, in the storing step of the transformation model generating method, the affective database 16 adds or updates the above-mentioned correspondence information for one or more types of sensory presentation. Next, in the extracting step, the processor 101 re-extracts physical parameters correlated with the affective parameters based on the correspondence information for each of the one or more types of sensory presentation in the affective database 16. Thereafter, in the generating step, the processor 101 updates the transformation model 15 based on the affective parameters and the newly extracted physical parameters.

[0019] The sense control system 100 executes the following sense control method. First, the sense control system 100 accepts input of affective parameters from a user or the like via the input unit 4 (acceptance step). Thereafter, the processor 101 converts the accepted affective parameters into physical parameters correlated with the affective parameters from among multiple types of physical parameters included in physical characteristics related to sensory presentation, based on the conversion model 15 (conversion step). Then, the processor 101 generates a sensory presentation signal based on the converted physical parameters and outputs it to the sense presentation unit 102 (output step). The sense presentation unit 102 presents a sensation to the user or the like based on the sensory presentation signal (sensation presentation step).

[0020] In this way, the sensory control system 100 can present sensations to a user or the like based on a sensory presentation signal that is based on physical parameters that correlate with the received sensory parameters, and can therefore present sensations to a user or the like that reflect human sensibilities.

[0021] (Tactile Control System 1) FIG. 2 shows the configuration of a haptic control system 1 as a first embodiment of the sensation control system 100 shown in FIG. 1, along with the flow of signals.

[0022] The haptic control system 1 shown in Fig. 2 has a main control device 10. The main control device 10 is a personal computer, a server, or the like, and has a processor (CPU) 14 and a storage unit 11 such as RAM or ROM. The main control device 10 is provided with calculation function units 12 and 13 that are executed by the processor 14.

[0023] The haptic control system 1 shown in Fig. 2 has an input / output device 3. The input / output device 3 includes an input unit 4, a display unit 5, and a processor that operates the input unit 4 and the display unit 5. The input / output device 3 and the main control device 10 are connected via various interfaces.

[0024] The tactile control system 1 includes a tactile presentation device 20. The tactile presentation device 20 includes a terminal processor 18 that controls its operation. The arithmetic function unit 13, which functions as the output unit of the main control device 10, and the tactile presentation device 20 are connected via interfaces such as cables and connectors, USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface, registered trademark), Ethernet (registered trademark), and Wi-Fi.

[0025] The storage unit 11 of the main control device 10 shown in FIG. 2 stores a conversion model 15. As described in the description of the sensory control system 100 in FIG. 1, the conversion model 15 is a conversion model capable of converting received sensory parameters into physical parameters correlated with the sensory parameters. The sensory parameters in this example are parameters indicating the degree of sensory expression for the tactile presentation. For example, the sensory parameters in this example may be a user's evaluation of the operating feel when operating a specific operating tool using a sensory expression. In other words, the sensory parameters in this example are input to reflect the operation of the specific operating tool. The physical parameters in this example are included in the physical characteristics related to tactile presentation, and there are multiple types of physical parameters. For example, the physical parameters in this example may be physical parameters included in the physical characteristics that realize the tactile presentation when the specific operating tool is operated. The physical parameters in this example can be used to operate the tactile presentation device 20 to reproduce the sensory expression of the specific operating tool.

[0026] The tactile presentation device 20 includes at least a tactile presentation unit 30. The tactile presentation device 20 controls the tactile presentation unit 30 based on a tactile presentation signal to present a tactile sensation to the user. Here, the tactile presentation unit 30 is an example of the sensation presentation unit 102 in FIG. 1.

[0027] The tactile sense providing unit 30 may provide a tactile sense by generating a resistance force or vibration. Examples of the tactile sense providing unit 30 that generates a resistance force or vibration include a voice coil motor (VCM), a linear actuator (either a resonant type or a non-resonant type), a piezoelectric element, an eccentric motor, a shape memory alloy, a magnetorheological fluid, and an electroactive polymer.

[0028] The tactile sense presentation unit 30 may present a tactile sensation by presenting a hot / cold sensation. An example of the tactile sense presentation unit 30 that presents a hot / cold sensation is a Peltier element. The Peltier element utilizes the heat transfer caused by the Peltier effect when a direct current is applied to two opposing metal plates, and the amount of heat on the surface of the metal plates changes depending on the direction of the current. By controlling the direction and amount of current, it is possible to make a part of the user's body, such as a finger, that touches the Peltier element feel a warm or cold temperature.

[0029] The tactile presentation unit 30 may be configured to present a tactile sensation by applying an electrical stimulus. An example of a tactile presentation unit 30 that applies an electrical stimulus is one that applies the electrical stimulus by capacitively coupling with a body part such as a user's fingertip. The tactile presentation unit 30 may be configured to present an aerial tactile sensation. An example of a tactile presentation unit 30 that presents an aerial tactile sensation is one that generates air vibrations using ultrasound or the like, and causes a body part such as a user's fingertip to resonate with the air vibrations, thereby presenting a tactile sensation.

[0030] 2, the tactile control system 1 may include an operation device 33, and the tactile sense providing unit 30 may provide a tactile sensation to a user operating the operation device 33. The tactile sense providing unit 30 may provide a predetermined operation feel to the user operating the operation device 33 by providing a tactile sensation. Specifically, the tactile sense providing unit 30 may provide an operation feel that imitates the operation feel of a predetermined operation tool. For example, examples of operation tools that may be used to simulate an operation feel include a push switch that accepts a pressing operation, a rotary switch that accepts a rotation operation, a joystick that accepts a tilt operation, a slide switch that accepts a slide operation on a slide operation unit, and a touch panel that accepts a touch operation, a pressing operation, a tracing operation, etc. on an operation panel.

[0031] Any type of device capable of performing operations similar to those of the above-described predetermined operating tool can be used as the operating device 33. Specifically, the operating device 33 may be in a form that imitates the predetermined operating tool, or may be in a form unrelated to the predetermined operating tool, such as an operating device such as an operating glove that is worn on the user's hand and receives operations by finger movements, etc.

[0032] The tactile sense providing unit 30 may provide a tactile sense to the user regardless of the operation of the operating device 33. In this case, the tactile sense control system 1 does not need to include the operating device 33.

[0033] 2, the tactile presentation device 20 may include various sensors such as a position sensor 27 and an acceleration sensor 28. By including various sensors, the tactile presentation device 20 can detect physical quantities of at least one of the tactile presentation device 20 itself, the operation device, and a body part of the user, and control the driving of the tactile presentation unit 30 based on the physical quantities. In addition to the above, other sensors that can be used include, for example, a torque sensor, an angular velocity sensor, a temperature sensor, a pressure sensor (including a barometric pressure sensor), a humidity sensor, a magnetic sensor, an optical sensor, an ultrasonic sensor, and an electromyographic sensor.

[0034] [An example of the tactile presentation unit 30] An example of the tactile sense presentation unit 30 included in the tactile control system 1 according to this embodiment will be described with reference to FIGS. 3 to 5. The tactile sense presentation unit 30 illustrated in FIGS. 3 to 5 reproduces the tactile sensation experienced when operating a pressure-type operating tool. The model pressure-type operating tool is a pressure-type operating tool such as a Tactile Switch (registered trademark) in which a dish-shaped leaf spring or a dome-shaped leaf spring generates an operating reaction force. The tactile sense presentation unit 30 reproduces a tactile sensation corresponding to a desired sensory parameter based on a tactile sense presentation signal provided by the main control device 10. By incorporating the tactile sense presentation unit 30 into the electronic circuitry of various devices, the tactile sense presentation unit 30 can be used in place of an actual pressure-type operating tool as a pressure-type operating tool that realizes a tactile sensation (here, an operating feel) corresponding to the desired sensory parameter. Furthermore, the tactile sense presentation device 20 can reproduce an operating reaction force to evaluate the relationship between the sensory parameter representing the operating feel and the physical parameter included in the physical characteristics that operate the tactile sense presentation device 20, and the evaluation can be used as a guideline for designing a pressure-type operating tool.

[0035] Fig. 4 shows an equivalent model illustrating an example of the components of the tactile sense presentation unit 30. Fig. 5 shows an equivalent circuit and internal structure of an actuator 39 included in the tactile sense presentation unit 30. The arrow F shown in Fig. 5 indicates the operation reaction force (vector quantity). Fig. 3 explains the operating principle of the tactile sense presentation unit 30 using an equivalent circuit that uses Laplace transform operators.

[0036] As shown in Fig. 4, the tactile sense presentation unit 30 may have a movable part 21. In this case, the operation device 33 shown in Fig. 2 is integrated with the movable part 21 shown in Fig. 4. Alternatively, the operation device 33 may be provided outside the system of the tactile sense presentation device 20, and the movable part 21 may be moved by operating the operation device 33. The tactile sense presentation unit 30 has an actuator 39. As shown in Fig. 5, the actuator 39 is provided with a bobbin 24 and a coil 25 wound around the bobbin 24. The bobbin 24 and the coil 25 are also part of the movable part 21.

[0037] As shown in FIG. 4, the tactile sense presentation unit 30 may include a spring member 26. The spring member 26 has a predetermined spring constant and is formed, for example, by a coil spring. The spring member 26 is held in a compressed state within the tactile sense presentation unit 30, for example, and applies an operation reaction force to the movable unit 21 in a direction opposite to the direction in which the movable unit 21 is pressed (upward in FIG. 4) under normal use. In FIG. 4, the spring constant of the spring member 26 is indicated by "Ks." As shown in FIG. 4, an operation reaction force acts on the movable unit 21 based on a viscosity coefficient "C" resulting from lubricating oil, sliding friction on the mechanism, and the like. Also, in FIG. 4, the stroke amount in the direction in which the movable unit 21 is pressed (downward in FIG. 4) is indicated by "x."

[0038] As shown in FIG. 5, the actuator 39 includes a cylindrical yoke 31 made of an iron-based magnetic material. The yoke 31 includes an outer yoke 31a and a center yoke 31b. A cylindrical magnet 32 ​​is fixed inside the outer yoke 31a. A cylindrical magnetic gap is formed between the center yoke 31b and the magnet 32, and a cylindrical bobbin 24 and coil 25 are inserted into the magnetic gap. As shown in FIG. 5, the amount of current flowing through the coil 25 is defined as "I," the magnetic flux density of the magnetic field generated by the magnet 32 ​​and crossing the coil 25 is defined as "B," the inductance of the coil 25 is defined as "L," and the electrical resistance including the coil 25 is defined as "R." The number of turns in the coil 25 is defined as "N." The operation reaction force "F" acting on the movable part 21 from the actuator 39 is controlled by a tactile presentation signal provided from the main control device 10 to the tactile presentation device 20.

[0039] In this example, a position sensor 27 provided in the tactile presentation device 20 shown in Fig. 2 detects the amount of movement (hereinafter referred to as "stroke amount") "x" of the movable part 21 in the pressing operation direction. In this example, an acceleration sensor 28 provided in the tactile presentation device 20 shown in Fig. 2 detects the acceleration of the movable part 21. In this example, an operation range variable unit 29 provided in the tactile presentation device 20 shown in Fig. 2 can change the total length of the stroke amount of the movable part 21 in the pressing operation direction.

[0040] The basic operation of the tactile presentation device 20 will be described with reference to Figures 3 to 5. The tactile presentation device 20 can present a tactile sensation to the movable part 21 via the operation device 33 by controlling the current "I" applied to the coil 25 of the tactile presentation unit 30. The tactile sensation presented here is a change in the operation reaction force "F" applied to a body part such as a user's finger pressing the movable part 21 in the pressing operation direction. This operation reaction force "F" is a resistance force that reproduces the operation reaction force of a pressure-type operation tool that generates an operation reaction force using a disc-shaped leaf spring or a dome-shaped leaf spring.

[0041] 4 shows a model of the tactile presentation unit 30. The following equation 1 expresses the operation of the tactile presentation device 20 as a "force" equation.

[0042]

number

[0043] The following equation 2 is obtained by transforming equation 1.

[0044]

number

[0045] In the equivalent circuit of the tactile presentation unit 30 shown in Fig. 5, the voltage acting on the coil 25 is indicated by "V" and the back electromotive force acting on the coil is indicated by "e". The following equation 3 shows the differential equation of "Ve" and an equation expressing this differential equation using the Laplace transform variable "s".

[0046]

number

[0047]

number

[0048] [Conversion model 15 generation process] Fig. 6 shows an example of a generation process (a conversion model generation method) for the conversion model 15 stored in the haptic control system 1 of Fig. 2. The conversion model generation method is executed by a conversion model generation system that includes at least an input unit, a storage unit, and a processor. "ST" in Fig. 6 indicates a processing step.

[0049] In STa, the conversion model generation system accepts input of sensory parameters from multiple users for each of one or more types of tactile presentation. Here, "one or more types of tactile presentation" is not limited to tactile presentations when the user operates a manipulation tool, but also includes tactile presentations given to the user when the user does not perform any manipulation. For example, one or more types of tactile presentations may be presented via a suit or gloves as tactile presentations corresponding to content such as a game or video, and sensory parameters based on how the user feels about each type may be accepted. This step is an example of a storage step in the conversion model generation method described for the sensory control system 100 of FIG. 1.

[0050] In STb of FIG. 6, the conversion model generation system extracts physical parameters correlated with sensory parameters from among physical characteristics related to various tactile presentations. This step is an example of an extraction step in the conversion model generation method described for the sensation control system 100 of FIG. 1. In STc, the conversion model generation system generates a conversion model 15. This step is an example of a generation step in the conversion model generation method described for the sensation control system 100 of FIG. 1. The conversion model 15 can be generated manually, by multiple regression analysis, machine learning, or various other analytical methods. The conversion model 15 may be a model capable of converting one type of sensory parameter into one type of physical parameter, a model capable of converting one type of sensory parameter into multiple types of physical parameters, a model capable of converting multiple types of sensory parameters into one type of physical parameter, or a model capable of converting multiple types of sensory parameters into multiple types of physical parameters. A model capable of converting multiple types of sensory parameters into multiple types of physical parameters may be generated by deriving information on complex correlations from information on the correlation between one type of sensory parameter and one type of physical parameter using machine learning or the like. The data structure of the conversion model 15 may be a correspondence table between the sensory parameters and the tactile parameters, or may be stored so that they can be calculated using a function.

[0051] Here, an example of a method for generating a conversion model 15 capable of converting multiple types of sensory parameters into multiple types of physical parameters will be described. In this example, first, in an extraction step, the conversion model generation system extracts information about the correlation between each of the multiple types of physical parameters and the sensory parameters for the multiple types of sensory parameters. In detail, the conversion model generation system extracts information about the multiple correlations by multiple regression analysis using each of the multiple types of sensory parameters as a response variable and the multiple types of physical parameters as explanatory variables. Here, examples of the information about the correlations include the coefficient of determination in the multiple regression analysis, a constant term, or values ​​derived from these.

[0052] Next, in a generation step, the conversion model generation system generates a first relational expression that explains each of the plurality of types of perceptual parameters using a plurality of types of physical parameters and information on a plurality of correlation degrees (first generation step). Specifically, the plurality of types of perceptual parameters are denoted by A1, A 2、 A n (n is a natural number), and multiple types of physical parameters are defined as P1, P2, ... P n The constant term and coefficient of determination in the multiple regression analysis are used as the sensitivity parameter A m (m is a natural number less than or equal to n) m1 , B m2 , B mn Then, the first relational expression can be expressed by the following equation 5.

[0053]

number

[0054] When Expression 5 is expressed as a matrix equation in which one side (here, the left side) is a column vector indicating a plurality of types of perceptual parameters and the other side (here, the right side) is the product of a coefficient matrix indicating information about a plurality of correlation degrees and a column vector indicating a plurality of types of physical parameters, the first relational expression is expressed as shown in Fig. 22. Here, the coefficient matrix is ​​a square matrix with n rows and n columns.

[0055] After a first generation step included in the generation step, the transformation model generation system generates second relational expressions that explain each of the multiple types of physical parameters based on the first relational expressions, using multiple types of sensory parameters and information on multiple correlation degrees (second generation step). Specifically, the transformation model generation system generates the second relational expressions by multiplying both sides of the first relational expression shown in Fig. 22 by the inverse matrix of the coefficient matrix from the left. As shown in Fig. 23, the second relational expression can be expressed with one side (here, the left side) being a column vector indicating the multiple types of physical parameters, and the other side (here, the right side) being the product of the inverse matrix of the coefficient matrix and the column vector indicating the multiple types of sensory parameters.

[0056] After the second generation step included in the generation step, the conversion model generation system generates a conversion model 15 capable of converting a plurality of types of sensory parameters into a plurality of types of physical parameters correlated with the plurality of types of sensory parameters based on the second relational expressions (third generation step). In this way, the conversion model generation system can generate a conversion model 15 capable of converting a plurality of types of sensory parameters into a plurality of types of physical parameters.

[0057] In the above example, the coefficient matrix is ​​described as a square matrix, but the coefficient matrix does not necessarily have to be a square matrix. For example, by using a pseudo-inverse matrix as the inverse matrix, it is possible to generate a conversion model 15 capable of converting multiple types of perceptual parameters into multiple types of physical parameters in the same way, even if the coefficient matrix is ​​not a square matrix.

[0058] The sensation control method by the sensation control system 100 shown in Fig. 1 can be executed as follows when using the conversion model 15 obtained in this example. First, in a receiving step, the sensation control system 100 receives input of multiple types of sensory parameters from a user or the like via the input unit 4. Then, in a converting step, the processor 101 converts the acquired multiple types of sensory parameters into multiple types of physical parameters correlated with the multiple types of sensory parameters based on the conversion model 15. Note that the outputting step and the sensation presenting step are similar to those described above, and therefore description thereof will be omitted.

[0059] [Examples of tactile presentation] The following describes an example of tactile presentation using the tactile presentation device 20 shown in FIG. 2, which simulates the feel of operating a predetermined operating tool. The sensory parameters of the conversion model 15 in this example are the expression levels of adjectives that express the feel of operating a pressure-type operating tool as the predetermined operating tool. The physical parameters of the conversion model 15 in this example are included in the physical characteristics that realize the sensation presented when operating a pressure-type operating tool as the predetermined operating tool. When the tactile control system 1 receives input of a specific sensory parameter via the input unit 4, it converts the received specific sensory parameter into a physical parameter using the conversion model 15. The sensory parameters assuming a pressure-type operating tool are the degree of sensory expression using adjectives, onomatopoeia, etc. that express the feel of operating a pressure-type operating tool when a person presses the pressure-type operating tool. The physical characteristics realized by the physical parameters include, for example, a displacement accompanying the operation (e.g., stroke amount), an operation reaction force (load), the speed, acceleration, and jerk of the movable part 21, the elasticity characteristics of a body part such as the operator's fingers, or quantities derived from these physical characteristics. A physical parameter is defined herein to include one or more variations of a physical property.

[0060] Fig. 7 is a flowchart illustrating a specific example of a conversion model generation method and a tactile presentation method. In the flowchart shown in Fig. 7, processing steps are indicated by "ST," with ST1, ST2, etc. including manual processing, and ST3, ST4, etc. including processing executed by processor 14 of main control device 10 shown in Fig. 2.

[0061] In ST1 in FIG. 7, multiple operating tools with the same function but different operating feel are prepared. In ST2, a sensory test is conducted by multiple users, and the operating feel of the prepared operating tools is classified by the expressiveness of adjectives as affective parameters. In ST3, the processor 14 of the haptic control system 1 associates the expressiveness of adjectives as affective parameters with physical parameters included in the physical characteristics that realize the sensory presentation when the operating tool is operated, based on a correlation coefficient or the like. Each affective parameter and physical parameter includes at least one variable. The associated affective parameter and physical parameter are stored as the affective database 16 shown in FIG. 1. In ST4, the processor 14 uses the conversion model 15 to convert the newly inputted affective parameter into a physical parameter correlated with the expressiveness of adjectives as affective parameters. The calculation function unit 12 generates a haptic presentation signal based on the physical parameters, and the calculation function unit 13 outputs the haptic presentation signal. This haptic presentation signal operates the haptic presentation device 20, and a haptic sensation is presented. By controlling at least one of the coefficients "Kv", "Ks", and "C" shown in Figure 4 using a tactile presentation signal based on a physical parameter, a tactile sensation corresponding to the sensory parameter is presented via the tactile presentation device 20.

[0062] In ST1 of FIG. 7 , for example, a plurality of pressure-type operating tools, such as Tact Switches (registered trademark), which are actual products having dish-shaped leaf springs or dome-shaped leaf springs, are prepared as operating tools. FIG. 11 schematically illustrates the change in the operation reaction force when the pressure-type operating tool is pressed. FIG. 11 illustrates physical characteristics that realize the sensation when the pressure-type operating tool is operated on a coordinate plane with the horizontal axis representing the displacement associated with the operation and the vertical axis representing the operation reaction force (load) acting on the user's body part, such as the fingers, performing the operation. In this specification, the "displacement associated with the operation of the operating tool" includes the operation amount of the operating tool, the operation time of the operating tool, or a combination of the operation amount and the operation time. In other words, the physical characteristics that realize the sensation when the operating tool is operated can be expressed as any of the relationship between the operation amount of the operating tool and the operation reaction force, the relationship between the operation time of the operating tool and the operation reaction force, and the relationship between the combination of the operation amount and the operation time of the operating tool and the operation reaction force. Furthermore, the "displacement associated with the operation of the manipulating tool" may also include a displacement resulting from the elasticity of the body part, such as the fingers of the operator operating the manipulating tool. In FIG. 11, the "displacement associated with the operation of the manipulating tool" is the operation amount of the pressure-type manipulating tool serving as the manipulating tool, which will be referred to as the "stroke amount 'x'" hereinafter as appropriate. The operation amount of the manipulating tool is a quantity in one-dimensional space, two-dimensional space, or three-dimensional space. In FIG. 11, the operation amount of the pressure-type manipulating tool serving as the manipulating tool is a quantity in one-dimensional space along the direction of the pressing operation. The manipulating tool may have a movable part that moves in association with the operation of the manipulating tool. The pressure-type manipulating tool serving as the manipulating tool has a knob portion as the movable part that is pressed by the user or the like. Therefore, the operation amount of the pressure-type manipulating tool may be the movement amount of the movable part of the pressure-type manipulating tool.

[0063] As shown in FIG. 11, a curve representing the physical characteristics that realize the sensation presented when a manipulation tool is operated on a coordinate plane with the horizontal axis representing the manipulation amount (the amount of movement of the movable part) of the manipulation tool and the vertical axis representing the manipulation reaction force is referred to as an FS curve (Force Stroke Curve), feeling curve, operating force curve, load-displacement curve, etc. Hereinafter, it will be referred to as a "load-displacement curve" as appropriate. As shown in FIG. 11, when a user presses the pressure-type manipulation tool, as the stroke amount "x" in the direction of the pressing manipulation increases, the manipulation reaction force gradually increases due to the compressive deformation of the disc-shaped leaf spring or dome-shaped leaf spring. When the stroke amount "x" reaches a maximum position Pmax, the manipulation reaction force reaches a maximum value Tmax. When the pressure-type manipulation tool is further pressed, the disc-shaped leaf spring or dome-shaped leaf spring buckles and deforms, inverting, and the manipulation reaction force suddenly decreases. When the stroke amount "x" reaches a minimum position Pmin, the manipulation reaction force reaches a minimum value Tmin. When the user subsequently presses the pressure-type operating tool further, the buckled plate-shaped or dome-shaped plate spring is compressed, and the reaction force continues to increase until the plate-shaped or dome-shaped plate spring reaches the final stroke position where it contacts the fixed contact. In Figure 11, the stroke at which the reaction force becomes equal to the maximum value Tmax while being pressed from the minimum position Pmin to the final position is defined as the load recovery position Pend.

[0064] When the user releases the pressure on the pressure-type operating tool after pushing it all the way to the final stroke position where the contacts are in contact, the knob portion, which serves as the movable part of the pressure-type operating tool, returns to its initial position due to the elastic restoring force of the disc-shaped leaf spring or dome-shaped leaf spring. The load-displacement curve when the operating device 33 returns has hysteresis with respect to the load-displacement curve when the displacement associated with the pressing operation increases, as shown in Figure 11. For ease of explanation, the operation will be described below using only the load-displacement curve when the displacement associated with the pressing operation increases.

[0065] Several pressure-type operating tools (23 in total) were classified into groups (A), (B), and (C) according to the total stroke length when pressed to the final stroke. Class (A) had a total stroke length of 0.25 mm or more and 0.35 mm or less, Class (B) had a total stroke length of 0.15 mm or more and less than 0.25 mm, and Class (C) had a total stroke length of less than 0.15 mm.

[0066] A sensory test was conducted by 25 users on the above-mentioned pressure-type operating tools. In the sensory test, the operating feel (tactile sensation) felt by the users was classified by expression frequency using the SD method. In this sensory test, a predetermined sensitivity parameter A was used as the sensitivity parameter, and evaluation was performed on a seven-point scale of "1," "2," "3," "4," "5," "6," and "7." In this sensory test, the sensitivity parameter A expression frequency of the pressure-type operating tools in category (A) varied widely from around "1" to around "6." The sensitivity parameter A expression frequency of the pressure-type operating tools in category (B) varied in the intermediate range from around "2.5" to around "3.5." The sensitivity parameter A expression frequency of the pressure-type operating tools in category (C) varied from around "3.5" to around "6." Here, the affective parameter A may be, for example, a parameter relating to "sense of decision," "comfort," "touch," etc. Specifically, if it is a parameter relating to "sense of decision," the parameter may be such that the lower the expression frequency, the "higher the sense of decision," and the higher the expression frequency, the "lower the sense of decision."

[0067] As mentioned above, the correlation between the sensory parameter A and the total stroke amount of the pressure-type operating tool as a physical parameter is not necessarily clear. Therefore, for the 23 pressure-type operating tools mentioned above, we focused on physical characteristics other than the classified total stroke amount and examined whether there was a correlation between the sensory parameter A and the physical parameters extracted from those physical characteristics. Figure 9 shows the load-displacement curves (i), (ii), and (iii) of three pressure-type operating tools with different total stroke amounts. In Figure 10(A), the area S4-1 of the depression in the load-displacement curve (i) and the area S4-2 of the depression in the load-displacement curve (ii) are extracted as variables of the physical quantity of the operation, and in Figure 10(B), the areas S4-1 and S4-2 are translated so that their respective minimum values ​​Tmin are the same and compared.

[0068] As shown in FIG. 11, area S4 is the area of ​​the depression in a coordinate plane with the horizontal axis representing the operation amount of the operating tool and the vertical axis representing the operation reaction force, from the maximum value Tmax through the minimum value Tmin to the point where the operation reaction force returns to the same maximum value Tmax. In other words, area S4 is the area of ​​the region defined in the coordinate plane by the load-displacement curve and a line parallel to the horizontal axis that passes through the maximum value Tmax of the load-displacement curve. The dimension representing area S4 is expressed as "distance (stroke amount) × load (of operation reaction force)," and this dimension is equivalent to energy (work volume). In other words, area S4 corresponds to the energy lost (energy lost) compared to the energy consumption anticipated by the user due to a decrease in the operation reaction force when the user operates the pressure-type operating tool. The presence of area S4 causes the user to feel a sense of being drawn in the direction of the pressing operation.

[0069] In addition, the operation reaction force shown by the load-displacement curve (iii) in Figure 9 has a preload when the stroke is zero. This preload causes so-called "play" in the operation. This "play" can also be used as one of the physical parameters.

[0070] FIG. 12 is a graph showing the relationship between the affective parameter A, which is the expression frequency measured by the SD method, and the area S4, which is a physical parameter extracted from the physical characteristics that realize the sensation presented when the operating tool is operated. The horizontal axis of FIG. 12 represents the affective parameter A, and the vertical axis represents the area S4, which is a physical parameter. As shown in FIG. 12, for a total of 23 press-type operating tools with total stroke amounts ranging from 0.35 to 0.15 mm, there is a correlation between the size of the area S4 shown in FIG. 11 and the expression frequency of the affective parameter A. That is, for the 23 press-type operating tools, there is a negative correlation, in which the expression frequency of the affective parameter A decreases as the area S4 increases. Here, when there is a correlation between the affective parameter and the physical parameter, the absolute value of the correlation coefficient between the affective parameter and the physical parameter is preferably 0.5 or greater, and more preferably 0.7 or greater.

[0071] In normalizing the area S4, which is a physical quantity, it is preferable to limit the total stroke amount of the pressure-type operating tool to a predetermined range. For example, the total stroke amount of the pressure-type operating tool is preferably 0.05 mm or more and less than 0.5 mm, and more preferably 0.05 mm or more and less than 0.35 mm.

[0072] In this way, in the above example, the change in the actuation reaction force in response to the displacement associated with the operation of the operating implement has at least a maximum portion and a minimum portion. The physical parameter includes a variable based on the area of ​​a depression in a coordinate plane whose axes are the displacement associated with the operation and the actuation reaction force, from the maximum portion through the minimum portion to the coordinate where the actuation reaction force transitions to the same magnitude as the maximum portion. Here, the maximum portion is the portion including the maximum value Tmax in the load-displacement curve shown in FIG. 11, and the minimum portion is the portion including the minimum value Tmin in the load-displacement curve shown in FIG. 11.

[0073] 2 uses the conversion model 15 to convert the expression frequency of the inputted sensory parameter A into an area S4, which is a physical parameter correlated with the inputted sensory parameter A, and the calculation function unit 12 calculates a load-displacement curve including the area S4 and sets one tactile presentation signal including the load-displacement curve. Alternatively, the calculation function unit 12 calculates multiple load-displacement curves having the same area S4 but different strokes or loads, and sets multiple tactile presentation signals including these load-displacement curves. Alternatively, the conversion model 15 may store multiple types of load-displacement curves associated with the size of the area S4 in advance in association with the expression frequency of the sensory parameter A, and the calculation function unit 12 may read out information on the load-displacement curve corresponding to the expression frequency of the inputted sensory parameter A from the storage unit 11 and generate a tactile presentation signal.

[0074] The input unit 4 of the input / output device 3 can accept input of integer representation frequencies such as "2," "3," and so on, or representation frequencies including decimals such as "2," "2.5," "3," "3.5," and so on, as well as numerical ranges of representation frequencies such as "2-2.5," "2.5-3," "3-3.5," "3.5-4," and so on. The haptic control system 1 uses the conversion model 15 to convert one or more force-displacement curves having area S4, which is a physical parameter corresponding to the representation frequency of the sensory parameter input received via the input unit 4. Information about the converted one or more force-displacement curves is output to the input / output device 3, which then displays one or more force-displacement curves on the display unit 5. The user can check one of the force-displacement curves displayed on the display unit 5 or select one of the multiple displayed force-displacement curves. When this confirmation command or selection command is given to the processor 14 from the input unit 4, a tactile presentation signal based on the selected load-displacement curve is set in the calculation function unit 12, and the tactile presentation signal is output from the calculation function unit 13 to the tactile presentation device 20. As a result, when the operation device 33 of the tactile presentation device 20 is operated, an operation feel corresponding to the expression frequency of the sensory parameter desired by the user can be presented.

[0075] Furthermore, physical parameters such as "stroke amount" and "magnitude of operation reaction force" may be directly specified as input items from the input unit 4 along with the expression frequency of the affective parameter A. For example, when the haptic control system 1 receives an input of "stroke amount 0.25-0.35 mm" as a physical parameter along with the expression frequency of the affective parameter A via the input unit 4, it selects a load-displacement curve having an area S4 that matches the expression frequency of the adjective from among the multiple load-displacement curves included in class (A), and generates a haptic signal based on this load-displacement curve. Alternatively, when the haptic control system 1 receives an input of a numerical item of "magnitude of operation reaction force" as a physical parameter along with the expression frequency of the affective parameter A via the input unit 4, it may generate a haptic signal based on both the expression frequency of the affective parameter A and the "magnitude of operation reaction force" as a physical parameter.

[0076] In the above description, the total stroke amount is limited to a range of, for example, 0.35-0.15 mm, and the physical parameter, which is the size of the area S4, and the affective parameter, which is the expressive frequency of the adjective, are associated with each other using this range as a reference. However, the size of the area S and the expressive frequency of the affective parameter may be associated with a numerical range other than the total stroke amount. For example, a predetermined numerical range may be set for the maximum value Tmax, minimum value Tmin, maximum value minus minimum value (Tmax-Tmin), click stroke (Pend-Pmax), push-in stroke (Pmax / (Pend-Pmax)), click stroke ratio (Pmax / Pend), and push-in stroke ratio (Pmax / (Pend-Pmax)) shown in FIG. 11, and this numerical range may be used as a reference. Alternatively, a predetermined numerical range may be set for the areas S1, S2, and S3 other than S4, or their ratios, and this numerical range may be used as a reference. Using these numerical ranges as a reference, it is possible to associate the size of the area S4, which is a physical parameter, with the expression frequency of the affective parameter.

[0077] For the 23 pressure-type operating tools described above, a sensory test of the operating feel was conducted by 25 users for sensory parameters other than sensory parameter A. The results are shown in Figures 13 to 15. Figures 13 to 15 show the relationship between the expression frequency of sensory parameters other than sensory parameter A and physical parameters other than area S4 that change depending on the expression frequency.

[0078] In Fig. 13, the horizontal axis represents the expression frequency of the affective parameter B. The vertical axis represents a variable related to the stroke amount of the pressure-type operating tool as a physical parameter, such as the "click stroke (Pend-Pmax)" shown in Fig. 11. Fig. 13 shows a negative correlation in which the expression frequency of the affective parameter B decreases as the "click stroke (Pend-Pmax)" as a physical parameter increases. The affective parameter B may be a parameter related to, for example, a "sense of decision," "comfort," or "tactile sensation," and specifically, when the parameter is related to "comfort," the parameter may be such that a lower expression frequency indicates "comfortable" and a higher expression frequency indicates "uncomfortable."

[0079] In this way, in the above example, the physical parameters include a variable related to the amount of displacement accompanying the operation. More specifically, it includes a "click stroke (Pend-Pmax)" which is the amount of displacement from the maximum point, through the minimum point, to the coordinate at which the operation reaction force changes to the same magnitude as the maximum point.

[0080] In Fig. 14, the horizontal axis represents the expression frequency of the sensory parameter C. The vertical axis represents a variable related to the load of the pressure-type operating tool as a physical parameter, such as Pmax shown in Fig. 11. Fig. 14 shows a positive correlation in which the expression frequency of the sensory parameter C decreases as Pmax as a physical parameter decreases. The sensory parameter C is a parameter related to, for example, "decisiveness," "comfort," "tactile sensation," etc., and specifically, when it is a parameter related to "tactile sensation," the smaller the expression frequency, the softer the perceived operation feel, and the greater the expression frequency, the harder the perceived operation feel.

[0081] In Fig. 15, the horizontal axis represents the expression frequency of the sensory parameter D. The vertical axis represents a variable related to the stroke amount of the pressure-type operating tool as a physical parameter, such as the "push-in stroke ratio (Pmax) / (Pend-Pmax)" shown in Fig. 11. Fig. 15 shows a positive correlation in which the expression frequency of the sensory parameter D increases as the "push-in stroke ratio (Pmax) / (Pend-Pmax)" as a physical parameter increases. The sensory parameter D is a parameter related to, for example, "decisiveness," "comfort," "tactile sensation," etc. Specifically, when the parameter is related to "tactile sensation," the higher the expression frequency, the sharper the tactile sensation is perceived, and the lower the expression frequency, the duller the tactile sensation is perceived.

[0082] Thus, in the above example, the physical parameters include a variable related to the amount of displacement accompanying the operation. More specifically, the physical parameters include a variable related to the "click stroke (Pend-Pmax)," which is the amount of displacement from the maximum point, through the minimum point, to the coordinate at which the operation reaction force transitions to the same magnitude as the maximum point, and the "push stroke ratio (Pmax) / (Pend-Pmax)," which is the ratio between "Pmax," which is the amount of displacement from the start of the operation to the maximum point.

[0083] The conversion model 15 may store a plurality of relationships as correlations between sensory parameters and physical parameters, including (1) the relationship between the expression frequency of sensory parameter A and the area S4, which is a physical parameter, shown in Fig. 12, (2) the relationship between the expression frequency of sensory parameter B and the click stroke, which is a physical parameter, shown in Fig. 13, (3) the relationship between the expression frequency of sensory parameter C and the maximum value minus the minimum value, which is a physical parameter, shown in Fig. 14, and (4) the relationship between the expression frequency of sensory parameter D and the push-in stroke ratio, which is a physical parameter, shown in Fig. 15. Any one or a plurality of relationships (1) to (4) are combined to calculate a physical parameter included in a physical quantity such as a load-displacement curve, and a tactile presentation signal is generated.

[0084] As mentioned above, the acceleration of the movable part 21 of the tactile sense providing unit 30 shown in Fig. 4 can be detected by the acceleration sensor 28. In an actual pressure-type operating tool, when a disc-shaped leaf spring or a dome-shaped leaf spring is pressed, it buckles, deforms, and inverts, generating vibrations, which are transmitted to the body part, such as a finger, that is performing the pressing operation, thereby providing an operating sensation.

[0085] Figures 16(A), (B), and (C) show simulation data showing the acceleration of the moving parts of three pressure-type operating tools when the three operating tools are pressed. A sensory test by users using the three pressure-type operating tools was conducted to investigate the relationship between the expression frequency of the sensory parameter E, which is related to the feel of the pressing operation, and the acceleration of the moving parts of the operating tools as a physical parameter. The peak-to-peak value of the acceleration when the disc-shaped leaf spring or dome-shaped leaf spring of the pressure-type operating tool undergoes buckling deformation was largest for the pressure-type operating tool in Figure 16(A), and decreased in the order of (B) and (C). Furthermore, in the sensory test by users, the expression frequency of the sensory parameter E for the operation of the pressure-type operating tool in Figure 16(A) was the smallest, and the expression frequency increased in the order of (B) and (C). The affective parameter E may be, for example, a parameter relating to "decisiveness," "comfort," "tactile sensation," etc. Specifically, if it is a parameter relating to "comfort," the parameter may be such that the smaller the expression frequency, the more "comfortable" it is, and the larger the expression frequency, the more "uncomfortable" it is.

[0086] Based on the sensory test described above, the conversion model 15 may store a correlation between the expression frequency of the sensory parameter E and the acceleration of the movable part of the operating tool, which is a physical parameter. The haptic control system 1 uses the conversion model 15 to convert the expression frequency of the sensory parameter E input via the input unit 4 into the acceleration of the movable part of the operating tool, which is a physical parameter, generate a haptic presentation signal based on this acceleration, and output the haptic presentation signal, thereby reproducing a desired operating feel with the haptic presentation device 20. For example, based on the physical parameters (amount of movement, speed, acceleration, jerk, etc.) of the movable part of the operating tool, a haptic presentation signal may be generated that controls the corresponding physical parameters of the movable part 21 of the haptic presentation device 20.

[0087] [Example of operation of the tactile presentation device 20] FIG. 8 shows a flowchart of an example of the control operation of the tactile presentation device 20. The processing shown in the flowchart is executed by the control operation of the processor 18 included in the tactile presentation device 20. In ST11 of FIG. 8, a tactile presentation signal is provided from the arithmetic function unit 13 to the processor 18 of the tactile presentation device 20, and in ST12, control based on a load-displacement curve selected based on the physical parameters is started. In ST13, when the operation device 33 is operated, detection signals related to the movable part 21 are obtained from the position sensor 27 and the acceleration sensor 28. The processor 18 calculates the difference between the movement profile of the load-displacement curve set corresponding to the expression frequency, which is an affective parameter, and the detected position of the movable part 21. In ST14, the current I provided to the coil 25 of the tactile presentation unit 30 is optimized, and a tactile sensation is presented so as to reproduce the expression frequency of the affective parameter desired by the user.

[0088] [Modification of the tactile presentation device 20] 17 to 19, a description will be given of a modified example of the tactile presentation device 20 included in the tactile control system 1. The tactile presentation device 40 shown in Fig. 19 reproduces the tactile sensation of a rotary operation tool. The rotary operation tool is, for example, a rotary switch.

[0089] 19 includes a processor 41, a tactile presentation unit 43, and a sensor 45. The tactile presentation device 40 presents a tactile sensation to a user who rotates an operation device 42. The operation device 42 may be mechanically incorporated into the tactile presentation device 40, or may be provided outside the tactile presentation device 40.

[0090] The tactile sense presentation unit 43 includes a resistance torque generator 43a and a rotational torque generator 43b. The resistance torque generator 43a variably applies a resistance torque in a direction opposite to the rotational direction in response to the rotation of the rotary operation unit of the operation device 42. The resistance torque generator 43a includes, for example, a yoke made of a magnetic material and a coil that applies a magnetic field to the yoke. A rotating plate that rotates in conjunction with the rotation of the rotary operation unit of the operation device 42 is located within the magnetic gap of the yoke, and a magnetorheological fluid is filled between the yoke and the rotating plate in the magnetic gap. Magnetic powder can also be used instead of the magnetorheological fluid. Controlling the current applied to the coil changes the coagulation state of the magnetorheological fluid, thereby varying the resistance torque. In addition to the above configuration, the resistance torque generator 43a can also include, for example, a rotary motor, which can vary the resistance torque. The rotational torque generator 43b variably applies a rotational torque in a rotational direction in response to the rotation of the rotary operation unit of the operation device 42. The rotational torque generator 43b can also include, for example, a rotary motor. The sensor 45 detects the rotation angle of the rotary operation part of the operation device 42 .

[0091] Figure 18 shows a load-displacement curve for the operational reaction force of a rotary switch, a rotary operating tool. A rotary switch divides 360 degrees (one rotation) into multiple divided angles, and the operational reaction force changes within each divided angle, with the same change in operational reaction force repeated within each divided angle. Figure 18 shows the change in operational reaction force within one divided angle. The horizontal axis of Figure 18 represents the rotation angle of the rotary operating part, which is the amount of operation of the rotary switch. The positive side of the vertical axis represents the magnitude of the resistance torque acting on the rotary operating part of the rotary switch in the direction opposite to the operating direction, and the negative side of the vertical axis represents the magnitude of the rotation torque acting on the rotary operating part in the same direction as the operating direction. The rotary switch has spring contacts within each divided angle. When a rotation operation begins within a divided angle, the spring contacts contract, increasing the resistance torque acting on the rotary operating part. When the resistance torque exceeds the maximum value Rmax, the restoring force of the spring contact pushes the rotary operation unit in the direction of rotation, reducing the resistance torque, and a rotational torque directed in the direction of operation acts on the rotary operation unit from the spring contact.As a result, when rotating the rotary operation unit, a tactile sensation is felt with the finger at each division angle.

[0092] The conversion model 15 stores correlations between the expression frequencies of sensory parameters related to rotation operations and physical parameters. The haptic control system 1 receives input of the expression frequencies of sensory parameters via the input unit 4. The processor 14 of the haptic control system 1 then converts the received sensory parameters into physical parameters using the conversion model 15 and generates a haptic presentation signal based on the physical parameters. The processor 14 then outputs the generated haptic presentation signal to a processor 41 included in a haptic presentation device 40 shown in FIG. 19. When a rotary operation unit of an operation device 42 is rotated by a body part such as a user's finger, the haptic presentation device 40 detects the rotation angle of the rotary operation unit using a sensor 45 and feeds back the detection output to the processor 41. The processor 41 controls the haptic presentation unit 43, thereby controlling the resistance torque and rotation torque when the rotary operation unit of the operation device 42 is rotated, and a haptic sensation simulating a rotary switch that reproduces the expression frequencies of the sensory parameters can be presented.

[0093] FIG. 17 is an explanatory diagram illustrating the change in resistance torque as an example of a physical characteristic related to the expression frequency of a sensory parameter. FIG. 17(A) shows the reaction force of four rotary switches when they are rotated as a load-displacement curve, and FIG. 17(B) shows the change in curvature on each of the load-displacement curves shown in FIG. 17(A). In a sensory test conducted by multiple users, it was found that when a rotary operation unit is rotated with a body part such as a finger, the resistance torque passes through a peak where the maximum value Rmax is reached. The conclusion was reached that the smaller the curvature of the change in the motion line at this peak, the higher the expression frequency of the sensory parameter F. That is, it was confirmed that the expression frequency of the sensory parameter F correlates with the curvature of the inflection point where the rotation load transitions from an increase to a decrease. Therefore, the conversion model 15 stores the correlation between the expression frequency of the sensory parameter F and a physical parameter whose variable is the curvature of the change in resistance torque, thereby enabling the tactile presentation device 40 to present a rotation operation feel that achieves the expression frequency of the sensory parameter F. The sensory parameter F may be, for example, a parameter relating to "decisiveness," "comfort," "tactile sensation," etc. Specifically, if it is a parameter relating to "tactile sensation," it may be a parameter such that the greater the expression frequency, the sharper the tactile sensation is perceived, and the smaller the expression frequency, the duller the tactile sensation is perceived.

[0094] In this way, in the above example, the change in the manipulation reaction force in response to the displacement caused by the manipulation of the manipulation implement has at least a maximum portion. Furthermore, the physical parameters include a variable related to the curvature of the maximum portion including the maximum value Rmax. Here, the maximum portion is the portion including the maximum value Rmax in the load-displacement curve shown in FIG. 18.

[0095] 18, physical parameters including variables related to the rise of the increase in rotational load, such as the angle of the resistance torque rise vector Tb from the starting point of the division angle in the rotation angle as the operation amount of the rotary switch, and the ratio of the areas Sa and Sb shown on the load-displacement curve at the rise portion of the resistance torque, can be associated with the frequency of expression of adjectives such as "stiff operation, resistance" as sensory parameters. In this way, in this example, the physical parameters include variables related to the rise of the operation reaction force from the start of the operation to the maximum portion.

[0096] Here, the area Sa shown in Fig. 18 is the area defined by the load-displacement curve, the horizontal axis, and a line parallel to the vertical axis that passes through the intersection of the load-displacement curve and the maximum value Rmax. In other words, the area Sa is the value obtained by integrating the load-displacement curve over the range of rotation angles from the start point of the division angle in the rotation angle as the operation amount to the maximum value Rmax of the actuation reaction force. The area Sb is the area defined by the load-displacement curve, the vertical axis, and a line parallel to the horizontal axis that passes through the maximum value Rmax. In other words, the area Sb is the area obtained by subtracting the area Sa from the area of ​​a rectangle whose one side is the rotation angle value at the intersection of the load-displacement curve and the maximum value Rmax and whose other side is the maximum value Rmax. That is, if the load-displacement curve changes linearly on the coordinate plane from the start of the operation until the actuation reaction force reaches its maximum value Rmax, as shown by the dashed line in FIG. 18, then the ratio of area Sa to area Sb is 1:1, and the smaller the area Sb is relative to area Sa, the more the load-displacement curve bulges toward the positive side of the vertical axis on the coordinate plane. In other words, the ratio of area Sa to area Sb indicates the degree of bulging of the load-displacement curve. Furthermore, the rising vector Tb of the resistance torque as a physical parameter shown in FIG. 18 includes a variable related to the derivative of the actuation reaction force with respect to the actuation amount. Similarly, the physical parameter may include a variable related to the derivative of the actuation reaction force with respect to the actuation time, or a variable related to the second-order derivative of the change in the actuation reaction force.

[0097] Furthermore, a variable related to the maximum value Dmax of the rotational torque (pull-in torque) acting in the same direction as the operation direction shown in Fig. 18, i.e., the magnitude of the pull-in amount at which the direction of the rotational load is reversed, can be associated with the frequency of describing adjectives such as "fast rotation." In this way, in this example, the physical parameters include a variable related to the magnitude of the pull-in amount whose minimum part has a negative sign. Here, the minimum part is the part of the load-displacement curve shown in Fig. 18 that includes the maximum value Dmax.

[0098] In the above example, FIG. 18 was described as showing a load-displacement curve related to the reaction force of a rotary switch, which is a rotary operating tool. However, FIG. 18 can also be used as a graph showing a load-displacement curve related to the reaction force of a slide switch that accepts a slide operation of a slide operation unit. That is, the horizontal axis of FIG. 18 represents the amount of slide operation of the slide operation unit, and the positive side of the vertical axis represents the reaction force to the slide operation of the slide operation unit. As the amount of slide operation of the slide operation unit increases, the reaction force gradually increases and reaches a maximum value Rmax. After exceeding Rmax, the reaction force begins to decrease, becomes a pulling force acting in the same direction as the operation direction, and reaches a minimum value Dmax (the maximum value on the negative side of the vertical axis). In this way, a tactile sensation can be presented in response to the operation of the slide switch. Note that the correlation between the sensory parameters and physical parameters described for the rotary switch also applies to slide switches.

[0099] [First Modification of Sensation Control Method] A first modified example of the sense control method executed by the sense control system 100 of the present disclosure further includes an acquisition step of acquiring a sensory stimulation signal, and a designation step of designating a sensory parameter based on the acquired sensory stimulation signal. The above-described reception step of receiving input of the sensory parameter is not limited to input from a user or the like, but is a step of receiving the sensory parameter designated in the designation step. This allows the sense control system 100 according to the first modified example to designate a sensory parameter based on the acquired sensory stimulation signal, and to output a sensory presentation signal based on a physical parameter correlated with the designated sensory parameter.

[0100] Here, the sensory stimulation signal is an auditory stimulation signal based on an auditory stimulation element such as sound, a visual stimulation signal based on a visual stimulation element such as an image or video, a tactile stimulation signal based on a tactile stimulation element such as an operation reaction force or vibration, or a signal based on any combination of these. Furthermore, the sensory control system 100 according to the first modified example may generate and acquire the sensory stimulation signal by sensing an auditory stimulation element, a visual stimulation element, a tactile stimulation element, or a combination of these in the acquisition step.

[0101] Furthermore, in the designation step, the sensation control system 100 according to the first modification may convert a physical parameter included in the physical characteristics of at least one of an auditory stimulus element, a visual stimulus element, and a tactile stimulus element (hereinafter collectively referred to as a sensory stimulus element) that forms the basis of the sensory stimulus signal into an affective parameter correlated with the physical parameter and designate the converted parameter. When converting the physical parameter into the affective parameter correlated with the physical parameter, the above-described conversion model 15 may be used, or a conversion model different from the conversion model 15 may be used. Similar to the conversion model 15, a conversion model different from the conversion model 15 can be generated by AI analysis, including machine learning, based on the correspondence information stored in the affective database 16. Furthermore, the physical parameters included in the physical characteristics of sensory stimulus elements such as sound, image, and video can be extracted by AI analysis, including machine learning, and the like.

[0102] As described above, the sensation control system 100 according to the first modification example can acquire a sensory stimulation signal based on a sensory stimulation element such as sound, an image, or a video, extract physical parameters included in the physical characteristics of the sensory stimulation element by AI analysis or the like, specify correlated sensory parameters, and output a sensory presentation signal based on the physical parameters correlated with the specified sensory parameters. Thus, for example, it can output a tactile presentation signal based on sensory parameters adjusted based on sound, an image, a video, or the like.

[0103] [Second Modification of Sensation Control Method] The operation device 33 of the present disclosure may have an operation surface that accepts a slide operation. The slide operation is an operation in which a user moves a contact position while keeping a body part, such as a finger, in contact with the operation surface of the operation device 33. In this case, the tactile sense providing unit 30 of the present disclosure generates an operation reaction force by vibrating the operation surface of the operation device 33. An example of a method for vibrating the operation surface of the operation device 33 is vibration of a weight using an actuator or the like. The sensation providing step in the second modified example of the sensation control method of the present disclosure can be a step of providing a tactile sensation by using such an operation device 33 and the tactile sense providing unit 30 to generate an operation reaction force from the tactile sense providing unit 30 in response to a slide operation of the operation device 33. In detail, the sensation providing step includes detecting a slide operation on the operation surface of the operation device 33 by the operation device 33 and causing the tactile sense providing unit 30 to generate an operation reaction force in response to the detected slide operation.

[0104] The physical parameters that can be converted based on the conversion model 15 stored in the memory unit 11 of the sensation control system 100 according to the second modification include parameters related to a change in the operation reaction force in response to a displacement accompanying a slide operation of the operation device 33, and the change in the operation reaction force includes at least a maximum portion or a minimum portion. Then, in the sensation providing step, by controlling the tactile providing unit 30 with a tactile providing signal based on such physical parameters, the change in the operation reaction force in response to a displacement accompanying a slide operation of the operation device 33 can be artificially synthesized to include the above-mentioned maximum portion or minimum portion. Here, the tactile providing unit 30 supplies a drive signal that causes vibration of the operation surface of the operation device 33 based on the received tactile providing signal, thereby driving the operation surface in a first direction at the rising edge of the drive signal and driving the operation surface in a second direction opposite to the first direction at the falling edge of the drive signal. Therefore, by varying the time changes of the rising and falling edges of the drive signal and increasing the power in a first direction corresponding to the rising edge or in a second direction corresponding to the falling edge averaged over a predetermined time, it is possible to artificially combine the above-mentioned maximum and minimum portions. Here, the drive signal that causes vibration of the operation surface of the operating device 33 may be, for example, a signal that drives a weight using an actuator or the like, and vibration of the weight may indirectly cause vibration of the operation surface.

[0105] 24A and 24B are diagrams showing examples of temporal changes in the strength of a drive signal supplied to a weight based on a tactile sensation signal. In the example shown in FIG. 24A, when the temporal change in the strength of the drive signal is positive, the weight is driven in a first direction, and when the temporal change in the strength of the drive signal is negative, the weight is driven in a second direction. As shown in FIG. 24A, when the temporal change in the rising edge of the drive signal of the weight is greater than the temporal change in the falling edge of the drive signal of the weight on average over a predetermined time, the force in the first direction corresponding to the rising edge of the drive signal is greater than the force in the second direction corresponding to the falling edge of the drive signal. On the other hand, as shown in FIG. 24B, when the temporal change in the falling edge of the drive signal of the weight is greater than the temporal change in the falling edge of the drive signal of the weight on average over a predetermined time, the force in the first direction corresponding to the rising edge of the drive signal is greater than the force in the second direction corresponding to the falling edge of the drive signal. In this way, by controlling the switching between a period in which the power in the first direction is increased as shown in Figure 24(a) and a period in which the power in the second direction is increased as shown in Figure 24(b), the above-mentioned maximum or minimum portions can be artificially combined.

[0106] The first direction and the second direction may be directions intersecting the operation surface of the operation device 33, or may be directions along (parallel to) the operation surface. For example, if the first direction and the second direction are directions intersecting the operation surface of the operation device 33, the resistance force in the pressing direction against the operation surface for a body part, such as a finger of a user, performing a slide operation on the operation surface changes, and the friction force between the body part and the operation surface due to the slide operation, i.e., the operation reaction force, can be changed. Furthermore, if the first direction and the second direction are directions along the operation surface of the operation device 33, the resistance force in the sliding operation direction on the operation surface for a body part, such as a finger of a user, performing a slide operation on the operation surface changes, and the friction force between the body part and the operation surface due to the slide operation, i.e., the operation reaction force, can be changed.

[0107] The transformation model 15 stored in the storage unit 11 of the sensation control system 100 according to the second modification may be obtained by a transformation model generation method including the following storage step. Specifically, in the storage step of the transformation model generation method according to the second modification, the sensibility database 16 stores, for each of one or more types of manipulation tools, correspondence information that associates physical characteristics that realize the sensation presented when a specific manipulation tool is operated with sensibility parameters that are input to reflect the operation of the manipulation tool. Here, the manipulation tool has a manipulation surface that accepts a slide operation. Furthermore, the change in the manipulation reaction force in response to a displacement accompanying the slide operation of the manipulation tool includes at least a maximum portion and a minimum portion. Here, the manipulation reaction force is generated by vibration of the manipulation surface of the manipulation tool. The vibration of the manipulation surface of the manipulation tool may be indirect vibration caused by, for example, vibration of a weight by an actuator, similar to the vibration of the manipulation surface of the manipulation device 33 described above. The maximum and minimum portions of the change in the reaction force due to the displacement caused by the sliding operation of the operating tool are artificially synthesized by varying the time changes of the rise and fall of the drive signal that causes the vibration of the operating surface of the operating tool, and by increasing the force in the direction corresponding to the rise or fall on average over a predetermined time period more than the other. By using such an operating tool, the conversion model 15 of this example can be generated more easily.

[0108] [Modification of Sensibility Database 16] As described above, the sensory database 16 of the present disclosure stores, for one or more sensory presentations, correspondence information that associates physical characteristics related to a predetermined sensory presentation with sensory parameters indicating the degree of sensory expression for that sensory presentation. While the sensory presentation has been described primarily in terms of tactile presentation, the term "tactile" referred to primarily in this specification refers to tactile in a broad sense, which includes tactile in a narrow sense, pressure, and force. In this specification, the term "tactile" refers to tactile in a broad sense. Here, tactile in a narrow sense refers to, for example, the sensation related to the texture of the surface of an object that a body part comes into contact with, and is highly correlated with sensory parameters related to sensory expressions such as unevenness and roughness. Pressure sensation refers to, for example, the sensation related to the resistance between a body part and an object, and is highly correlated with sensory parameters related to sensory expressions such as hardness. Force sensation refers to, for example, the sensation related to an external force acting on a body part, such as the sensation of being pulled or pushed. It is known that the receptors primarily responsible for the narrow sense of touch, pressure, and force are different, and that the response characteristics of each receptor also differ.

[0109] Furthermore, physical characteristics related to tactile presentation include static characteristics and dynamic characteristics. Static characteristics are physical characteristics obtained, for example, when an operating tool is operated at a constant operating speed using an instrument or the like with high rigidity to the extent that elasticity can be ignored (hereinafter simply referred to as a "rigid body"). Dynamic characteristics are physical characteristics obtained, for example, when an operating tool made of a flexible material that imitates a body part such as a human finger is operated while changing the operating speed. Unlike static characteristics, dynamic characteristics are physical characteristics that also include physical parameters such as the elasticity of the body part, operating speed, operating acceleration, operating jerk, and frictional force.

[0110] The correspondence information stored in the sensitivity database 16 may be information related to at least one of information related to tactile sensations in the narrow sense, pressure sensations, and force sensations included in tactile sensations in the broad sense, and information related to static characteristics and dynamic characteristics included in physical characteristics. For example, the correspondence information stored in the sensitivity database 16 may be information in which the weighting of the static characteristics and dynamic characteristics of tactile sensations in the narrow sense, pressure sensations, and force sensations changes depending on the stage of operation of the operating tool. More specifically, for example, the weighting of the static characteristics may be set to be greater than the weighting of the dynamic characteristics in an operation stage immediately after the start of the operation, and the weighting of the dynamic characteristics may be set to be greater than the weighting of the static characteristics in an operation stage in which the change in the operation reaction force relative to the displacement associated with the operation becomes greater (for example, an operation stage corresponding to the maximum portion of the load-displacement curves shown in FIGS. 11 and 18 or the minimum portion of the load-displacement curve shown in FIG. 11). This is because, in the operation stage immediately after the start of an operation, the influence of factors such as operation speed may be small, and in that case, approximating the physical characteristics with static characteristics can reproduce the physical characteristics with high accuracy. However, in the operation stage when the change in the operation reaction force due to the displacement caused by the operation becomes large, the influence of factors such as operation speed may be large, and in that case, approximating the physical characteristics with dynamic characteristics can reproduce the physical characteristics with higher accuracy. Furthermore, the correspondence information stored in the sensitivity database 16 may be information including physical characteristics that reflect the differences in the response characteristics of receptors mainly involved in the narrow sense of touch, pressure, and force. Generating the conversion model 15 based on such correspondence information enables tactile presentation that better reflects human sensibilities.

[0111] (Tactile Control System 2) FIG. 20 shows the configuration of a haptic control system 2 as a second embodiment of the sensation control system 100 shown in FIG. 1, along with the signal flow.

[0112] The haptic control system 2 shown in FIG. 20 includes a terminal device 80 and a communication device 70, which are communicatively connected to each other via a network 9. The terminal device 80 includes a main control device 6, an input / output device 3, and a haptic presentation device 20. The main control device 6 includes a processor 7 and a storage unit 8, and controls the operations of the input / output device 3 and the haptic presentation device 20. The haptic presentation device 20 includes a haptic presentation unit 30, an operation range variable unit 29, and sensors such as a position sensor 27 and an acceleration sensor 28. The communication device 70 is, for example, a server device, and includes a processor 14, a storage unit 11, a calculation function unit 12, and a calculation function unit 13. A conversion model 15 is stored in the storage unit 11.

[0113] Of the components included in the haptic control system 2, the input / output device 3, the haptic presentation device 20, the processor 14, the storage unit 11, the calculation function unit 12, and the calculation function unit 13 are the same as the components denoted by the same reference numerals included in the haptic control system 1 shown in Fig. 2, and therefore descriptions thereof will be omitted. Similar to the haptic control system 1, the terminal device 80 may include an operation device 33, and the haptic presentation unit 30 may present a haptic sensation to a user operating the operation device 33. Furthermore, the haptic control system 2 may include a haptic presentation unit 43 shown in Fig. 19 instead of the haptic presentation unit 30, and may include an operation device 42 shown in Fig. 19 instead of the operation device 33.

[0114] FIG. 21 is a sequence diagram showing the operation of the haptic control system 2. In FIG. 21, the processes executed by the terminal device 80 and the communication device 70 included in the haptic control system 2 are explained in steps (ST). First, in ST31, the terminal device 80 accepts input of affective parameters. Specifically, the terminal device 80 accepts affective parameters input by a user or the like via the input unit 4 of the input / output device 3. Next, in ST32, the terminal device 80 encodes information about the affective parameters and transmits the encoded information about the affective parameters to the communication device 70 via the network 9. The terminal device 80 may include an encoder for encoding the information about the affective parameters. Furthermore, the terminal device 80 may encode the entire information about the affective parameters, or may encode only a portion of the information about the affective parameters.

[0115] After ST32, in ST21, the communication device 70 decodes the information received from the terminal device 80 to obtain information on the affective parameters. The communication device 70 may include a decoder for decoding the information on the affective parameters. Next, in ST22, the communication device 70 converts the affective parameters into physical parameters correlated with the affective parameters using the conversion model 15. Next, in ST23, the communication device 70 encodes the converted physical parameters and transmits information on the encoded physical parameters to the terminal device 80 via the network 9. The communication device 70 may include an encoder for encoding the information on the physical parameters. Furthermore, the communication device 70 may encode the entire information on the physical parameters, or only a portion of the information on the physical parameters.

[0116] After ST23, in ST33, the terminal device 80 decodes the received information to obtain information on the physical parameters. The terminal device 80 may include a decoder for decoding the information on the physical parameters. Thereafter, in ST34, the terminal device 80 generates a tactile presentation signal based on the physical parameters and operates the tactile presentation device 20. Note that the encoding and decoding processes in ST32, ST21, ST23, and ST33 are not essential.

[0117] In this way, when a sensory parameter is input to the terminal device 80, the haptic control system 2 according to this embodiment can receive information on a physical parameter correlated with the sensory parameter from the communication device 70 via the network 9 and present a haptic sensation in the form of a haptic presentation signal based on the physical parameter. Thus, the haptic control system 2 makes it possible to present a haptic sensation that reflects human sensibilities by communicating haptic information via the network 9. The haptic control system 2 is particularly useful in the field of the tactile internet.

[0118] Furthermore, when communicating all physical parameters included in the physical characteristics related to haptic presentation, problems such as communication delays are likely to occur due to an increase in data volume. However, the haptic control system 2 according to this embodiment extracts and communicates physical parameters correlated with sensory parameters, thereby reducing the data volume. This can contribute to faster communication and a reduced load on each processor, etc. This effect is also true for the haptic control system 1 according to the first embodiment, but is particularly useful in the haptic control system 2 according to this embodiment, which uses the haptic Internet.

[0119] The haptic control system 2 according to this embodiment may include a plurality of terminal devices 80. That is, the communication device 70 may be connected to each of the plurality of terminal devices 80 via the network 9. In this case, the communication device 70 may store identification information such as an address or ID that specifies each of the plurality of terminal devices 80, and a conversion model 15 associated with each identification information. This allows the conversion model 15 to be optimized and configured for each user who uses each terminal device 80.

[0120] Furthermore, the communication device 70 of the haptic control system 2 according to this embodiment may store a plurality of conversion models 15 depending on, for example, the application (for games, for vehicle use, etc.), and different conversion models may be used depending on the application, etc. required by the terminal device 80. This allows the conversion model 15 to be optimized and configured depending on the application, etc., so that different physical parameters can be selected depending on the application, etc., even if the physical parameters are converted from the same sensory parameters.

[0121] 20 shows an example in which the conversion model 15 is stored in the storage unit 11 of the communication device 70, but the conversion model 15 may also be stored in the storage unit 8 of the main control device 6 of the terminal device 80. In this case, for example, information (including encoded information) related to affective parameters may be distributed from the communication device 70, and the affective parameters may be converted into physical parameters in the main control device 6 of the terminal device 80, thereby generating a tactile presentation signal based on the physical parameters that correlate with the affective parameters.

[0122] (Application example of haptic control systems 1 and 2) The haptic control system 1 according to the first embodiment can be used for entertainment purposes such as games, videos, and music. When the haptic control system 1 is used for entertainment purposes, a haptic sensation from the haptic presentation device 20 may be presented to the user through an operation unit such as a button, joystick, or trigger switch included in an operation device 33 such as a game controller. Furthermore, a haptic sensation from the haptic presentation device 20 may be presented to a part other than the operation unit of the operation device 33, for example, to the entire or part of the user's body part such as the hand holding the operation device 33. The game controller may be, for example, a steering controller that imitates the steering wheel of an automobile.

[0123] Examples of timings for presenting a tactile sensation to the user through the operation device 33 include timings when an operation on an operation unit included in the operation device 33 is detected, timings when an operation such as movement, rotation, acceleration or deceleration on all or part of the operation device 33 is detected, and timings when presenting a tactile sensation according to content. The timings for presenting a tactile sensation according to content are timings for presenting a tactile sensation that are set in advance to enhance the sense of realism within each piece of content such as a game, video, or music, and may be timings when no operation from the user is detected.

[0124] When the haptic control system 1 is used for entertainment purposes, the haptic presentation from the haptic presentation device 20 is not limited to being performed through the above-mentioned operation device 33. The haptic presentation from the haptic presentation device 20 may be performed, for example, through a seat on which the user sits, a suit worn specifically by the user, a headset used for virtual reality (VR) or augmented reality (AR), a glove or other wearable device worn by the user on a body part such as the hand, or other wearable device. For example, the feeling of operating a virtual switch in a VR or AR space may be presented through a wearable device.

[0125] The haptic control system 1 according to the first embodiment can be used, for example, in an in-vehicle application. When used in an in-vehicle application, the haptic presentation device 20 may present a tactile sensation to an occupant through, for example, devices used for driving operations such as a steering wheel, pedals, or shifter, an operating device 33 such as an infotainment system, an air conditioning unit, or a decorative panel, or a seat. Here, the decorative panel is provided at any location inside the vehicle, such as a door trim, pillar, glove box, center console, dashboard, or overhead console, and is a device that constitutes the interior of the vehicle and can display information through a touch operation or a proximity operation.

[0126] When the haptic control system 1 is used in an in-vehicle application, the main purpose of presenting a haptic sensation is to notify the occupants that an input operation has been performed on the operating device 33 or the like, as well as to warn the occupants of lane departure, approaching another vehicle, etc. In other words, the purpose may differ from the above-mentioned entertainment application, which mainly aims to present a sense of realism. Therefore, the haptic control system 1 may store a conversion model 15 that can convert physical parameters converted from the same sensory parameters into different physical parameters depending on the application.

[0127] When the tactile control system 1 is used in an in-vehicle application, the timing for presenting tactile sensations to the occupants may include the timing when an input operation to the operating device 33 or the like is detected, or the timing when a danger such as lane departure or approaching another vehicle is detected.

[0128] The haptic control system 2 according to the second embodiment can be used for the same purposes as the haptic control system 1 according to the first embodiment. That is, the haptic control system 2 can be used for entertainment purposes such as games, videos, and music, and for in-vehicle purposes.

[0129] When the haptic control system 2 according to the second embodiment is used for entertainment purposes, in addition to being used in the same manner as the haptic control system 1 of FIG. 1 , haptic presentation signals may be transmitted, received, or distributed in conjunction with live distribution (including broadcasting) of content via a network 9, content data updates, or user interactions or competitions. For example, when a communication device 70 communicates with multiple terminal devices 80, common affective parameters may be set for each terminal device 80, or individual affective parameters may be set for each terminal device 80. Alternatively, some affective parameters may be set in common among the terminal devices 80 while other affective parameters may be set individually among the terminal devices 80. For example, when users of multiple terminal devices 80 are to work in a common VR or AR environment, the environment can be individually adjusted by setting common affective parameters indicating the intensity of the sensation among the terminal devices 80 and adjusting affective parameters indicating the sharpness of the sensation according to the preferences of the users of each terminal device 80.

[0130] When the haptic control system 2 is used for in-vehicle applications, in addition to being used in the same manner as the haptic control system 1, it may also receive haptic presentation signals for warnings and the like based on communication between vehicles via the network 9, communication with road objects such as traffic signs, and traffic information distributed from a server. Communication between vehicles and communication with road objects can also be achieved with the haptic control system 1 according to the first embodiment, as long as direct communication is possible without using the network 9.

[0131] The haptic control system 2 according to the second embodiment can be used, for example, for medical and industrial applications. Medical applications include the transmission of haptic information associated with remote medical care. Industrial applications include the transmission of haptic information associated with the remote operation of industrial robots. If the haptics transmitted in these applications could be customized based on sensitivity values, it would be possible to present a more realistic haptic sensation to the user and enable comfortable operation.

[0132] The haptic control system 2 according to the second embodiment can be used, for example, for internet shopping. For example, the tactile sensations of a product, the feel of wearing it, and the writing comfort of a writing implement can be presented to the user through haptic transmission. Furthermore, the tactile sensations of a product and the feel of wearing it can be customized based on the sensitivity value, and products that are closer to the tactile sensations and the feel of wearing them that the user desires can be suggested to the user.

[0133] The haptic control system 2 according to the second embodiment can be used for communication between users in remote locations. It can present the feeling of shaking hands between users in remote locations, the feeling of touching each other, and the like. It can also present the feeling of touching an animal such as a pet. In these applications, it is particularly useful to use or combine a heat sensation presentation as the haptic presentation unit 30, as it can convey warmth. [Aspect 2] [Background technology] Conventionally, operating tools that provide a sense of sensation by applying some kind of stimulus to a person have been known. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating tools.

[0134] There is known a technology for producing products according to user preferences (see, for example, Patent Document 2). Patent Document 2 discloses a technology in which the user selects a reference model, and in a subsequent process, the color, size, material, position, etc. are added or changed based on the user selection.

[0135] [Summary of the Invention] [Problem to be solved by the invention] However, conventional technologies have a problem in that they cannot adjust the sensory presentation based on sensory input. That is, although users' preferred sensations vary, they may express their preferences through sensory input. However, conventionally, this sensory expression has not been utilized to change the sensory presentation.

[0136] In view of the above-mentioned problems, an object of the present invention is to provide a tactile control device that can adjust the operational feel through sensory input.

[0137] [Explanation of Mode 2] In the first aspect, a sensory control method for converting sensory parameters into physical parameters using the conversion model 15 has been described. However, even if a manufacturer prototypes an operating tool in which physical parameters converted from sensory parameters are applied to tactile presentation, it often takes several rounds of trial and error to obtain an operating feel that the user prefers. Because prototyping an operating tool requires many steps, it may take a long time to complete an operating tool that has an operating feel that the user prefers.

[0138] Therefore, in this embodiment, a tactile control device that can reproduce the user's preferred operation feel in real time and a tactile control method performed by the tactile control device will be described.

[0139] [Example of a haptic control device] Fig. 25 is a perspective view of the tactile control device 50. Fig. 25 shows a stand-alone type tactile control device 50. As shown in Fig. 25, the tactile control device 50 has three reference operating tools 51a to 51c (plurality of reference operating tools), a reproduction operating tool 52, a touch panel 53, and a display 260. Note that, hereinafter, any reference operating tool among the reference operating tools 51a to 51c will be referred to as a "reference operating tool 51." There may be two or more reference operating tools 51.

[0140] The display 260 displays instructions for using the haptic control device 50, an operation menu, etc. The touch panel 53 displays affective parameters (e.g., adjectives) for which expressiveness levels are input, allowing the user to input the expressiveness level for each affective parameter. When reproducing the user's preferred operation feel, the haptic control device 50 accepts input of the expressiveness level for each affective parameter multiple times, and therefore displays on the touch panel 53 the affective parameters for which the expressiveness level can be input each time.

[0141] The three standard operating tools 51a to 51c are operating tools that are prepared as standards and have different operating feel, that is, the three standard operating tools 51a to 51c each have a different load-displacement curve.

[0142] By inputting the expression level that the user prefers, the operation feel of the reference operation tool 51 selected by the haptic control device 50 from among the three reference operation tools 51a to 51c is reproduced in the reproduction operation tool 52. In other words, the haptic control device 50 copies the physical parameters of one of the three reference operation tools 51a to 51c to the reproduction operation tool 52. The user can adjust the operation feel to their preference by operating this reproduction operation tool 52 and inputting the expression level.

[0143] Therefore, the user can adjust the operation feel in real time by operating the reproduction operation device 52 and checking the operation feel, inputting the expression level, and repeating the process of adjusting the operation feel of the reproduction operation device 52. In addition, the user can compare the operation feel of the adjusted reproduction operation device 52 with the operation feel of the reference operation devices 51a to 51c, making it easier for the user to adjust the expression level to their preference.

[0144] The shape and appearance of FIG. 25 are merely an example, and a general-purpose system configuration in which the reference operating tool 51 and the reproduction operating tool 52 are connected to a PC or tablet terminal via a USB cable or the like may also be used.

[0145] FIG. 26 shows a client-server type haptic control system 2. In the haptic control system 2 of FIG. 26, a terminal device 80 and a server 200 can communicate via a network. The terminal device 80 may run, for example, a web browser or a dedicated application. The terminal device 80 displays a screen required for inputting the expression frequency for each affective parameter and accepts the expression frequency input from the user. The terminal device 80 transmits the expression frequency to the server 200, and the server 200 transmits the selection results of the reference operating tools 51a to 51c, the physical parameters corresponding to the reference operating tools 51a to 51c, and the adjusted physical parameters to the terminal device 80.

[0146] In this way, even in a client-server type system, the user can adjust the operational feel in real time, just like the haptic control device 50.

[0147] <First form of haptic control device> First, an outline of the operation of the haptic control device 50 will be described with reference to Figures 27 and 28. Figures 27 and 28 show an outline of the process by which the user adjusts the operational feel using the haptic control device 50.

[0148] (1) First, the user inputs expression frequencies (an example of first expression frequencies) that represent the user's preferences for a plurality of affective parameters (for example, adjectives) (FIG. 27(a)). The touch panel 53 displays a first input screen 281 of FIG. 27(a), which has an affective parameter presentation field 282 and a reference operating tool field 112. In the affective parameter presentation field 282, the user can input the expression frequency for each affective parameter (an example of a first affective parameter) using a slide bar (an example of an input means). The reference operating tool field 112 displays the probability of the reference operating tools 51a to 51c being selected for the input expression frequencies.

[0149] (2) The tactile control device 50 selects the reference operating tools 51a to 51c that are closest to the user's preference (the inputted expression level of each affective parameter) based on the correspondence between the expression level of each affective parameter and the reference operating tools 51a to 51c that has been learned in advance (FIG. 27(b)). This process is called STEP 1.

[0150] (3) The tactile control device 50 reproduces the operation feel of the reference operation tools 51a to 51c with the reproduction operation tool 52 (FIG. 27(c)). In FIG. 27, there are three reference operation tools 51a to 51c, but this is merely an example. The user tries operating the reproduction operation tool 52 to check whether the operation feel is what they prefer.

[0151] (4) If the operating feel is different from the user's preference, the user again inputs expression levels (an example of second expression levels) that represent the user's preference for a plurality of affective parameters (FIG. 28(a)). The touch panel 53 displays the second input screen 120 of FIG. 28(a), which has an affective parameter presentation field 121. The user can input the expression level for each affective parameter (an example of a second affective parameter) in the affective parameter presentation field 121 using a slide bar. The number of affective parameters in the affective parameter presentation field 121 may be smaller than the number of affective parameters in the affective parameter presentation field 282. This is because the user's preferred reference operating tool 51 has already been selected in the affective parameter presentation field 282. Furthermore, having a smaller number of affective parameters in the affective parameter presentation field 121 reduces the user's workload.

[0152] In the initial state of the affective parameter presentation field 121, the expression frequency of the slide bar indicates the median value. Even if the user sets the expression frequency of the same affective parameter to the minimum or maximum value in the affective parameter presentation field 282, the expression frequency of the slide bar in the initial state of the affective parameter presentation field 121 is the median value. This makes it easy for the user to adjust the expression frequency in the affective parameter presentation field 121 to a range including or including the expression frequency input in the affective parameter presentation field 282. Furthermore, the expression frequency in the initial state of the affective parameter presentation field 121 is the expression frequency corresponding to the physical parameter set in the reference operating tool 51. The user can adjust from this initial state to an expression frequency either before or after the value.

[0153] (5) The tactile control device 50 converts the expression frequency of each sensory parameter input by the user into a physical parameter based on the correspondence (e.g., a regression model) between the expression frequency of each sensory parameter and the physical parameter that has been learned in advance, and reflects the converted physical parameter in the reproduction operating tool 52 (Fig. 28(b)). This process is called STEP 2. (6) The user operates the reproduction operation tool 52 to check whether the operation feel is to his / her liking (FIG. 28(c)).

[0154] Thereafter, the user repeats steps (4) to (6), allowing the tactile control device 50 to determine the physical parameters of the user's preferred operation feel.

[0155] [About the function of the haptic control device] Fig. 29 is a functional block diagram illustrating the functions of the haptic control device 50. As shown in Fig. 29, the haptic control device 50 has a display control unit 61, a first input receiving unit 62, a second input receiving unit 63, a classification unit 64, a first conversion model 65a, a second conversion model 65b, a third conversion model 65c, and a physical parameter setting unit 66. Each of these functions of the haptic control device 50 is realized by a CPU or processor included in the information processing device executing a program loaded in RAM. Alternatively, each function may be realized by a hardware circuit.

[0156] The display control unit 61 displays preset affective parameters and five or seven levels of expression levels set for the affective parameters on the touch panel 53 in a selectable manner (displaying the first input screen 281 and the second input screen 120). The expression levels can be adjusted in any number of levels or continuously. The user may select the expression level by tapping on the touch panel 53 or by sliding a slide bar. The user may also select the expression level by voice input or button input. The display control unit 61 displays different affective parameters in STEP 1 and STEP 2. The number of affective parameters in STEP 1 may be greater than the number of affective parameters in STEP 2.

[0157] The first input receiving unit 62 receives an input of the expression frequency of each affective parameter in response to a user operation in STEP 1. The second input receiving unit 63 receives an input of the expression frequency of each affective parameter in response to a user operation in STEP 2.

[0158] The classification unit 64 is an identification model that has learned the correspondence between the expression frequencies of the affective parameters received by the first input receiving unit 62 and three conversion models. There are many types of classification learning methods, such as deep learning, decision trees, and support vector machines, but in this embodiment, any learning method may be used for learning. The classification unit 64 outputs identification information of the first conversion model 65a to the third conversion model 65c for the expression frequencies of the affective parameters received by the first input receiving unit 62 (identifying the conversion model 15 that is closest to the user's preference from the multiple conversion models 15).

[0159] As described in the first aspect, the first to third conversion models 65a to 65c are conversion models capable of converting perceptual parameters into physical parameters correlated with the perceptual parameters. The first to third conversion models 65a to 65c correspond to the three reference operating tools 51a to 51c, and can convert the expression frequencies of perceptual parameters into physical parameters for each of the reference operating tools 51a to 51c. Examples of physical parameters include the stroke amount of the operating tool, the operation reaction force (load), the speed, acceleration, and jerk of a movable part, and the elastic properties of a body part such as the operator's fingers. The first to third conversion models 65a to 65c are generated by multiple regression or the like based on the expression frequencies of sensory tests in which load-displacement curves correspond to different physical parameters in order to reproduce different operation feels.

[0160] Then, the first conversion model 65a to the third conversion model 65c convert into different physical parameters the expression frequencies of the affective parameters received by the second input receiving unit 63. In this way, the reference conversion model selected in STEP 1 can convert into physical parameters the expression frequencies that are close to the user's preferences input in STEP 2.

[0161] The physical parameter setting unit 66 sets the physical parameters output by any of the first conversion model 65a to the third conversion model 65c in the reproduction operation tool 52. Therefore, the tactile control device 50 can reproduce the operation feel desired by the user in real time.

[0162] [Classification unit generation, learning the correspondence between the expression frequency of sensory parameters and physical parameters] Next, the generation of the classification unit 64 will be described with reference to Fig. 30 etc. Fig. 30 is a flowchart showing the flow of learning in the generation of the classification unit 64. Note that although various types of learning are performed by the haptic control device 50, the learning can be performed by any information processing device.

[0163] In ST41, the haptic control device 50 accepts input of the expression frequency. The affective parameters used for generation by the classifier 64 are shown in Fig. 27(a). The affective parameters are, for example, the following 24. 24 is just an example, and there may be fewer or more. "Light (heavy) operating force" "There is no sense of decision" "Inaccurate (accurate)" "Clear (ambiguous)" "Soft (hard)" "Blurred (clear)" "Smooth" "Tired (not tired)" "Strict (kind)" "Coarse (fine)" "There is no sensation of being sucked in (there is)" "Innovative (traditional)" "Cheap (luxury)" "Durable (not)" "I don't want to operate it again (I want to)" "Fun (boring)" "Not comfortable (good)" "I hate (like)" "There is no bouncy feeling (there is)." "Mild (Sharp)" "Dry (wet)" "Bright (dark)" "Cold (warm)" "With (or without) play" These affective parameters may be automatically generated by web analysis, tweet analysis, SNS analysis, papers, market clustering analysis, feature and adjective extraction, etc. In other words, the affective parameters may not be fixed but may be dynamically changeable.

[0164] In ST42, the haptic control device 50 learns the correspondence between the expression frequencies of the affective parameters and the reference operating tools 51a to 51c by machine learning. The classification unit 64 has this correspondence.

[0165] Machine learning is a technology that allows computers to acquire human-like learning capabilities. It is a technology in which a computer autonomously generates algorithms necessary for judgments such as data classification from previously acquired training data, and applies these algorithms to new data to make predictions. The learning method for machine learning can be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or it can be a combination of these learning methods. Machine learning techniques include perceptrons, deep learning, support vector machines, logistic regression, naive Bayes, decision trees, and random forests, and are not limited to these learning methods. Deep learning and decision trees will be explained later as examples of learning methods.

[0166] In ST43, the classifier 64 generated by machine learning is incorporated into the haptic control device 50.

[0167] FIG. 31 is a flowchart showing the flow of learning the correspondence between the expression frequency of the affective parameter and the physical parameter.

[0168] In ST51, the haptic control device 50 receives input of expression frequencies. The sensory parameters used to learn the correspondence between the expression frequencies of sensory parameters and physical parameters are shown in FIG. 28(a). The sensory parameters are, for example, the following five. Five is just an example, and there may be fewer or more. "Mild (Sharp)" "Coarse (fine)" "Bright (dark)" "Soft (hard)" "Light (heavy)" In ST52, the haptic control device 50 determines the correspondence between the expression frequencies of the sensory parameters and the physical parameters by multiple regression analysis. In this embodiment, three reference operating tools 51a to 51c are prepared, and therefore a force-displacement curve is obtained for each of the three reference operating tools 51a to 51c. The physical parameters that realize this force-displacement curve are also known. The user operates the reference operating tools 51a to 51c and inputs the operation feel of the reference operating tools 51a to 51c as the expression frequencies. Once the expression frequencies of a sufficient number of people have been input, the haptic control device 50 performs multiple regression analysis using Equation 5. Multiple regression analysis has been described with reference to Equation 5 in Embodiment 1, and FIGS. 22 and 23. Therefore, the coefficient of determination B for each of the three reference operating tools 51a to 51c is 11 ~B mn can be determined, and a transformation model 15 as shown in Fig. 23 is obtained for each of the reference operating tools 51a to 51c. The transformation models for the three reference operating tools 51a to 51c are the first transformation model 65a to the third transformation model 65c.

[0169] In ST53, the first conversion model 65a to the third conversion model 65c generated by the multiple regression analysis are incorporated into the haptic control device 50.

[0170] [Tactile presentation flow] FIG. 32 is a flowchart showing the flow in which the tactile control device 50 presents the user's preferred operation feel using the classification unit 64 and the first to third conversion models 65a to 65c.

[0171] In ST61, the first input receiving unit 62 receives an input of the expression frequency of the affective parameter on the first input screen 281 in order to select the standard operating tools 51a to 51c (STEP 1).

[0172] In ST62, the classification unit 64 identifies the reference operating tools 51a to 51c based on the expression frequency of each affective parameter input on the first input screen 281. Once the reference operating tools 51a to 51c are determined, one of the first to third conversion models 65a to 65c is also determined.

[0173] In ST63, the physical parameter setting unit 66 sets the physical parameters of the selected reference operating tools 51a to 51c in the reproduction operating tool 52. The user can operate the reproduction operating tool 52 to check whether the operation feel is what he or she prefers.

[0174] In S64, the user determines whether to adjust the operation feel to a different one from the reference operation tools 51a to 51c depending on whether the operation feel is the user's preference. The tactile control device 50 receives an instruction from the user to start readjustment.

[0175] In S65, when the user adjusts the operation feel to be different from that of the reference operation tools 51a to 51c, the second input receiving unit 63 receives input of the expression level of the affective parameter on the second input screen 120 (STEP 2). n (corresponding to the first to third conversion models 65a to 65c selected in ST63) is used to calculate the physical parameters P1 to P n The physical parameter setting unit 66 converts the physical parameters P1 to P n is set in the reproduction operation tool 52. The user can operate the reproduction operation tool 52 again to check whether the operation feel is what the user prefers.

[0176] Thereafter, the user can repeatedly use the second input screen 120 to adjust the preferred operation feel until the preferred operation feel is obtained.

[0177] In this way, the tactile control device 50 of this embodiment can reproduce the operation feel preferred by the user in real time.

[0178] <Second form of haptic control device> Next, a second embodiment of the haptic control device 50 will be described.

[0179] First, an outline of the operation of the haptic control device 50 of the second embodiment will be described with reference to Fig. 33. Fig. 33 shows an outline of the process by which the user adjusts the operational feel using the haptic control device 50.

[0180] (1) First, the user inputs the expression levels that represent his / her preferences for a plurality of affective parameters on a first input screen 281 (FIG. 33(a)). The first input screen 281 may be the same as that shown in FIG. 27(a).

[0181] (2) The tactile control device 50 determines a physical parameter (an example of a second physical parameter) corresponding to the expression frequency based on the correspondence between the expression frequency of each sensory parameter and the physical parameter (load-displacement curve) that has been learned in advance by regression (Figure 33(b)).

[0182] (3) The tactile control device 50 performs curve fitting on the load-displacement curves of the reference operating tools 51a to 51c, which have been prepared in advance, using an appropriate fitting model (FIG. 33(c)). This fitting model is, for example, a polynomial with physical parameters as coefficients. Therefore, physical parameters (an example of first physical parameters) that represent the load-displacement curve for each of the reference operating tools 51a to 51c are obtained. The tactile control device 50 compares the physical parameters of (2) with the physical parameters of (3).

[0183] (4) If the physical parameters of (2) and (3) are similar, the tactile control device 50 presents a similar reference operating tool 51; if they are not similar, it suggests adjusting the new sensation using the reproduction operating tool 52 (Figure 33(d)).

[0184] [About the function of the haptic control device] Fig. 34 is a functional block diagram illustrating the functions of the haptic control device 50. Note that the explanation of Fig. 34 may mainly focus on the differences from Fig. 29. The haptic control device 50 has a display control unit 61, a first input receiving unit 62, a second input receiving unit 63, a physical parameter conversion unit 67, a curve fitting unit 68, a comparison unit 69, a first conversion model 65a, a second conversion model 65b, a third conversion model 65c, and a physical parameter setting unit 66. Each of these functions of the haptic control device 50 is realized by a CPU included as an information processing device executing a program loaded in RAM. Alternatively, each function may be realized by a hardware circuit.

[0185] The physical parameter conversion unit 67 uses the correspondence between the expression frequency and the physical parameters obtained by the multiple regression analysis to determine the physical parameters corresponding to the expression frequency accepted by the first input acceptance unit 62. Note that, since determining the physical parameters also determines the load-displacement curve, it may be said that the physical parameter conversion unit 67 determines the load-displacement curve.

[0186] The curve fitting unit 68 fits the load-displacement curves of the reference operating tools 51a to 51c (first conversion model 65a to third conversion model 65c) with an appropriate fitting model (for example, a polynomial). Note that curve fitting is a form of multiple regression analysis. By setting physical parameters as coefficients of the polynomial, the curve fitting unit 68 can estimate the physical parameters for each of the reference operating tools 51a to 51c. Therefore, it is preferable to select a fitting model that allows fitting of the load-displacement curves with physical parameters.

[0187] The comparison unit 69 compares the physical parameters determined by the physical parameter conversion unit 67 with the physical parameters determined by the curve fitting unit 68, and determines whether they are similar. For example, the comparison unit 69 compares the physical parameters P1 to P n The comparison unit 69 calculates the sum of squares of the differences for each reference operating tool 51a to 51c and determines whether it is less than a threshold value. If there are similar physical parameters, the comparison unit 69 indicates the physical parameters corresponding to the reference operating tools 51a to 51c to the physical parameter setting unit 66.

[0188] The physical parameter setting unit 66 sets the specified physical parameters of the reference operating tool 51 in the reproduction operating tool 52 .

[0189] [Learning the physical parameters (load-displacement curve) corresponding to the expression frequency, and curve fitting of the load-displacement curve of the reference operating tool] Next, learning of physical parameters (load-displacement curves) corresponding to expression frequencies will be described with reference to Fig. 35 etc. Fig. 35 is a flowchart showing the flow of learning physical parameters (load-displacement curves) corresponding to expression frequencies.

[0190] In ST71, the haptic control device 50 receives an input of the expression frequency. The affective parameters used for generation by the classifier 64 are shown in FIG.

[0191] In ST72, the haptic control device 50 determines the correspondence between the expression frequency of the sensory parameters and the physical parameters by multiple regression analysis. The user inputs the expression frequency of the operation feel for the operation tool whose physical parameters are known. The operation tool whose physical parameters are known may be the reference operation tool 51 or any operation tool. When the expression frequencies of a sufficient number of people have been input, the haptic control device 50 performs multiple regression analysis using Equation 5. Multiple regression analysis has been explained with reference to Equation 5 of aspect 1, and Figs. 22 and 23. Therefore, the haptic control device 50 uses the coefficient of determination B 11 ~B mn can be determined, and a transformation model 15 as shown in FIG. 23 is obtained.

[0192] In ST73, the physical parameter conversion section 67 generated by the multiple regression analysis is incorporated into the haptic control device 50.

[0193] FIG. 36 is a flowchart showing the flow of curve fitting the load-displacement curves of the standard operating tools 51a to 51c.

[0194] In ST81, the curve fitting unit 68 performs curve fitting on the load-displacement curves of the reference operating devices 51a to 51c. As shown in FIG. 9, the correspondence between the stroke amount x and the operation reaction force is obtained for each of the reference operating devices 51a to 51c. The curve fitting unit 68 extracts pairs of stroke amount and operation reaction force, preferably at regular intervals, from x=0 to the maximum stroke amount. The curve fitting unit 68 performs curve fitting by applying the pairs of stroke amount x and operation reaction force y to a fitting model. The fitting model is an equation that determines the operation stress from the stroke amount x using physical parameters as coefficients. The fitting model below is an example, and any appropriate model (equation) that determines the operation reaction force y from the stroke amount x using physical parameters as coefficients may be used. Fitting model: y=P1×x 0 +P2×x 1 +P3×x 2 +......P n ×x n The curve fitting unit 68 can obtain P1 to Pn by multiple regression analysis. n corresponds to a physical parameter.

[0195] In ST82, the physical parameters of the reference operating tools 51a to 51c generated by curve fitting are set in the comparison unit 69.

[0196] [Tactile presentation flow] FIG. 37 is a flowchart showing the flow in which the haptic control device 50 uses the physical parameter conversion unit 67 and the comparison unit 69 to present the user's preferred operation feel.

[0197] In ST91, the first input receiving unit 62 receives an input of the expression frequency of the affective parameter for selecting the standard operating tools 51a to 51c.

[0198] In ST92, the physical parameter conversion section 67 converts each affective parameter into a physical parameter (load-displacement curve) based on the expression frequency of each affective parameter.

[0199] In ST93, the comparison section 69 compares the physical parameters determined by the physical parameter conversion section 67 with the physical parameters determined by the curve fitting section 68 for each of the reference operating tools 51a to 51c.

[0200] In S94, the comparison unit 69 determines whether there are any of the reference operating tools 51a to 51c having physical parameters similar to the physical parameters converted by the physical parameter conversion unit 67. This determination is made based on the physical parameters P1 to P2 determined by the physical parameter conversion unit 67, as described above. n and the physical parameters P1 to P2 obtained by curve fitting the reference operating tools 51a to 51c. n There is a method for determining whether the sum of squares of the differences is less than a threshold value.

[0201] If the determination in S94 is Yes, in S95, the physical parameter setting unit 66 sets the physical parameters of the reference operating tools 51a to 51c that are similar to the physical parameters determined by the physical parameter conversion unit 67 in the replica operating tool 52.

[0202] If the determination in S94 is No, in S96, the physical parameter setting unit 66 sets the physical parameters of the reference operating tool 51a to 51c having the highest similarity to the reproduced operating tool 52. Alternatively, a first type of classification unit 64 may be provided, and the classification unit 64 may determine the reference operating tools 51a to 51c (first conversion model 65a to third conversion model 65c).

[0203] Thereafter, the user can repeatedly use the second input screen 120 to adjust the preferred operation feel until the preferred operation feel is obtained.

[0204] In this way, the tactile control device 50 of this embodiment can reproduce the operation feel preferred by the user in real time.

[0205] [Classification learning example] A classification learning method will be described with reference to Fig. 38 and other figures. Fig. 38 shows an example of a neural network in which the classification unit 64 is realized by a neural network. In the neural network of Fig. 38, three nodes in the output layer 133 each output an output value yi for data input to the input layer 131. This output value yi is a probability, and y1 + y2 + y3 is 1.0. In this embodiment, the three nodes in the output layer 133 correspond to the three reference operating tools 51a to 51c, and output the probability that one of the three reference operating tools 51a to 51c is likely, depending on the expression frequency.

[0206] FIG. 38 shows a neural network in which L layers (for example, three layers) from the input layer 131 to the output layer 133 are fully connected. A neural network with deep layers is called a DNN (Deep Neural Network). The layer between the input layer 131 and the output layer 133 is called an intermediate layer 132. The number of layers and the number of nodes in the intermediate layers can be set arbitrarily, so the number of layers and the number of nodes 130 in each layer are merely examples. In this embodiment, the number of nodes 130 in the input layer is the number of affective parameters (24 in FIG. 27(a)). Note that the expression frequency may be set for each affective parameter in any step adjustment, such as five steps or three steps, or may be continuously adjustable.

[0207] In a neural network, all nodes 130 in the l-1 layer are connected to one node 130 in the l-th layer (l: 2, 3) excluding the input layer, and the product of the output z of node 130 in the l-1 layer and the connection weight w is input to the node in the l-1 layer. Equation (1) shows how to calculate the output signal of node 130.

[0208]

number

[0209]

number

[0210] Each node in the output layer 133 receives z output from each node in the intermediate layer 132. i is input, and each node of the output layer 133 is z i Then, an activation function for the output layer is used for the nodes of the output layer 133. In the case of multi-value classification (selection of the reference operating tools 51a to 51c), the activation function of the output layer 133 is generally a softmax function. Each node of the output layer 133 is assigned an output value y i During learning, a teacher signal (1 or 0) is set after each node of the output layer 133 is associated with a reference operating tool. If learning is performed appropriately, each node of the output layer 133 can output the probability of the reference operating tools 51a to 51c corresponding to the 24 affective parameters. In the figure, the nodes correspond to the reference operating tools 51a to 51c from the top. However, if the output value is less than the threshold, it may be determined to be unclassified.

[0211] We will now explain the learning of the neural network. Multiple users operate the three reference operating tools 51a to 51c and input the expression frequency for each of the reference operating tools 51a to 51c. In this way, training data consisting of pairs of 24 affective parameters and one teacher signal (which reference operating tool) is obtained in the number of users x the number of reference operating tools. The teacher signal is either (1,0,0), (0,1,0), or (0,0,1).

[0212] The neural network processes the expression frequencies input to the input layer 131 and outputs an output value yi from the output layer 133. A teacher signal contained in the training data paired with the input expression frequencies is input to the nodes of the output layer 133. During learning, the error between the output value yi of the nodes of the output layer 133 and the teacher signal is calculated using a loss function. When the activation function of the output layer 133 is a softmax function, the loss function is cross entropy. The error between the teacher signal calculated by the loss function and the output value is propagated to the nodes of the input layer 131 using a calculation method called backpropagation. The weights w between the nodes are learned during the propagation process. Details of backpropagation will be omitted.

[0213] As a result of learning, for example, when an expression frequency is input for the reference operating tool 51a, the neural network is expected to have the node 130 corresponding to the reference operating tool 51a in the output layer 133 output a value close to 1.0, and the nodes 130 corresponding to the reference operating tools 51b and 51c output values ​​close to 0.0.

[0214] In FIG. 38, the nodes are fully connected, but convolutional layers, pooling layers, etc. may also be included.

[0215] Fig. 39 shows an example of a decision tree when the classification unit 64 is realized by a decision tree. A decision tree is a machine learning technique that finds clusters of data in which specific features frequently appear and generates classification rules for them. In this embodiment, learning corresponds to determining the affective parameters that frequently appear for each of the three reference operating tools 51a to 51c and their expression frequencies. A method using entropy is known as one method for learning the structure of a decision tree.

[0216] As machine learning suitable for classification, in addition to neural networks and decision trees, support vector machines, random forests, logistic regression, etc. may be used.

[0217] [Additional information on the first input screen] FIG. 40 is a diagram illustrating the first input screen 281 for the expression frequency in STEP 1. The user operates the slide bar to input the expression frequency for each affective parameter. The classification unit 64 described in the first embodiment uses the learning results to calculate the probability that each of the reference operating devices 51a to 51c will be selected for the current expression frequency. The display control unit 61 displays the probability for each of the reference operating devices 51a to 51c in the reference operating device field 112. Therefore, the user can understand which of the reference operating devices 51a to 51c the current expression frequency is closest to by operating the reference operating device 51a to 51c. The probability may be displayed either in real time or when the user inputs a confirmation operation.

[0218] Furthermore, when the user presses the icon of the reference operating tool 51a to 51c in the reference operating tool field 112, the display control unit 61 initializes the slide bar in the affective parameter presentation field 282 to the expression frequency set for the reference operating tool 51a to 51c. This allows the user to easily check the expression frequency of each reference operating tool 51a to 51c. Note that the expression frequency at the time of initialization may be the median or average value of the expression frequencies input for that reference operating tool 51 in a sensory test, for example.

[0219] [Operation of client-server system] Next, the operation of the client-server system will be described with reference to Fig. 41 etc. Fig. 41 is a functional block diagram of a haptic control system 2 in which the haptic control device 50 of the first form is applied to a client-server system. The explanation of Fig. 41 will mainly focus on the differences from Fig. 29. As shown in Fig. 41, the terminal device 80 and the server 200 have the same functions as the haptic control device 50 of Fig. 29, except that the terminal device 80 and the server 200 have a first communication unit 71 and a second communication unit 72, respectively.

[0220] Fig. 42 is a sequence diagram illustrating the operation of the haptic control system 2. In the explanation of Fig. 42, differences from Fig. 32 will be mainly explained.

[0221] In ST101, the first input receiving unit 62 receives an input of the expression frequency of the affective parameter for selecting the standard operating tools 51a to 51c inputted into the first input screen 281 (STEP 1).

[0222] In ST102, the first communication unit 71 of the terminal device 80 transmits the expression frequency of each affective parameter to the server 200.

[0223] In ST103, the classification unit 64 of the server 200 selects the reference operating tools 51a to 51c based on the expression frequency of each affective parameter.

[0224] In ST104, the second communication unit 72 of the server 200 transmits the physical parameters of the reference operating tools 51a to 51c to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a to 51c, and the physical parameter setting unit 66 sets them in the reproduction operating tool 52.

[0225] In ST105, the user determines whether to adjust the operation feel to a different one from those of the reference operating tools 51a to 51c depending on whether the operation feel is their preference. If the user adjusts the operation feel to a different one from those of the reference operating tools 51a to 51c, the second input receiving unit 63 receives input of the expression frequency of the affective parameter input on the second input screen 120 (STEP 2).

[0226] In ST106, the first communication unit 71 of the terminal device 80 transmits the expression frequency of the affective parameter to the server 200.

[0227] In ST107, one of the first conversion model 65a to the third conversion model 65c of the server 200 (already selected in ST103) is used to calculate the expression frequency based on the physical parameters P1 to P n Convert to.

[0228] In ST108, the physical parameter setting unit 66 of the server 200 transmits the converted physical parameters to the terminal device 80 via the second communication unit 72. The physical parameters of the reference operating tools 51a to 51c received by the first communication unit 71 of the terminal device 80 are set in the reproduction operating tool 52.

[0229] In this way, the haptic control system 2 of this embodiment can reproduce the user's preferred operation feel in real time even in a client-server system.

[0230] Figure 43 is a functional block diagram of a haptic control system 2 in which the second form of haptic control device 50 is applied to a client-server system. The explanation of Figure 43 will mainly focus on the differences from Figure 34. As shown in Figure 43, the terminal device 80 and the server 200 have the same functions as the haptic control device 50 in Figure 34, except that the terminal device 80 and the server 200 have a first communication unit 71 and a second communication unit 72, respectively.

[0231] 44 is a sequence diagram illustrating the operation of the second embodiment of the haptic control system 2. In the explanation of FIG. 44, differences from FIG. 37 will be mainly explained.

[0232] In ST111, the first input receiving unit 62 receives an input of the expression frequency of the affective parameter on the first input screen 281.

[0233] In ST112, the first communication unit 71 of the terminal device 80 transmits the expression frequency of each affective parameter to the server 200.

[0234] In ST113, the physical parameter conversion unit 67 of the server 200 converts each affective parameter into a physical parameter (load-displacement curve) based on the expression frequency of each affective parameter.

[0235] In ST114, the comparison unit 69 of the server 200 compares the physical parameters converted by the physical parameter conversion unit 67 with the physical parameters determined in advance by the curve fitting unit 68 for each of the reference operating tools 51a to 51c.

[0236] In ST115, if there are reference operating tools 51a-51c with physical parameters similar to the physical parameters determined by the physical parameter conversion unit 67, the second communication unit 72 transmits the physical parameters of one of the similar reference operating tools 51a-51c to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a-51c, and the physical parameter setting unit 66 sets them in the reproduction operating tool 52.

[0237] In ST116, if there is no reference operating tool 51a-51c with physical parameters similar to those determined by the physical parameter conversion unit 67, the second communication unit 72 transmits the physical parameters of one of the reference operating tools 51a-51c with the highest similarity to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a-51c, and the physical parameter setting unit 66 sets them in the reproduced operating tool 52. Alternatively, the first type of classification unit 64 may be provided, and the classification unit 64 may determine the reference operating tools 51a-51c (first conversion model 65a-third conversion model 65c).

[0238] In this way, the haptic control system 2 of this embodiment can reproduce the user's preferred operation feel in real time even in a client-server system.

[0239] [Additional note for mode 2] [Claim 1] A tactile control device that controls the operating feel of an operating tool, a display control unit that displays an input means for inputting a first expression frequency associated with the first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a physical parameter setting unit that sets a pre-prepared physical parameter in the reproduction operating tool based on the first expression frequency, the display control unit displays an input means for inputting a second expression frequency associated with the second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation; a conversion unit that converts the second expression frequency into a physical parameter using a regression model, The haptic control device is characterized in that the physical parameter setting unit sets the physical parameters converted by the conversion unit to the reproduction operation tool. [Claim 2] a classification unit that classifies the first expression frequency into one of a plurality of reference operating tools; 2. The haptic control device according to claim 1, wherein the physical parameter setting unit sets the physical parameters set for the reference operating tool classified by the classification unit to the replica operating tool. [Claim 3] a curve fitting unit that performs curve fitting on a load-displacement curve realized by first physical parameters of a plurality of reference operating tools, and estimates the first physical parameters for each of the plurality of reference operating tools; a physical parameter conversion unit that converts the first expression frequency into a second physical parameter using a regression model; The tactile control device according to claim 1, characterized in that the physical parameter setting unit sets the first physical parameter of the reference operating tool having the first physical parameter that is most similar to the second physical parameter for the reproduction operating tool. [Claim 4] The tactile control device according to claim 2, characterized in that the input means for the second expression frequency can take on an expression frequency corresponding to the physical parameters set for the reference operating tool classified by the classification unit, as well as values ​​before and after that. [Claim 5] The haptic control device according to claim 1, characterized in that there are a plurality of the first sensory parameters and a plurality of the second sensory parameters, and the number of the first sensory parameters is greater than the number of the second sensory parameters. [Claim 6] 3. The tactile control device according to claim 2, wherein the classifier is generated by learning a correspondence between the operation feels of the plurality of reference operating tools and the expression levels input for each of the first affective parameters by a user operating each of the plurality of reference operating tools. [Claim 7] 3. The haptic control device according to claim 2, wherein the regression model is generated by performing a regression analysis on the correspondence between the physical parameters of the plurality of reference operating tools and the expression levels input for each of the second affective parameters by a user operating the plurality of reference operating tools. [Claim 8] 4. The haptic control device according to claim 3, wherein the regression model is generated by performing a regression analysis on a correspondence between physical parameters of an arbitrary reference operating tool and an expression frequency input for each of the first affective parameters by a user operating the arbitrary reference operating tool. [Claim 9] The haptic control device according to claim 3, characterized in that the curve fitting unit performs curve fitting on the load-displacement curve using a fitting model that calculates the operation stress from the stroke amount using the first physical parameter as a coefficient, and estimates the first physical parameter. [Claim 10] the first affective parameter and the second affective parameter are adjectives, 2. The haptic control device according to claim 1, wherein the first expression frequency and the second expression frequency are values ​​indicating the degree of the adjective. [Claim 11] 11. The tactile control device according to claim 1, wherein the first expression frequency and the second expression frequency are tactile information obtained when a user operates an operating tool, respectively. [Claim 12] The tactile control device according to claim 11, wherein in the regression model, the first expression frequency and the second expression frequency are correlated with actuation forces as tactile sensations obtained when operating the respective operating tools. [Claim 13] a tactile control device for controlling the operating feel of an operating tool, a display control unit that displays an input means for inputting a first expression frequency associated with the first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a physical parameter setting unit that sets a pre-prepared physical parameter in the reproduction operating tool based on the first expression frequency; the display control unit displays an input means for inputting a second expression frequency associated with the second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation; a conversion unit that converts the second expression frequency into a physical parameter using a regression model; The physical parameter setting unit sets the physical parameters converted by the conversion unit to the reproduction operating tool. [Claim 14] A tactile control method in which a tactile control device that controls an operation feel of an operation tool controls a tactile sense, a step of displaying an input means for a first expression frequency associated with the first affective parameter; receiving an input of the first expression frequency in response to a user operation; a step of setting a physical parameter prepared in advance in a reproduction operating tool based on the first expression frequency; a step of displaying an input means for a second expression frequency associated with the second affective parameter; receiving an input of the second expression frequency in response to a user operation; converting the second expression frequency into a physical parameter by a regression model; setting the converted physical parameters to the reproduction manipulation tool; A tactile control method comprising: [Claim 15] A haptic control system in which a terminal device and a server communicate via a network, The terminal device a display control unit that displays an input means for inputting a first expression frequency associated with the first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a first communication unit that transmits the first expression frequency to the server; a physical parameter setting unit that sets the physical parameters transmitted from the server in the reproduction operating tool, the display control unit displays an input means for inputting a second expression frequency associated with the second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation, the first communication unit transmits the second expression frequency to the server; The server a second communication unit that determines the physical parameter prepared in advance based on the first expression frequency received from the terminal device and transmits the determined physical parameter to the terminal device; a conversion unit that converts the second expression frequency received from the terminal device into a physical parameter using a regression model, The haptic control system is characterized in that the second communication unit transmits the physical parameters converted by the conversion unit to the terminal device. [Claim 16] a display control unit that displays an input means for inputting a first expression frequency associated with the first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a first communication unit that transmits the first expression frequency to a server; and a physical parameter setting unit that sets the physical parameters transmitted from the server to a reproduction operating tool. the display control unit displays an input means for inputting a second expression frequency associated with the second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation, and the first communication unit communicates with a terminal device that transmits the second expression frequency to the server via a network, a second communication unit that determines a pre-prepared physical parameter based on the first expression frequency received from the terminal device and transmits the determined physical parameter to the terminal device; a conversion unit that converts the second expression frequency received from the terminal device into the physical parameter using a regression model, The server is characterized in that the second communication unit transmits the physical parameters converted by the conversion unit to the terminal device. [Aspect 3] [Background technology] Conventionally, operating units that provide a sense of sensation by providing some kind of stimulus to a person have been known. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating units.

[0240] A game controller is known in which buttons and the like incorporating vibration devices are interchangeable (see, for example, Patent Document 3). Patent Document 3 discloses a technique for exchanging the vibration device itself to achieve different vibration intensities.

[0241] [Summary of the Invention] [Problem to be solved by the invention] However, conventional technologies have a problem in that they do not adequately present sensations according to the physical characteristics of the operation unit. For example, in the case of a rotary operation unit, the sensation conveyed to the user operating the operation unit varies depending on the size and mass of the operation unit, even if the actuator is driven in the same way.

[0242] In view of the above-mentioned problems, an object of this aspect is to provide a technology for presenting sensations according to the physical characteristics of an operation unit.

[0243] [Effects of the invention] It is possible to provide a technology that presents sensations according to the physical characteristics of the operating unit.

[0244] [Explanation of Aspect 3] In this embodiment, a sensation control method for making adjustments based on the physical characteristics of an operation unit (for example, an operation device 33 in FIG. 45 described later) will be described. When the tactile presentation device 20 generates a tactile sensation through the operation unit by driving an actuator, depending on the physical characteristics (size, mass, etc.) of the operation unit, the sensation (operation sensation perceived by the user) conveyed to the user (an example of an operator) operating the operation unit will differ even if the actuator is driven in the same way.

[0245] In other words, the physical parameters correlated with the sensory parameters are composed of a composite physical parameter of the operation unit and the physical parameter of the actuator. Therefore, the tactile presentation device 20 of this embodiment adjusts the tactile presentation signal to be suitable for physical parameters such as the size and mass of the operation unit. The tactile control system 110 includes an adjustment unit that adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation based on the physical characteristics of the operation unit.

[0246] For example, the difference in the physical characteristics of the operation unit is detected as follows. The user inputs information about the differences in the physical characteristics of the operation unit into the input / output device 3. The size and mass of the operation unit are identified. The tactile presentation device 20 detects the ID, size, mass, etc., which represent the differences in physical characteristics of the operation unit, using a sensor. The sensor that detects the difference in the physical characteristics of the control panel is a camera that reads one-dimensional and two-dimensional codes. The camera also identifies the control panel by recognizing its image. Alternatively, the sensor is an IC tag reader that reads the ID.

[0247] [Tactile Control System 110] 45 is a diagram showing the configuration of the haptic control system 110 of the sensation control system 100 in this embodiment. In this embodiment, components with the same reference numerals as in FIG. 2 perform similar functions, so only the main components of this embodiment will be mainly described.

[0248] The tactile presentation device 20 in FIG. 45 newly includes an operation unit sensor 254, a torque sensor 251, and a communication unit 256. When a detachable operation unit is attached to the tactile presentation device 20, the operation unit sensor 254 detects that the operation unit has been attached and detects information that can identify the operation unit. Information that can identify the operation unit includes an IC tag built into the operation unit, a one-dimensional code or a two-dimensional code attached to the operation unit, and the appearance of the operation unit. When the information that can identify the operation unit is an IC tag, the operation unit sensor 254 is an IC tag reader and acquires the ID (identification information) of the operation unit from the IC tag. When the information that can identify the operation unit is a one-dimensional code or a two-dimensional code, the operation unit sensor 254 is a camera and acquires the ID of the operation unit from the one-dimensional code or the two-dimensional code. When the information that can identify the operation unit is the appearance of the operation unit, the operation unit sensor 254 is a camera and a classifier and identifies the operation unit using the classifier that has learned the correspondence between image data of the appearance of the operation unit and the ID (the ID of the operation unit is determined).

[0249] The operation unit is exemplified by the operation device 33, and may be an attachment unit that is detachable from at least a part (which may be the whole or a part) of the operation device 33. The main control device 10 and the tactile presentation device 20 are also examples of a sensory control device.

[0250] The torque sensor converts the current that drives the actuator into torque during calibration to estimate the mass of the operation unit, as will be described in detail later.

[0251] The communication unit 256 communicates with the mobile terminal 60 to receive the size of the operation unit from the mobile terminal 60. Details will be described later.

[0252] Furthermore, main control device 10 in Fig. 45 newly includes operation unit parameters 54, a calibration unit 55, and a mass correction unit 261. Operation unit parameters 54 will be described in Fig. 46. Calibration unit 55 estimates the mass of the operation unit through calibration. Mass correction unit 261 corrects the mass of the operation unit. Calibration unit 55 and mass correction unit 261 will be described later.

[0253] 46 shows an example of the operation unit parameters 54. In the operation unit parameters 54, the ID of the operation unit is associated with mass, size, and other physical parameters. Mass and size are physical properties of the operation unit 201, and in this embodiment, mass and size are included in the physical parameters.

[0254] The size may be the radius, diameter, or total length (length of the largest part) if the operation unit is a rotary operation unit that accepts rotational operation. If the operation unit is a pressure-type operation unit, the size may be the length in the pressing direction. If the operation unit is a slide operation unit that accepts sliding operation, the size may be any of the length of the slide amount, height, width, or thickness. If the operation unit is a pivot operation unit that accepts tilting operation, the size may be the length of the operation unit.

[0255] Other physical parameters are as explained in embodiment 1. As shown in Fig. 46, when the ID of the operation unit is detected by the operation unit sensor 254, the physical parameters are known. [Operation section detection by operation section sensor] A method for detecting an operation unit by operation unit sensor 254 will be described with reference to Figs. 47 and 48. First, Fig. 47 is a diagram illustrating the differences in physical characteristics of rotary operation units. Fig. 47(a) shows small operation unit 201a, and Fig. 47(b) shows large operation unit 201b. Note that, hereinafter, any of operation units 201a and 201b will be referred to as "operation unit 201."

[0256] The operation units 201a and 201b in Fig. 47 are rotary type, but the tactile sensation transmitted to the user operating the operation units will differ depending on the size (diameter) and mass of the operation units 201a and 201b, even if the processor 14 drives the actuators in the same way. For example, the torque required to rotate the operation unit 201 can be smaller as the diameter increases. For this reason, if the reaction force against the rotation operation of the operation units 201a and 201b is made the same, the user may feel that it is difficult to turn the operation unit 201a, or feel that there is no operation sensation when operating the operation unit 201b.

[0257] Even if the sizes of the operation units 201 differ, each operation unit usually has a similar shape, so there is a certain relationship between size and mass. For example, mass is proportional to the cube of the size (e.g., radius), and an approximate proportionality constant can also be calculated. Therefore, as will be described later, it is possible to find the mass from the size of the operation unit 201, or find the size from the mass, using a conversion formula.

[0258] 48A and 48B are diagrams illustrating several methods for detecting the size and mass of the operation unit 201 using the operation unit sensor 254. In FIG. 48A, the operation unit 201 has an IC tag 202 built in or attached thereto. In this case, the operation unit sensor 254 is an IC tag reader 204 that energizes the IC tag 202 using electromagnetic waves, communicates with the IC tag 202, and receives the ID of the operation unit from the IC tag 202. The IC tag reader 204 is preferably installed in the tactile presentation device 20, but may be an external device such as a mobile terminal 60.

[0259] 48(b), a barcode 203 is attached to the operation unit 201. In this case, the operation unit sensor 254 acquires the ID of the operation unit by capturing an image of the barcode 203 with the camera 205 and decoding the barcode 203. The camera 205 is preferably installed in the tactile presentation device 20, but may be an external device such as the mobile terminal 60.

[0260] In FIG. 48(c), the operation unit sensor 254 captures an image of the operation unit 201 itself with a camera. The operation unit sensor 254 estimates the size of the operation unit sensor 254 from the image data based on the preset distance between the camera 205 and the operation unit 201 and the focal length of the camera 205. A classifier that has learned the correspondence between the ID and the image data of the distance, focal length, and appearance of the operation unit can output the ID of the operation unit from the image data. Regarding mass, a conversion formula is used to calculate mass from size.

[0261] 48 may be built into the tactile presentation device 20, or may exist separately from the tactile presentation device 20. For example, the operation sensor 254 may be an information processing device carried by the user, such as a mobile terminal 60.

[0262] [When the control unit detected by the control unit sensor is not included in the control unit parameters] There may be a case where the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters. For example, In the case of controllers used by users, such as game controllers, the user may be able to change the knobs or other operating parts, and may wish to obtain an operating feel that is suited to the attached part. In some cases, a user may wish to change the operating handle of a vehicle or the like to obtain an operating feel that is suitable for the installed handle.

[0263] If the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters, the mobile terminal 60 estimates the physical parameters. The user starts a predetermined application on the mobile terminal 60. The user takes an image of the operation unit 201 attached to the tactile presentation device 20 with a camera controlled by the application. The application then detects the size of the operation unit 201 from the image data of the operation unit 201. Therefore, it is preferable that the camera included in the mobile terminal 60 is a stereo camera or a LiDAR (lidar) scanner. The application transmits the size of the operation unit 201 to the tactile presentation device 20. The communication unit 256 receives the size of the operation unit 201.

[0264] Even if the communication unit 256 receives the size, the mass is unknown, so a conversion formula for calculating the mass from the size is used to calculate the mass of the operation unit 201. Alternatively, the application calculates the mass from the size using the conversion formula and transmits the result to the tactile presentation device 20.

[0265] [Estimating the mass of the control unit through calibration] Next, we will explain how the calibration unit 55 estimates the mass through calibration. When the operation unit 201 is attached, the calibration unit 55 operates the operation unit using a current pattern (rotates the operation unit in the case of a rotary type), and estimates the mass of the operation unit from the correspondence between the current and the position.

[0266] Fig. 49 is a diagram illustrating a method for estimating the mass of the operation unit by calibration. First, Fig. 49(a) is a diagram illustrating the position of the rotary operation unit 201. In the case of the rotary operation unit 201, the position may be the rotation angle of the center of rotation. If the top surface of the operation unit 201 is circular, the center of rotation is the center of the circle. When the calibration unit 55 rotates the rotary operation unit 201, the heavier the mass, the larger the current required.

[0267] FIG. 49(b) is a diagram illustrating the relationship between the current required to change the position of the operation unit 201 and the position. The relationship between the current and the position shown in FIG. 49(b) is an example for illustrative purposes. Generally, the greater the change in position, the greater the current required. Furthermore, the current has a certain relationship with the torque that rotates the operation unit, and the torque required to rotate the operation unit can be found from the current. It is known that the current required to change the position is greater as the mass of the operation unit increases. The torque sensor 251 converts this current into torque.

[0268] If the relationship "I = αM" between the current I required to rotate the operation unit 201 to a certain position and the mass M is known, then the mass M of the attached operation unit 201 can be estimated by measuring the current I required when the calibration unit 55 rotates the operation unit 201 to a certain position. Note that α can be easily found by measuring the current required to rotate several operation units 201 with known masses to a certain position.

[0269] In this way, the mass M of the operation unit 201 equipped with the calibration unit 55 is estimated. Regarding the size, a conversion formula for finding the mass from the size is used.

[0270] Therefore, if the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters, the size and mass of the attached operation unit can be estimated by calibration in addition to using an application on the mobile terminal 60.

[0271] [Mass correction according to the installation location of the control unit] The degree of inclination of the operation unit 201 varies depending on the installation location. For example, the inclination of the operation unit 201 varies depending on whether the operation unit 201 is attached to a steering wheel or a center console. When the inclination varies, the effect of gravity causes a difference in the feel of operation, particularly of the pressure-type operation unit. Therefore, the tactile presentation device 20 uses the acceleration sensor 28 to measure the inclination of the installation location of the operation unit 201 and corrects the mass of the operation unit 201.

[0272] FIG. 50 is a diagram for explaining the correction of the mass of the operation unit 201. FIG. 50(a) shows the operation reaction force F1 when the operation unit 201 placed at an installation location with zero tilt is pressed down. The operation reaction force F1 is, for example, the maximum value Tmax in FIG. 11. FIG. 50(b) shows the operation reaction force F2 when the operation unit 201 placed at an installation location with a tilt θ is pressed down. Due to the relationship between the operation reaction forces F1 and F2 and the tilt θ shown in FIG. 50(b), the operation reaction force F2 is as follows: F2=F1 / cosθ As described above, a larger operational reaction force is required in an installation location with an inclination, but there is a correlation between the operational reaction force and mass. Therefore, the mass correction unit 261 corrects the mass of the operation unit 201 by regarding the difference in operational reaction force as a difference in mass. The mass correction unit 261 corrects the mass of the operation unit 201 using a relationship such as "mass after correction = original mass / cosθ." In this way, a preferable operation feel can be controlled even if the operation unit 201 is installed in an inclined location.

[0273] [Operations and Processing] FIG. 51 is a flowchart showing the process of adjusting the haptic presentation signal in accordance with the physical parameters of the operation unit to which the haptic control system 110 is attached.

[0274] First, the haptic control system 110 obtains correspondence between physical parameters including the mass and size of the operation unit and sensory parameters by SD method or the like (ST121).

[0275] Next, when the user wears the operation unit, the operation unit sensor 254 detects the operation unit worn by the user (ST122).

[0276] The tactile presentation device 20 determines whether or not the detected operation unit is included in the operation unit parameters 54 (ST123). The case where the operation unit sensor 254 cannot detect an ID is also included in the case where the operation unit parameters 54 do not include the case where the detected operation unit is not included in the operation unit parameters 54.

[0277] If the determination in step ST123 is Yes, the conversion model 15 converts the physical parameters registered in the operation unit parameters 54 into perceptual parameters (ST124). Note that the conversion model 15 in this embodiment calculates perceptual parameters from physical parameters as shown in FIG. 22.

[0278] If the determination in step ST123 is No, the user takes a picture of the operation unit using an application on the mobile terminal 60 and transmits the size and mass to the tactile presentation device (ST125).

[0279] The communication unit 256 receives the size and mass from the application of the mobile terminal 60 (ST126). As described above, the size and mass determined by the calibration unit 55 through calibration may be used.

[0280] The conversion model 15 converts the estimated physical parameters (size, mass) into perceptual parameters (ST127).

[0281] Then, the arithmetic function unit 12 generates a tactile sensation presentation signal using physical parameters such as size and mass (which have been registered in the operation unit parameters 54 or have been estimated) (ST128).

[0282] The arithmetic function unit 13 transmits a tactile presentation signal to the tactile presentation device 20. When the user performs a rotation operation or the like on the operation unit 201, the processor 18 generates an operation signal. If the operation unit is a rotary operation unit, the operation signal is, for example, a rotation angle. If the operation unit is another type of operation unit, the operation signal is an amount of operation of the operation unit. The tactile presentation unit 30 controls the actuator by the tactile presentation signal corresponding to the operation signal (ST129).

[0283] The arithmetic function unit 12 may convert the sensory parameters converted from the physical parameters in step ST127 back into physical parameters to generate a tactile sensation signal. A dedicated conversion model 15 may be prepared for the second conversion.

[0284] In this way, the haptic control system 110 can estimate the physical parameters even when an unregistered operation unit is attached. The calculation function unit 12 as an adjustment unit generates a haptic presentation signal based on the physical parameters, and therefore can adjust the haptic presentation signal according to the attached operation unit.

[0285] The adjustment unit is not limited to adjusting the "tactile sensation providing signal," but may also be capable of adjusting the "operation signal," the "sensory providing signal," the "sensory presentation" itself, or any combination thereof. Specifically, there are the following cases. The processor 18 (an example of an operation detection unit) functions as an adjustment unit and reflects the adjustment in the "operation signal." A case in which the calculation function unit 12 (an example of a signal generation unit) functions as an adjustment unit and reflects the adjustment in the "sensory presentation signal." The tactile presentation unit 30 functions as an adjustment unit and reflects the adjustment in the "sensory presentation."

[0286] Furthermore, in this embodiment, since the conversion model estimates the sensory parameters from the physical parameters, it is possible to construct a correlation between the sensory parameters and the physical parameters that also reflects the physical parameters of the operation unit. Supplementally, this content can also be applied to content related to "adjusting the sensory presentation signal." That is, when the physical parameters of the operation unit change due to, for example, replacing the operation unit, driving the actuator in the same way as before the operation unit 201 was replaced will result in a different sensation, i.e., a different sensory parameter. When the sensory parameters to be realized are constant, it is possible to adjust the sensory presentation signal to adjust the physical parameters of the actuator and present a sensory presentation that is in line with the set sensory parameter.

[0287] Moreover, Fig. 52 is a flowchart showing a process of adjusting a tactile presentation signal in accordance with physical parameters of an operation unit to which the tactile control system 110 is attached, as a modified example of Fig. 51. In the explanation of Fig. 52, differences from Fig. 51 will be mainly explained.

[0288] In FIG. 52, if the determination in step ST123 (an example of a predetermined condition) is No, the arithmetic function unit 12 stops generating the sensation providing signal (ST130).

[0289] In this way, when an operating unit with unknown physical parameters is attached and it is difficult to generate an appropriate sensory signal, the sensory signal can be stopped.

[0290] Furthermore, the arithmetic function unit 12 may generate a predetermined sensation providing signal such as an initial value, instead of stopping the generation of the sensation providing signal.

[0291] [Haptic control system with communication device (server) and terminal device] Next, a haptic control system 111 having a communication device 70 (server) and a terminal device 80 will be described with reference to Fig. 53. Fig. 53 shows the configuration of the haptic control system 111 as a second embodiment of the sensation control system 100 shown in Fig. 45, along with the signal flow. Note that the description of Fig. 53 will mainly focus on the differences from Fig. 45.

[0292] 53, the tactile presentation device 20 of the terminal device 80 has a torque sensor 251, an operation unit sensor 254, and a communication unit 256. Furthermore, the communication device 70 has an operation unit parameter 54, a calibration unit 55, and a mass correction unit 261. The torque sensor 251, the operation unit sensor 254, the communication unit 256, the operation unit parameter 54, the calibration unit 55, and the mass correction unit 261 may be the same as those described in FIG.

[0293] FIG. 54 is a sequence diagram showing how the communication device 70 (server) and the terminal device 80 communicate with each other to estimate the affective parameters of the attached operation unit.

[0294] In step ST131, the haptic control system 111 obtains correspondence between physical parameters including the mass and size of the operation unit and sensory parameters by SD method or the like.

[0295] In step ST132, when the user wears the operation unit, the operation unit sensor 254 detects the operation unit worn by the user.

[0296] In step ST133, the terminal device 80 transmits the ID of the operation unit detected by the operation unit sensor 254 to the communication device 70. If the operation unit sensor 254 cannot detect the ID, the terminal device 80 transmits to the communication device 70 a message indicating that the ID was not detected.

[0297] In step ST134, the communication device 70 determines whether or not the operation unit parameter 54 contains an attached operation unit based on the ID of the operation unit received.

[0298] If an operation unit is registered in the operation unit parameters 54, the conversion model 15 converts the physical parameters registered in the operation unit parameters 54 into perceptual parameters in step ST135.

[0299] If the operation unit is not registered in the operation unit parameter 54, in step ST136, a message is sent to the terminal device 80 indicating that the communication device 70 is not registered.

[0300] In step ST137, the user takes a picture of the operation unit using an application on the mobile terminal 60, and transmits the size and mass to the tactile presentation device 20.

[0301] In step ST138, the communication unit 256 receives the size and mass from the application of the mobile terminal 60.

[0302] In step ST139, the portable terminal 60 transmits the size and mass to the communication device .

[0303] In step ST140, the conversion model 15 converts the estimated physical parameters (size, mass) into perceptual parameters.

[0304] In step ST141, the arithmetic function unit 12 generates a tactile sensation providing signal using physical parameters such as size and mass (which have been registered in the operation unit parameters 54 or estimated).

[0305] In step ST142, the communication device 70 transmits a haptic presentation signal to the terminal device 80.

[0306] In step ST143, in response to an operation signal generated by a user's operation, the haptic sense providing unit 30 controls the actuator with a haptic sense providing signal corresponding to the operation signal. At least one of the operation signal, the sensory providing signal, and the sensory presentation may be adjusted by either the communication device 70 or the terminal device 80.

[0307] [Major effects] According to the tactile control systems 110 and 111 of this embodiment, the physical parameters of the operating unit are adjusted according to the size, mass, etc. of the operating unit, so that even if the size or mass of the operating unit changes, the feeling conveyed to the user operating it can be controlled to a feeling that is preferable to the user.

[0308] [others] For example, the operation unit of aspect 3 is not limited to being detachable. For example, in a system in which multiple operation units are implemented, if multiple operation units with different knob sizes and designs are arranged, it is possible to recognize the differences and generate appropriate tactile sensations.

[0309] Furthermore, the operation unit sensor 254 may estimate the size and mass of the attached operation unit by comparing it with a reference operation unit, rather than directly determining the size and mass through an application or calibration of the mobile terminal 60. For example, if an operation unit whose ID is registered in the operation unit parameters 54 and an operation unit whose ID is not registered are placed close to each other, the image data will show two operation units. The processor 18 calculates the ratio between the size of the operation unit whose ID is registered and the size of the operation unit whose ID is not registered, and multiplies this ratio by the size and mass of the operation unit whose ID is registered to estimate the size and mass of the operation unit whose ID is not registered.

[0310] The processor 18 is an example of an operation detection unit, the arithmetic function unit 12 is an example of a signal generation unit, and the tactile sense providing unit 30 is an example of a sensation providing unit.

[0311] [Additional Note for Mode 3] [Claim 1] An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a signal generating unit that generates a sensation providing signal based on the operation signal; a sensation providing unit that provides a sensation to an operator based on the sensation providing signal; A sensation control device comprising: an adjustment unit that adjusts at least one of the operation signal, the sensation presentation signal, and the sensation presentation based on the physical characteristics of the operation unit. [Claim 2] The sensory control device according to claim 1, wherein the physical characteristics of the operating portion include at least one physical parameter of a mass, a diameter, a radius, or a total length of at least a part of the operating portion. [Claim 3] an operation unit sensor that detects the attached operation unit; The operation unit sensor identifies a physical characteristic of the operation unit by acquiring identification information of the operation unit, or 2. The sensory control device according to claim 1, wherein the physical characteristics of the operation unit are identified from image data of the operation unit. [Claim 4] 2. The sensory control device according to claim 1, wherein the sensory providing unit stops generating the sensory providing signal when a physical characteristic of the operation unit satisfies a predetermined condition. [Claim 5] 2. The sensory control device according to claim 1, wherein the operation unit is a pressure-type operation unit that accepts a pressing operation. [Claim 6] 2. The sensory control device according to claim 1, wherein the operation unit is a slide operation unit that accepts a slide operation. [Claim 7] 2. The sensory control device according to claim 1, wherein the operation unit is a pivot operation unit that receives a tilt operation. [Claim 8] 2. The sensory control device according to claim 1, wherein the operation unit is a rotary operation unit that accepts a rotation operation. [Claim 9] The sensory control device according to claim 1, wherein the sensory presentation signal is correlated with a sensory parameter. [Claim 10] 2. The sensation control device according to claim 1, wherein the sensation providing unit is a tactile sensation providing unit that provides a tactile sensation to an operator. [Claim 11] 2. The sensory control device according to claim 1, wherein at least a part of the operation unit is detachable. [Claim 12] a torque sensor that detects a torque required when the operation unit is driven by an actuator; a calibration unit that estimates the mass of the operation unit from the torque detected by the torque sensor based on a relationship between torque and mass that is prepared in advance; The sensory control device according to claim 1, characterized in that it comprises: [Claim 13] an acceleration sensor that detects the tilt of the operation unit; a mass correction unit that corrects the mass of the operation unit in accordance with the tilt detected by the acceleration sensor; The sensory control device according to claim 1, characterized in that it comprises: [Claim 14] A sensory control method performed by a device having an operation unit, detecting an operation of the operation unit and generating an operation signal; generating a sensation providing signal based on the operation signal; providing a sensation to an operator based on the sensation providing signal; adjusting at least one of the operation signal, the sensation providing signal, and the sensation provided based on a physical characteristic of the operation unit; A sensation control method comprising: [Claim 15] A sensory control system including a communication device and a terminal device that can communicate with each other, The terminal device An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a sensation providing unit that provides sensation to an operator based on a sensation providing signal transmitted from the communication device, The communication device a signal generating unit that generates the sensation providing signal based on the operation signal; The terminal device or the communication device is a sensory control system comprising an adjustment unit that adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation based on the physical characteristics of the operation unit. [Aspect 4] [Background technology] Conventionally, operating tools that provide a sense of sensation by applying some kind of stimulus to a person have been known. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating tools.

[0312] A tactile system that provides a click sensation and the like that takes into account a fingertip model is known (see, for example, Patent Document 4). Patent Document 4 discloses a technique for parameter evaluation of the response to shear vibrations generated by the fingertip while pressing a key by applying a mass-spring-damper system approximation to the fingertip.

[0313] [Summary of the Invention] [Problem to be solved by the invention] However, conventional technologies do not take into account deformation of elastic bodies such as fingers in the direction of operation, such as buckling due to pressing operations, which results in a narrow range of expressive power for sensory presentation. That is, although fingers contain elastic bodies such as skin and flesh, buckling phenomena caused by elastic bodies are not reflected in sensory presentation.

[0314] In view of the above-mentioned problems, the present aspect aims to provide a technology that further expands the range of expressive power of sensory presentation.

[0315] [Effects of the invention] It is possible to provide a technology that further expands the range of expressiveness of sensory presentation.

[0316] [Explanation of Aspect 4] In this embodiment, a tactile control system 1 and a sensory control method for outputting a sensory stimulation signal based on physical parameters including dynamic characteristics are described. Dynamic characteristics are physical characteristics including a time factor, and for example, physical characteristics change over time.

[0317] In this embodiment, the block diagram of FIG. 1, the hardware configuration diagram of the haptic control system 1 of FIG. 2, and other necessary explanations explained in the above embodiment 1 are used for the explanation.

[0318] Conventionally, the load-displacement curve when a user presses an operating tool such as a switch is based on the static characteristics of a rigid body, which does not include a time factor. Therefore, it is not possible to obtain information on the correspondence between sensory parameters and physical parameters in a state that reproduces the buckling phenomenon that occurs when a user actually presses a switch with their finger.

[0319] In this embodiment, to simulate the situation when a user presses a manipulation tool with their finger, a finger model pressing tool is used, which has an elastic body (corresponding to the flesh and skin of the finger) integrated between a rigid body (corresponding to the bones of the finger) and the manipulation tool. When the finger model pressing tool presses the manipulation tool, the position change [mm] of the manipulation tool and the values ​​[N] of two force sensors between the elastic body and the manipulation tool are analyzed, and measurements and evaluations are performed using the SD method with a configuration that takes into account a human finger. The new physical parameters obtained in this way include dynamic characteristics, so correspondence information between sensory parameters and physical parameters is generated in a state that reproduces the buckling phenomenon that occurs when a user actually presses the manipulation tool with their finger.

[0320] Specifically, the following correlation is obtained: Note that the buckling period T1, fingertip impact period T3, and fingertip vibration period T4 are the periods shown in Fig. 60(b) to be described later, and details will be given later. Correlation between physical parameters (movement distance of the operating tool during the fingertip impact period T3, change in the force sensor value during the fingertip impact period T3, fingertip impact period T3) and sensory parameters (return feeling) Correlation between physical parameters (positional change during buckling period T1) and sensory parameters (suction sensation) Correlation between physical parameters (fingertip vibration period T4) and sensory parameters (fatigue sensation) [Example of configuration of finger model pressing tool and operation tool] 55 is a diagram illustrating the static characteristics obtained by a rigid pressing tool and the dynamic characteristics obtained by a finger model pressing tool 252 in which a rigid body and an elastic body are integrated. First, the load-displacement curve of the operating tool 250 by a rigid body 253 pressing tool can only express the static characteristics that do not include a time factor. The load-displacement curve 75 does not include the influence of the elastic body corresponding to the flesh part 257 of the finger, and therefore does not fully express the physical characteristics that contribute to the tactile sensation perceived by the operator.

[0321] Next, the pressing of the operating tool 250 by the finger model pressing tool 252 will be described. First, the flesh 257 of the finger is an elastic body that deforms under stress. Inside the finger, there is also a bone 255, which can be considered a rigid body. As will be described later, the finger model pressing tool 252 is designed to have the characteristics of the flesh 257 and the bone 255. When the finger model pressing tool 252, which is an integrated rigid body and elastic body, presses the operating tool 250, the operation reaction force, position change, and other dynamic characteristics over time are included. In FIG. 55, the dynamic characteristics 270 include the position change and two force sensor values ​​A and B. The two force sensor values ​​A and B respectively detect the operation reaction force generated by the finger model pressing tool 252 on the operating tool 250. The two force sensor values ​​A and B are measured by different force sensors, which are located at the location where the flesh 257 of the finger contacts the operating tool 250 and at the rigid part inside the finger (corresponding to the bone 255). Details will be explained in Fig. 59. As shown in the dynamic characteristics 270, the pressing of the operating tool 250 by the finger model pressing tool 252 can capture the finger movement taking time into consideration, that is, the occurrence and change of sensation, and therefore a correlation close to the actual situation when a user presses with their finger can be obtained.

[0322] Fig. 56 is a diagram illustrating the relative positions of a finger and operating tool 250 when the finger is deformed. The upper part of Fig. 56 shows periods A to C that can be read from load-displacement curve 75. The lower part of Fig. 56 schematically shows deformation of the flesh of the finger corresponding to periods A to C.

[0323] As shown in the lower part of FIG. 56, during period A, the position of button portion 56 of operating tool 250 gradually drops as the pushing force and repulsive force of the finger balance each other.

[0324] During period B, deformation (buckling) of metal contact 57 of operating tool 250 occurs, and the repulsive force disappears. Button portion 56 falls downward while maintaining the downward force. The operation reaction force becomes different from that during period A. Therefore, the operation reaction force at the contact point between the finger and the button is reduced.

[0325] During period C, the finger and button portion 56 collide again with metal contact 57. At this time, the maximum operation reaction force is again generated at the contact point between the fingertip and button portion 56. The collision also causes vibration of button portion 56.

[0326] 57 is a diagram illustrating a finger model pressing tool 252. As explained in FIG. 56, a finger is an elastic body in which flesh 257 deforms. Also, bones 255 that can be considered as rigid bodies exist inside the finger. Therefore, finger model pressing tool 252, which has elastic body 59 that contacts button portion 56 and rigid body 58 that presses button portion 56 via elastic body 59, serves as an appropriate model when a finger presses operating tool 250.

[0327] [Generation of a click-like sensory signal] Figure 58 is a diagram explaining the generation of a click sensation signal. The click sensation refers to the reaction when an input is made on an input device such as a button, or the feeling of pressing a switch. In the case of a mechanical switch, the click sensation is obtained by the resistance or deformation of the metal contact 57, etc. However, how the click sensation is generated varies depending on the button structure.

[0328] Furthermore, in the operating tool 250 in which the sensation providing signal is generated electrically as in this embodiment, the click feeling is controlled by the current supplied to the actuator.

[0329] FIG. 58(a) shows the current value of the actuator over time, and FIG. 58(b) shows the operation reaction force over time. When the current value suddenly decreases in frame 283, the operation reaction force also suddenly decreases. Convex portion 284 in FIG. 58(b) corresponds to the time when the current value suddenly decreases. Therefore, when the user presses operating tool 250 with their finger, they can feel a click, similar to pressing a mechanical switch. The timing when the current value suddenly decreases and the amount of current decrease shown in FIG. 58(a) are merely examples and can be adjusted as appropriate.

[0330] Fig. 59(a) shows a functional configuration diagram of a pressure-type operating tool, and Fig. 59(b) shows a block diagram of the pressure-type operating tool. The button portion 271 in Fig. 59 is an example of the operating device 33 in Fig. 2, and the VCM 263 is an example of the tactile sense providing unit 30 in Fig. 2. As shown in Fig. 59(a), two force sensors A and B are arranged on the finger model pressing tool 252. Force sensor A is arranged at a position where the elastic body 59 of the finger model pressing tool 252 and the button portion 271 come into contact, and force sensor B is arranged inside the rigid body 58 of the finger model pressing tool 252. In this way, the buckling phenomenon can be monitored by the force sensor value A detected by force sensor A.

[0331] The block diagram in Figure 59(b) is merely an example of a pressure-type operating tool, but will be briefly explained. The MCU circuit 262 is an example of the processor 18 in Figure 2, and the position sensor 264 is an example of the position sensor 27 in Figure 2. As shown in Figure 59(b), the MCU circuit 262 outputs a current to a VCM (Voice Coil Motor) 263 that corresponds to the amount of depression (position change) of a button portion 271 of the operating tool 250. The VCM 263 applies an artificial reaction force proportional to the current to the button portion 271. The finger model pressing tool 252 presses the button portion 271 from the opposite side of the VCM 263, so that an artificial reaction force is transmitted to the finger model pressing tool 252. The artificial reaction force is measured by force sensors A and B.

[0332] [Dynamic characteristics obtained using a finger model pressing tool] FIG. 60 is a diagram illustrating the dynamic characteristics when the operating tool 250 is pressed by the finger model pressing tool 252. FIG. 60(a) is a load-displacement curve 75 shown for reference, and FIG. 60(b) is an example of the dynamic characteristics 270 when the operating tool 250 is pressed by the finger model pressing tool 252. In FIG. 60(b), the horizontal axis represents time, and the vertical axis represents the two force sensor values ​​A and B and the position change 211. The unit of time is [msec], and the unit of the force sensor values ​​A and B is [N]. Note that the dynamic characteristics 270 vary greatly depending on the operating tool 250, and FIG. 60(b) is merely an example.

[0333] The dynamic characteristics (buckling period T1, fingertip drop period T2, fingertip collision period T3, fingertip vibration period T4) extracted from two force sensor values ​​A and B over time and position change 211 over time will be described with reference to Fig. 60 and Fig. 61. Fig. 61 is a diagram illustrating the transition over time of the relative position of the finger model pressing tool 252 and the operating tool 250.

[0334] The buckling period T1 is the period from the peak of the force sensor value B to the peak of the position change 211. Although it is difficult to see due to the scale, the force sensor value B is not constant and has a peak at the start of the buckling period T1. This peak will be explained in Figure 62. The peak of the force sensor value B corresponds to the maximum value of the operation reaction force on the load-displacement curve 75. Therefore, the buckling period T1 is the period from when the operation reaction force reaches its maximum value to when the position change 211 reaches its maximum. Figure 61(a) shows the relative positions of the finger model pressing tool 252 and the button portion 56 at the start of the buckling period T1. At the start of the buckling period T1, the maximum value of the operation reaction force on the load-displacement curve 75 is reached, and therefore the elastic body 59 of the finger model pressing tool 252 is significantly pressed down. The arrow to the left of the button portion 56 indicates the direction of the position change.

[0335] The fingertip drop period T2 is the period from the peak of force sensor value B to the downward peak of force sensor value A. As shown in load-displacement curve 75, after the maximum value of the operational reaction force is reached, the operational reaction force suddenly decreases to produce a clicking sensation. Therefore, the operational reaction force on the finger model presser 252 decreases, and the elastic body 59 of the finger model presser 252 begins to restore after the start of the fingertip drop period T2. This causes the force sensor value A to decrease during the fingertip drop period T2. Therefore, the fingertip drop period T2 is the period from the peak value of the operational reaction force to the elastic body of the finger model presser 252 returning to its maximum. Figure 61(b) shows the relative positions of the finger model presser 252 and the button portion 56 at the end of the fingertip drop period T2. A comparison with Figure 61(a) reveals that the elastic body 59 of the finger model presser 252 has restored to its original state.

[0336] The fingertip impact period T3 is the period from the downward peak of the force sensor value A to the upward peak of the force sensor value A. During the fingertip impact period T3, after the elastic body 59 of the finger model pressing tool 252 has fully restored in FIG. 61(b), the finger model pressing tool 252 continues to be pressed, causing the force sensor value A to increase rapidly. Therefore, the fingertip impact period T3 is the period from when the elastic body 59 of the finger model pressing tool 252 has fully restored to when the elastic body 59 is fully depressed. FIG. 61(c) shows the relative positions of the finger model pressing tool 252 and the button portion 56 at the end of the fingertip impact period T3. Comparing this with FIG. 61(b), it can be seen that the elastic body 59 of the finger model pressing tool 252 is being depressed.

[0337] The fingertip vibration period T4 is the period from the upward peak of the force sensor value A until the fluctuation of the force sensor value A settles within a certain value. Because the operation reaction force has already been reduced to produce a click feeling, the force sensor value A rapidly decreases even if the position change 211 continues to increase due to the pressing force. After that, the position change 211 stops increasing (the finger model pressing tool 252 also stops moving), so the force sensor value B also becomes less likely to change, and the force sensor value A oscillates like chattering. Therefore, the fingertip vibration period T4 is the period until the most depressed elastic body returns to its original position and stabilizes. Figure 61(d) shows the relative positions of the finger and button portion 56 at the end of the fingertip vibration period T4. A comparison with Figure 61(c) reveals that the elastic body 59 of the finger model pressing tool 252 has returned to its original position.

[0338] The above-described buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4 are examples of dynamic characteristics. Furthermore, during each of the buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4, changes in the force sensor values ​​A and B and position change 211 can be extracted. In this embodiment, these can also be used as dynamic characteristics.

[0339] In this way, the dynamic characteristic may be a physical characteristic including a time change in at least one of the operation reaction force and the operation amount accompanying the operation of the predetermined operation tool 250. This physical characteristic is a physical characteristic that realizes the presentation of a sensation when the elastic body 59 of the finger model pressing tool 252 is brought into contact with the operation tool 250 and operated.

[0340] FIG. 62 is a diagram explaining the dynamic characteristics in more detail, along with the periods A to C mentioned above. The upper left portion of FIG. 62 is an overall view including the dynamic characteristics from start to finish, and the lower right portion of FIG. 62 is an enlarged view of the dynamic characteristics within frame 212 in the upper left portion of FIG. 62. The lower right portion of FIG. 62 shows the correspondence between the dynamic characteristics and periods A to C. Force sensor value A detected by force sensor A changes significantly as elastic body 59 is pressed and restored. Force sensor value B detected by force sensor B is less affected by the deformation of elastic body 59, and therefore changes only slightly.

[0341] The lower right part of Figure 62 also shows the buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4, which are as explained in Figure 60. The peaks of force sensor B (the start points of the buckling period T1 and fingertip drop period T2), which were not clear in Figure 60, are now clearly visible.

[0342] [Dynamic characteristics correlated with sensory parameters] Some of the dynamic characteristics explained in Figures 60 and 62 are correlated with the sensory parameters. Appropriate dynamic characteristics correlated with the sensory parameters are the physical parameters of this embodiment.

[0343] The haptic control system 1 performs evaluation using the SD method in order to evaluate appropriate dynamic characteristics correlated with the sensitivity parameters. For this purpose, a plurality of operation tools 250 with different dynamic characteristics are prepared.

[0344] Fig. 63 shows the dynamic characteristics of a plurality of operating tools 250 with different dynamic characteristics when pressed by a finger model pressing tool 252. In this embodiment, for the purpose of explanation, 25 operating tools 250 were prepared, and the dynamic characteristics were measured for each of the 25 operating tools 250. Fig. 63 shows the dynamic characteristics of four of the operating tools 250. In each of Figs. 63(a) to (d), the upper diagram shows the dynamic characteristics 270 for the entire period (approximately 1 second) when pressed, and the lower diagrams are enlarged views of the dynamic characteristics 270 before and after the buckling period T1, the fingertip dropping period T2, the fingertip impact period T3, and the fingertip vibration period T4.

[0345] <Determining physical parameters correlated with sensory parameters> FIG. 64 is a flowchart illustrating the flow of determining physical parameters correlated with subjective parameters.

[0346] In step ST151, the haptic control system 1 measures the dynamic characteristics when the finger model pressing tool 252 presses each of the 25 operation tools 250.

[0347] Next, in step ST152, the input unit 4 receives the expression frequency for each affective parameter for the 25 operating tools 250 using the SD method.

[0348] Next, in step ST153, the processor 101 acquires a pair of the dynamic characteristic and the expression frequency of each operation tool 250 for each affective parameter.

[0349] Next, in step ST154, the processor 101 calculates the correlation coefficient between the dynamic characteristic and the expression frequency for each affective parameter.

[0350] Next, in step ST155, the processor 101 determines a dynamic characteristic having a large absolute value of the correlation coefficient. The large absolute value of the correlation coefficient may be, for example, 0.5 or more.

[0351] Next, in step ST156, the processor 101 applies the multiple regression analysis explained in Equation 5 to the physical parameters and the perceptual parameters that have a high correlation with the perceptual parameters, thereby creating the conversion model 15.

[0352] Figure 65 is a scatter plot of pairs of dynamic characteristics and expression frequencies of each operating tool 250 for a certain affective parameter acquired by processor 101 in step ST153. In Figure 65, the horizontal axis represents the affective parameter "there is (is not) a sense of return" and the vertical axis represents buckling period T1. The buckling period T1 and the expression frequencies of "there is (is not) a sense of return" generally have an upward trend. The correlation coefficient is 0.82.

[0353] Figure 66 is a scatter plot of pairs of dynamic characteristics and expression frequencies of each operating tool 250 for a certain affective parameter acquired by processor 101 in step ST153. In Figure 66, the horizontal axis represents the affective parameter "feeling of being sucked in (or not)," and the vertical axis represents the position change during fingertip impact period T3. There is a general downward trend between the position change during fingertip impact period T3 and the expression frequency of "feeling of being sucked in (or not)." The correlation coefficient is 0.65.

[0354] Figure 67 is a scatter plot of pairs of dynamic characteristics and expression frequencies of each operating tool 250 for a certain affective parameter acquired by processor 101 in step ST153. In Figure 67, the horizontal axis represents the affective parameter "there is (is not) a sense of return" and the vertical axis represents the change in operation reaction force (force sensor value A) during fingertip vibration period T4. The change in operation reaction force during fingertip vibration period T4 and the expression frequency of "there is (is not) a sense of return" generally have an upward trend. The correlation coefficient is 0.78.

[0355] The processor 101 correlates the affective parameters and dynamic characteristics shown in Figures 65, 66, and 67 using the least squares method (an example of regression analysis) or the like. By the least squares method, the strength of the correlation between the affective parameters and dynamic characteristics is estimated using a correlation coefficient.

[0356] Fig. 68 shows a list of correlation coefficients between each sensory parameter and each dynamic characteristic. In Fig. 68, the row headings are sensory parameters, and the column headings are dynamic characteristics of the operating tool 250. In Fig. 68, correlation coefficients of 0.5 or more are highlighted with diagonal lines. It can therefore be seen that dynamic characteristics with large correlation coefficients are suitable for physical parameters.

[0357] In this way, when each operating tool 250 is pressed by the finger model pressing tool 252, physical parameters highly correlated with the perceptual parameters are determined, and the processor 101 applies the multiple regression analysis explained in Equation 5 to the physical parameters highly correlated with the perceptual parameters and the perceptual parameters to create the conversion model 15. The physical parameters P1 to Pn used in Equation 5 are those with large correlation coefficients determined in step ST154. The multiple regression analysis has been explained in Equation 5, FIGS. 22 and 23 of the first embodiment. Therefore, the coefficient of determination B 11 ~B mn can be determined, and a transformation model 15 as shown in FIG. 23 can be obtained for each operation tool 250.

[0358] [Haptic control system with communication device (server) and terminal device] Next, a haptic control system 2 having a communication device 70 (server) and a terminal device 80 will be described with reference to Fig. 69. The block diagram of the haptic control system 2 may be the same as that in Fig. 20.

[0359] FIG. 69 is a sequence diagram showing how the communication device 70 (server) and the terminal device 80 communicate with each other to estimate the affective parameters of the attached operating tool 250. In FIG.

[0360] In step ST161, the communication device 70 and the terminal device 80 communicate with each other, and the finger model pressing tool 252 presses the 25 operation tools 250, thereby measuring the dynamic characteristics of each operation tool 250.

[0361] Next, in step ST162, the input unit 4 receives the expression frequency for each affective parameter for the 25 operating tools 250 using the SD method.

[0362] Next, in step ST163, the terminal device 80 transmits the expression frequency to the communication device .

[0363] Next, in step ST164, the processor 14 obtains a pair of the dynamic characteristic and the expression frequency of each operation tool 250 for each affective parameter.

[0364] Next, in step ST165, the processor 14 calculates the correlation coefficient between the dynamic characteristic and the expression frequency for each affective parameter.

[0365] Next, in step ST166, the processor 14 determines a dynamic characteristic having a large absolute value of the correlation coefficient. The large absolute value of the correlation coefficient may be, for example, 0.5 or more.

[0366] Next, in step ST167, the processor 14 applies the multiple regression analysis explained in Equation 5 to the physical parameters and the perceptual parameters that are highly correlated with the perceptual parameters, thereby creating the conversion model 15.

[0367] [Major effects] As described above, the tactile control system 1 of this embodiment can extract dynamic characteristics correlated with the sensory parameters by pressing the operating tool 250 with the finger model pressing tool 252. Therefore, a conversion model that converts the sensory parameters into these dynamic characteristics can be created, and a sensory presentation signal with desirable dynamic characteristics can be generated.

[0368] [others] For example, although a pressure-type operating tool has been described in the second aspect, the present invention can be similarly applied to a rotary operating tool that accepts rotational operation. In the case of a rotary operating tool, the rotation angle is the position change, and the resistance force against the rotation is the operation reaction force.

[0369] Furthermore, although the finger model pressing tool 252 has been described as having only one type of elastic body 59, the finger model pressing tool 252 may have multiple types of elastic bodies with different elastic forces on the side that comes into contact with the button portion 56. The multiple types of elastic bodies with different elastic forces may be, for example, elastic bodies corresponding to skin, elastic bodies corresponding to flesh, etc. Furthermore, the multiple types of elastic bodies with different elastic forces may be arranged in layers so that the closer they are to the rigid body 58, the greater the elastic force. In this way, a finger model pressing tool 252 can be constructed that exhibits dynamic characteristics closer to human tactile sensation.

[0370] The shape of the finger model pressing tool 252 may be a simple cube or may be an imitation of the shape of a finger. The finger shape may be of different sizes and shapes to accommodate the fingers of men, women, adults, children, and people of all races.

[0371] [Additional Note for Mode 4] [Claim 1] a receiving step of receiving an input of a sensitivity parameter indicating a degree of sensory expression when the operating tool is operated; a conversion step of converting the received sensory parameter into a physical parameter correlated with the sensory parameter from among a plurality of types of physical parameters included in the physical characteristics related to the sensory stimulus; and outputting a sensory stimulation signal based on the converted physical parameter; The sensory control method, wherein the physical property includes a dynamic property. [Claim 2] 2. The sensation control method according to claim 1, wherein the dynamic characteristics are physical characteristics including time variations of at least one of an operation reaction force and an operation amount accompanying the operation of a predetermined operating tool. [Claim 3] The sensation control method according to claim 2, wherein the physical property is a physical property that realizes sensation presentation when a finger model pressing tool including a rigid body and an elastic body is brought into contact with the predetermined operating tool and operated. [Claim 4] The sensation control method according to claim 1 , wherein the physical parameter is a buckling period. [Claim 5] The sensation control method according to claim 1 , wherein the physical parameter is a fingertip drop duration. [Claim 6] The sensation control method according to claim 1 , wherein the physical parameter is a fingertip impact duration. [Claim 7] The sensation control method according to claim 1 , wherein the physical parameter is a fingertip vibration period. [Claim 8] The sensation control method according to claim 1 , wherein the physical parameter has a correlation with the subjective parameter. [Claim 9] 2. The sensation control method according to claim 1, wherein the operation tool is a pressure-type operation tool that accepts a pressing operation. [Claim 10] 2. The sensation control method according to claim 1, wherein the operating tool is a rotary operating tool that accepts rotational operation. [Claim 11] an input unit that receives an input of a sensitivity parameter that indicates the degree of sensory expression when the operating tool is operated; a conversion model for converting the sensory parameter received by the input unit into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in physical characteristics related to sensory stimuli; a sensation providing unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model, The device wherein the physical characteristics include dynamic characteristics. [Claim 12] A sensory control system including a communication device and a terminal device that can communicate with each other, the terminal device has an input unit that receives an input of an affective parameter that indicates a degree of sensory expression when the operating tool is operated, the communication device has a conversion model that converts the affective parameter transmitted from the terminal device into a physical parameter correlated with the affective parameter from among a plurality of types of physical parameters included in physical characteristics related to sensory stimuli, the terminal device includes a sensation providing unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model; The sensory control system wherein the physical characteristics include dynamic characteristics. [Claim 13] The device, an input unit that receives an input of a sensitivity parameter that indicates the degree of sensory expression when the operating tool is operated; a conversion model for converting the sensory parameter received by the input unit into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in physical characteristics related to sensory stimuli; a sensory presentation unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model; The program, wherein the physical characteristics include dynamic characteristics.

[0372] [others] Although the best mode for carrying out the present invention has been described above using various aspects, the present invention is not limited to these aspects and various modifications and substitutions can be made without departing from the scope of the present invention. For example, the functions included in each component, step, etc. can be rearranged so as not to cause logical contradictions, and multiple components, steps, etc. can be combined into one or divided.

[0373] This application claims priority based on Patent Application No. 2021-084696 filed with the Japan Patent Office on May 19, 2021, Patent Application No. 2022-079095 filed with the Japan Patent Office on May 12, 2022, Patent Application No. 2022-079099 filed with the Japan Patent Office on May 12, 2022, and Patent Application No. 2022-079128 filed with the Japan Patent Office on May 13, 2022, and the entire contents of Patent Application No. 2021-084696, Patent Application No. 2022-079095, Patent Application No. 2022-079099, and Patent Application No. 2022-079128 are incorporated herein by reference. [Explanation of symbols]

[0374] 1, 2 Haptic control system 3 Input / Output Devices 4 Input section 5 Display section 6, 10 Main control unit 7, 14, 18, 41, 101 processors 8, 11 Storage section 9 Network 12, 13 Calculation function unit 15. Sensibility parameter-physical parameter conversion model 16 Sensitivity Database 20, 40 Tactile presentation device 21 Moving parts 24 bobbins 25 coils 26 Spring member 27 Position Sensor 28 Acceleration Sensor 29 Variable operation range section 30, 43 Tactile display unit 31 York 31a Periphery yoke 31b Center York 32 Magnet 33, 42 Operating device 39 Actuator 43a Resistance torque generator 43b Rotational torque generator 45 sensors 70 Communication equipment 80 Terminal Equipment 100 Sensory Control System 102 Sensory presentation unit

Claims

1. An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a signal generating unit that generates a sensation providing signal based on the operation signal; a sensation providing unit that provides a sensation to an operator based on the sensation providing signal, an adjustment unit that adjusts at least one of the operation signal, the sensation providing signal, and the sensation provided when operating the operation unit, based on the mass and size of the operation unit as physical characteristics, the operation unit is a slide operation unit, The adjustment unit is characterized in that it adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when sliding the slide operation unit based on the mass and any one of the sliding amount, height, width, and thickness of the operation unit, which are physical characteristics.

2. an operation unit sensor that detects the attached operation unit; The operation unit sensor identifies a physical characteristic of the operation unit by acquiring identification information of the operation unit, or The sensory control device according to claim 1 , wherein the physical characteristics of the operation unit are identified from image data of the operation unit.

3. The sensory control device according to claim 1 , wherein the sensory providing unit stops generating the sensory providing signal when the physical characteristics of the operation unit satisfy a predetermined condition.

4. The sensory control device according to claim 1 , wherein the sensory presentation signal is correlated with a sensory parameter.

5. The sensory control device according to claim 1 , wherein the sensory presentation unit is a tactile presentation unit that presents a tactile sensation to the operator.

6. The sensory control device according to claim 1 , wherein at least a part of the operation unit is detachable.

7. a torque sensor that detects a torque required when the operation unit is driven by an actuator; a calibration unit that estimates the mass of the operation unit from the torque detected by the torque sensor based on a relationship between torque and mass that is prepared in advance; The sensory control device according to claim 1, further comprising:

8. The operation unit is installed at a predetermined installation location, an acceleration sensor that detects a tilt of the operation unit at the installation location; a mass correction unit that corrects the operation reaction force of the operation unit based on the action of gravity corresponding to the tilt detected by the acceleration sensor; The sensory control device according to claim 1, further comprising:

9. A sensory control method performed by a device having an operation unit, detecting an operation of the operation unit and generating an operation signal; generating a sensation providing signal based on the operation signal; providing a sensation to an operator based on the sensation providing signal; and adjusting at least one of the operation signal, the sensation providing signal, and the sensation provided when operating the operation unit based on the mass and size of the operation unit as physical characteristics, the operation unit is a slide operation unit, In the adjusting step, the mass of the operation unit and any one of the sliding distance, height, width, and thickness are set as physical properties, and at least one of the operation signal, the sensory presentation signal, and the sensory presentation when the slide operation unit is slid is adjusted based on the mass, the sliding distance, height, width, and thickness. A method for controlling sensations.

10. A sensory control system including a communication device and a terminal device that can communicate with each other, The terminal device An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a sensation providing unit that provides sensation to an operator based on a sensation providing signal transmitted from the communication device, The communication device a signal generating unit that generates the sensation providing signal based on the operation signal; the terminal device or the communication device includes an adjustment unit that, with a mass and a size of the operation unit as physical characteristics, adjusts at least one of the operation signal, the sensation provision signal, and the sensation provision when operating the operation unit based on the mass and the size; the operation unit is a slide operation unit, The adjustment unit uses the mass of the operation unit and any one of the sliding distance, height, width, and thickness as physical characteristics, and adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation when sliding the slide operation unit based on the mass and any one of the sliding distance, height, width, and thickness.

Images