Information processing apparatus, control method, and haptic feedback device

Abstract

The present disclosure relates to an information processing apparatus, a control method, and a haptic feedback apparatus capable of providing a common user experience. To provide an information processing apparatus including a control unit configured to perform control to acquire sensing data including information about a body part of a user in contact with a surface of a haptic presentation unit configured to present a cold or warm stimulus, and control operation of a drive unit to adjust a contact area that is an area in which a surface of the haptic presentation unit and a part of the user's body are in contact with each other, based on the acquired sensing data. The present disclosure can be applied to, for example, a system that presents a cold or warm stimulus in a virtual space.

Description

Technical Field

[0001] This disclosure relates to information processing devices, control methods, and haptic feedback devices, and in particular, to information processing devices, control methods, and haptic feedback devices that can provide a common user experience. Background Technology

[0002] One type of tactile stimulation is the stimulation of temperature. For example, by presenting users experiencing virtual spaces with temperature stimuli and reproducing the ambient temperature, temperature sensation, and material feel of virtual objects, the sense of realism can be enhanced.

[0003] As a technique related to the presentation of warm and cold stimuli, for example, there is the technique disclosed in PTL 1. PTL 1 discloses a technique for matching the timing of a specific frame of a motion image perceived by a user with the timing of a warm or cold stimuli perceived by the user.

[0004] Reference List

[0005] Patent documents

[0006] [PTL 1]JP 2020-135191 A Summary of the Invention

[0007] Technical issues

[0008] When presenting hot and cold stimuli, the state of contact between the user and the position of the stimulus varies from user to user, and therefore the temperature perceived by each user is different, and there may be differences in experience between users.

[0009] In view of the above, this disclosure was made and is intended to provide a common user experience.

[0010] Solution to the problem

[0011] According to one aspect of this disclosure, the information processing device includes a control unit configured to perform control of acquiring sensing data including information about a body part of a user in contact with a surface of a tactile presentation unit configured to present hot and cold stimuli, and to control the operation of a drive unit based on the acquired sensing data to adjust the contact area of ​​the area where the surface of the tactile presentation unit and the user's body part are in contact with each other.

[0012] According to one aspect of the present disclosure, the control method includes: acquiring sensing data, including information about a body part of a user that is in contact with the surface of a tactile presentation unit configured to present hot and cold stimuli, by an information processing device; and controlling the operation of a drive unit based on the acquired sensing data to adjust the contact area of ​​the area in contact between the surface of the tactile presentation unit and the user's body part.

[0013] In the information processing apparatus and control method according to one aspect of the present disclosure, sensing data including information about a user's body part in contact with the surface of a tactile presentation unit configured to present hot and cold stimuli is acquired, and based on the acquired sensing data, the operation of a drive unit is controlled to adjust the contact area of ​​the area where the surface of the tactile presentation unit and the user's body part are in contact with each other.

[0014] According to one aspect of this disclosure, a haptic feedback device is a haptic feedback device comprising: a haptic presentation unit configured to present warm or cold stimuli; a sensing unit configured to sense information about a body part of a user that is in contact with a surface of the haptic presentation unit; and a driving unit configured to perform an operation to adjust the contact area of ​​the area in contact between the surface of the haptic presentation unit and the user's body part based on driving data obtained from the sensing data acquired by the sensing unit.

[0015] A haptic feedback device according to one aspect of the present disclosure includes: a haptic presentation unit configured to present warm or cold stimuli; a sensing unit configured to sense information about a user's body part in contact with the surface of the haptic presentation unit; and a driving unit configured to perform an operation to adjust the contact area of ​​the surface of the haptic presentation unit and the user's body part in contact with each other based on driving data based on the sensing data acquired by the sensing unit.

[0016] It should be noted that the information processing device and haptic feedback device according to one aspect of this disclosure can be independent devices or constitute an internal block of a device. Attached Figure Description

[0017] Figure 1 This is a block diagram illustrating a configuration example of an information processing system to which this disclosure is applied.

[0018] Figure 2 It shows the composition Figure 1 A block diagram illustrating the functional configuration of devices in an information processing system.

[0019] Figure 3 This is a diagram illustrating an example configuration of a haptic feedback device including a haptic presentation unit composed of flexible Peltier elements.

[0020] Figure 4 It is shown in Figure 3 The figure shows an example of a flexible Peltier element deforming to change the contact area with a user's body part.

[0021] Figure 5 This is a diagram illustrating an example configuration of a haptic feedback device including a haptic presentation unit composed of multiple Peltier elements.

[0022] Figure 6 It is shown in Figure 5 The diagram illustrates an example of a scenario where multiple Peltier elements move to change the contact area with a user's body part.

[0023] Figure 7 This is a diagram illustrating an example configuration where a flexible Peltier element deforms by the expansion and contraction of an air bladder to adjust the contact area with a user's body part.

[0024] Figure 8 This is a diagram illustrating an example configuration in which multiple Peltier elements move by the expansion and contraction of the airbag to adjust the contact area with the user's body parts.

[0025] Figure 9 This is a diagram illustrating an example configuration of a haptic feedback device including a haptic presentation unit consisting of a Peltier element.

[0026] Figure 10 This is a diagram illustrating an example where the Peltier element is lifted by the rotation of a cam to change the contact area between the surface of the curved component and the user's body part.

[0027] Figure 11 This is a diagram illustrating an example where the Peltier element is lifted by the rotation of a cam to change the contact area between the surface of the curved component and the user's body part.

[0028] Figure 12 This is a diagram illustrating an example configuration where the Peltier element is raised by the expansion and contraction of an air bladder to adjust the contact area between the surface of the curved component and the user's body part.

[0029] Figure 13 This diagram illustrates an example of altering a user's temperature perception by changing the contact area while maintaining a constant temperature at the tactile presentation location.

[0030] Figure 14 This is a diagram illustrating an example of how the intensity of hot and cold stimuli can be homogenized by changing the contact area of ​​each tactile presentation location when there are multiple tactile presentation locations.

[0031] Figure 15 This is a diagram illustrating an example of how the contact area changes over time due to the adaptation effect.

[0032] Figure 16 This is a diagram illustrating an example of how the contact area is varied depending on the part of the user's body that is being exposed to hot or cold stimuli.

[0033] Figure 17 This is a diagram illustrating an example configuration of a haptic feedback device that includes a sensing unit consisting of physical sensors.

[0034] Figure 18 It is shown Figure 17 A diagram illustrating an example of surface pressure sensor measurements.

[0035] Figure 19 This is a diagram illustrating a configuration example of a haptic feedback device comprising a sensing unit consisting of multiple physical sensors.

[0036] Figure 20 It is shown Figure 19 A diagram illustrating an example of measurements taken by multiple pressure sensors.

[0037] Figure 21 This is a diagram illustrating an example of measurement using a drive unit or a haptic presentation unit.

[0038] Figure 22 This is a diagram illustrating an example of estimating the contact area using the encoder or resistance values ​​of a Peltier element.

[0039] Figure 23 This is a flowchart describing a first example of applying the warm and cold stimulus presentation processing disclosed herein.

[0040] Figure 24 This is a diagram illustrating an example of how different warm and cold stimuli are presented for each user.

[0041] Figure 25 This is a flowchart describing a second example of applying the warm and cold stimulus presentation processing disclosed herein.

[0042] Figure 26 It is shown Figure 1 A block diagram of another example of the functional configuration of an information processing system.

[0043] Figure 27 This is a diagram showing another configuration example of the shape of the housing and the flexible Peltier element.

[0044] Figure 28 This is a block diagram illustrating an example of a computer's hardware configuration. Detailed Implementation

[0045] System Configuration

[0046] Figure 1 This is a block diagram illustrating a configuration example of an embodiment of the information processing system to which this disclosure is applied. Figure 1 In this system, the information processing system 1 consists of an information processing device 11, a visual presentation device 12, a sensing device 13, a tactile presentation device 14, and an auditory presentation device 17.

[0047] Information processing device 11 is a central processing unit that controls the operation of the devices constituting information processing system 1. For example, information processing device 11 may be composed of devices such as personal computers (PCs), servers, or smartphones. Information processing device 11 may exchange data with visual presentation device 12, sensing device 13, tactile presentation device 14, and auditory presentation device 17 according to predetermined communication methods.

[0048] The information processing device 11 performs control to ensure that the presentation of video content visually presented to the user via the visual presentation device 12 and the presentation of warm and cold stimuli of video content tactilely presented to the user via the tactile presentation device 14 are executed at the correct timing. For example, the content is cross-reality (XR) content that includes virtual reality (VR), augmented reality (AR), and mixed reality (MR), and includes video (images) of the content to be presented to the user. The information processing device 11 performs control to ensure that the presentation of video content visually presented to the user via the visual presentation device 12 and the presentation of sound (speech, etc.) of content auditorily presented to the user via the auditory presentation device 17 are executed at the correct timing.

[0049] The visual presentation device 12 is a device for presenting video content to a user. For example, the visual presentation device 12 may consist of a device such as a display device, a head-mounted display (HMD), or a smartphone. The visual presentation device 12 displays the video content based on control data input from the information processing device 11. The content data may be received via a network or recorded by the visual presentation device 12 in a memory or the like. Alternatively, the content data may be input from the information processing device 11.

[0050] Sensing device 13 is a device that senses information about the user. For example, sensing device 13 consists of various sensors and is installed on a device such as a haptic glove, controller, or smartphone. Sensing device 13 outputs the sensed data obtained through sensing to information processing device 11. Information processing device 11 generates control data for controlling haptic presentation device 14 based on the sensed data input from sensing device 13.

[0051] The tactile presentation device 14 is a device that presents warm and cold stimuli to a user. For example, the tactile presentation device 14 includes elements (electronic components) such as flexible Peltier elements or Peltier components that present warm and cold stimuli and is mounted on a device such as a tactile glove, controller, or smartphone. The tactile presentation device 14 presents warm and cold stimuli to a part of the user's body (skin) based on control data input from the information processing device 11. Warm and cold stimuli (temperature stimulation) include at least one of warm or cold stimulation.

[0052] The auditory presentation device 17 is a device that presents sound (speech, etc.) of content to a user. For example, the auditory presentation device 17 is composed of devices such as headphones, speakers, or earphones. The auditory presentation device 17 outputs the sound of the content based on sound data and control data input from the information processing device 11. The sound of the content presented by the auditory presentation device 17 can be the sound synchronized with the video of the content presented by the visual presentation device 12.

[0053] Figure 1 The configuration of the information processing system 1 is an example, and another configuration may be adopted. For example, the sensing device 13 and the haptic presentation device 14 may be mounted on a single device (e.g., a haptic feedback device), and the information processing system 1 may consist of an information processing device 11, a visual presentation device 12, an auditory presentation device 17, and a haptic feedback device. The information processing device 11, the visual presentation device 12, the sensing device 13, the haptic presentation device 14, and the auditory presentation device 17 may be configured as a device included in a single housing (e.g., a device such as a smartphone).

[0054] In information processing system 1, another device can be further installed. In information processing system 1, it is not necessary to provide... Figure 1 All devices included in the illustrated configuration. For example, at least one of a visual presentation device 12 and an auditory presentation device 17 may be provided. In the information processing system 1, various communication methods such as wireless local area networks (LAN), short-range wireless communications such as Bluetooth (registered trademark), and cellular wireless communications (LTE-Advanced, 5G) may be used as communication methods between devices. Alternatively, not limited to wireless communications, wired communication may be performed by connecting the devices with cables.

[0055] Figure 2 It shows the composition Figure 1 A block diagram illustrating the functional configuration example of the equipment in information processing system 1. Figure 2 In this system, information processing system 1 consists of information processing device 11, visual presentation device 12, haptic feedback device 15, and auditory presentation device 17. The haptic feedback device 15 is equipped with… Figure 1 The device includes the sensing device 13 and the tactile presentation device 14.

[0056] exist Figure 2 In the information processing device 11, there are an information processing unit 21, a communication unit 22, a sensing information processing unit 23, a presentation data generation unit 24, a tactile data processing unit 25, and a communication unit 26. The visual presentation device 12 includes a communication unit 31 and a display unit 32. The tactile feedback device 15 includes a sensing unit 41, a communication unit 42, a driving unit 43, and a tactile presentation unit 44. In the tactile feedback device 15, the sensing unit 41 corresponds to... Figure 1 The sensing device 13 in the middle, and the tactile presentation unit 44 corresponding to Figure 1 The tactile presentation device 14 is included. The auditory presentation device 17 includes a communication unit 35 and a sound presentation unit 36.

[0057] exist Figure 2 In the information processing device 11, the information processing unit 21, the sensing information processing unit 23, the presentation data generation unit 24, and the tactile data processing unit 25 perform some functions of a control unit 20 composed of a processor such as a central processing unit (CPU). The control unit 20 may include a memory such as random access memory (RAM) that appropriately records the data required for processing. The communication unit 22 and the communication unit 26 may be configured as the same communication unit.

[0058] Information processing unit 21 performs processing to present an experience to the user in virtual space through vision, hearing, and touch, based on content. For example, information processing unit 21 manages various location information and various types of parameters such as physical parameters. Information processing unit 21 generates control data including various information required for presenting the video content (e.g., information about the user's position in virtual space) and outputs the control data to communication unit 22. Communication unit 22 is composed of communication circuits following a predetermined communication method. Communication unit 22 sends the control data input from information processing unit 21 to visual presentation device 12. Information processing unit 21 controls communication unit 22 to send the data required for the sound of the presented content (sound data, control data, etc.) to auditory presentation device 17.

[0059] Information processing unit 21 outputs various information (e.g., information about physical parameters) required for presenting the haptic feedback of the video content to sensing information processing unit 23. Various information from information processing unit 21 and sensing data from communication unit 26 are input to sensing information processing unit 23. Sensing information processing unit 23 estimates the contact area based on the sensing data. The contact area is the area of ​​contact between the surface of haptic presentation unit 44 and the user's body part (skin). Sensing information processing unit 23 outputs various information from information processing unit 21 and information about the estimated contact area to presentation data generation unit 24.

[0060] Various information from the sensing information processing unit 23, as well as information about the contact area, are input to the presentation data generation unit 24. The presentation data generation unit 24 generates presentation data for the tactile presentation unit 44 to present warm and cold stimuli based on the necessary information for presenting tactile sensation. The presentation data generation unit 24 also generates drive data for controlling the drive unit 43 based on information about the contact area. For example, when generating drive data, the drive data is generated by comparing the estimated contact area with a preset set value to execute the drive so that the contact area matches the set value. The presentation data generation unit 24 outputs the generated presentation data and drive data to the tactile data processing unit 25.

[0061] Presentation data and driving data from presentation data generation unit 24 are input to tactile data processing unit 25. Tactile data processing unit 25 processes the presentation data according to the characteristics of the components (electronic assemblies) constituting tactile presentation unit 44. That is, tactile presentation unit 44 is composed of flexible Peltier elements, etc., but because these components have individual differences, the presentation data is processed to match the characteristics of the components. Tactile data processing unit 25 outputs the processed presentation data and control data, including driving data, to communication unit 26.

[0062] The communication unit 26 is composed of communication circuits and the like that follow a predetermined communication method. The communication unit 26 receives sensing data sent from the haptic feedback device 15 and outputs the sensing data to the sensing information processing unit 23. The communication unit 26 also sends control data input from the haptic data processing unit 25 to the haptic feedback device 15. Note that in the information processing device 11, at least a portion of the processing performed by the sensing information processing unit 23, the presentation data generation unit 24, and the haptic data processing unit 25 can be executed by the information processing unit 21 or the control unit 20.

[0063] exist Figure 2 In the visual presentation device 12, the communication unit 31 is composed of a communication circuit or the like that follows the same communication method as the communication unit 22. The communication unit 31 receives control data sent from the information processing device 11 and outputs the control data to the display unit 32. The display unit 32 includes a display such as a liquid crystal display or an organic EL display. The display unit 32 displays video content based on the control data input from the communication unit 31. The content data can be received from another device (e.g., the information processing device 11, a server, etc.) connected via a network, or it can be recorded in a memory by the visual presentation device 12. Alternatively, the visual presentation device 12 can read the content data recorded on a recording medium.

[0064] exist Figure 2In the haptic feedback device 15, the sensing unit 41 is composed of a surface pressure sensor, a pressure sensor, etc. The sensing unit 41 senses the user's body parts (hand, fingers, etc.) in contact with the surface of the haptic presentation unit 44, and outputs sensing data, including the measurement values ​​obtained through sensing, to the communication unit 42. The communication unit 42 is composed of a communication circuit, etc., following the same communication method as the communication unit 26. The communication unit 42 sends the sensing data input from the sensing unit 41 to the information processing device 11. The communication unit 42 receives control data sent from the information processing device 11. The communication unit 42 outputs drive data included in the control data to the drive unit 43, and outputs presentation data included in the control data to the haptic presentation unit 44.

[0065] The drive unit 43 is composed of a motor or the like. The drive unit 43 adjusts the contact area between the user's body part (skin) and the surface of the tactile presentation unit 44 by driving based on drive data input from the communication unit 42. The tactile presentation unit 44 includes elements (electronic components) such as flexible Peltier elements or Peltier components that present warm and cold stimuli. The tactile presentation unit 44 presents warm and cold stimuli to the user's body part (skin) in contact with the surface based on presentation data input from the communication unit 42.

[0066] exist Figure 2 In the auditory presentation device 17, the communication unit 35 is composed of a communication circuit, connectors, etc., that follow the same communication method as the communication unit 22. The communication unit 35 receives data (control data, sound data, etc.) sent from the information processing device 11 and outputs the data to the sound presentation unit 36. The sound presentation unit 36 ​​outputs sound (speech, etc.) based on the data received from the communication unit 35.

[0067] Note that in Figure 2 In the description, the communication unit 26 of the information processing device 11 and the communication unit 42 of the haptic feedback device 15 have been described using the same communication method to send and receive control data and sensing data, but different communication methods may also be used to send and receive control data and sensing data. Figure 2 In the description, both the communication unit 31 of the visual presentation device 12 and the communication unit 35 of the auditory presentation device 17 send and receive data such as control data to and from the communication unit 22 of the information processing device 11. However, the communication methods between the communication unit 22 and the communication unit 31, as well as the communication methods between the communication unit 22 and the communication unit 35, can be different or the same.

[0068] Methods for adjusting contact area

[0069] Figure 3This is a diagram illustrating an example configuration of a haptic feedback device 15, including a haptic presentation unit 44 composed of flexible Peltier elements. Figure 3 In this diagram, the directions of the x-axis, y-axis, and z-axis are orthogonal to each other. It can also be said that the x-axis is the left-right direction, the y-axis is the front-back direction, and the z-axis is the up-down direction. Figure 3 A portion of the components of the haptic feedback device 15 is shown in a cross-sectional view in the y-direction (front-back direction), and a haptic presentation unit 44 composed of flexible Peltier elements is shown.

[0070] like Figure 3 As shown, the housing 51 of the haptic feedback device 15 has a recessed shape along the z-direction in a top view, and a flexible Peltier element 52 is disposed on the bottom surface of the recess. That is, an opening 51A is formed at a portion of the surface of the housing 51, and a flexible Peltier element 52 with a shape according to the opening 51A is disposed thereon. The flexible Peltier element 52 is a type of Peltier element capable of performing temperature control for cooling and heating; it is flexible, can be repeatedly deformed, and retains its properties even when its shape is deformed. Figure 3 In the process, the flexible Peltier element 52 has a curved surface shape, wherein the central portion of the heat dissipation surface protrudes along the z-direction.

[0071] In the flexible Peltier element 52, when the heat dissipation surface on the side in contact with the user's body is defined as the upper surface, and the heat dissipation surface opposite to the upper surface is defined as the lower surface, a plurality of heat sinks 53, such as heat sinks 53-1 to 53-3, are attached to the lower surface side. The heat sinks 53 have a shape such as a plate and are heat dissipation mechanisms that dissipate heat transferred by the flexible Peltier element 52 to the outside air, etc. In the plurality of heat sinks 53, a spring 54 is attached to the lower surface opposite to the upper surface of the flexible Peltier element 52. The spring 54 is an example of an elastic body.

[0072] exist Figure 3 The haptic feedback device 15 includes a drive unit 43 that can be operated to deform the flexible Peltier element 52 constituting the haptic presentation unit 44. The drive unit 43 is composed of, for example, a motor. The motor is a small motor that can be mounted on the haptic feedback device 15. The motor can be a voice coil motor (VCM), an ultrasonic motor, or the like. One end of each of a plurality of springs 54, such as springs 54-1 to 54-3, is fixed to the lower surface side of the heat sink 53, and the other end is fixed to the drive unit 43 side. The flexible Peltier element 52 can be deformed by driving the drive unit 43, such as a motor, to apply an external force to the lower surface side of the flexible Peltier element 52 via the springs 54, the heat sink 53, etc.

[0073] Figure 4 It is shown in Figure 3The diagram illustrates an example of a flexible Peltier element 52 deforming to change the contact area with a user's body part. Figure 4 The diagram schematically illustrates the state in which the skin S of a user's body part (e.g., hand or fingers) is in contact with the upper surface of the flexible Peltier element 52. For example, Figure 4 The status indication of B in the middle is more timely than Figure 4 A in the equation is a later state.

[0074] The area of ​​the skin S in contact with the upper surface of the flexible Peltier element 52 is Figure 4 A and B are different. Specifically, Figure 4 The contact area in B is greater than Figure 4 The contact area in A. Figure 4 In B, the flexible Peltier element 52 is deformed by driving a drive unit 43, such as a motor, to push the peripheral portion (which is the periphery of the central portion of the flexible Peltier element 52) ​​upward via springs 54-1, spring 54-3, etc. Specifically, the flexible Peltier element 52 has a curved shape protruding from its central portion in the z-direction. Figure 4 The convex curved shape in A is deformed into a curved surface shape in which the central part is concave in the z-direction. Figure 4 (The concave curved shape in B).

[0075] As described above, in the haptic feedback device 15, the contact area (contact area) between the user's skin S and the upper surface of the flexible Peltier element 52 can be changed by deforming the flexible Peltier element 52. Although details will be described later, in this case, the control unit 20 can estimate the contact area based on sensing data from the sensing unit 41 and perform control to adjust the contact area between the upper surface of the flexible Peltier element 52 and the user's skin S to a desired contact area. By performing such control to adjust the contact area, the contact area at the presentation position of the hot or cold stimulus presented by the flexible Peltier element 52 can be changed in real time.

[0076] exist Figure 3 The image shows a case where the tactile presentation unit 44 is composed of flexible Peltier elements, but similarly, control of adjusting the contact area can also be performed when multiple Peltier elements are used. Figure 5 This is a diagram illustrating an example configuration of a haptic feedback device 15 including a haptic presentation unit 44 composed of multiple Peltier elements. Figure 5 In the middle, the x, y, and z directions are... Figure 3 The same as in [the previous sentence].

[0077] exist Figure 5 In, with Figure 3In contrast, multiple Peltier elements 61, such as Peltier elements 61-1 to 61-3, are provided instead of flexible Peltier elements 52. Figure 5 In, with Figure 3 The same parts are given the same reference numerals, and their descriptions will be omitted as appropriate. A heat sink 53 is attached to the lower surface side of the Peltier element 61. The heat sink 53 is a heat dissipation mechanism that dissipates heat transferred by the Peltier element 61 to the outside air, etc. In the heat sink 53, a spring 54 is attached to the lower surface on the side opposite to the upper surface of the Peltier element 61, and the Peltier element 61 can be moved by driving a drive unit 43, such as a motor.

[0078] Figure 6 It is shown in Figure 5 A diagram illustrating an example of a scenario where multiple Peltier elements 61 move to change the contact area with a user's body part. Figure 6 In A, multiple Peltier elements 61 are moved by the drive unit 43, and the skin S of the user's body parts (hands, fingers, etc.) is in contact with the upper surfaces of Peltier elements 61-1, 61-2, and 61-3. On the other hand, in Figure 6 In B, multiple Peltier elements 61 are moved by the drive unit 43, and among Peltier elements 61-1 to 61-3, only the upper surface of Peltier element 61-2 is in contact with the skin S of the user's body part.

[0079] In this case, with Figure 6 Compared to the contact area in B, in Figure 6 In the contact area of ​​A, the area (contact area) where the user's skin S contacts the upper surface of the Peltier element 61 is larger. As described above, although the Peltier element 61 is not as flexible as the flexible Peltier element 52, by providing multiple Peltier elements 61 and adjusting the contact portions with the user's body parts, the overall contact area between the upper surface of the Peltier element 61 and the user's skin S can be adjusted to the desired contact area. A component with a predetermined shape (e.g., a convex curved shape) can be attached to the upper surface of the Peltier element 61.

[0080] exist Figure 3 and Figure 5 The diagram shows a drive unit 43 consisting of a motor or the like. However, a configuration consisting of another mechanism can be used, as long as the contact area between the surface of the haptic presentation unit 44 (such as flexible Peltier elements 52 and multiple Peltier elements 61) and the user's body part (skin) can be adjusted. For example, an airbag, a balloon, a solenoid, etc., can be used as the mechanism constituting the drive unit 43.

[0081] Figure 7 This diagram illustrates an example configuration where the flexible Peltier element 52 is deformed by the expansion and contraction of the airbag to adjust the contact area with the user's body parts. Figure 7 In, with Figure 3 and Figure 4 In contrast, multiple airbags 71, such as airbags 71-1 to 71-3, and micropumps are provided instead of springs 54 and motors. The airbags 71 are attached to the lower surface side of the radiator 53 and expand or contract by supplying or venting air from the micropumps.

[0082] In the haptic feedback device 15, the flexible Peltier element 52 can be deformed by inflating or contracting the airbag 71 using a micro-pump, and the contact area between the user's skin S and the upper surface of the flexible Peltier element 52 can be changed. For example, as Figure 7 As shown, by inflating airbags 71-1 and 71-3 to push the peripheral portion of the flexible Peltier element 52 upward, the shape of the flexible Peltier element 52 deforms into a curved shape in which the central portion is concave in the z-direction (a concave curved shape), and thus, the contact area can be increased. Alternatively, when airbags 71-1 and 71-3 contract, although not shown, the shape of the flexible Peltier element 52 changes to a curved shape in which the central portion protrudes in the z-direction (a convex curved shape), and thus, the contact area can be reduced.

[0083] exist Figure 7 The image shows a haptic presentation unit 44 composed of flexible Peltier elements 52. However, similarly, when the haptic presentation unit 44 is composed of multiple Peltier elements 61, control of adjusting the contact area via a drive unit 43, such as an airbag or a micropump, can also be performed. Figure 8 This is a diagram illustrating an example configuration in which multiple Peltier elements 61 are moved by the expansion and contraction of the airbag to adjust the contact area with the user's body parts.

[0084] exist Figure 8 In this device, a heat sink 53 is attached to the lower surface of a Peltier element 61. An airbag 71, which can be expanded or contracted using a micropump, is attached to the heat sink 53. In the haptic feedback device 15, the contact portion of the upper surface of multiple Peltier elements 61, such as Peltier elements 61-1 to 61-3, with the user's body part can be adjusted by using a micropump to expand or contract multiple airbags 71, such as airbags 71-1 to 71-3.

[0085] For example, such as Figure 8As shown, by inflating airbags 71-1 and 71-3, the skin S of the user's body part comes into contact with the upper surfaces of Peltier elements 61-1 and 61-3, thereby increasing the contact area. Alternatively, when airbags 71-1 and 71-3 are deflating, although not shown, the skin S of the user's body part does not come into contact with the upper surfaces of Peltier elements 61-1 and 61-3, thus reducing the contact area.

[0086] exist Figure 3 The image shows a case where the tactile presentation unit 44 is composed of a flexible Peltier element 52, but similarly, control of adjusting the contact area can also be performed when using a single Peltier element. Figure 9 This is a diagram illustrating an example configuration of a haptic feedback device 15 including a haptic presentation unit 44 consisting of a Peltier element.

[0087] exist Figure 9 In the middle, the x, y, and z directions are... Figure 3 The same as in. In Figure 9 In this context, when the surface of the Peltier element that contacts the user's body part is defined as the upper surface and the surface opposite to the upper surface is defined as the lower surface, the direction from the lower surface toward the upper surface is the z-direction, the direction parallel to the upper surface is the x-direction, and the direction orthogonal to the x-direction and parallel to the upper surface is the y-direction. Figure 9 A portion of the components of the haptic feedback device 15 is shown in a cross-sectional view along the y-direction. That is, in Figure 9 The diagram shows the drive unit 43, which consists of a cam, a motor, etc., and the tactile presentation unit 44, which consists of a Peltier element, in the components of the tactile feedback device 15.

[0088] like Figure 9 As shown, in the haptic feedback device 15, an opening 51A with a predetermined shape in a plan view viewed from above is formed on the surface of the housing 51. The Peltier element 82 has a shape in the plan view corresponding to the shape of the opening 51A. For example, when the opening 51A has a circular shape in the plan view, the Peltier element 82 can have a circular shape with a diameter smaller than the diameter of the circle of the opening 51A in the plan view. A curved member 81 is fixed to the upper surface of the Peltier element 82. The curved member 81 has a shape approximately the same as the Peltier element 82 in the plan view, and has a curved shape (convex curved shape) in the cross-sectional view in which the central portion protrudes in the z-direction. For example, as the material of the curved member 81, a material capable of transmitting temperature through the Peltier element 82 for cooling and heating can be used. Since the user's body part (skin) is in contact with the surface of the curved member 81, a deformable member can be used.

[0089] A heat sink 83 is attached to the lower surface of a Peltier element 82. The heat sink 83 has a plate-like shape and is a heat dissipation mechanism that dissipates heat transferred by the Peltier element 82 to the outside air. When the surface of the heat sink 83 attached to the Peltier element 82 is defined as the upper surface, one end of each of springs 84-1 and 84-2 is fixed to the upper surface. The other ends of springs 84-1 and 84-2 are fixed to the rear surface of the housing 51 and serve as a buffer. The lower surface of the heat sink 83 contacts a portion of a cam 85. The cam 85 has a generally elliptical shape in cross-section. The cam 85 can push the heat sink 83 upward by rotating shaft 85A via a drive motor, and can also move the position of the curved member 81 and the Peltier element 82 in the z-direction.

[0090] exist Figure 9 In the cross-sectional view, the curved component 81 and the Peltier element 82 are located below the surface of the housing 51 and are embedded within the housing 51. In this state, the cam 85 rotates about axis 85A, thus pushing the curved component 81 and the Peltier element 82 upward in the z-direction, and the curved component 81 can be exposed from the opening 51A. Figure 9 In this configuration, the curved component 81 is the part that contacts the user's body part (skin), and the exposed portion of the curved component 81 can be adjusted by adjusting the rotation amount of the cam 85. That is, in the drive unit 43, by controlling the drive of the motor to adjust the rotation amount of the cam 85, the contact area between the user's body part (skin) and the surface of the curved component 81 fixed to the upper surface of the Peltier element 82 can be changed.

[0091] Figure 10 and Figure 11 These figures illustrate how the Peltier element 82 is lifted by the rotation of the cam 85, which pushes the heat sink 83 upward according to the rotation of the shaft 85A, and how the contact area between the surface of the curved member 81 and the user's body changes. Figure 10 A and Figure 10 In B, the user's skin S is in contact with the surface of the casing 51. Figure 10 Figure A shows the state before cam 85 rotates, and Figure 10 Figure B shows the state after cam 85 has rotated. Similarly, Figure 11 Figure A shows the state before cam 85 rotates, and Figure 11 B shows the state after cam 85 has rotated.

[0092] here, Figure 10 B and Figure 11The comparison between the states of cam 85 after rotation in B is as follows. That is, focusing on the horizontal axis H in the x-direction indicated by the dashed line in the figure, cam 85 in... Figure 10 Rotate approximately 45° in B, while Figure 11 Cam 85 in B rotates approximately 90°, and Figure 11 The rotation of cam 85 in B is greater than Figure 10 The rotation amount of cam 85 in B. In this case, the magnitude relationship of the rotation amounts is R2>R1, where... Figure 10 In B, the amount of rotation of cam 85 is indicated by arrow R1, and... Figure 11 In B, the lifting amount of the curved component 81 and the Peltier element 82 is indicated by arrow U1, and the rotation amount of the cam 85 is indicated by arrow R2. The lifting amount of the curved component 81 and the Peltier element 82 is indicated by arrow U2. Therefore, the relationship between the lifting amounts is U2>U1.

[0093] That is, when Figure 11 The curved surface component 81 and Peltier element 82 in B are... Figure 10 When comparing the curved surface component 81 and the Peltier element 82 in B, Figure 11 The increase in B is large, therefore Figure 11 The contact area between the curved component 81 in B and the skin S is greater than Figure 10 Surface component 81 in B. Conversely. Figure 10 The contact area between the curved component 81 in B and the skin S is less than Figure 11 The curved component 81 in B. In this example, the curved component 81 and the Peltier element 82 are shown raised, but the contact area can be adjusted by reversing the rotation direction of the axis 85A and pushing the heat sink 83 downward.

[0094] As described above, in the haptic feedback device 15, even with the haptic presentation unit 44 consisting of Peltier elements 82 installed, the contact area can be adjusted by rotating the cam 85 and raising or lowering the curved member 81 and the Peltier elements 82 by driving the motor in the drive unit 43. Note that in Figure 10 A and Figure 11 In A, since the skin S does not contact the upper surface of the curved component 81, temperature control via cooling or heating through the Peltier element 82 can be stopped. Figure 10 B and Figure 11 In B, since the skin S is in contact with the upper surface of the curved member 81, temperature control via cooling or heating through the Peltier element 82 can be performed. For example, the state of cooling or heating through the Peltier element 82 can be controlled based on the elevation of the curved member 81 and the Peltier element 82.

[0095] exist Figure 9 The diagram shows a drive unit 43 consisting of a cam, motor, etc., but it can be configured with other mechanisms for moving (raising or lowering) the curved component 81 and the Peltier element 82. For example, an airbag, balloon, etc., can be used as the mechanism constituting the drive unit 43.

[0096] Figure 12 This diagram illustrates an example configuration of the curved member 81 and the Peltier element 82, which are raised or lowered by the expansion and contraction of an airbag to adjust the contact area between the surface of the curved member 81 and a user's body part. Figure 12 In, with Figure 9 In contrast, instead of a cam 85 and a motor, an airbag 91 and a micropump are provided. The airbag 91 is attached to the lower surface side of the radiator 83 and expands or contracts by supplying or venting air from the micropump.

[0097] Figure 12 Figure A shows the state of airbag 91 before it inflates, and Figure 12 Figure B shows the state of airbag 91 after inflation. Figure 12 In case A, since the airbag 91 does not inflate, the curved member 81 does not contact the skin S on the surface of the housing 51. On the other hand, in Figure 12 In B, the radiator 83 is pushed upwards along the z-direction by the expansion of the airbag 91, and therefore, the curved member 81 and the Peltier element 82 move (rise) along the z-direction. Figure 12 In the diagram, the amount of upward movement is represented by arrow U3. As described above, the contact area can be increased by inflating the airbag 91 to raise the curved member 81 and the Peltier element 82. The contact area can be decreased by contracting the airbag 91 to lower the curved member 81 and the Peltier element 82.

[0098] Temperature sensing changes are adjusted by contact area.

[0099] In the haptic feedback device 15, the user's temperature perception can be altered by adjusting the contact area while maintaining a constant temperature presented by the haptic presentation unit 44. For example, as Figure 13 As shown, when the user's fingertip skin F comes into contact with the surface of the tactile presentation unit 44, even when the surface temperature is constant, the user's temperature perception will change due to different contact areas. Figure 13 In the diagram, the user's fingertip skin F is marked with a thick dot pattern, and the contact area A S A M and A L The skin and contact areas are marked with a fine dot pattern, and portions of the patterns overlap each other. This representation of skin and contact area is identical in the other figures described later.

[0100] More specifically, Figure 13 A shows a temperature of 15°C and a contact area of ​​A presented by the tactile presentation unit 44. S For a case of 50mm². Figure 13 B shows a temperature of 15°C and a contact area of ​​A presented by the tactile presentation unit 44. M 100mm 2 The situation. Figure 13 C shows that the temperature is 15°C and the contact area is A, as presented by the tactile presentation unit 44. L 200mm 2 The situation is as follows: The temperature is constant at 15℃, and the contact area has an A... S M L The relationship. In this case, the user's temperature perception is as follows. That is, in Figure 13 In state A, the user felt slightly cold. Figure 13 In state B, the user feels cold. Figure 13 In C, the user felt very cold.

[0101] As described above, in the haptic feedback device 15, the user's temperature perception can be altered by adjusting the contact area through the operation of the drive unit 43 while maintaining a constant temperature presented by the haptic presentation unit 44. In this case, since the temperature does not need to be changed, power consumption can be reduced in the haptic feedback device 15. Note that research on this type of temperature perception has been extensive, and, for example, the relationship between fingertip pressure and temperature sensing characteristics is described in reference A below.

[0102] Reference A: Relationship between fingertip pressure and temperature sensation characteristics, Nishimura Takahiro et al., International Journal of Affective Engineering, Vol. 13, No. 3, pp. 433-439 (2014).

[0103] Contact area adjustment at multiple presentation locations

[0104] In the haptic feedback device 15, multiple haptic presentation units 44 can be set up, and when there are multiple haptic presentation positions, the intensity of hot and cold stimulation can be adjusted by adjusting the contact area of ​​each haptic presentation position. Figure 14 This is a diagram illustrating an example of adjusting the contact area when multiple tactile presentation locations are present.

[0105] For example, such as Figure 14 ​​As shown in Figure A, assuming the haptic feedback device 15 is worn near the neck of user U, and haptic presentation units 44-1 and 44-2 are arranged at a predetermined interval in the back of the neck (neck area). In this case, the contact area between the surface of haptic presentation unit 44-1 and the skin is 100 mm². 2 Furthermore, the surface of the tactile presentation unit 44-2 has a contact area of ​​200 mm² with the skin. 2 For example, by controlling the operation of the drive unit 43 through the control unit 20, the contact area can be adjusted as follows.

[0106] That is, firstly, the average contact area at the two locations is obtained, and the contact areas of tactile presentation units 44-1 and 44-2 can be adjusted to make them average areas. More specifically, the contact areas of tactile presentation units 44-1 and 44-2 are adjusted to (100mm) 2 +200mm 2 ) / 2=150mm 2 Secondly, the contact area of ​​the haptic presentation unit 44-1 and the haptic presentation unit 44-2 can be adjusted to any value set by the user (e.g., 120mm). 2 Third, the contact area of ​​one of the contact areas of tactile presentation units 44-1 and 44-2 can be adjusted to match the contact area of ​​the other. More specifically, when tactile presentation unit 44-2, which is arranged at the back of the user U's neck, can present a better temperature sensation, in order to make the contact area of ​​tactile presentation unit 44-1 match the contact area of ​​tactile presentation unit 44-2 (200mm²), 2 Matching, the contact area of ​​the haptic presentation unit 44-1 ranges from 100mm². 2 Become 200mm 2 .

[0107] like Figure 14 As shown in Figure B, when the haptic feedback device 15 is a controller 15A, it is assumed that the haptic presentation unit 44-1 is arranged in the portion grasped by the user's left hand, and the haptic presentation unit 44-2 is arranged in the portion grasped by the user's right hand. Similarly, in this case, the intensity of the warm and cold stimuli can be made uniform by adjusting the contact area A1-1 between the haptic presentation unit 44-1 and the left hand, and the contact area A1-2 between the haptic presentation unit 44-2 and the right hand. Figure 14 The method for adjusting the contact area in B can be used with... Figure 14 The adjustment method in A is the same as the adjustment method, such as adjusting using the average value or the set value.

[0108] like Figure 14As shown in Figure C, in the case where the haptic feedback device 15 is a device 15B with a cylindrical shape, it is assumed that when the haptic feedback device 15 is grasped by the user, the haptic presentation unit 44-1 is arranged in the fingertip portion and the haptic presentation unit 44-2 is arranged in the palm portion. Similarly, in this case, the intensity of the warm and cold stimulation can be made uniform by adjusting the contact area A2-1 between the haptic presentation unit 44-1 and the fingertip, and the contact area A2-2 between the haptic presentation unit 44-2 and the right hand. Figure 14 The method for adjusting the contact area in C can be used with... Figure 14 The adjustment method in A is the same as the adjustment method, such as adjusting using the average value or the set value.

[0109] As described above, in the haptic feedback device 15, when multiple haptic presentation locations exist, the intensity of warm and cold stimuli can be made uniform by changing the contact area of ​​each presentation location. Note that in Figure 14 The example shows the case where there are two tactile presentation locations, but the same applies to cases where there are three or more tactile presentation locations. For example, when there are three tactile presentation locations, the average contact area of ​​the three tactile presentation locations can be obtained, and the contact area of ​​each tactile presentation location can be adjusted to the average value.

[0110] Adjust the contact area according to the adaptation effect.

[0111] When the same warm or cold stimulus (temperature stimulus) is repeatedly applied to a user, an adaptation effect may occur, in which sensitivity to the stimulus decreases. For example, when a warm stimulus is presented continuously for a long period, the user is less likely to feel warm. When a cold stimulus is presented continuously for a long period, the user is less likely to feel cold. Extensive research has been conducted on this type of adaptation effect, and, for example, the relationship between the temperature limit of adaptation and time is described in reference B below.

[0112] Reference B: The time process of thermal adaptation, Dan R. Kenshalo, Harley A. Scott and Jr., Science, New Series, Vol. 151, No. 3714 (March 4, 1966), paragraphs 1095-1096, American Association for the Advancement of Science.

[0113] In the haptic feedback device 15, when the same hot and cold stimuli are continuously presented to the user through the haptic presentation unit 44, the operation of the drive unit 43 can be controlled by the control unit 20 to increase the contact area according to the passage of time. Figure 15This is a diagram illustrating an example of how the contact area changes over time due to adaptation. Figure 15 In the graph, when the horizontal axis represents time and the vertical axis represents the contact area, the curve of the function represented by the straight line L1 shows that the contact area increases over time.

[0114] For example, when a warm stimulus is presented continuously for a long time, a user is unlikely to feel warmth. However, the difficulty of feeling warmth can be reduced and the likelihood of feeling warmth can be suppressed by increasing the contact area over time. Similarly, when a cold stimulus is presented continuously, the likelihood of feeling cold can be suppressed by increasing the contact area over time.

[0115] Note that since the assumption of adaptation through multiple types of stimuli is also made, the passage of time is not limited to the time associated with a single stimulus, and the cumulative time of multiple types of stimuli can be used. Figure 15 The diagram shows a function represented by the line L1, but this function is not limited to linear functions and can use nonlinear functions such as quadratic functions.

[0116] Adjust according to the contact area of ​​the user's body parts

[0117] Because sensory sensitivity depends on the user's body part, the sensitivity to hot and cold stimuli can vary from body part to body part. For example, when comparing the sensitivity of the feet (soles) to the palms, the palms are more likely to feel cold than the feet. Extensive research has been conducted on this type of sensory sensitivity, and, for example, the sensory sensitivity of each body part to warmth or cold is described in reference C below.

[0118] Reference C: Temperature sensitivity of body surface throughout lifespan, Joseph C. Stevens and Kennett HK.CHO, John B. Pierce Laboratory and Yale University, New Haven, CT. 06519, USA

[0119] In the haptic feedback device 15, when hot and cold stimuli are presented to different parts of the user's body through the haptic presentation unit 44, and it is desired that the same temperature sensation be perceived for each body part, the operation of the drive unit 43 can be controlled by the control unit 20 to adjust the temperature sensation by the contact area. Figure 16 This is a diagram illustrating an example of how the contact area changes based on the body part of the user being presented with hot or cold stimuli. Figure 16 The image shows an example of a situation where the skin F of the palm shown on the left and the skin T of the foot shown on the right perceive the same temperature.

[0120] like Figure 16 As shown in Figure A, the area of ​​contact between the skin F of the palm and the surface of the tactile presentation unit 44 is the contact area A. S Furthermore, the contact area between the skin T of the foot and the surface of the tactile presentation unit 44 is the contact area A. M That is, the temperature is constant, and the contact area has A. S M The relationship. In this case, the user's perceived sensitivity is, for example, Figure 16 As shown in B. That is, since the palms are more susceptible to cold than the feet, the user can sense the cold through the contact area A of the palms. S Smaller than the contact area A of the foot M Use your palms and feet to perceive the same temperature sensation. Figure 16 Region P in B is represented by a fine dot pattern. Figure 16 The image shows the palms and feet (soles), but the contact area can be adjusted for other body parts in the same way.

[0121] Sensing methods

[0122] In the information processing system 1, the control unit 20 estimates the contact area between the surface of the tactile presentation unit 44 and the user's body part based on the sensing data from the sensing unit 41 and generates control data (drive data), so that the drive unit 43 can operate and adjust the contact area between the surface of the tactile presentation unit 44 and the user's body part based on the control data (drive data).

[0123] That is, in information processing system 1, the perceived temperature of the presentation location is changed by sensing the contact area and changing the contact area, rather than sensing the temperature and changing the intensity of the warm or cold stimulus at the presentation location. For example, it is described in document D below that even when the presentation temperature is the same, the perceived temperature of the presentation location can be changed by changing the contact area.

[0124] Reference D: Warm or Cool, Big or Small? The Challenges of Thermal Display, Lynette A. Jones, IEEE Fellow, and Hsin-Ni Ho, IEEE TRANSACTIONS ON HAPTICS, Vol. 1, No. 1, January-June 2008

[0125] Figure 17 This is a diagram illustrating an example configuration of a tactile feedback device 15, including a sensing unit 41 composed of physical sensors. Figure 17 In the middle, the x, y, and z directions are... Figure 3 Those are the same as those in [the text]. Figure 17 In, with Figure 3 ​In contrast, the surface pressure sensor 111 is attached to the upper surface of the flexible Peltier element 52 and stacked thereon. Figure 17 In, with Figure 3 The same parts in the figures are given the same reference numerals, and their descriptions will be omitted as appropriate.

[0126] The surface pressure sensor 111 is a sensor that measures the pressure distribution on the upper surface of the flexible Peltier element 52. For example, the surface pressure sensor 111 is constructed of a pressure-measuring sheet, wherein the local resistance value of the membrane varies according to the applied pressure, and is attached to the upper surface of the flexible Peltier element 52. The surface pressure sensor 111 is an example of a physical sensor constituting the sensing unit 41. The tactile feedback device 15 transmits sensing data, including measurements indicating the pressure distribution measured by the surface pressure sensor 111, to the information processing device 11. In the information processing device 11, the control unit 20 can use the measurements included in the sensing data to estimate the contact area.

[0127] Figure 18 It is shown Figure 17 A diagram illustrating an example of measurement by the surface pressure sensor 111. Figure 18 The image schematically illustrates the state in which the skin S of a user's body part (e.g., hand or finger) is in contact with the upper surface of the surface pressure sensor 111. Figure 18 A and Figure 18 In B, the pressure distribution of each skin S pressed against the upper surface of the surface pressure sensor 111 is different. The surface pressure sensor 111 outputs a measured value indicating the measured pressure distribution. Specifically, it indicates... Figure 18 The measured pressure distribution in B is greater than the indicated value. Figure 18 The measured values ​​of the pressure distribution in A.

[0128] The control unit 20 estimates the contact area based on sensing data, including measurements taken by the surface pressure sensor 111. In this example, based on indications from... Figure 18 The pressure distribution of the surface pressure sensor 111 in A is measured to estimate Figure 18 The contact area A3 in A, and based on the indication from Figure 18 The pressure distribution of the surface pressure sensor 111 in B is measured to estimate Figure 18 The contact area A4 in B. Figure 18 The value of the contact area A4 in B is greater than Figure 18 The value of the contact area A3 in A.

[0129] exist Figure 17The image shows a configuration in which a surface pressure sensor 111, which has a sheet shape, is attached to the upper surface of a flexible Peltier element 52, but multiple physical sensors can be provided. Figure 19 This is a diagram illustrating an example configuration of a haptic feedback device 15, which includes a sensing unit 41 composed of multiple physical sensors. Figure 19 In the middle, the x, y, and z directions are... Figure 17 The same as in. In Figure 19 In, with Figure 17 In contrast, pressure sensors 121-1 to 121-3 are provided instead of surface pressure sensor 111. Figure 19 In, with Figure 17 Identical parts are given the same reference numerals, and their descriptions will be omitted appropriately.

[0130] Pressure sensor 121-1 is a physical sensor attached to the upper surface of the flexible Peltier element 52 and measures the pressure applied to the upper surface. Pressure sensors 121-2 and 121-3 are constructed in the same manner as pressure sensor 121-1, but they are located at different positions. Figure 19 For ease of description, a configuration with three pressure sensors 121 shown in a cross-sectional view is illustrated; however, more pressure sensors 121 can be used, and for example, the pressure sensors can be arranged in a predetermined pattern in a plan view. By using more pressure sensors 121, the contact area can be estimated more accurately.

[0131] Figure 20 It is shown Figure 19 A diagram illustrating an example of measurements taken by multiple pressure sensors 121. Figure 20 The image schematically illustrates the state in which the skin S of a user's body part (such as a hand or fingers) is in contact with the upper surface of the pressure sensor 121. Figure 20 A and Figure 20 In B, the pressure distribution of skin S pressed against the upper surfaces of multiple pressure sensors 121 is different from each other. The output includes sensing data indicating the measured values ​​of the pressure distribution measured by the multiple pressure sensors 121. Specifically, indicating Figure 20 The measured pressure distribution in B is greater than the indicated value. Figure 20 The measured values ​​of the pressure distribution in A.

[0132] In control unit 20, based on the indication from Figure 20 The pressure distribution of multiple pressure sensors 121 in A is measured to estimate Figure 20 The contact area A5 in A, and based on the indication from Figure 20 The pressure distribution of multiple pressure sensors 121 in B is measured to estimate Figure 20 The contact area A6 in B. Figure 20 The value of the contact area A6 in B is greater than Figure 20 The value of the contact area A5 in A.

[0133] exist Figure 17 and Figure 19 The text shows how to do this by sending... Figure 3 The configuration shown is an example of adding physical sensors, such as surface pressure sensor 111 and multiple pressure sensors 121, to sense the contact area, but in Figure 9 In the configuration shown, information for estimating the contact area can be sensed using physical quantities obtained from the mechanism constituting the drive unit 43 (such as a motor), physical quantities obtained from the Peltier element constituting the tactile presentation unit 44, etc., without providing physical sensors.

[0134] Figure 21 This is a diagram illustrating an example of measurement using either the drive unit 43 or the haptic presentation unit 44. Figure 21 In the middle, the x, y, and z directions are... Figure 9 The same as in. In Figure 21 In, with Figure 9 Identical parts are given the same reference numerals, and their descriptions will be omitted appropriately. Figure 21 In this process, the drive motor rotates shaft 85A, the radiator 83 is pushed upward by cam 85, and the curved member 81 and Peltier element 82 move (lift) in the z-direction (the amount of lifting is indicated by arrow U4). In this situation, because the encoder value output by the encoder included in the motor constituting the drive unit 43 changes, sensing data including the encoder value is transmitted. For example, the encoder value includes values ​​such as the motor's rotation angle, rotational speed, motor speed, and motor load. Therefore, the control unit 20 can estimate the contact area based on the encoder value included in the sensing data.

[0135] Figure 22 Figure A is a diagram illustrating an example of estimating the contact area using encoder values. Figure 22 In diagram A, when the horizontal axis represents the encoder value and the vertical axis represents the contact area, the graph of the function represented by the straight line L2 shows that the contact area increases as the encoder value increases. For example, in the haptic feedback device 15, as the shaft 85A continues to rotate via the drive motor, the encoder value increases, the heat sink 83 is pushed upward by the cam 85, the curved member 81 and the Peltier element 82 rise, and the contact area increases. Therefore, in the information processing device 11, the control unit 20 can estimate the contact area based on the encoder value from the haptic feedback device 15.

[0136] Alternatively, when the Peltier element 82 is raised by the operation of the drive unit 43 and the surface of the curved member 81 comes into contact with the user's body part, the resistance value of the Peltier element 82 changes, and thus sensing data including the resistance value of the Peltier element 82 can be transmitted. Therefore, the control unit 20 can estimate the contact area based on the resistance value included in the sensing data.

[0137] Figure 22 Figure B is an example illustrating the estimation of contact area using the resistance value of the Peltier element 82. Figure 22 In diagram B, when the horizontal axis represents the resistance value of the Peltier element 82 and the vertical axis represents the contact area, the graph of the function represented by the straight line L3 shows that the contact area increases as the resistance value increases. For example, in the haptic feedback device 15, as the shaft 85A continues to rotate via the drive motor, the curved member 81 and the Peltier element 82 rise, the contact area increases, and the resistance value of the Peltier element 82 increases. Therefore, in the information processing device 11, the control unit 20 can estimate the contact area based on the resistance value from the haptic feedback device 15.

[0138] The first example of handling hot and cold stimuli

[0139] Figure 23 It is a description of Figure 1 A flowchart of a first example of warm / cold stimulus presentation processing performed by the information processing system 1. As a premise of warm / cold stimulus presentation processing, assume a user is wearing a haptic feedback device 15, such as a haptic glove, and watching a video of content (such as VR content) presented on a visual presentation device 12, such as an HMD. In the information processing device 11, such as a PC, a preset value X for the contact area is established, and this information is recorded in memory or the like. The preset value X for the contact area is the contact area intended by the content creator during creation, and is, for example, a value set for the area expected to be touched by the user to apply warm / cold stimuli.

[0140] In step S11, the control unit 20 determines whether the user has interacted with a specific object in the virtual space based on the content presented on the visual presentation device 12. If it is determined in step S11 that the user has interacted with the specific object, the processing in step S12 and subsequent steps is executed.

[0141] In step S12, the tactile presentation unit 44 presents warm or cold stimuli to the user under the control of the control unit 20. Here, warm or cold stimuli can be presented to the user who has already come into contact with a specific object, depending on the attributes of that object. In step S13, the control unit 20 estimates the contact area between the user's body part (e.g., the skin of the fingers) and the surface of the tactile presentation unit 44 based on sensing data from the sensing unit 41.

[0142] In step S14, the control unit 20 determines whether the estimated contact area is less than a set value (X). If, in step S14, the estimated contact area is determined to be less than the set value (X), the process proceeds to step S15. In step S15, the control unit 20 controls the operation of the drive unit 43 to increase the contact area to the set value (X).

[0143] For example, such as Figure 3 and Figure 4 As shown in Figure B, when the drive unit 43 is composed of a motor or the like and the tactile presentation unit 44 is composed of a flexible Peltier element 52, the contact area can be increased by deforming the shape of the flexible Peltier element 52 into a concave, curved shape through the drive motor. Figure 9 and Figure 11 As shown, when the drive unit 43 is composed of a motor, a cam 85, etc., and the tactile presentation unit 44 is composed of a curved component 81 and a Peltier element 82, the contact area can be increased by driving the motor to lift the curved component 81 and the Peltier element 82.

[0144] In step S14, when it is determined that the estimated contact area is not less than the set value (X), the process proceeds to step S16. In step S16, the control unit 20 determines whether the estimated contact area is greater than the set value (X). In step S16, when it is determined that the estimated contact area is greater than the set value (X), the process proceeds to step S17. In step S17, the control unit 20 controls the operation of the drive unit 43 to reduce the contact area to the set value (X).

[0145] For example, such as Figure 3 and Figure 4 As shown in Figure A, when the drive unit 43 is composed of a motor or the like and the tactile presentation unit 44 is composed of a flexible Peltier element 52, the contact area can be reduced by driving the motor to deform the shape of the flexible Peltier element 52 into a convex curved shape. Figure 9 and Figure 10 As shown, when the drive unit 43 is composed of a motor, a cam 85, etc., and the tactile presentation unit 44 is composed of a curved component 81 and a Peltier element 82, the contact area can be reduced by driving the motor to lower the curved component 81 and the Peltier element 82.

[0146] When the processing in step S15 or step S17 is completed, the process returns to step S13 and repeats the above processing. In step S16, when it is determined that the estimated contact area is not greater than the set value (X), the process proceeds to step S18. In this case, the contact area matches the set value (X), and the warm and cold stimulation desired by the creator can be presented to the user's body parts.

[0147] In step S18, the control unit 20 determines whether the user has moved away from the specific object in the virtual space. If it is determined in step S18 that the user has made contact with the specific object, the process proceeds to step S19. In step S19, the tactile presentation unit 44, under the control of the control unit 20, stops presenting warm and cold stimuli. When the process in step S19 is completed, a series of processes are finished.

[0148] Figure 24 This is a diagram illustrating an example of how different warm and cold stimuli are presented for each user. Figure 24 An example is shown where different users (User A and User B) touch the surface of the tactile presentation unit 44 with their fingers. In this example, User B's finger is thicker than User A's finger, and the thicknesses of User A's and User B's fingers are different from each other, but users with different finger thicknesses perceive the same temperature sensation. Figure 24 In the middle, the skin F of user A's finger A The state of contact between the tactile presentation unit 44 and the surface of user B's finger F is shown on the left, and the skin of user B's finger F B The state of contact with the surface of the tactile presentation unit 44 is shown on the right.

[0149] exist Figure 24 In A, the skin F of user A's finger A The area in contact with the surface of the tactile presentation unit 44 is the contact area A. S And the skin of user B's finger F B The area in contact with the surface of the tactile presentation unit 44 is the contact area A. L That is, the contact area has A S L The relationship. In this case, the contact area is adjusted by operating the drive unit 43, so that the contact area changes as follows. Figure 24 The state shown in B.

[0150] exist Figure 24 In B, user A's finger skin F A The contact area from the contact area A S Adjust to contact area A M And the skin of user B's finger F B The contact area from the contact area A L Adjust to contact area A M Skin F A Contact area and skin F B The contact area of ​​all becomes contact area A. M For example, when the preset contact area setting (X) is 100mm... 2 And skin F A ​Contact area A S Estimated to be 50mm 2 And skin F B Contact area A L Estimated to be 200mm 2 At that time, skin F A and skin F B The contact area between them was adjusted to 100mm. 2 Contact area A M .

[0151] In this case, the perceptual sensitivity of user A and user B is, for example, as follows: Figure 24 As shown in C. That is, the contact area of ​​user A with thin fingers is adjusted to increase, while the contact area of ​​user B with thick fingers is adjusted to decrease. Therefore, despite the difference in finger thickness, users A and B can perceive the same temperature sensation (as indicated by...). Figure 24 The area P is represented by the fine dot pattern in C. Therefore, users with different finger thicknesses can experience the same temperature sensation.

[0152] As described above, in the information processing system 1, when the tactile presentation unit 44 presents warm and cold stimuli based on the content presented on the visual presentation device 12 and the user's actions in the virtual space, the contact area of ​​the surface of the tactile presentation unit 44 is sensed, and control is executed based on the sensing data to make the contact area of ​​the presentation position of the warm and cold stimuli variable. Therefore, a similar temperature perception experience can be provided even for different users. In the information processing system 1, when the tactile presentation unit 44 presents warm and cold stimuli to the user's body parts in contact with the surface of the tactile presentation unit 44, control for adjusting the contact area is executed in real time (immediately). Therefore, the user experience desired by the creator can be provided by matching the tactile presentation with the visual presentation.

[0153] The second example of handling hot and cold stimuli

[0154] Figure 25 It is a description of Figure 1 A flowchart of a second example of the warm / cold stimulus presentation processing performed by the information processing system 1. As a premise of the warm / cold stimulus presentation processing, it is assumed that a user wears a haptic feedback device 15 and watches a video of content presented on a visual presentation device 12. In the information processing device 11, preset values ​​X and Y for the contact area are established.

[0155] In step S31, the control unit 20 determines whether the user has entered the area with temperature A in the virtual space based on the content presented on the visual presentation device 12. In step S31, when the control unit 20 determines that the user has entered the area with temperature A, step S32 and subsequent steps are executed.

[0156] In step S32, the tactile presentation unit 44, under the control of the control unit 20, presents the user with warm or cold stimuli based on temperature A. In step S33, the control unit 20 estimates the contact area between the user's body part and the surface of the tactile presentation unit 44 based on sensing data from the sensing unit 41.

[0157] In step S34, the control unit 20 determines whether the estimated contact area is less than or greater than a set value (X). If, in step S34, the estimated contact area is determined to be less than or greater than the set value (X), the process proceeds to step S35. In step S35, the control unit 20 controls the operation of the drive unit 43 so that the contact area becomes the set value (X).

[0158] In other words, when the control unit 20 determines that the estimated contact area is less than the set value (X), the control unit 20 controls the operation of the drive unit 43 to increase the contact area to the set value (X). When the control unit 20 determines that the estimated contact area is greater than the set value (X), the control unit 20 controls the operation of the drive unit 43 to decrease the contact area to the set value (X).

[0159] For example, such as Figure 3 and Figure 4 As shown, when the drive unit 43 is composed of a motor or the like and the tactile presentation unit 44 is composed of a flexible Peltier element 52, the contact area can be increased or decreased by deforming the shape of the flexible Peltier element 52 into a concave or convex curved shape by driving the motor. Figures 9 to 11 As shown, when the drive unit 43 is composed of a motor, a cam 85, etc., and the tactile presentation unit 44 is composed of a curved component 81 and a Peltier element 82, the contact area can be increased or decreased by raising or lowering the curved component 81 and the Peltier element 82 by driving the motor.

[0160] When the processing in step S35 is completed, the process returns to step S33 and repeats the above processing. In step S34, when it is determined that the estimated contact area is neither greater than nor less than the set value (X), that is, when it is determined that the estimated contact area matches the set value (X), the process proceeds to step S36.

[0161] In step S36, the control unit 20 determines whether the user has come into contact with an object having temperature B in the virtual space. If it is determined in step S36 that the user has come into contact with an object having temperature B, the process proceeds to step S37. In step S37, the control unit 20 controls the operation of the drive unit 43 so that the contact area becomes a set value (Y).

[0162] That is, by presetting two set values X and Y, the cold and warm stimuli according to temperature A are presented by the contact area according to set value X, and then the cold and warm stimuli according to temperature B are presented by the contact area according to set value Y. In this example, temperature A is the temperature (ambient temperature) of a specific area in the virtual space, and temperature B is the temperature of a specific object (virtual object) in the virtual space. For example, when the user is playing a game such as VR content, when the user enters the area of the volcanic region in the game, the cold and warm stimuli according to temperature A (ambient temperature) can be presented by the contact area according to set value X. Thereafter, when an enemy character (e.g., a dragon) appears in the volcanic region in the game and when under a flame attack, the cold and warm stimuli according to temperature B (A < B) can be presented by the contact area according to set value Y (X < Y).

[0163] For example, as Figure 3 and Figure 4 shown, when the drive unit 43 is composed of a motor or the like and the tactile presentation unit 44 is composed of the flexible Peltier element 52, the contact area can be changed to a preset set value (Y) by driving the motor to deform the flexible Peltier element 52. As Figure 9 and Figure 11 shown, when the drive unit 43 is composed of a motor, a cam 85, etc. and the tactile presentation unit 44 is composed of the curved surface member 81 and the Peltier element 82, the contact area can be changed to a preset set value (Y) by driving the motor to lift or lower the curved surface member 81 and the Peltier element 82.

[0164] When the processing in step S37 is completed, the processing proceeds to step S38. In step S36, when it is determined that the user is not in contact with the object having temperature B, step S37 is skipped, and the processing proceeds to step S38. In step S38, the control unit 20 determines whether the user has moved away from the area having temperature A in the virtual space. In step S38, when it is determined that the user has moved away, the processing proceeds to step S39. In step S39, the tactile presentation unit 44 stops presenting the cold and warm stimuli under the control of the control unit 20. Then, a series of processing is completed.

[0165] Modify

[0166] Figure 2 The functional configuration of the devices constituting the information processing system 1 shown in Figure 26 is an example, and other configurations can be adopted. Figure 1 is a block diagram showing another example of the functional configuration of the information processing system 1 in Figure 26 In Figure 26 In, and Figure 2 Identical parts are given the same reference numerals, and their descriptions will be omitted appropriately.

[0167] exist Figure 26 In the above, the information processing device 11 includes an information processing unit 21, a communication unit 22, a sensing information processing unit 23, and a communication unit 26. The haptic feedback device 15 includes a sensing unit 41, a communication unit 42, a driving unit 43, a haptic presentation unit 44, a presentation data generation unit 45, and a haptic data processing unit 46. Figure 26 Information processing equipment 11 and Figure 2 The difference in the information processing device 11 is that the presentation data generation unit 24 and the tactile data processing unit 25 have been removed. Figure 26 The haptic feedback device 15 in the middle and Figure 2 The difference in the haptic feedback device 15 is that it adds a presentation data generation unit 45 and a haptic data processing unit 46.

[0168] exist Figure 26 In the information processing device 11, the information processing unit 21 and the sensing information processing unit 23 perform processing as some functions of the control unit 20, which is composed of a processor or the like. The sensing information processing unit 23 estimates the contact area based on sensing data received by the communication unit 26. The communication unit 26 sends control data, including various information from the information processing unit 21 and information about the estimated contact area from the sensing information processing unit 23, to the haptic feedback device 15.

[0169] exist Figure 26 In the haptic feedback device 15, the presentation data generation unit 45 and the haptic data processing unit 46 perform processing as functions of the control unit 40, which is composed of a processor, etc. Figure 2 The presentation data generation unit 45 performs the same processing as the presentation data generation unit 24. That is, the presentation data generation unit 45 generates presentation data and driving data based on information included in the control data received by the communication unit 42, and outputs the presentation data and driving data to the haptic data processing unit 46. The haptic data processing unit 46 performs the same processing as the presentation data generation unit 24. Figure 2 The haptic data processing unit 46 performs the same processing as the haptic data processing unit 25. That is, the haptic data processing unit 46 processes the presentation data according to the characteristics of the components (electronic components) constituting the haptic presentation unit 44. The haptic data processing unit 46 outputs driving data to the driving unit 43 and presentation data to the haptic presentation unit 44.

[0170] As described above, at least a portion of the processing performed by the control unit 20 of the information processing device 11 can be performed by the control unit 40 of the haptic feedback device 15. By performing the processing on the haptic feedback device 15 side, the delay after generating presentation data and drive data can be reduced compared to the case where the processing is performed on the information processing device 11 side. As described above, in the information processing system 1, at least one of a visual presentation device 12 and an auditory presentation device 17 can be provided, and if the auditory presentation device 17 is provided but the visual presentation device 12 is not provided, the control unit 20 or the control unit 40 can perform, for example, the following control. That is, the control unit 20, etc., performs control such that the haptic presentation unit 44 presents warm and cold stimuli based on the sound (speech, music, etc.) of the auditory content presented to the user by the auditory presentation device 17, and acquires sensing data including information about the user's body parts in contact with the surface of the haptic presentation unit 44. Therefore, the control unit 20, etc., can control the operation of the drive unit 43 based on the acquired sensing data to adjust the contact area of ​​the user's body parts.

[0171] Note that the configuration is not limited to Figure 26 The configuration shown is provided, but another configuration may be adopted. For example, the sensing information processing unit 23 of the control unit 20 may be disposed in the control unit 40. When the control unit 40 includes the sensing information processing unit 23, sensing data from the sensing unit 41 may be input to the sensing information processing unit 23 of the control unit 40 without being sent to the information processing device 11. When the device is configured such that the information processing device 11 and the haptic feedback device 15 are included in a housing, or when the information processing device 11, the visual presentation device 12, the haptic feedback device 15 and the auditory presentation device 17 are included in a housing, the control unit 20 and the control unit 40 may perform processing in cooperation with each other in one device, or a control unit may include the information processing unit 21, the sensing information processing unit 23, the presentation data generation unit 45 and the haptic data processing unit 46.

[0172] In the above description, Figure 3 The arrangement of the tactile presentation unit 44, which is composed of flexible Peltier elements 52, etc., shown is an example, and the arrangement of the tactile feedback device 15 relative to the surface of the housing 51 can be another arrangement. Although Figure 3 The housing 51 shown has a shape including a recessed portion along the z-direction, but for example, when the surface of the housing 51 has a shape including a protrusion along the z-direction, the arrangement position of the flexible Peltier element 52 can be as follows. That is, as Figure 27As shown, an opening is formed in the portion protruding along the z-direction on the surface of the housing 51, and a flexible Peltier element 52 having a shape according to the shape of the opening can be arranged there. As described above, the portion of the surface of the housing 51 other than the stimulation presentation position does not need to have a shape according to the user's body part (fingers, etc.), and can have a shape different from the user's body part.

[0173] As described above, in the information processing device 11 employing this disclosure, the control unit 20 acquires sensing data including information about a user's body part in contact with the surface of the tactile presentation unit 44, which presents warm or cold stimuli based on at least one content (video or sound) presented to the user's vision or hearing, and controls the operation of the control drive unit 43 based on the acquired sensing data. The drive unit 43 adjusts the contact area, which is the area of ​​contact between the surface of the tactile presentation unit 44 and the user's body part. This control can provide a consistent user experience. Therefore, the warm or cold stimuli intended by the content creator can be presented to the user, and the value of the experience can be ensured.

[0174] Note that the presentation methods of related technologies, including those disclosed in PTL 1 above, do not allow the range of the presented warm or cold stimuli to be variable. Therefore, when presenting warm or cold stimuli to a user, the state of contact (contact area) with the stimulus presentation location changes depending on the user, and thus the temperature perceived by each user changes, potentially resulting in different experiences for each user. In contrast, in the information processing device 11 applying this disclosure, by changing the presentation range of the warm or cold stimuli based on data including information about the sensed contact area, a common user experience that is difficult to achieve with the presentation methods of related technologies can be provided.

[0175] In related art presentation methods, there are methods that control heat transfer by changing the output to a Peltier element. In this method, it is impossible to eliminate fundamental individual differences between users caused by varying contact areas, and the same problem may occur every time a different temperature is presented. In contrast, when this disclosure is applied, these individual differences caused by varying contact areas can be eliminated. In related art presentation methods, changing the output to a Peltier element may require time to increase or decrease the temperature. In contrast, when this disclosure is applied, even when the temperature of the presented surface does not change, the user's perceived temperature can be changed instantaneously and in real-time by changing the contact area. Furthermore, in related art presentation methods, changing the output to a Peltier element may increase power consumption and heat dissipation. In contrast, when this disclosure is applied, stable operation can be achieved by reducing the opportunities to change the output of the Peltier element, thus reducing power consumption and heat dissipation, which are particularly important considerations when using a Peltier element.

[0176] Computer Configuration

[0177] The above series of processing steps can be performed by hardware or software. When the series of processing steps are performed by software, the program that constitutes the software is installed on the computer. Figure 28 This is a block diagram illustrating an example configuration of the hardware of a computer used to perform the above series of processing steps in a program.

[0178] In the computer, the central processing unit (CPU) 1001, read-only memory (ROM) 1002, and random access memory (RAM) 1003 are interconnected via bus 1004. Input / output interface 1005 is also connected to bus 1004. Input / output interface 1005 is connected to input unit 1006, output unit 1007, storage unit 1008, communication unit 1009, and driver 1010.

[0179] Input unit 1006 consists of a keyboard, mouse, microphone, etc. Output unit 1007 includes a display, speakers, etc. Storage unit 1008 includes a hard disk, non-volatile memory, etc. Communication unit 1009 includes a network interface, etc. Driver 1010 drives removable recording media 1011 such as semiconductor memory, disk, optical disk, or magneto-optical disk.

[0180] In the computer configured as described above, the CPU 1001 loads the program recorded in the ROM 1002 or storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes the program to perform the series of processing steps described above.

[0181] The program executed by the computer (CPU 1001) can be recorded on, for example, a removable recording medium 1011 such as a packet medium and provided. Alternatively, the program can be provided via wired or wireless transmission media (such as a local area network, the Internet, or digital satellite broadcasting).

[0182] In a computer, a program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting a removable recording medium 1011 into the drive 1010. Alternatively, the program can be received and installed in the storage unit 1008 via a wired or wireless transmission medium through the communication unit 1009. Additionally, the program can be pre-installed in the ROM 1002 or the storage unit 1008.

[0183] Depending on the type of processing performed by the computer, the processing can be either parallel or individual (e.g., parallel processing or processing by objects). A program can be processed by a single computer (processor) or by multiple computers in a distributed manner.

[0184] The embodiments disclosed herein are not limited to those described above, and various modifications can be made without departing from the spirit of this disclosure. For example, this disclosure may employ a cloud computing configuration, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0185] Each step described in the flowchart above can be executed by a single device, or it can be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes included in that single step can be executed by a single device, or they can be shared and executed by multiple devices. Note that the effects described in this specification are merely illustrative and are not limiting; other effects can be obtained.

[0186] In addition, this disclosure may have the following configurations.

[0187] (1) An information processing device, comprising:

[0188] The control unit is configured to perform the following controls:

[0189] Acquire sensing data, including information about a user's body part in contact with a surface of a tactile presentation unit configured to present warm and cold stimuli, and

[0190] The operation of the control drive unit is based on the acquired sensing data to adjust the contact area between the surface of the haptic presentation unit and the user's body parts.

[0191] (2) The information processing device according to (1), wherein the control unit is configured as follows:

[0192] The contact area is estimated based on the acquired sensing data, and

[0193] The operation of the control drive unit causes the estimated contact area to become a preset value.

[0194] (3) The information processing device according to (2), wherein the control unit is configured as follows:

[0195] The estimated contact area is compared with the set value, and

[0196] The operation of the drive unit is controlled based on the comparison results so that the estimated contact area becomes the set value.

[0197] (4) The information processing apparatus according to any one of (1) to (3), wherein

[0198] The control unit is configured to control the operation of the drive unit to adjust the contact area in real time when the tactile presentation unit presents hot or cold stimuli to a part of the user's body that is in contact with the surface of the tactile presentation unit.

[0199] (5) The information processing apparatus according to any one of (1) to (3), wherein

[0200] The tactile presentation unit is composed of flexible Peltier elements.

[0201] The drive unit is capable of operating to deform the flexible Peltier element, and

[0202] The control unit is configured to control the operation of the drive unit to deform the flexible Peltier element and adjust the contact area.

[0203] (6) The information processing apparatus according to any one of (1) to (3), wherein

[0204] The tactile presentation unit is composed of Peltier elements.

[0205] The drive unit is operable to raise or lower the Peltier element in a direction from the lower surface of the Peltier element toward the upper surface of the Peltier element, and

[0206] The control unit is configured to control the operation of the drive unit to raise or lower the Peltier element and adjust the contact area.

[0207] (7) The information processing device according to (5), wherein

[0208] The sensing data includes measurements taken by sensors positioned on the surface of the flexible Peltier element, and

[0209] The control unit is configured to estimate the contact area based on the measured values.

[0210] (8) The information processing device according to (6), wherein

[0211] The sensed data includes physical quantities obtained from the mechanisms constituting the drive unit or from Peltier elements, and

[0212] The control unit is configured to estimate the contact area based on physical quantities.

[0213] (9) The information processing apparatus according to any one of (1) to (8), wherein

[0214] The tactile presentation unit is configured to present warm or cold stimuli based on at least one of the visual or auditory content presented to the user.

[0215] (10) The information processing device according to (9), wherein

[0216] The haptic presentation unit is configured to present warm and cold stimuli based on the user's actions in the virtual space in accordance with the content presented by the visual presentation device.

[0217] (11) The information processing apparatus according to any one of (1) to (10), wherein

[0218] The information processing device is configured as a haptic feedback device, which includes a haptic presentation unit, a driving unit, and a sensing unit, the sensing unit being configured to sense information about a body part of a user that is in contact with the surface of the haptic presentation unit.

[0219] (12) A control method, comprising:

[0220] Sensing data is acquired by an information processing device, the sensing data including information about a user's body part in contact with a surface of a tactile presentation unit configured to present warm and cold stimuli; and

[0221] The information processing device controls the operation of the drive unit based on the acquired sensing data to adjust the contact area between the surface of the tactile presentation unit and the user's body parts.

[0222] (13) A haptic feedback device, comprising:

[0223] The tactile presentation unit is configured to present warm and cold stimuli;

[0224] The sensing unit is configured to sense information about a part of the user's body that is in contact with the surface of the haptic presentation unit; and

[0225] The driving unit is configured to perform operations to adjust the contact area of ​​the surface of the haptic presentation unit and the user's body parts in contact with each other based on driving data based on sensing data acquired by the sensing unit.

[0226] List of reference numerals

[0227] 1 Information processing system, 11 Information processing device, 12 Visual presentation device, 13 Sensing device, 14 Tactile presentation device, 15 Tactile feedback device, 17 Auditory presentation device, 20 Control unit, 21 Information processing unit, 22 Communication unit, 23 Sensing information processing unit, 24 Presentation data generation unit, 25 Tactile data processing unit, 26 Communication unit, 31 Communication unit, 32 Display unit, 35 Communication unit, 36 Sound presentation unit, 40 Control unit, 41 Sensing unit, 42 Communication unit, 43 Drive unit, 44 Tactile presentation unit, 45 Presentation data generation unit, 46 Tactile data processing unit, 51 Housing, 52 Flexible Peltier element, 53 Heat sink, 54 Spring, 61 Peltier element, 71 Airbag, 81 Curved component, 82 Peltier element, 83 Heat sink, 84 Spring, 85 Cam, 85A Shaft, 91 airbag, 111 surface pressure sensor, 121 pressure sensor, 1001 CPU.

Claims

1. An information processing device, comprising: Control unit configured to perform the following controls: Acquire sensing data, including information about a user's body part in contact with a surface of a tactile presentation unit configured to present warm and cold stimuli, and The operation of the drive unit is controlled based on the acquired sensing data to adjust the contact area between the surface of the haptic presentation unit and the user's body parts.

2. The information processing device according to claim 1, wherein the control unit is configured as follows: The contact area is estimated based on the acquired sensing data, and The operation of the control drive unit causes the estimated contact area to become a preset value.

3. The information processing device according to claim 2, wherein the control unit is configured as follows: The estimated contact area is compared with the set value, and The operation of the drive unit is controlled based on the comparison results so that the estimated contact area becomes the set value.

4. The information processing device according to claim 1, wherein... The control unit is configured to control the operation of the drive unit to adjust the contact area in real time when the tactile presentation unit presents hot or cold stimuli to a part of the user's body that is in contact with the surface of the tactile presentation unit.

5. The information processing device according to claim 2, wherein... The tactile presentation unit is composed of flexible Peltier elements. The drive unit is capable of operating to deform the flexible Peltier element, and The control unit is configured to control the operation of the drive unit to deform the flexible Peltier element and adjust the contact area.

6. The information processing device according to claim 2, wherein... The tactile presentation unit is composed of Peltier elements. The drive unit is operable to raise or lower the Peltier element in a direction from the lower surface of the Peltier element toward the upper surface of the Peltier element, and The control unit is configured to control the operation of the drive unit to raise or lower the Peltier element and adjust the contact area.

7. The information processing device according to claim 5, wherein... The sensing data includes measurements taken by sensors positioned on the surface of the flexible Peltier element, and The control unit is configured to estimate the contact area based on the measured values.

8. The information processing device according to claim 6, wherein The sensed data includes physical quantities obtained from the mechanisms constituting the drive unit or from Peltier elements, and The control unit is configured to estimate the contact area based on physical quantities.

9. The information processing device according to claim 1, wherein... The tactile presentation unit is configured to present warm or cold stimuli based on at least one of the visual or auditory content presented to the user.

10. The information processing device according to claim 9, wherein The haptic presentation unit is configured to present warm and cold stimuli based on the user's actions in the virtual space in accordance with the content presented by the visual presentation device.

11. The information processing device according to claim 1, wherein The information processing device is configured as a haptic feedback device, which includes tactile presentation unit drive unit, and The sensing unit is configured to sense information about a part of the user's body that is in contact with the surface of the haptic presentation unit.

12. A control method, comprising: Sensing data is acquired by an information processing device, the sensing data including information about body parts of a user that are in contact with the surface of a tactile presentation unit configured to present warm and cold stimuli; as well as The information processing device controls the operation of the drive unit based on the acquired sensing data to adjust the contact area between the surface of the tactile presentation unit and the user's body parts.

13. A haptic feedback device, comprising: The tactile presentation unit is configured to present warm and cold stimuli; The sensing unit is configured to sense information about a part of the user's body that is in contact with the surface of the haptic presentation unit; as well as The driving unit is configured to perform operations to adjust the contact area of ​​the surface of the haptic presentation unit and the user's body parts in contact with each other based on driving data based on sensing data acquired by the sensing unit.

Images