Thermal sensation presentation device, thermal sensation presentation control device and thermal presentation control program

The non-contact thermal sensation presentation system using optical fibers and LED light emitting devices addresses discomfort and direction inaccuracies by adjusting light intensity based on headset position, ensuring accurate thermal sensation direction changes.

JP2024162610APending Publication Date: 2024-11-21NIPPON HOSO KYOKAI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023078300
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing thermal sensation presentation methods using Peltier elements or LED light for virtual reality headsets face issues such as discomfort due to contact and incorrect direction presentation when the headset's posture changes.

Method used

A non-contact thermal sensation presentation system using optical fibers and LED light emitting devices, controlled by a brightness calculation unit to adjust light intensity based on the headset's position and orientation, ensuring accurate direction presentation of virtual heat sources.

Benefits of technology

The system provides a comfortable and accurate presentation of thermal sensations from virtual heat sources without physical contact, maintaining direction accuracy even as the headset's posture changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162610000001_ABST
    Figure 2024162610000001_ABST
Patent Text Reader

Abstract

To provide a thermal sensation presentation device capable of presenting a virtual direction of a heat source in a non-contact manner even when a posture of an HMD changes.SOLUTION: A thermal sensation presentation device 2 comprises: a plurality of optical fibers 20 that transmit LED light; an optical fiber holding unit 21 that is provided at a lower part of a head mounted display (HMD) 1, and holds output ends of the plurality of optical fibers 20 around the cheek of a person so as to face a person side; and an LED light emitting device 22 that makes LED light incident into an input end of each optical fiber while varying brightness, based on brightness control information that controls the brightness of each optical fiber according to the position and posture of the HMD 1.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a thermal sensation presentation device, a thermal sensation presentation control device, and a thermal sensation presentation control program. [Background technology]

[0002] With the development of virtual reality (VR) technology, it is now possible to experience highly immersive all-around video content by wearing a head-mounted display (HMD). In recent years, various systems have been disclosed that incorporate not only audiovisual but also haptic media into HMDs (e.g., Non-Patent Document 1), and it is expected that we will be able to experience VR spaces or the metaverse with a greater sense of immersion than ever before. Regarding thermal sensation in particular, a technology has been disclosed that attempts to present a hot / cold sensation by controlling the skin surface temperature of the contact surface using a Peltier element (Non-Patent Document 2). In this technology, a Peltier element is placed around the person's neck in contact with the skin, and the direction is presented by heat by heating the Peltier element that corresponds to the direction of an object that is a virtual heat source in the VR space. A method has also been disclosed in which LED light is irradiated onto the hand to increase the skin surface temperature and provide a sense of warmth (Non-Patent Document 3). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Takayuki Kameoka, “Haptics and HMD: Proposal of Haptopus”, Journal of the Virtual Reality Society of Japan, Vol. 26, No. 3, pp. 17-20, published September 30, 2021 [Non-Patent Document 2] Ryuichi Ono, Nobuteru Ito, “Proposal for providing direction to VR users by thermal stimulation to the neck”, Proceedings of Entertainment Computing Symposium 2018, Vol. 2018, pp. 220-223, published on September 6, 2018 [Non-Patent Document 3] Kai Eihiro, Yamaguchi Tsutomu, Mitake Hiroharu, Hasegawa Shoichi, “Non-contact thermal display to the hand using visible light LEDs for HMDVR”, Transactions of the Virtual Reality Society of Japan, Vol. 24, No. 1, pp. 83-92, published March 31, 2019 Summary of the Invention [Problem to be solved by the invention]

[0004] When using a Peltier element as in the technology described in Non-Patent Document 2, the Peltier element comes into contact with the skin, causing an uncomfortable feeling, so a method of presenting a non-contact thermal sensation has been desired. Furthermore, in the technique described in non-patent document 2, where a Peltier element is placed around a person's neck, if a person moves their head and the position of the HMD changes, the front direction of the HMD and the front direction of the neck will no longer match, which creates the problem that the direction of the heat source cannot be correctly displayed. Furthermore, the technology described in Non-Patent Document 3 is capable of presenting the thermal sensation of a virtual heat source in a non-contact manner, but has the problem of being unable to present the direction of the heat source.

[0005] Therefore, an object of the present invention is to provide a thermal sensation presentation device, a thermal sensation presentation control device, and a thermal sensation presentation control program that are capable of presenting the direction of a virtual heat source in a non-contact manner even if the posture of the HMD changes. [Means for solving the problem]

[0006] In order to solve the above problems, the warmth sensation presentation device of the present invention is a warmth sensation presentation device that presents warmth using LED light, and is configured to include an optical fiber, an optical fiber holding unit, and an LED light-emitting unit.

[0007] In such a configuration, the warm sensation presentation device holds the emission ends of multiple optical fibers that transmit LED light around a person's cheeks, facing towards the person, using an optical fiber holding section provided at the bottom of the head-mounted display. This allows the warmth sensation display device to fix the emission ends of the optical fibers in a non-contact manner at a distance around the person's cheek. The thermal sensation presentation device uses an LED light-emitting unit to input LED light with varying brightness to the input end of each optical fiber based on brightness control information that controls the brightness of each optical fiber according to the position and posture of the head-mounted display. This allows the thermal sensation presentation device to control the brightness of each optical fiber using brightness control information, even if the position or posture of the head-mounted display changes, and therefore can present a thermal sensation corresponding to the direction of a virtual heat source.

[0008] In addition, in order to solve the above-mentioned problems, the warmth presentation control device of the present invention is a warmth presentation control device that controls a warmth presentation device having an optical fiber holding unit at the bottom of a head-mounted display that holds the exit ends of multiple optical fibers facing toward the person around at least a person's cheek, and an LED light-emitting unit that emits LED light at the entrance end of each of the optical fibers, and is equipped with an optical fiber-specific brightness calculation unit and a brightness control information generation unit.

[0009] In such a configuration, the thermal sensation presentation control device calculates the brightness for each optical fiber using the optical fiber-specific brightness calculation unit based on the strength of the virtual heat source and the angle between the direction from the irradiation point of the LED light emitted from the emission end to the virtual heat source and the normal direction at the irradiation point. The warm sensation presentation control device then generates, via the brightness control information generation unit, brightness control information that controls the brightness emitted by the LED based on the ratio of the brightness calculated by the optical fiber-specific brightness calculation unit to the maximum brightness of the optical fiber, and outputs the brightness control information to the LED light emission unit of the warm sensation presentation device. As a result, the thermal sensation presentation control device can provide a non-contact thermal sensation in the direction of the virtual heat source via the thermal sensation presentation device by increasing the brightness the smaller the angle between the direction from the irradiation point to the virtual heat source and the normal direction at the irradiation point. The thermal sensation presentation control device can be operated by a program for causing a computer to function as the thermal sensation presentation control device. Effect of the Invention

[0010] According to the present invention, the direction of a virtual heat source can be presented in a non-contact manner even if the posture of the HMD changes. [Brief description of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram of a thermal sensation presentation system including a thermal sensation presentation device and a thermal sensation presentation control device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a perspective view of the optical fiber holding part of the thermal sensation display device seen obliquely through the HMD. [Diagram 3] FIG. 2 is a top view of an optical fiber holding section of the thermal sensation presentation device. [Figure 4] FIG. 2 is a diagram showing the configuration of an LED light-emitting section of the thermal sensation presentation device. [Diagram 5] 1 is a block diagram showing the configuration of a temperature sensation presentation control device according to a first embodiment of the present invention. [Figure 6] 3 is a schematic diagram showing the relationship between the arrangement of optical fibers and point heat sources relative to a person's head in the first embodiment. FIG. [Figure 7] FIG. 11 is a diagram showing a head modeled as an elliptical cylinder, where (a) is a diagram showing the major and minor axis radii of the ellipse relative to the origin of the head coordinate system, and (b) is a diagram showing the downward irradiation position of the optical fiber relative to the origin of the head coordinate system. [Figure 8] FIG. 2 is a schematic diagram showing the relationship between one optical fiber and a point heat source with respect to a person's head. [Figure 9] 4 is a flowchart showing the operation of the temperature sense presentation control device according to the first embodiment of the present invention. [Figure 10] FIG. 11 is a block diagram showing the configuration of a temperature sensation presentation control device according to a second embodiment of the present invention. [Figure 11] 13 is a schematic diagram showing the relationship between the arrangement of optical fibers and parallel heat sources relative to a person's head in the second embodiment. FIG. [Figure 12] 10 is a flowchart showing the operation of the temperature sense presentation control device according to the second embodiment of the present invention. [Figure 13] FIG. 13 is a configuration diagram showing a modified example of the thermal sensation presentation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Overall configuration of the thermal display system] First, with reference to FIG. 1, the overall configuration of a thermal sensation presentation system according to an embodiment of the present invention will be described.

[0013] The thermal sensation presentation system 100 presents a thermal sensation from a virtual heat source to a person H wearing a head mounted display (hereinafter, HMD) 1. Here, the thermal sensation is a temperature sensation in which the person H feels the warmth of an object that is a virtual heat source. The warmth sensation display system 100 includes an HMD 1, a warmth sensation display device 2, and a warmth sensation display control device 3.

[0014] The HMD 1 is worn on the head of person H and displays an image of a virtual space (VR [Virtual Reality] image) according to the position and posture (direction) of the head of person H. Here, the HMD 1 will be described as an example of a VR HMD that displays VR images, but it may be an MR HMD that displays MR (Mixed Reality) images in which virtual objects are superimposed on images of the real world.

[0015] The thermal sensation providing device 2 provides a thermal sensation by using LED light. Here, the warmth sensation presentation device 2 presents a warmth sensation by irradiating LED light onto the skin of person H and generating heat on the skin in a non-contact manner using light energy. The warm sensation supplying device 2 irradiates LED light emitted by an LED light emitting device 22 from the emission ends of a plurality of optical fibers 20 fixed to the lower part of the HMD 1 around the face (cheeks) by optical fiber holding parts 21 .

[0016] The warmth sense presentation control device 3 controls the brightness of the LED light emitted by the warmth sense presentation device 2. The thermal sensation display control device 3 controls the brightness of the LED light emitted by the LED light emitting device 22 with respect to the virtual heat source according to the position and posture of the HMD 1 through the optical fiber 20 (201, 202, ..., 20 n ;n is a predetermined number of optical fibers) is controlled.

[0017] As a result, the thermal sensation presentation system 100 can present the thermal sensation from a virtual heat source to the person H in a non-contact manner by controlling the LED light emitted from the optical fiber 20 in accordance with the position and posture of the HMD 1 worn by the person H. Each component of the thermal sensation display system will be described below. Note that since a general known HMD 1 can be used, a detailed description will be omitted here.

[0018] [Configuration of the thermal display device] First, the configuration of the thermal sensation supply device 2 will be described. As shown in FIG. 1, the thermal sensation display device 2 includes optical fibers 20 (201, 202, ..., 20 n ), an optical fiber holding section 21, and an LED light emitting device 22.

[0019] (Optical Fiber) Optical fiber 20 (201, 202, ..., 20 n ) transmits the LED light emitted by the LED light-emitting device 22. The optical fiber 20 has an LED light incident end incorporated inside the LED light emitting device 22, and an output end held by an optical fiber holding section 21 at a position below the HMD 1 where it irradiates the area around the face of the person H.

[0020] Here, in order to protect the optical fiber 20, the optical fibers 201, 202, ..., 20 n are bundled together by a braided tube 20T from the top of the HMD 1 to the LED light emitting device 22. In this example, the optical fiber 20 passes through the front surface of the HMD 1, but if the HMD 1 is an MR HMD, the optical fiber 20 may be bundled together at the top by passing through the side surface of the HMD 1 so as not to be in the field of view of the person H.

[0021] (Optical fiber holding part) The optical fiber holding section 21 is disposed at the bottom of the HMD 1 and holds the emission ends of the multiple optical fibers 20 around the face (cheeks) of the person H at a distance from the skin. As shown in FIGS. 2 and 3, the optical fiber holding unit 21 includes an optical fiber termination holding unit 210, an irradiation position fixing unit 211, and an HMD connection unit 212. Fig. 2 is a perspective view of the optical fiber holding unit 21 seen through the HMD 1. Fig. 3 is a top view of the optical fiber holding unit 21.

[0022] The optical fiber end holder 210 holds the output end of the optical fiber 20, which is the end from which light is output. The optical fiber termination holder 210 holds the output end of the optical fiber 20 and has an opening 210A that allows light to pass through. Although not shown, the optical fiber termination holder 210 is preferably configured to prevent bending load on the optical fiber 20 by doubling the optical fiber 20 in the axial direction or increasing the thickness in the axial direction.

[0023] The irradiation position fixing unit 211 is for fixing and disposing the emission end of the optical fiber 20 along the vicinity of the cheek in the horizontal direction of the person's face. For example, the irradiation position fixing unit 211 is a part of an ellipse (here, a semi-ellipse) whose diameter is several centimeters (2 to 3 cm) larger than an ellipse whose major axis is the head length, which is the length from front to back of the head of an average young male, and whose minor axis is the zygomatic arch width, which is the width of the face. This shape may be a part of a circle (for example, a semi-circle). The modeling of the head will be described later with reference to FIG.

[0024] The irradiation position fixing unit 211 arranges the optical fiber end holders 210 at intervals so that the emission ends of the optical fibers 20 face inward (toward the person) according to a predetermined number n (e.g., n=7) of the optical fibers 20. Here, the irradiation direction of light from each optical fiber 20 faces inward along the normal of an ellipse that is an ellipse schematically representing the head.

[0025] The HMD connection part 212 connects the optical fiber holding part 21 main body to the lower part of the HMD 1. For example, the HMD connection part 212 is screwed to the lower part of the HMD 1 via the screw hole 212A. Of course, the HMD connection part 212 may be configured to be detachable from the HMD 1 using something other than the screw. This allows the optical fiber holding section 21 to arrange the emission ends of the optical fibers 20 in a line along an elliptical shape in a non-contact manner around the cheek of a person, which is the lower part of the HMD 1, for example.

[0026] (LED light emitting device) Next, the configuration of the LED light emitting device 22 will be described with reference to FIG. The LED light emitting device (LED light emitting section) 22 changes the luminance of LED light and emits it into the entrance end of each optical fiber 20 according to luminance control information input from the thermal sense presentation control device 3. The LED light emitting device 22 includes an LED light source array 220 and an LED luminance control section 230.

[0027] The LED light source array 220 has LED light source units 221 arranged in an amount equal to the number of optical fibers 20 to be connected. The LED light source section 221 emits light with the LED brightness adjusted based on a brightness control signal input from the LED brightness control section 230 . The LED light source unit 221 includes a light emitting unit 222 and a light collecting unit 223 .

[0028] The light emitting unit 222 emits light and is composed of a plurality of LEDs. For example, the light emitting unit 222 is an LED module such as a flat light emitting LED. The wavelength distribution of the light emitted by the light emitting portion 222 is in the visible light region, but may include the infrared region when the optical fiber 20 capable of transmitting the infrared region is used. The light emitting unit 222 adjusts the luminance of light emitted in response to a luminance control signal input from an LED luminance control unit 230 .

[0029] Here, the light emitting unit 222 receives a PWM (Pulse Width Modulation) signal as a brightness control signal input from the LED brightness control unit 230, and emits light according to a duty ratio (ratio of on time to off time). The light emitting section 222 irradiates the emitted light onto the light collecting section 223 .

[0030] The light collecting section 223 collects the light emitted from the light emitting section 222 . The light collecting section 223 collects the light emitted from the light emitting section 222 onto the incident end of the optical fiber 20 . The light collecting section 223 may be configured, for example, by a Fresnel lens that functions as a convex lens. As an example of the arrangement, the light collecting section 223 is arranged at a distance of the focal length of the light collecting section 223 from the incident end of the optical fiber 20. It is preferable that the light emitting portion 222 and the light collecting portion 223 are covered on the optical path with a white cover or the like that does not absorb visible light, so that the light is efficiently collected into the optical fiber 20 . Furthermore, the LED light emitting device 22 is preferably configured so as to have a filter for blocking ultraviolet rays disposed on the optical path to prevent adverse effects on the human body.

[0031] The LED luminance control section 230 controls the luminance of the LED light emitted by each LED light source section 221 based on luminance control information for each LED light source section 221 input from the thermal sense presentation control device 3. Here, the brightness control information is the duty ratio (0.0 to 1.0) of the PWM signal. The LED luminance control unit 230 generates a PWM signal according to the duty ratio and outputs it to each of the LED light source units 221 .

[0032] With the configuration described above, the warmth sensation presentation device 2 can present a warmth sensation to the person H by irradiating the skin of the person H with light of different brightness for each optical fiber 20 based on brightness control information input from the warmth sensation presentation control device 3.

[0033] [Configuration of the thermal display control device] Next, the configuration of the thermal sense presentation control device 3 will be described with reference to FIG. The thermal sense presentation control device 3 receives heat source information and head information from the outside, and outputs brightness control information for controlling the brightness of the LED light for each optical fiber 20.

[0034] FIG. 6 shows a schematic diagram of the relationship between the arrangement of the optical fiber 20 with respect to the person's head HD and a point heat source Po, which is a virtual heat source. As shown in FIG. 6, with respect to a person's head HD having an elliptical shape around the cheeks, the direction of the emission ends of the optical fibers 20 is the same as the direction of the normal N (normal direction) of the elliptical shape. The warmth sensation presentation control device 3 controls the luminance of each optical fiber 20 so as to present a warmth sensation due to a ray R from a point heat source Po at the irradiation point EP, based on the illuminance of the irradiation point EP of the LED light of the optical fiber 20 on the head HD. For example, the warmth sensation presentation control device 3 controls the luminance of the optical fiber 201 so as to present a warmth sensation due to a ray R from the point heat source Po irradiated from a direction at an angle θ1 with the normal N at the irradiation point EP1, based on the LED light emitted by the optical fiber 201.

[0035] Here, as shown in Figure 5, the thermal sensation presentation control device 3 includes a memory unit 30, a heat source information acquisition unit 31, a head information acquisition unit 32, a heat source position calculation unit 33, an optical fiber-specific brightness calculation unit 34, and a brightness control information generation unit 35.

[0036] The storage unit 30 stores in advance the position of the irradiation point on the head in the head coordinate system, the normal direction of the irradiation point, and the position of the output end of the optical fiber 20. The head is modeled as an elliptical cylinder. Specifically, as shown in Fig. 7(a), the origin HDorg is the center of the ellipse, the major axis radius in the front-to-back direction of the head HD is a, and the minor axis radius is b. Also, as shown in Fig. 7(b), the output end of the optical fiber 20 is located at a position spaced downward from the origin HDorg by a distance z. For example, a = 189.1 / 2 [mm], b = 145.1 / 2 [mm], and z = 53.0 [mm]. Here, the head coordinate system is a coordinate system whose origin is HDorg.

[0037] The storage unit 30 stores the irradiation point EP i (i is the identification number of the optical fiber 20) i and the irradiation point EP i Normal vector n, which is the direction of the normal N of i (unit vector) and optical fiber 20 i The position vector p i are calculated and stored in advance. In addition, the irradiation point EP i Position vector q i is the irradiation point EP from the front direction of the head HD with HDorg as the origin. i The angle to the direction of is φ i Then, it can be calculated using the following formula (1).

[0038]

number

[0039] In addition, the irradiation point EP i Normal vector n in i can be calculated using the following formula (2).

[0040]

number

[0041] In addition, the irradiation point EP i 20 compatible optical fibers i Position vector p of the exit end i 20 i The exit end and irradiation point EP i The distance from i Then, it can be calculated using the following formula (3).

[0042]

number

[0043] The heat source information acquiring unit 31 acquires heat source information from an external source. The heat source information is information that indicates the strength (specified in luminous intensity [cd]) of a point heat source, which is a virtual heat source in virtual space, and its position in the world coordinate system. The world coordinate system is a coordinate system whose origin is a predetermined position in the virtual space. This heat source information is information added to the VR content by the VR content creator. If the strength or position of the virtual heat source changes, the heat source information is input at the timing of the change. This heat source information is input by wired or wireless communication from a content presentation device (not shown) that displays VR video on the HMD1.

[0044] The heat source information acquisition unit 31 outputs the intensity of the point heat source from among the acquired heat source information to the optical fiber-specific luminance calculation unit . Furthermore, the heat source information acquiring unit 31 outputs the position of the point heat source in the world coordinate system from the acquired heat source information to the heat source position calculating unit 33.

[0045] The head information acquisition unit 32 acquires head information from an external source. The head information is information indicating the position of the HMD 1 in a world coordinate system (a predetermined origin position of the head) and the attitude (head attitude) of the HMD 1, which change with the movement of the HMD 1. This head information is information obtained by a head tracking function of the HMD 1, and is obtained by the head information acquisition unit 32 from the HMD 1 via wireless communication or wired communication. The head information acquisition unit 32 outputs the acquired head information to the heat source position calculation unit 33.

[0046] The heat source position calculation unit 33 calculates the position of a point heat source in the head coordinate system. The heat source position calculation unit 33 calculates the position of the point heat source in the head coordinate system based on the position of the point heat source in the world coordinate system input from the heat source information acquisition unit 31 and the origin position and posture of the head in the world coordinate system input from the head information acquisition unit 32.

[0047] Here, the position of the point heat source in the world coordinate system is expressed by the position vector p heat W The origin position of the head in the world coordinate system is expressed as the position vector p head W The head posture (3D posture) seen from the world coordinate system is expressed as a rotation matrix R H W Then, the heat source position calculation unit 33 calculates the position vector p heat H is calculated using the following formula (4).

[0048]

number

[0049] The heat source position calculation unit 33 calculates the position of the point heat source in the head coordinate system (position vector p heat H ) is output to the optical fiber-specific luminance calculation unit 34.

[0050] The optical fiber-specific brightness calculation unit 34 calculates the brightness of each optical fiber based on the strength of the virtual heat source and the angle between the direction from the irradiation point of the LED light emitted from the output end of the optical fiber to the virtual heat source and the normal direction at the irradiation point. Here, the optical fiber-specific luminance calculation unit 34 includes a heat source direction calculation unit 340 , an irradiation point illuminance calculation unit 341 , and a luminance calculation unit 342 .

[0051] The heat source direction calculation section 340 calculates the direction of a point heat source for each irradiation point on the head. As shown in Figure 8, the irradiation point EP i (i is the identification number of the optical fiber 20) to the point heat source Po. i is the position vector p of the point heat source Po as shown in the following equation (5). heat H and the position vector q of the irradiation point EP in the head coordinate system i It can be calculated by the difference between

[0052]

number

[0053] Then, the heat source direction calculation unit 340 calculates the heat source direction from the angle between the light vector from the irradiation point to the point heat source and the normal line at the irradiation point. That is, the heat source direction calculation unit 340 calculates the irradiation point EP i Ray vector d from to point heat source Po i and irradiation point EP i Normal vector n in i The angle between i Then, the value indicating the direction of the point heat source Po is calculated as cosθ i It is calculated as follows.

[0054]

number

[0055] The heat source direction calculation unit 340 calculates the irradiation point EP i The direction of the point heat source Po (cosθ i ) to the irradiation point illuminance calculation unit 341.

[0056] The irradiation point illuminance calculation unit 341 calculates the illuminance at the irradiation point based on the direction (cos θ) of the point heat source, the strength of the point heat source, and the distance from the point heat source to the irradiation point. Here, the value indicating the direction of the point heat source is input from the heat source direction calculation unit 340. The strength of the point heat source is input from the heat source information acquisition unit 31. The distance from the point heat source to the irradiation point is calculated by the light vector d i is input from the heat source direction calculation unit 340, or the irradiation point illuminance calculation unit 341 calculates the light vector d i can be calculated as the length of the light vector.

[0057] The irradiation point illuminance calculation unit 341 calculates a value cosθ indicating the direction of the point heat source for each optical fiber identification number i. i, the intensity (luminosity) of the point heat source I [cd], and the distance from the point heat source to the irradiated point (ray vector d i The irradiation point EP i Illuminance E at i Calculate [lx].

[0058]

number

[0059] From this equation (7), θ i = 0°, illuminance E i is the maximum value, and the closer it is to ±90° the weaker the illuminance becomes. The irradiation point illuminance calculation unit 341 calculates the irradiation point EP i Illuminance E at i to the luminance calculation unit 342.

[0060] The luminance calculation section 342 calculates the luminance of the optical fiber corresponding to the irradiation point based on the illuminance of the irradiation point calculated by the irradiation point illuminance calculation section 341 . Specifically, as shown in FIG. 8, the luminance calculation unit 342 calculates the illumination point EP i From optical fiber 20 i The distance to the exit end of i [m], optical fiber 20 i The radius of the irradiation point EP i Let E be the illuminance at i [lx], the following formula (8) is used to calculate the optical fiber 20 i Luminance L i [cd / m 2 ] is calculated.

[0061]

number

[0062] In addition, optical fiber 20 i The radius r is stored in advance in the storage unit 30. The luminance calculation unit 342 outputs the calculated luminance for each optical fiber to the luminance control information generation unit 35.

[0063] The luminance control information generating unit 35 generates luminance control information for controlling the luminance of the LED emitted by the LED light emitting device 22 (FIG. 4) from the luminance for each optical fiber calculated by the optical fiber-specific luminance calculating unit 34. Here, the luminance control information generating unit 35 generates the luminance control information based on the ratio of the luminance calculated by the optical fiber luminance calculating unit 34 to the maximum luminance of the LED provided in the LED light emitting device 22 (FIG. 4). That is, the brightness control information generating unit 35 uses the duty ratio of the PWM signal as the brightness control information. There is a linear relationship between the duty ratio of the PWM signal and the brightness. Therefore, the luminance control information generating unit 35 calculates and generates the luminance control information k for each optical fiber by the following formula (9).

[0064]

number

[0065] Here, L i is the luminance for each optical fiber calculated by the luminance calculation unit 342. max is the maximum brightness of the optical fiber. The maximum brightness can be obtained by measurement in advance. If the maximum brightness differs for each optical fiber, L max As the maximum brightness value per optical fiber, L i,max can be used. In the above formula (9), L i ≦L max It is. Therefore, the strength (luminous intensity) of the virtual heat source I [cd] is calculated from the above equations (7) to (9) as follows: i ≦L max It is preferable to calculate I that satisfies the above in advance and set it in the VR content. The luminance control information generator 35 outputs the generated luminance control information to the LED light-emitting device 22 (FIG. 4) of the thermal sensation presentation device 2 wirelessly or via wire.

[0066] With the configuration described above, the thermal sensation presentation control device 3 can control the thermal sensation presentation device 2 so that light of different brightness is irradiated onto the skin of the person H wearing the HMD 1 for each optical fiber 20 in accordance with the movement of the head of the person H. The thermal sensation presentation control device 3 can be operated by a program (thermal sensation presentation control program) for causing a computer (not shown) to function as each unit.

[0067] [Operation of the thermal display control device] Next, the operation of the thermal sensation presentation control device 3 will be described with reference to Fig. 9 (and Fig. 5 and Fig. 8 as appropriate). It is assumed that the storage unit 30 stores in advance the position of the irradiation point on the head in the head coordinate system, the normal direction of the irradiation point, the position of the exit end of the optical fiber 20, and the radius of the optical fiber.

[0068] In step S1, the heat source information acquisition unit 31 acquires, as heat source information, the position of a point heat source in the world coordinate system and the strength of the point heat source. In step S2, the head information acquisition section 32 acquires, as head information, the position of the HMD 1 in the world coordinate system (head position: a predetermined origin position of the head) and the orientation of the HMD 1 (head orientation).

[0069] In step S3, heat source position calculation unit 33 calculates the position of the point heat source in the head coordinate system based on the position of the point heat source in the world coordinate system acquired in step S1 and the head position and head posture in the world coordinate system acquired in step S2. That is, heat source position calculation unit 33 calculates the position of the point heat source in the head coordinate system using equation (4) above.

[0070] In step S4, the heat source direction calculation unit 340 calculates the direction of the point heat source for each irradiation point on the head. i Ray vector d to i and irradiation point EP i Normal vector n in i The angle betweeni Then, from the above equations (5) and (6), the value indicating the direction of the point heat source is cosθ i It is calculated as follows.

[0071] In step S5, the irradiation point illuminance calculation unit 341 calculates the illuminance at the irradiation point based on the direction of the point heat source, the strength of the point heat source, and the distance from the point heat source to the irradiation point. That is, the irradiation point illuminance calculation unit 341 calculates the illuminance at the irradiation point EP i Illuminance E at i Calculate.

[0072] In step S6, the luminance calculation unit 342 calculates the luminance of the optical fiber corresponding to the irradiation point based on the illuminance of the irradiation point. i Luminance L i Calculate.

[0073] In step S7, the brightness control information generating unit 35 generates brightness control information for controlling the brightness of the LEDs emitted by the LED light emitting device 22 (FIG. 4) from the brightness of each optical fiber. Here, the brightness control information generating unit 35 generates the duty ratio of the PWM signal as the brightness control information by the above-mentioned formula (9).

[0074] In step S8, the luminance control information generator 35 outputs the generated luminance control information to the LED light-emitting device 22 of the warmth sensation presentation device 2 (FIG. 4). This makes it possible to control the luminance of the light irradiated onto the skin of the person H for each optical fiber 20 in the warmth sensation supplying device 2.

[0075] In step S9, the thermal sense presentation control device 3 determines whether or not an instruction to end the operation has been issued by a switch or the like (not shown). If the end of the operation is not instructed (No in step S9), the warmth sense presentation control device 3 returns to step S1 and continues the operation. Note that, if the position of the point heat source is fixed in the VR content, the warmth sense presentation control device 3 may return to step S2 and continue the operation. On the other hand, if an instruction to end the operation is given (Yes in step S9), the thermal sense presentation control device 3 ends the operation.

[0076] By the above operation, the warmth sense presentation control device 3 can control the luminance of each optical fiber of the warmth sense presentation device 2 in accordance with the movement of the head of the person H wearing the HMD 1. This allows the warmth sense presentation control device 3 to control the warmth sense presentation device 2 so as to present the direction of a virtual heat source even if the attitude of the HMD 1 changes.

[0077] [Configuration of the temperature sensation display control device (second embodiment)] The warmth sensation presentation control device 3 described in Fig. 5 presents a warmth sensation by using a virtual heat source as a point heat source. However, the virtual heat source may be a parallel heat source that simulates sunlight. The following describes a thermal sensation presentation control device in which the virtual heat source is a parallel heat source.

[0078] With reference to FIG. 10, the configuration of a thermal sense presentation control device 3B in the case where the virtual heat source is a parallel heat source will be described. The thermal sense presentation control device 3B inputs heat source information and head information from the outside, and outputs brightness control information for controlling the LEDs of each optical fiber 20.

[0079] FIG. 11 shows a schematic diagram of the relationship between the arrangement of the optical fiber 20 with respect to the person's head HD and a parallel heat source Pa, which is a virtual heat source. As shown in FIG. 11, with respect to a person's head HD having an elliptical shape around the cheeks, the direction of the emission end of each optical fiber 20 is the same as the direction of the normal line N of the elliptical shape. The warmth sensation presentation control device 3B controls the luminance of each optical fiber 20 so as to present a warmth sensation due to a ray R from the parallel heat source Pa at the irradiation point EP, based on the illuminance of the irradiation point EP of the LED light of the optical fiber 20 on the head HD. For example, the warmth sensation presentation control device 3B controls the luminance of the optical fiber 201 so as to present a warmth sensation due to a ray R from the parallel heat source Pa irradiated from the direction of the angle θ1 with the normal N at the irradiation point EP1, based on the LED light emitted by the optical fiber 201.

[0080] Here, as shown in FIG. 10, the thermal sensation presentation control device 3B includes a memory unit 30, a heat source information acquisition unit 31B, a head information acquisition unit 32, an optical fiber-specific brightness calculation unit 34B, and a brightness control information generation unit 35. The storage unit 30, the head information acquisition unit 32, and the brightness control information generation unit 35 have the same configuration as the thermal sense presentation control device 3 described in FIG. 5, and therefore a description thereof will be omitted.

[0081] The heat source information acquiring unit 31B acquires heat source information from the outside. The heat source information is information that indicates the strength (specified as illuminance [lx]) of a parallel heat source, which is a virtual heat source in virtual space, and the direction (unit vector) of the parallel heat source in the world coordinate system. This heat source information is information added to the VR content by the VR content creator. If the strength or direction of the parallel heat source changes, the heat source information is input at the timing of the change. This heat source information is input by wired or wireless communication from a content presentation device (not shown) that displays VR images on the HMD1. The heat source information acquisition unit 31B outputs the acquired heat source information to the optical fiber-specific luminance calculation unit 34B.

[0082] The optical fiber-specific brightness calculation unit 34B calculates the brightness of each optical fiber based on the strength of the virtual heat source and the angle between the direction from the irradiation point of the LED light emitted from the output end of the optical fiber to the virtual heat source and the normal direction at the irradiation point. Here, the optical fiber-specific luminance calculation unit 34B includes a heat source direction calculation unit 340B, an irradiation point illuminance calculation unit 341B, and a luminance calculation unit 342. The luminance calculation unit 342 has the same configuration as that of the thermal sense presentation control device 3 described in FIG. 5, and therefore a description thereof will be omitted.

[0083] The heat source direction calculation section 340B calculates the direction of the parallel light beam of the parallel heat source for each irradiation point on the head. The heat source direction calculation unit 340B converts the direction of the parallel heat source in the world coordinate system into a direction in the head coordinate system, and then calculates the heat source direction for each irradiation point based on the angle between the direction of the parallel heat source in the head coordinate system and the normal at the irradiation point. Here, the direction of the parallel heat source in the world coordinate system is expressed as the unit vector e d W The head orientation (3D orientation) in the world coordinate system is expressed as a rotation matrix R H W Then, the heat source direction calculation unit 340B calculates a unit vector d H is calculated using the following formula (10).

[0084]

number

[0085] Then, the heat source direction calculation unit 340B calculates the irradiation point EP i (i is the identification number of the optical fiber 20) to the parallel heat source Pa, and the ray vector (unit vector) from the parallel heat source Pa to the irradiation point EP i The direction of the heat source is calculated based on the angle with the normal line N at the point EP i The ray vector (unit vector) from to the parallel heat source Pa is the unit vector d calculated by equation (10). H It is. The heat source direction calculation unit 340B detects the irradiation point EP i Unit vector d from to the point heat source Po H and irradiation point EP i Normal vector n in i The angle between i Then, the value indicating the direction of the parallel heat source Pa is calculated as cosθ i It is calculated as follows.

[0086]

number

[0087] The heat source direction calculation unit 340B calculates the irradiation point EP i The direction of the parallel heat source Pa (cosθ i ) to the irradiation point illuminance calculation unit 341B.

[0088] The irradiation point illuminance calculation unit 341B calculates the illuminance at the irradiation point based on the direction and strength of the parallel heat source. Here, the direction and strength of the parallel heat source are input from the heat source information acquiring unit 31B. The irradiation point illuminance calculation unit 341B calculates a value cosθ indicating the direction of the parallel heat source for each optical fiber identification number i. i and the intensity (illuminance) of the parallel heat source E p [lx], the irradiation point EP is calculated by the following equation (12). i Illuminance E at i Calculate [lx].

[0089]

number

[0090] From this equation (12), θ i = 0°, illuminance E i is the maximum value, and the closer it is to ±90° the weaker the illuminance becomes. The irradiation point illuminance calculation unit 341B calculates the irradiation point EP i Illuminance E at i to the luminance calculation unit 342.

[0091] The luminance calculation unit 342 and the luminance control information generation unit 35 perform the calculations of the above-mentioned formulas (8) and (9), respectively. In the above-mentioned formula (9), L i ≦L max It is. Therefore, the strength (illuminance) of the virtual heat source E p [lx] is L from the above formulas (8), (9), and (12). i ≦L max Satisfying E p It is preferable to calculate and set the above in advance in the VR content.

[0092] Furthermore, when the virtual heat source is a parallel heat source simulating sunlight, it is preferable to adjust the strength of the heat source according to attenuation due to weather or shading. In this case, the luminance calculation unit 342 calculates the luminance L i can be adjusted by coefficients that represent the attenuation effects of weather and occlusion. For example, let K be the coefficient that indicates the effect of attenuation of sunlight due to weather. weather (0 to 1), and K is a coefficient that indicates the attenuation of sunlight due to other objects such as occlusion (e.g., frosted glass, etc.). shadow (0 to 1), maximum illuminance of sunlight (approximately 1.0×10 6 [lx]) to E sun , the maximum illuminance [lx] of the illuminated point is E max Then, the luminance calculation unit 342 calculates the adjusted luminance L i,con The coefficient K weather , K shadow is information input corresponding to the head position from a content presentation device (not shown) that displays VR video on the HMD 1.

[0093]

number

[0094] With the configuration described above, the thermal sensation presentation control device 3B can control the thermal sensation presentation device 2 so that light of different brightness is irradiated onto the skin of the person H wearing the HMD 1 for each optical fiber 20 in accordance with the movement of the head of the person H. The thermal sensation presentation control device 3B can be operated by a program (thermal sensation presentation control program) for causing a computer (not shown) to function as each unit.

[0095] [Operation of the temperature feedback control device (second embodiment)] Next, the operation of the thermal sense presentation control device 3B will be described with reference to Fig. 12 (and Fig. 10 as necessary). It is assumed that the storage unit 30 stores in advance the position of the irradiation point on the head in the head coordinate system, the normal direction of the irradiation point, the position of the exit end of the optical fiber 20, and the radius of the optical fiber.

[0096] In step S10, the heat source information acquisition unit 31B acquires, as heat source information, the direction and strength of the parallel heat source in the world coordinate system. In step S11, the head information acquisition section 32 acquires, as head information, the position of the HMD 1 in the world coordinate system (head position: a predetermined origin position of the head) and the orientation of the HMD 1 (head orientation).

[0097] In step S12, the heat source direction calculation unit 340B calculates the direction of the parallel light beam of the parallel heat source for each irradiation point on the head. That is, the heat source direction calculation unit 340B calculates a unit vector d H is calculated by the above formula (10). Then, the heat source direction calculation unit 340B calculates the unit vector d H and irradiation point EP i Normal vector n in i The angle between i Then, according to the above formula (11), the value indicating the direction of the parallel heat source Pa is cosθ i It is calculated as follows.

[0098] In step S13, the irradiation point illuminance calculation unit 341B calculates the illuminance at the irradiation point based on the direction of the parallel heat source and the strength of the parallel heat source. i Illuminance E at i Calculate [lx]. The operations from step S14 onwards are the same as those from step S6 onwards of the thermal sense presentation control device 3 described with reference to FIG. 9, and therefore will not be described here.

[0099] By the above operation, the warmth sense presentation control device 3B can control the luminance of each optical fiber of the warmth sense presentation device 2 when the virtual heat source is a parallel heat source, according to the movement of the head of the person H wearing the HMD 1. This allows the warmth sense presentation control device 3B to control the warmth sense presentation device 2 so as to present the direction of a virtual heat source even if the attitude of the HMD 1 changes.

[0100] The above describes the thermal sensation presentation system 100 according to an embodiment of the present invention, but the present invention is not limited to these embodiments and also includes design modifications and the like that do not deviate from the gist of the present invention.

[0101] (Variation 1) For example, here, as shown in Figures 1 to 3, in the optical fiber holding unit 21 of the temperature sensation presentation device 2, the irradiation position fixing unit 211 and the optical fiber terminal holding unit 210 are separate components, and the optical fiber terminal holding unit 210 is arranged at a distance from the irradiation position fixing unit 211. However, the thickness of the irradiation position fixing portion 211 in the vertical direction of the face may be increased, and only the opening 210A of the optical fiber terminal holding portion 210 may be positioned away from the irradiation position fixing portion 211, so that the irradiation position fixing portion 211 also serves as the optical fiber terminal holding portion 210.

[0102] (Variation 2) Further, here, the optical fiber holding section 21 of the warm sensation supplying device 2 is configured to place the emission end of the optical fiber 20 at least in the front part of the face around the cheeks of a person as a part of the elliptical shape. However, as shown in a warm sense supplying device 2B in FIG. 13, a second optical fiber holding part 21B may be further provided in which the emission ends of the optical fibers 20 are arranged as part of an elliptical shape around the back side of the neck of the person H. This allows the warmth sensation presentation device 2B to present a virtual heat source located behind as a warm sensation.

[0103] (Variation 3) Here, the optical fiber-specific luminance calculation units 34, 34B of the thermal sense presentation control devices 3, 3B performed various calculations in the head coordinate system. However, the optical fiber-specific luminance calculation units 34 and 34B may perform calculations in the world coordinate system. In that case, the heat source position calculation unit 33 can be omitted from the configuration. In that case, the optical fiber-specific luminance calculation units 34 and 34B may convert the head irradiation point position and the normal direction of the irradiation point in the head coordinate system stored in the storage unit 30 into the world coordinate system and use them.

[0104] (Variation 4) Moreover, here, the warmth sensation presentation control device 3 presents a warmth sensation by targeting one point heat source as a virtual heat source. However, there may be multiple point heat sources or a collection of point heat sources of a certain size. In this case, the thermal sensation presentation control device 3 calculates the luminance of each optical fiber for each point heat source in the optical fiber luminance calculation unit 34, and calculates the maximum luminance of the same optical fiber as the luminance of that optical fiber. [Explanation of symbols]

[0105] 1 HMD 2 Temperature presentation device 20 Optical Fiber 21 Optical fiber holder 210 Optical fiber end holder 211 Irradiation position fixing part 212 HMD connection part 22 LED light emitting device (LED light emitting part) 220 LED light source array 221 LED light source section 222 Light emitting part 223 Light collecting part 230 LED brightness control unit 3. Thermal display control device 30 Storage section 31 Heat source information acquisition section 32 Head information acquisition unit 33 Heat source position calculation section 34 Optical fiber brightness calculation section 340 Heat source direction calculation section 341 Irradiation point illuminance calculation unit 342 Luminance Calculation Unit 35 Brightness control information generator 100 Thermal display system

Claims

1. A thermal sensation presentation device that presents a thermal sensation using LED light, A plurality of optical fibers for transmitting the LED light; an optical fiber holding unit provided at a lower portion of the head mounted display and configured to hold the emission ends of the optical fibers around a person's cheek such that the emission ends face the person; an LED light emitter that changes the luminance of the LED light and inputs the LED light to an input end of each of the optical fibers according to luminance control information that controls the luminance of each of the optical fibers according to a position and a posture of the head mounted display; A thermal sensation presentation device comprising:

2. The thermal sensation presentation device described in claim 1, further comprising a second optical fiber holding portion provided at the bottom of the head-mounted display and holding the emission ends of the multiple optical fibers facing toward the person around the back of the person's neck.

3. The brightness control information is a duty ratio of the LED light for each of the optical fibers, The LED light emitting unit is an LED light source unit corresponding to the number of the optical fibers; an LED luminance control unit that generates a PWM signal based on a duty ratio of the LED light, which is the luminance control information, and controls the luminance of the LED light for each optical fiber of the LED light source unit based on the PWM signal; The thermal sensation presentation device according to claim 1 or 2, further comprising:

4. A thermal sensation presentation control device is provided at a lower part of a head mounted display with an optical fiber holding unit that holds multiple optical fibers around at least a person's cheek with their emission ends facing the person, and controls a thermal sensation presentation device having an LED light emitting unit that directs LED light to the entrance ends of each of the optical fibers, an optical fiber luminance calculation unit that calculates luminance for each of the optical fibers based on the intensity of the virtual heat source and the magnitude of the angle between a direction from an irradiation point of the LED light emitted from the output end to the virtual heat source and a normal direction at the irradiation point; a brightness control information generating unit that generates brightness control information for controlling the brightness of the LED in the LED light emitting unit based on a ratio of the brightness calculated by the optical fiber brightness calculating unit to the maximum brightness of the optical fiber; A thermal sensation presentation control device comprising:

5. The virtual heat source is a point heat source, and the optical fiber-specific luminance calculation unit a heat source direction calculation unit that calculates a value of cos θ for each of the irradiation points, where θ is an angle between a direction from the irradiation point to the point heat source and a normal direction at the irradiation point; an irradiation point illuminance calculation unit that calculates an illuminance at the irradiation point based on the value of cos θ, the strength of the point heat source, and the distance from the point heat source to the irradiation point; a luminance calculation unit that calculates the luminance of the optical fiber corresponding to the irradiation point from the illuminance of the irradiation point; The thermal sensation presentation control device according to claim 4, further comprising:

6. The virtual heat source is a parallel heat source, and the optical fiber-specific luminance calculation unit a heat source direction calculation unit that calculates a value of cosθ for each irradiation point, where θ is an angle between a direction from the irradiation point to the parallel heat source and a normal direction at the irradiation point; an irradiation point illuminance calculation unit that calculates an illuminance at the irradiation point based on the value of cos θ and the strength of the parallel heat source; a luminance calculation unit that calculates the luminance of the optical fiber corresponding to the irradiation point from the illuminance of the irradiation point; The thermal sensation presentation control device according to claim 4, further comprising:

7. A thermal sensation control program for causing a computer to function as the thermal sensation control device according to any one of claims 4 to 6.