Audio output device and head-mounted display device

Abstract

To provide a voice output device with improved user-friendliness.SOLUTION: A head-mounted display device comprises: an image display section 20 mounted on the head of a user and comprising a plurality of right speakers 520 and a plurality of left speakers 540; and a DP control section 120 controlling the right speakers 520 and the left speakers 540. The right speakers 520 and the left speakers 540 include: the first speakers arranged at positions closer to the ear positions of the user; and the second speakers arranged at positions different from those of the first speakers and also more separately arranged from the ear positions of the user than the first speakers. The DP control section 120 outputs voice by varying voice setting between the first speakers and the second speakers.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an audio output device and a head-mounted display device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, devices that are worn on a user's head and output sound are known. For example, Patent Document 1 comprises an optical element placed near the user's eye, a contact element extending from the optical element and placed along the user's head, a number of fixing parts placed on the contact element for fixing a sound-producing body at different positions, and an output control part for controlling the volume of the sound-producing body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP2014-78868A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the head-mounted display device disclosed in Patent Document 1 requires that the position of the sound generator be adjusted to match the position of the ears of each user, making the adjustments cumbersome. [Means for solving the problem]

[0005] One aspect that solves the above problem is an audio output device that includes a main body unit that is worn on a user's head and has a plurality of speakers, and a control unit that controls the plurality of speakers, wherein the plurality of speakers include a first speaker that is positioned close to the position of the user's ears, and a second speaker that is positioned differently from the first speaker and farther away from the position of the user's ears than the first speaker, and the control unit outputs audio by differentiating the audio settings of the first speaker and the second speaker.

[0006] One aspect of the invention that solves the above problem is a head-mounted display device that is worn on a user's head and has a display unit that displays images visible through the outside world, the head-mounted display device including a plurality of speakers installed on the display unit and a control unit that controls the plurality of speakers, the plurality of speakers including a first speaker that is positioned close to the position of the user's ears and a second speaker that is positioned differently from the first speaker and farther away from the position of the user's ears than the first speaker, and the control unit outputs sound by differentiating the sound settings of the first speaker and the second speaker. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a display system. [Diagram 2] FIG. 2 is a plan view of a main portion showing the configuration of an optical system of an image display unit. [Diagram 3] FIG. 1 is a block diagram of a head-mounted display device. [Figure 4] FIG. [Diagram 5] 4 is a flowchart showing the operation of the head-mounted display device. [Figure 6] FIG. 11 is a diagram showing another example of speaker arrangement. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] [1. Display system configuration] Hereinafter, embodiments to which the present invention is applied will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a display system 1. As shown in FIG. The display system 1 includes an HMD 100 and a control device 300. The HMD 100 is a head-mounted display device that allows a user to visually recognize an image or video, and is an example of the audio output device of the present invention. HMD is an abbreviation for Head Mounted Display.

[0009] The HMD 100 has an image display unit 20 that is worn on the head of a user, and a connection device 10 that is connected to the image display unit 20. The connection device 10 functions as an interface that connects the HMD 100 to a device different from the HMD 100. In the display system 1, a control device 300 is connected to the connection device 10. In the following description and drawings, for convenience of explanation, the prefix DP is added to the names of some of the functional units constituting the HMD 100, and the prefix CO is added to the names of some of the functional units constituting the control device 300.

[0010] The control device 300 is a portable terminal device, for example a smartphone, that includes a display screen that displays characters and images, and a touch panel 328 that functions as an operation unit that detects touch operations and pressing operations. The touch panel 328 is composed of a display panel and a touch sensor. The display panel is composed of, for example, an LCD. LCD is an abbreviation for Liquid Crystal Display. The control device 300 may be a desktop personal computer, a notebook personal computer, a tablet personal computer, or the like.

[0011] The connection device 10 includes a box-shaped case and a connector 11A and a connector 11D. The image display unit 20 is connected to the connector 11A via a connection cable 40, and the control device 300 is connected to the connector 11D via a USB cable 46. This allows the image display unit 20 and the control device 300 to be connected so as to be able to transmit and receive data to each other. For example, the control device 300 outputs video data for the image display unit 20 to display an image and audio data to the image display unit 20. In addition, the image display unit 20 transmits detection data of various sensors included in the image display unit 20 to the control device 300 as described later. The control device 300 may be capable of supplying power to the image display unit 20. USB is an abbreviation for Universal Serial Bus.

[0012] The configuration in which the connection device 10 and the control device 300 are connected using the USB cable 46 is merely an example, and the specific connection form between the connection device 10 and the control device 300 is not limited. For example, they may be connected by wire using other types of cables, or may be connected via wireless communication. For example, in a configuration in which the USB cable 46 is connected to the connector 11D of the USB-TypeC standard, the USB cable 46 can supply a direct current of 20 volts, and can transmit video data of the HDMI standard as a function of an alternative mode of the USB-TypeC. HDMI is a registered trademark.

[0013] The image display unit 20 corresponds to an example of a main body, and includes a main body having a right holding portion 21, a left holding portion 23, and a front frame 27, a right display portion 22, a left display portion 24, a right light guide plate 26, and a left light guide plate .

[0014] The right holding portion 21 and the left holding portion 23 correspond to the temples of the eyeglass-shaped image display unit 20, extend rearward from both ends of the front frame 27, and hold the right display unit 22 or the left display unit 24 on the user's head.

[0015] A right speaker unit 510 is disposed on the right holding unit 21. The right speaker unit 510 is provided on an inner surface of the right holding unit 21 that faces the face of a user when the user wears the image display unit 20 on his / her head. The right speaker unit 510 may also be provided on a lower surface of the right holding unit 21. The lower surface refers to the surface of the right holding unit 21 that faces the floor surface when the user wears the image display unit 20 on his / her head.

[0016] As shown in FIG. 3, the right speaker unit 510 includes five speakers, namely, a right speaker 521, a right speaker 522, a right speaker 523, a right speaker 524, and a right speaker 525. Hereinafter, the right speaker 521, the right speaker 522, the right speaker 523, the right speaker 524, and the right speaker 525 are collectively referred to as a right speaker 520. The right speaker 521, the right speaker 522, the right speaker 523, the right speaker 524, and the right speaker 525 are arranged in a horizontal row at predetermined intervals along the extension direction of the right holding part 21. In this embodiment, a case will be described in which the right speaker 520 includes five speakers, namely, the right speakers 521 to 525, but the number of the right speakers 520 may be two or more.

[0017] Any one of right speaker 521, right speaker 522, right speaker 523, right speaker 524, and right speaker 525 is selected as the first speaker arranged at a position close to the position of the user's ears. The method of selecting the first speaker may include a configuration for detecting right speaker 520 arranged at a position closest to the position of the user's ears. Alternatively, the method may be configured to select the first speaker by an operation by the user.

[0018] Also, any one of the right speaker 521, the right speaker 522, the right speaker 523, the right speaker 524, and the right speaker 525 is selected as the second speaker. The second speaker is a speaker that is arranged at a position different from the first speaker and arranged at a position farther away from the position of the user's ear than the first speaker. In this embodiment, a case will be described in which the right speaker 520 arranged at a position next to the first speaker closest to the user's ear is selected as the second speaker. The right speaker 520 selected as the second speaker does not have to be the right speaker 520 arranged at a position next to the first speaker closest to the user's ear. For example, the right speaker 520 arranged at a position third closest to the user's ear may be selected as the second speaker.

[0019] A left speaker unit 530 is disposed in the left holding unit 23. The left speaker unit 530 is provided on an inner surface of the left holding unit 23 that faces the face of the user when the user wears the image display unit 20 on the head. The left speaker unit 530 may also be provided on the lower surface of the left holding unit 23. The left speaker unit 530 includes five speakers, a left speaker 541, a left speaker 542, a left speaker 543, a left speaker 544, and a left speaker 545. Hereinafter, the left speaker 541, the left speaker 542, the left speaker 543, the left speaker 544, and the left speaker 545 are collectively referred to as a left speaker 540. The left speaker 541, the left speaker 542, the left speaker 543, the left speaker 544, and the left speaker 545 are disposed in a row at predetermined intervals along the extension direction of the left holding unit 23. In this embodiment, a case will be described in which the left speaker 540 includes five speakers, left speakers 541 to 545, but the number of left speakers 540 may be two or more.

[0020] One of left speaker 541, left speaker 542, left speaker 543, left speaker 544, and left speaker 545 is selected as the first speaker located at a position close to the position of the user's ear. The method of selecting the first speaker is the same as that of right speaker 520.

[0021] Also, any one of left speaker 541, left speaker 542, left speaker 543, left speaker 544, and left speaker 545 is selected as the second speaker. The second speaker is a speaker that is arranged at a position different from the first speaker and arranged at a position farther away from the position of the user's ear than the first speaker. In this embodiment, a case will be described in which left speaker 540 arranged at a position next to the first speaker closest to the user's ear is selected as the second speaker. The left speaker 540 selected as the second speaker does not have to be left speaker 540 arranged at a position next to the first speaker closest to the user's ear. For example, left speaker 540 arranged at a position third closest to the user's ear may be selected as the second speaker.

[0022] The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is located in front of the right eye of the user when the image display unit 20 is attached, allowing the right eye to view an image. The left light guide plate 28 is located in front of the left eye of the user when the image display unit 20 is attached, allowing the left eye to view an image. The right light guide plate 26 and the left light guide plate 28 are optical units formed of a light-transmitting resin or the like, and guide image light output by the right display unit 22 and the left display unit 24 to the user's eyes. The right light guide plate 26 and the left light guide plate 28 are, for example, prisms.

[0023] The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other, and this connection position corresponds to the space between the user's eyebrows when the user wears the image display unit 20. The front frame 27 may be provided with a nose pad that comes into contact with the user's nose when the image display unit 20 is worn, or may be configured such that a belt is connected to the right holding portion 21 and the left holding portion 23 to hold the image display unit 20 on the user's head.

[0024] The right display unit 22 and the left display unit 24 are each a module in which an optical unit and peripheral circuits are unitized. The right display unit 22 displays an image using a right light guide plate 26, and the left display unit 24 displays an image using a left light guide plate 28. The right display unit 22 is provided in the right holding unit 21, and the left display unit 24 is provided in the left holding unit 23. The right display unit 22 and the left display unit 24 are each connected to the connection device 10 by a connection cable 40.

[0025] The image light guided by the right light guide plate 26 and the external light transmitted through the right light guide plate 26 are incident on the right eye of the user. Similarly, the image light guided by the left light guide plate 28 and the external light transmitted through the left light guide plate 28 are incident on the left eye. The image light from the right light guide plate 26 and the left light guide plate 28 and the external light transmitted through the right light guide plate 26 and the left light guide plate 28 are incident on the user's eye. In other words, the user can view the external scene while wearing the image display unit 20 on the head. As a result, the user views the image displayed by the image display unit 20 superimposed on the external scene transmitted through the right light guide plate 26 and the left light guide plate 28.

[0026] A DP illuminance sensor 65, a DP outer camera 61 and an LED indicator 67 are disposed on the right end of the front frame 27.

[0027] The DP illuminance sensor 65 is a sensor that receives external light from the front of a user wearing the image display unit 20. The DP illuminance sensor 65 can detect the illuminance and amount of external light that passes through the right light guide plate 26 and the left light guide plate 28 and enters the user's eyes.

[0028] The DP outer camera 61 is provided in the front frame 27 at a position that does not block external light passing through the right light guide plate 26 and the left light guide plate 28. The DP outer camera 61 is a digital camera equipped with an imaging element such as a CCD or a CMOS, an imaging lens, and the like, and may be a monocular camera or a stereo camera. The angle of view of the DP outer camera 61 includes at least a part of the range of the external scene that a user wearing the image display unit 20 visually recognizes through the right light guide plate 26 and the left light guide plate 28. The DP outer camera 61 may be a wide-angle camera, and may be capable of capturing an image of the entire external scene that a user wearing the image display unit 20 visually recognizes. CCD stands for Charge Coupled Device, and CMOS stands for Complementary Metal Oxide Semiconductor. An LED indicator 67 that lights up when the DP outer camera 61 is in operation is disposed on the front frame 27 .

[0029] The front frame 27 is provided with a distance sensor 64 that detects the distance to a measurement target located in a preset measurement direction. The distance sensor 64 is, for example, a light reflection type distance sensor using an LED or a laser diode, an infrared depth sensor, an ultrasonic distance sensor, or a laser range scanner. The distance sensor 64 may be a distance detection unit that combines image detection and sound detection, or a device that detects the distance by processing images obtained by stereo photography using a camera. The measurement direction of the distance sensor 64 is, for example, the direction of the outside scene visually recognized by the user through the right light guide plate 26 and the left light guide plate 28.

[0030] [2. Optical system configuration of the image display unit] Fig. 2 is a plan view of the main part showing the configuration of the optical system of the image display unit 20. For the purpose of explanation, Fig. 2 illustrates the left eye LE and right eye RE of the user. The right display section 22 and the left display section 24 are configured, for example, symmetrically. As a configuration for allowing the right eye RE to view an image, the right display unit 22 includes an OLED unit 221 that emits image light, and a right optical system 251 that guides the image light L emitted by the OLED unit 221 to the right light guide plate 26. OLED is an abbreviation for Organic Light Emitting Diode.

[0031] The OLED unit 221 has an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display panel in which light-emitting elements that emit, for example, R, G, and B color lights are arranged. The OLED drive circuit 225 drives the OLED panel 223 under the control of the DP control unit 120. The OLED drive circuit 225 is mounted on, for example, a substrate (not shown) that is fixed to the rear surface of the OLED panel 223, and the temperature sensor 217 shown in FIG. 3 is mounted on this substrate.

[0032] The right optical system 251 converts the image light L emitted from the OLED panel 223 into a parallel light beam by a collimator lens, and causes the image light L to enter the right light guide plate 26. Inside the right light guide plate 26, the image light L is reflected by a plurality of reflecting surfaces, reflected by a half mirror 261 located in front of the right eye RE, and emitted from the right light guide plate 26 toward the right eye RE.

[0033] As a configuration for allowing the left eye LE to view an image, the right display unit 22 includes an OLED unit 241 that emits image light, and a left optical system 252 that guides the image light L emitted by the OLED unit 241 to the left light guide plate .

[0034] The OLED unit 241 has an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The OLED panel 243 is a self-luminous display panel in which light-emitting elements that emit, for example, R, G, and B color lights are arranged. The OLED drive circuit 245 drives the OLED panel 243 under the control of the DP control unit 120. The OLED drive circuit 245 is mounted on a substrate (not shown) that is fixed to the rear surface of the OLED panel 243, for example, and a temperature sensor 239 shown in FIG. 3 is mounted on this substrate.

[0035] The left optical system 252 converts the image light L emitted from the OLED panel 243 into a parallel light beam by a collimator lens, and causes the image light L to enter the left light guide plate 28. Inside the left light guide plate 28, the image light L is reflected by a plurality of reflecting surfaces, reflected by a half mirror 261 located in front of the left eye LE, and emitted from the left light guide plate 28 toward the left eye LE.

[0036] The HMD100 functions as a transmissive display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the right eye RE of the user. The image light L reflected by the half mirror 281 and the external light OL transmitted through the half mirror 281 are incident on the left eye LE. The HMD100 superimposes the image light L of an image processed inside and the external light OL and causes them to enter the user's eye. Therefore, the user can see the outside scene through the right light guide plate 26 and the left light guide plate 28, and an image by the image light L is visually recognized superimposed on this outside scene. The half mirrors 261 and 281 are image extraction units that reflect the image light output by the right display unit 22 and the left display unit 24, respectively, and extract the image, and constitute the display unit.

[0037] [3. HMD control system] FIG. 3 is a block diagram of the display system 1, and in particular shows the configuration of the HMD 100 in detail. In the image display unit 20, the right display unit 22 has a right display unit board 210. On the right display unit board 210, a right I / F unit 211 connected to the connection cable 40, a receiving unit 213 that receives data input from the connection device 10 via the right I / F unit 211, and an EEPROM 215 are mounted. The right I / F unit 211 connects the receiving unit 213, the EEPROM 215, the temperature sensor 217, the DP outer camera 61, the distance sensor 64, the DP illuminance sensor 65, the LED indicator 67, and the right speaker unit 510 to the connection device 10. The receiving unit 213 connects the OLED unit 221 to the connection device 10.

[0038] The left display unit 24 has a left display unit board 230. On the left display unit board 230, a left I / F unit 231 connected to the connection cable 40 and a receiving unit 233 that receives data input from the connection device 10 via the left I / F unit 231 are mounted. On the left display unit board 230, a DP 6-axis sensor 235 and a DP magnetic sensor 237 are mounted. The left I / F unit 231 connects the receiving unit 233, the DP 6-axis sensor 235, the DP magnetic sensor 237, the temperature sensor 239, and the left speaker unit 530 to the connection device 10. The receiving unit 233 connects the OLED unit 241 to the connection device 10.

[0039] In the description and drawings of this embodiment, I / F is an abbreviation for interface. EEPROM is an abbreviation for Electrically Erasable Programmable Read-Only Memory. The receiving unit 213 and the receiving unit 233 may be written as Rx213 and Rx233, respectively.

[0040] The EEPROM 215 stores various data in a non-volatile manner. For example, the EEPROM 215 stores data related to the light emission characteristics and display characteristics of the OLED units 221 and 241 included in the image display unit 20, data related to the characteristics of the sensors included in the right display unit 22 or the left display unit 24, etc. Specifically, the EEPROM 215 stores parameters related to gamma correction of the OLED units 221 and 241, data for compensating for the detection values ​​of the temperature sensors 217 and 239, etc. in a manner readable by the DP control unit 120.

[0041] The DP outer camera 61 captures an image according to a signal input via the right I / F unit 211, and outputs the captured image to the right I / F unit 211. The DP illuminance sensor 65 receives external light, and outputs a detection value corresponding to the amount or intensity of received light. The LED indicator 67 lights up according to a control signal or drive current input via the right I / F unit 211. The temperature sensor 217 detects the temperature of the OLED unit 221, and outputs a voltage value or a resistance value corresponding to the detected temperature as a detection value.

[0042] The distance sensor 64 outputs a signal indicating the result of the distance detection to the connection device 10 via the right I / F unit 211.

[0043] The receiving unit 213 receives image data for display transmitted from the connection device 10 via the right I / F unit 211, and outputs the image data to the OLED unit 221. The OLED unit 221 displays an image based on the image data transmitted by the connection device 10. The receiving unit 233 receives image data for display transmitted from the connection device 10 via the left I / F unit 231, and outputs the image data to the OLED unit 241. The OLED units 221 and 241 display images based on the image data transmitted by the connection device 10.

[0044] The DP six-axis sensor 235 is a motion sensor including a three-axis acceleration sensor and a three-axis gyro sensor. The DP magnetic sensor 237 is, for example, a three-axis geomagnetic sensor. The DP six-axis sensor 235 and the DP magnetic sensor 237 may be an IMU in which the above-mentioned sensors are modularized, or may be a module in which the DP six-axis sensor 235 and the DP magnetic sensor 237 are integrated. IMU is an abbreviation of Inertial Measurement Unit. The temperature sensor 239 detects the temperature of the OLED unit 241. The DP six-axis sensor 235, the DP magnetic sensor 237, and the temperature sensor 239 each output a detection value to the connection device 10.

[0045] Each part of the image display unit 20 operates with power supplied from the connection device 10 via the connection cable 40. The image display unit 20 includes a power supply unit 229 in the right display unit 22 and a power supply unit 249 in the left display unit 24. The power supply unit 229 distributes and supplies the power supplied by the connection device 10 via the connection cable 40 to each part of the right display unit 22, including the right display unit board 210. The power supply unit 249 distributes and supplies the power supplied by the connection device 10 via the connection cable 40 to each part of the left display unit 24, including the left display unit board 230. The power supply units 229, 249 may include a conversion circuit that converts voltage, or the like.

[0046] The right speaker 521, the right speaker 522, the right speaker 523, the right speaker 524, and the right speaker 525 each include a D / A converter and an amplifier. The D / A converter and the amplifier are not shown. The audio signal is D / A converted by the D / A converter and amplified by the amplifier and output from the right speaker 520.

[0047] Detector 515 is disposed adjacent to right speaker 521. Detector 150 of this embodiment is configured with a microphone. Detector 515 detects a reflected wave of ultrasonic waves output from right speaker 521 reflected by the user's ear. When detector 515 detects ultrasonic waves, it outputs a notification signal notifying DP control unit 120 that ultrasonic waves have been detected.

[0048] The left speaker 541, the left speaker 542, the left speaker 543, the left speaker 544, and the left speaker 545 each include a D / A converter and an amplifier. The D / A converter and the amplifier are not shown. The audio signal that is D / A converted by the D / A converter and amplified by the amplifier is output from the left speaker 540.

[0049] Detector 535 is disposed adjacent to left speaker 541. Detector 535 is configured with a microphone. Detector 535 detects a reflected wave of an ultrasonic wave output from left speaker 541 reflected by the ear of the user. When detector 535 detects an ultrasonic wave, it outputs a notification signal notifying DP control unit 120 that the ultrasonic wave has been detected.

[0050] The connection device 10 includes an I / F unit 110, a DP control unit 120, a sensor control unit 122, a display control unit 123, a power control unit 126, a non-volatile storage unit 130, an operation unit 140, a connection unit 145, and an audio processing unit 147.

[0051] The I / F unit 110 includes a connector 11D and an interface circuit that executes a communication protocol conforming to various communication standards via the connector 11D. The I / F unit 110 is, for example, an interface board on which the connector 11D and the interface circuit are mounted. The I / F unit 110 may include an interface for a memory card to which an external storage device or storage medium can be connected, or the I / F unit 110 may be configured as a wireless communication interface.

[0052] The DP control unit 120 includes a processor such as a CPU or a microcomputer, and the processor executes a program to control each part of the connection device 10. The DP control unit 120 may include a RAM that constitutes the work area of ​​the processor. RAM is an abbreviation for Random Access Memory.

[0053] The DP control unit 120 is connected to a non-volatile storage unit 130, an operation unit 140, a connection unit 145, and an audio processing unit 147. The non-volatile storage unit 130 is a ROM that non-volatilely stores programs and data executed by the DP control unit 120. ROM is an abbreviation for Read Only Memory. The non-volatile storage unit 130 stores distance information. The distance information is information on the distance from a reference position of each right speaker 520. In this embodiment, the reference position is set to the position of the right speaker 521. The distance information is information on the distance from the right speaker 521 to each of the right speakers 520, that is, the right speaker 522, the right speaker 523, the right speaker 524, and the right speaker 525. The distance information is information on the distance from a reference position of each left speaker 540. In this embodiment, the reference position is set to the position of left speaker 541. The distance information is information on the distance from left speaker 541 to each of left speakers 540, that is, left speaker 542, left speaker 543, left speaker 544, and left speaker 545. In this embodiment, the reference positions are set to the positions of right speaker 521 and left speaker 541, but the reference positions are not limited to the positions of right speaker 521 and left speaker 541. For example, the positions of right speaker 523 and left speaker 543 may be set to the reference positions, or may be set to preset positions of right holding part 21 and left holding part 23.

[0054] The sensor control unit 122 operates each sensor included in the image display unit 20. Here, each sensor refers to each of the DP outer camera 61, the distance sensor 64, the DP illuminance sensor 65, the temperature sensor 217, the DP 6-axis sensor 235, the DP magnetic sensor 237, and the temperature sensor 239. Each sensor includes at least one or more of the DP outer camera 61, the DP illuminance sensor 65, the DP 6-axis sensor 235, and the DP magnetic sensor 237. The sensor control unit 122 sets and initializes the sampling period of each sensor according to the control of the DP control unit 120, and executes energization of each sensor, transmission of control data, acquisition of detection values, etc. in accordance with the sampling period of each sensor.

[0055] The sensor control unit 122 outputs detection data indicating the detection values ​​and detection results of each sensor to the I / F unit 110 at a preset timing. The sensor control unit 122 may include an A / D converter that converts analog signals into digital data. In this case, the sensor control unit 122 converts the analog signals of the detection values ​​and detection results acquired from the sensors of the image display unit 20 into detection data and outputs it. The sensor control unit 122 may acquire digital data of the detection values ​​and detection results from the sensors of the image display unit 20, convert the data format, adjust the output timing, etc., and output the detection data to the I / F unit 110.

[0056] By operation of the sensor control unit 122 , the control device 300 connected to the I / F unit 110 can obtain detection values ​​of each sensor of the HMD 100 and captured images of the DP outer camera 61 .

[0057] The sensor control unit 122 may output the result of arithmetic processing based on the detection values ​​of each of the above-mentioned sensors as detection data. For example, the sensor control unit 122 may integrally process the detection values ​​and detection results of a plurality of sensors and function as a so-called sensor fusion processing unit. In this case, the sensor control unit 122 may generate detection data of a virtual sensor not included in each sensor of the image display unit 20 by sensor fusion. For example, the sensor control unit 122 may output trajectory data indicating the trajectory of the movement of the image display unit 20, coordinate data indicating the position of the image display unit 20 in a three-dimensional space, and direction data indicating the direction of the image display unit 20 as detection data. Here, the coordinate data may be data indicating relative coordinates based on the position of the connection device 10, or may be data indicating a position relative to a reference position set in a space in which the image display unit 20 exists. The direction data may be data indicating a direction based on the position or direction of the connection device 10, or may be data indicating a direction relative to a reference position set in a space in which the image display unit 20 exists. The sensor control unit 122 executes a communication protocol between the device connected to the connector 11D via the USB cable 46, and outputs detection data.

[0058] The display control unit 123 executes various processes for displaying an image based on the image data input to the I / F unit 110 on the image display unit 20. In this embodiment, an image signal output by the control device 300 is input to the connector 11D. The display control unit 123 executes various processes such as frame extraction, resolution conversion, intermediate frame generation, frame rate conversion, etc. Resolution conversion includes so-called scaling. The display control unit 123 outputs video data corresponding to each of the OLED unit 221 and the OLED unit 241 to the connection unit 145. The video data input to the connection unit 145 is transmitted from the connector 11A to the right I / F unit 211 and the left I / F unit 231 as a video signal.

[0059] In this embodiment, the connector 11D is configured as a USB-Type C connector. The display control unit 123 receives video data transmitted in the alternative mode of USB-Type C via the connector 11D.

[0060] The sensor control unit 122 and the display control unit 123 may be realized by a collaboration between software and hardware by a processor executing a program. That is, the sensor control unit 122 and the display control unit 123 are configured by a processor, and execute the above operations by executing a program. In this example, the sensor control unit 122 and the display control unit 123 may be realized by a processor configuring the DP control unit 120 executing a program. In other words, the processor may execute a program to function as the DP control unit 120, the display control unit 123, and the sensor control unit 122. The sensor control unit 122 and the display control unit 123 may also be configured by programmed hardware such as a DSP or FPGA. DSP stands for Digital Signal Processor. FFPGA stands for Field Programmable Gate Array.

[0061] The power supply control unit 126 is connected to the connector 11D. The power supply control unit 126 supplies power to each unit of the connection device 10 and the image display unit 20 based on the power supplied from the connector 11D. The power supply control unit 126 may also be configured to include a voltage conversion circuit (not shown) and convert the voltage and supply it to each unit of the connection device 10 and the image display unit 20. The power supply control unit 126 may be configured as a programmed semiconductor device such as a logic circuit or an FPGA. The power supply control unit 126 may also be configured as hardware common to the sensor control unit 122 and the display control unit 123.

[0062] The sensor control unit 122, the display control unit 123, and the power control unit 126 may be provided with a work memory for performing data processing, or may perform processing using the memory of the DP control unit 120.

[0063] The operation unit 140 detects an operation on an operated unit included in the connection device 10, and outputs to the DP control unit 120 data indicating the operation content or an operation signal indicating the operated operated unit.

[0064] The audio processing unit 147 generates an audio signal according to the audio data input from the DP control unit 120, and outputs the audio signal to the connection unit 145. This audio signal is output from the connection unit 145 to the right speaker unit 510 and the left speaker unit 530. The audio processing unit 147 also adjusts the volume of the audio signal according to the control of the DP control unit 120.

[0065] [4. Control device configuration] FIG. 4 is a block diagram of the control device 300. The control device 300 includes a CO control unit 310. The CO control unit 310 includes a processor 311, a memory 312, and a non-volatile memory 313. The processor 311 is composed of a CPU, a microcomputer, a DSP, etc., and controls each unit of the control device 300 by executing a program. The memory 312 forms a work area for the processor 311. The non-volatile memory 313 is composed of a semiconductor memory device, etc., and stores in a non-volatile manner the programs executed by the processor 311 and various data processed by the processor 311. For example, the non-volatile memory 313 stores an operating system as a basic control program executed by the processor 311, application programs that run on the operating system, etc. The non-volatile memory 313 stores data that is processed when the application program is executed and data of the processing result. The operating system is hereinafter abbreviated as OS. The CO control unit 310 may be an SoC that integrates a processor 311, a memory 312, and a non-volatile memory 313.

[0066] The CO control unit 310 is connected to a GNSS 321, a CO camera 322, a CO six-axis sensor 323, a CO magnetic sensor 324, a CO illuminance sensor 325, an audio output unit 326, a CO display unit 327, a battery 329, a communication unit 330, and an I / F unit 331.

[0067] The GNSS 321 performs positioning using a satellite positioning system, and outputs the position of the control device 300 to the CO control unit 310. GNSS is an abbreviation for Global Navigation Satellite System.

[0068] CO camera 322 is a digital camera provided on the main body of control device 300, and is disposed, for example, adjacent to touch panel 328 and captures an image in the direction facing touch panel 328. CO camera 322 captures images according to the control of CO control unit 310, and outputs the captured images to CO control unit 310.

[0069] The CO six-axis sensor 323 is a motion sensor equipped with a three-axis acceleration sensor and a three-axis gyro sensor, and outputs detection data indicating the detection values ​​to the CO control unit 310.

[0070] The CO magnetic sensor 324 is, for example, a three-axis geomagnetic sensor, and outputs detection data indicating a detection value to the CO control unit 310. The CO six-axis sensor 323 and the CO magnetic sensor 324 may be an IMU in which the above-mentioned sensors are modularized, or the CO six-axis sensor 323 and the CO magnetic sensor 324 may be integrated into a module.

[0071] The CO illuminance sensor 325 receives external light and outputs to the CO control unit 310 detection data indicating a detection value corresponding to the amount or intensity of received light.

[0072] The audio output unit 326 includes a speaker and outputs audio from the speaker under the control of the CO control unit 310. The audio output unit 326 may include an amplifier that amplifies the audio signal output by the CO control unit 310 and outputs the amplified audio signal to the speaker. When the CO control unit 310 is configured to output digital audio data, the audio output unit 326 may include a D / A converter that converts the digital audio data into an analog audio signal.

[0073] The CO display unit 327 has a touch panel 328 and displays characters and images on the touch panel 328 under the control of the CO control unit 310.

[0074] The battery 329 is a secondary battery built into the main body of the control device 300, and supplies power to each unit of the control device 300. The battery 329 may include a control circuit (not shown) that controls the output of power and the charging of the secondary battery.

[0075] The communication unit 330 supports wireless communication protocols such as Bluetooth and Wi-Fi, and performs wireless communication with devices external to the display system 1. Bluetooth and Wi-Fi are registered trademarks. The communication unit 330 may be configured to perform mobile data communication using a mobile communication network such as LTE or a fifth generation mobile communication system. LTE is a registered trademark.

[0076] The I / F unit 331 includes a connector (not shown) to which a data communication cable is connected, and an interface circuit that executes a communication protocol conforming to various communication standards via the connector. For example, the I / F unit 331 includes a connector and an interface circuit conforming to the USB standard, and transmits and receives data via the USB cable 46.

[0077] [5. Operation of DP control unit] The DP control unit 120 corresponds to a control unit. When the DP control unit 120 receives a preset operation, it estimates the position of the user's ears to determine the first speaker and the second speaker. The preset operation may be an operation of an operation button provided on the operation unit 140 of the HMD 100, or an operation of the touch panel 328 of the control device 300. The operation button is a button for setting the right speaker unit 510 and the left speaker unit 530 that output sound.

[0078] First, the DP control unit 120 sequentially outputs ultrasonic waves from the preset right speaker 520 and left speaker 540. In this embodiment, the right speaker 521 and the left speaker 541 are set as the preset speakers. The right speaker 521 is provided at a position close to the joint between the right holding part 21 and the front frame 27 or at the right end of the front frame 27 so that the ultrasonic waves output from the right speaker 521 hit the user's ear, particularly the earlobe. In addition, the installation angle of the right speaker 521 is adjusted so that the ultrasonic waves output from the right speaker 521 hit the user's ear, particularly the earlobe. Similarly, left speaker 541 is provided at a position close to the joint between left holding part 23 and front frame 27, or at the left end of front frame 27, so that the ultrasonic waves output from left speaker 541 hit the user's ear, particularly the earlobe. Also, the installation angle of left speaker 541 is adjusted so that the ultrasonic waves output from left speaker 541 hit the user's ear, particularly the earlobe.

[0079] The ultrasonic waves output from the right speaker 521 are detected by the detector 515, and the ultrasonic waves output from the left speaker 541 are detected by the detector 535. When the detectors 515 and 535 detect ultrasonic waves, they each output a notification signal notifying the DP control unit 120 that the ultrasonic waves have been detected.

[0080] The DP control unit 120 estimates the positions of the left and right ears of the user based on the time from when the right speaker 521 and the left speaker 541 output ultrasonic waves to when the ultrasonic waves are detected by the detectors 515 and 535. The DP control unit 120 estimates the distance from the right speaker 521 to the right ear, and estimates the distance from the left speaker 541 to the left ear.

[0081] After estimating the positions of the left and right ears of the user, the DP control unit 120 determines the first speaker and the second speaker based on the estimated positions of the left and right ears and the distance information stored in the non-volatile storage unit 130. The first speaker is the speaker closest to the position of the user's ears, and the second speaker is the speaker next closest to the position of the user's ears after the first speaker. The DP control unit 120 compares the estimated distance from the right speaker 521 to the right ear with information on the distance from the right speaker 521 to each right speaker 520 indicated by the distance information, and determines the first speaker and the second speaker. Similarly, the DP control unit 120 compares the estimated distance from the left speaker 541 to the left ear with information on the distance from the left speaker 541 to each left speaker 540 indicated by the distance information, to determine the first speaker and the second speaker.

[0082] Also, the DP control unit 120 may select multiple right speakers 520 as the first speakers, and select multiple right speakers 520 as the second speakers. Similarly, the DP control unit 120 may select multiple left speakers 540 as the first speakers, and select multiple left speakers 540 as the second speakers. For example, the DP control unit 120 obtains the difference between the estimated distance from the right speaker 521 to the right ear and the distance from the right speaker 521 to each right speaker 520 indicated by the distance information. The DP control unit 120 determines the right speaker 520 for which the obtained difference is equal to or less than a preset first threshold as the first speaker, and determines the right speaker 520 for which the obtained difference is greater than the preset first threshold and equal to or less than a preset second threshold as the second speaker. The second threshold is set to a value greater than the first threshold. Similarly, the DP control unit 120 finds the difference between the estimated distance from the left speaker 541 to the left ear and the distance from the left speaker 541 indicated by the distance information to each left speaker 540. The DP control unit 120 determines the left speaker 540 for which the found difference is within a preset first threshold as the first speaker, and determines the left speaker 540 for which the found difference is greater than the preset first threshold and less than or equal to a preset second threshold as the second speaker.

[0083] When the control device 300 starts playing back content and audio data is input from the control device 300, the DP control unit 120 sets the output destination of the audio data to the determined first speaker and second speaker. The DP control unit 120 controls the audio processing unit 147 and the right I / F unit 211 to set the right speaker 520 from which the audio data is output to the right speaker 520 determined as the first speaker and the right speaker 520 determined as the second speaker. Similarly, the DP control unit 120 controls the audio processing unit 147 and the left I / F unit 231 to set the left speaker 540 from which the audio data is output to the left speaker 540 determined as the first speaker and the left speaker 540 determined as the second speaker. Since the speakers from which sound is output are limited to the first speaker closest to the user's ears and the second speaker next closest to the user's ears after the first speaker, sound leakage to the user's surroundings can be reduced.

[0084] Furthermore, the DP control unit 120 outputs the sound by differentiating the sound settings for the first speaker and the second speaker. The DP control unit 120 sets the amplifier output of the right speaker 520 determined as the first speaker and the right speaker 520 determined as the second speaker, and changes the sound pressure of the sound output from the first speaker and the sound pressure of the sound output from the second speaker. Similarly, the DP control unit 120 sets the amplifier output of the left speaker 540 determined as the first speaker and the left speaker 540 determined as the second speaker, and changes the sound pressure of the sound output from the first speaker and the sound pressure of the sound output from the second speaker. For example, assume that the right speaker 523 is set as the first speaker and the right speaker 522 is set as the second speaker. Similarly, assume that the left speaker 543 is set as the first speaker and the left speaker 542 is set as the second speaker. The DP control unit 120 sets the amplifiers of right speaker 523, which is the first speaker, so that the sound pressure of these speakers is greater than the sound pressure of right speaker 522, which is the second speaker. Similarly, the DP control unit 120 sets the amplifiers of these speakers so that the sound pressure of left speaker 543, which is the first speaker, is greater than the sound pressure of left speaker 542, which is the second speaker.

[0085] Furthermore, when audio data is input from the control device 300, the DP control section 120 causes the audio processing section 147 to perform equalizer processing on the input audio data, thereby changing the frequency of the audio output to the first speaker and the second speaker. For example, assume that the right speaker 523 is set as the first speaker and the right speaker 522 is set as the second speaker. Similarly, assume that the left speaker 543 is set as the first speaker and the left speaker 542 is set as the second speaker. Audio processing unit 147 performs equalizer processing on the audio data so that the frequency of the audio output from right speaker 523, which is the first speaker, is lower than the frequency of the audio output from right speaker 522, which is the second speaker. Specifically, audio processing unit 147 outputs audio in a low frequency band to right speaker 523, which is the first speaker, and outputs audio in a high frequency band to right speaker 522, which is the second speaker. Similarly, audio processing unit 147 performs equalizer processing on the audio data so that the frequency of the audio output from left speaker 543, the first speaker, is lower than the frequency of the audio output from left speaker 542, the second speaker. Specifically, audio processing unit 147 outputs audio in a low frequency band to left speaker 543, the first speaker, and outputs audio in a high frequency band to left speaker 542, the second speaker.

[0086] Since low-frequency sound is output from the first speaker, which is closer to the user's ears, the user can easily hear the low-frequency sound. Also, high-frequency sound is output from the second speaker, which is located further away from the first speaker. Since high-frequency sound is easier for the user to hear, the user can still hear the sound even if high-frequency sound is output from the second speaker, which is farther away from the first speaker. In addition, since low-frequency band sound is output from the first speaker and high-frequency band sound is output from the second speaker, sound without frequency bias can be heard.

[0087] The DP control unit 120 may also output sound from speakers other than the first speaker and the second speaker. In this case, the DP control unit 120 controls the sound processing unit 147, the right I / F unit 211, and the left I / F unit 231 so that sound is not output from speakers arranged behind the user's ears. Sound output from speakers arranged behind the ears is difficult for the user to hear, and may cause sound leakage to the user's surroundings. The DP control unit 120 detects the distance from the right speaker 521 to the right ear, and therefore can determine the right speaker 520 arranged behind the right ear. Similarly, the DP control unit 120 can determine the left speaker 540 arranged behind the left ear. When the DP control unit 120 determines that the right speaker is located behind the right ear and the left speaker is located behind the left ear, it controls the audio processing unit 147, the right I / F unit 211, and the left I / F unit 231 so that no audio is output from these speakers.

[0088] FIG. 5 is a flowchart showing the operation of the DP control unit 120. The operation of the DP control unit 120 will be described with reference to the flow chart shown in FIG. First, the DP control unit 120 determines whether or not a setting button for setting the right speaker unit 510 and the left speaker unit 530 has been pressed (step S1). If the setting button has not been pressed (step S1 / NO), the DP control unit 120 waits until the setting button is pressed.

[0089] When the setting button is pressed (step S1 / YES), the DP control unit 120 detects the distance from the right speaker 521, which is the reference position, to the right ear, and the distance from the left speaker 541, which is the reference position, to the left ear (step S2). DP control unit 120 causes right speaker 521 and left speaker 541 to sequentially output ultrasonic waves, and causes detectors 515 and 535 to detect reflected waves of the ultrasonic waves reflected by the user's ears. DP control unit 120 detects the distance from right speaker 521 and left speaker 541 to the ears based on the time from when ultrasonic waves are output from right speaker 521 and left speaker 541 to when the reflected waves are detected by detectors 515 and 535.

[0090] When the DP control unit 120 detects the distance from the right speaker 521 to the right ear, it determines the first speaker and the second speaker based on the information of the detected distance to the right ear and the information of the distance from the right speaker 521 to each right speaker 520 (step S3). In addition, the DP control unit 120 determines the right speakers 520 other than the right speakers 520 determined as the first speaker and the second speaker as right speakers 520 that do not output sound (step S3).

[0091] Similarly, when the DP control unit 120 detects the distance from the left speaker 541 to the left ear, it determines the first speaker and the second speaker (step S3). The DP control unit 120 determines the first speaker and the second speaker based on information on the detected distance to the left ear and information on the distance from the left speaker 541 to each left speaker 540. In addition, the DP control unit 120 determines the left speakers 540 other than the left speaker 540 determined as the first speaker and the second speaker as left speakers 540 that do not output sound (step S3).

[0092] Next, the DP control unit 120 judges whether or not a notification to start playing the content has been received from the CO control unit 310 (step S4). If the notification to start playing the content has not been received (step S4 / NO), the DP control unit 120 waits until a notification to start playing the content is received.

[0093] Furthermore, when the DP control unit 120 receives a notification from the CO control unit 310 to start playing the content (step S4 / YES), it causes the right speaker 520 and the left speaker 540 determined as the first speaker to output sound. The DP control unit 120 causes the right speaker 520 and the left speaker 540 determined as the first speaker to output sound in the set frequency band at the set sound pressure (step S5). For example, it is assumed that the right speaker 523 and the left speaker 543 are determined as the first speakers, and the right speaker 522 and the left speaker 542 are determined as the second speakers. The DP control unit 120 sets the amplifiers of the right speaker 523 and the left speaker 543 so that the sound pressure of the sound output from these speakers is greater than the sound pressure of the sound output from the right speaker 522 and the left speaker 542. The DP control unit 120 also performs equalizer processing on the audio data so that the frequency of the sound output from the right speaker 523 and the left speaker 543 is lower than the frequency of the sound output from the right speaker 522 and the left speaker 542. Specifically, the DP control unit 120 causes the right speaker 523 and the left speaker 543 to output low-frequency band sound, and causes the right speaker 522 and the left speaker 542 to output high-frequency band sound.

[0094] Next, the DP control unit 120 judges whether or not the playback of the content has stopped (step S6). If the playback of the content has not stopped (step S6 / NO), the DP control unit 120 returns to step S6. The DP control unit 120 continues outputting sound to the right speaker 520 and the left speaker 540 determined as the first speaker, and the right speaker 520 and the left speaker 540 determined as the second speaker. Furthermore, if the playback of the content has stopped (step S6 / YES), the DP control unit 120 ends this processing flow.

[0095] [6. Modifications] In the above explanation, an example was described in which the right speaker unit 510 is provided in the right holding portion 21 and the left speaker unit 530 is provided in the left holding portion 23, but the installation positions of the right speaker unit 510 and the left speaker unit 530 are not limited to the right holding portion 21 and the left holding portion 23. Fig. 6 is a diagram showing another example of installation of the left speaker unit 530. Fig. 6 shows an HMD 600 that is worn on a wearing part such as the user's head or helmet. 6, the HMD 600 includes a headband unit 610 that is attached to the user's head or a helmet or the like that is attached to the head, and an image display unit 620 that displays a virtual image. The HMD 600 includes a pair of arms 630 that are rotatably attached to the headband unit 610 and connect the headband unit 610 and the image display unit 620. Further, a right speaker unit 710 and a left speaker unit 730 are provided on the arm unit 630. The left speaker unit 730 shown in FIG. The left speakers 741, 742, 743, and 744 are disposed diagonally. That is, the position of the left speaker 741 disposed near the headband unit 610 is disposed further back than the position of the left speaker 744 disposed near the image display unit 620. FIG. 6 shows an example in which the left speakers 741, 742, 743, and 744 are disposed diagonally along the arm unit 630, but the left speakers 741, 742, 743, and 744 may be disposed vertically. The left speakers 741, 742, 743, and 744 may be disposed in a staggered manner. The right speaker unit 710 and the left speaker unit 730 may be disposed at the end of the image display unit 620 near the connection portion with the arm unit 630.

[0096] In addition, in the above description, the position of the user's ears is estimated, the first speaker and the second speaker are determined based on the estimated ear position, and sound is output from only the first speaker and the second speaker, but the number of speakers that output sound may be increased. As an example, the right speaker 520 closest to the user's ear may be set as the first speaker, and all right speakers arranged in front of the user's ear may be set as the second speaker. For example, it is assumed that, among the right speakers 520, the right speaker 525 is arranged behind the user's ear, and the right speaker 524 is arranged at a position closest to the user's ear. In this case, the DP control unit 120 sets the right speaker 524 as the first speaker, and the right speakers 521, 522, and 523 as the second speakers. The DP control unit 120 sets the amplifiers of the right speakers 520 so that the sound pressure of the right speaker 524 is greater than the sound pressure of the right speakers 521, 522, and 523. In addition, the DP control unit 120 controls the sound processing unit 147 so that the right speaker 524 outputs a low-frequency band sound, and the right speakers 521, 522, and 523 output a high-frequency band sound. The DP control unit 120 performs the same process on the left speaker unit 530.

[0097] It is also assumed that the sound pressure of the sound output by the right speaker unit 510 and the left speaker unit 530 is set to 80 dB by the user. In this case, the DP control unit 120 does not cause all of the right speaker unit 510 to output sound at 80 dB, but controls the right speaker 521 to the right speaker 524 so that the total sound pressure output by the right speaker 521 to the right speaker 524 is 80 dB. The DP control unit 120 also sets the right speaker 521 to the right speaker 524 so that the sound pressure is higher as the right speaker is placed closer to the ear and lower as the right speaker is placed farther from the ear. The sound output of the right speaker farther from the user's ear can be significantly reduced to improve the effect of preventing sound leakage. Furthermore, the DP control unit 120 outputs low-frequency sound to the right speaker 520 located near the user's ear, and outputs high-frequency sound to the right speaker 520 located away from the user's ear. This makes it possible to output sound including frequencies from low to high frequencies while suppressing power consumption. The DP control unit 120 performs the same process on the left speaker unit 530.

[0098] In the above-described embodiment, microphones for receiving reflected ultrasonic waves are provided as detectors 515 and 535. The position of the ears is estimated based on the time from when ultrasonic waves are output from right speaker 521 and left speaker 541 to when the reflected waves of the ultrasonic waves reflected by the user's ears are detected. Alternatively, touch sensors may be provided on the right holding part 21 and the left holding part 23 of the image display part 20 to allow the user to touch the position of the ear. The DP control part 120 may determine the first speaker and the second speaker based on the positions of the right holding part 21 and the left holding part 23 touched by the user's hands.

[0099] [7. Effects] As described above, the HMD 100 of this embodiment is equipped with a right speaker unit 510 and a left speaker unit 530 and is equipped with an image display unit 20 that is worn on the user's head, and a DP control unit 120 that controls the right speaker unit 510 and the left speaker unit 530. Right speaker section 510 includes right speaker 521, right speaker 522, right speaker 523, right speaker 524, and right speaker 525. These multiple right speakers include a first speaker arranged in a position close to the user's ear, and a second speaker arranged in a position different from the first speaker and farther away from the position of the user's ear than the first speaker. Similarly, left speaker section 530 includes left speaker 541, left speaker 542, left speaker 543, left speaker 544, and left speaker 545. These multiple left speakers include a first speaker arranged at a position close to the user's ear, and a second speaker arranged at a position different from the first speaker and farther away from the position of the user's ear than the first speaker. The DP control unit 120 outputs the sound by differentiating the sound settings for the first speaker and the second speaker. For example, by making the sound pressure of the second speaker greater than that of the first speaker, the user can easily hear the sound output from the second speaker, which is located away from the user's ears. This eliminates the need to adjust the speaker to match the position of the user's ears, reducing the inconvenience to the user.

[0100] The DP control unit 120 sets the sound pressure of the first speaker higher than the sound pressure of the second speaker. Therefore, the sound pressure of the first speaker, which is closer to the user's ear, is set high, making it easier for the user to hear the sound from the first speaker.

[0101] Furthermore, the DP control unit 120 sets the sound of the first speaker and the second speaker so that the frequency of the sound output from the first speaker is lower than the frequency of the sound output from the second speaker. Therefore, low-frequency sound that is difficult to hear is output from the first speaker close to the user's ear, and high-frequency sound that is easy to hear is output from the second speaker that is farther away than the first speaker, making it possible to produce sound that is easy for the user to understand.

[0102] The image display unit 20 has a eyeglasses shape. The right speaker section 510 and the left speaker section 530 are disposed on the temples of the eyeglass-shaped image display section 20 along the extension direction of the temples. Although the position of the eyes and ears and the distance between the eyes and ears vary from person to person, the speakers can be placed close to the user's ears regardless of individual differences.

[0103] The HMD 100 also includes a detector 515 that detects the reflected sound of the sound wave output from the right speaker 521 , and a detector 535 that detects the reflected sound of the sound wave output from the left speaker 541 . The DP control unit 120 estimates the position of the user's ears based on the time from when sound waves are output from at least one of the multiple speakers to when reflected waves are detected by the detectors 515 and 535. The DP control unit 120 determines the first speaker and the second speaker based on the estimated position of the user's ears. Therefore, the first speaker and the second speaker arranged in positions close to the user's ears can be determined, and the sound heard by the user can be controlled to have an optimal volume and sound quality.

[0104] The DP control unit 120 also estimates the position of the user's ear based on the time from when sound waves are output from at least one of the multiple speakers to when the detector detects the reflected waves. The DP control unit 120 determines a speaker that is located behind the position of the user's ear in the extension direction of the temple based on the estimated position of the user's ear and distance information, and does not output sound from the determined speaker. Therefore, power consumption can be reduced by outputting sound only from speakers that are positioned in front of the user's ears and in a position that makes it easy for the user to hear the sound, and not outputting sound from speakers that are positioned in front of the user's ears.

[0105] The present invention is not limited to the configurations described in the above embodiments, and can be embodied in various forms without departing from the spirit and scope of the present invention. For example, in the above-described embodiment, the right speaker 521 and the left speaker 541 output ultrasonic waves, and the detectors 515 and 535 detect reflected waves of the ultrasonic waves. As another configuration, a light output unit that outputs detection light such as infrared light, and a light receiving sensor that receives the reflected light of the detection light reflected by the user's ear may be provided. Alternatively, the right speaker 521 and the left speaker 541 may output audible sound, and the detectors 515 and 535 may detect reflected waves of the audible sound output from the right speaker 521 and the left speaker 541.

[0106] In addition, in the above-described embodiment, audio output to speakers other than those determined as the first and second speakers is stopped, but it may be set to output audio from all speakers when certain conditions are met, such as when a disaster occurs.

[0107] In addition, the DP control unit 120 changed the amplifier and equalizer processing so that the sound pressure and frequency of the audio output from the speaker determined to be the first speaker are changed from the sound pressure and frequency of the audio output from the speaker determined to be the second speaker. In addition, the DP control unit 120 may set the amplifier so that the sound pressure of the audio to be output from the first speaker and the second speaker is the same, and perform equalizer processing so that the frequencies of the audio to be output from the first speaker and the second speaker are different. In addition, the DP control unit 120 may perform equalizer processing so that the frequencies of the sounds output from the first speaker and the second speaker are the same, and may set the amplifier so that the sound pressures of the sounds output from the first speaker and the second speaker are different.

[0108] As a method for determining the first and second speakers, the right speakers 521-525 may be made to output audio at a constant volume in sequence, and the user, upon hearing the audio output from each of the right speakers 521-525, may operate the operation unit 140 or touch panel 328 to set the speakers to be used as the first and second speakers. Similarly, the left speakers 541-545 may be caused to output audio at a constant volume, and a user who hears the audio output from each of the left speakers 541-545 may operate the operation unit 140 or touch panel 328 to set the speakers to be used as the first speaker and the second speaker.

[0109] Further, the display system 1 is exemplified as having the HMD 100, which is a head-mounted display device, but the present invention is not limited thereto, and various display devices can be adopted. For example, instead of the image display unit 20, other types of image display units, such as an image display unit worn like a hat, may be adopted, and it is sufficient to have a display unit that displays an image corresponding to the left eye of the user and a display unit that displays an image corresponding to the right eye of the user. Furthermore, the display device may be configured as a head-mounted display mounted on a vehicle, such as an automobile or an airplane. Furthermore, it may be configured as a head-mounted display built into a body protector, such as a helmet. In this case, the part that positions the display device relative to the user's body and the part that is positioned relative to the part may be the wearing part.

[0110] The HMD 100 is an example of a display device to which the present invention is applied, and is not limited to the configuration shown in FIG. 3. For example, in the above embodiment, the image display unit 20 and the connection device 10 are separated from each other, but the connection device 10 and the image display unit 20 may be integrally configured and worn on the user's head. The optical system of the image display unit 20 may be configured arbitrarily, and may use, for example, an optical member that is located in front of the user's eyes and overlaps a part or all of the user's field of vision. Alternatively, a scanning optical system that scans laser light or the like to generate image light may be adopted. Alternatively, the optical system may not be limited to one that guides image light inside the optical member, but may only have the function of refracting and / or reflecting the image light toward the user's eyes. The display device may be a liquid crystal monitor or a liquid crystal television that displays an image on a liquid crystal display panel. A display device equipped with a plasma display panel or an organic EL display panel may also be used. The display device may also be a projector that projects image light onto a screen or the like.

[0111] Also, for example, in the HMD 100 shown in FIG. 3, the connection device 10 may be configured using a USB-TypeC connector, a USB-TypeC controller, and a USB hub. In this case, the DP outer camera 61 and other sensors may be connected to the USB hub. Also, as a controller for controlling the display of the right display unit 22 and the left display unit 24 in the image display unit 20, an FPGA that outputs display data to the right display unit 22 and the left display unit 24 may be disposed in either the right display unit 22 or the left display unit 24. In this case, the connection device 10 may include a bridge controller that connects the USB-TypeC controller and the FPGA. Also, in the image display unit 20, the DP 6-axis sensor 235, the DP magnetic sensor 237, the EEPROM 215, and the like may be mounted on the same board as the FPGA. The arrangement of the other sensors can also be changed as appropriate. For example, the distance sensor 64 and the DP illuminance sensor 65 may be disposed at positions suitable for measurement or detection and connected to the FPGA and the USB-TypeC controller. Furthermore, there is no limitation on the specific specifications of the display device including the OLED units 221 and 241. For example, the OLED units 221 and 241 may have a common configuration.

[0112] At least some of the functional blocks shown in Figures 3 and 4 may be realized by hardware, or may be realized by a combination of hardware and software, and are not limited to configurations in which independent hardware resources are arranged as shown in the figures. In addition, the program executed by the processor 311 may be configured to acquire and execute a program stored in an external device via the communication unit 330 or the I / F unit 331. [Explanation of symbols]

[0113] 1...display system, 10...connection device, 11A...connector, 11D...connector, 20...image display unit, 21...right holding unit, 22...right display unit, 23...left holding unit, 24...left display unit, 26...right light guide plate, 27...front frame, 28...left light guide plate, 40...connection cable, 46...USB cable, 61...DP outer camera, 64...distance sensor, 65...DP illuminance sensor, 67...LED indicator, 100...head-mounted display device, 110...I / F unit, 120...DP control unit, 122...sensor control unit, 1 23...display control unit, 126...power control unit, 130...non-volatile storage unit, 140...operation unit, 145...connection unit, 147...audio processing unit, 210...right display unit board, 211...right I / F unit, 213...receiving unit, 215...EEPROM, 217...temperature sensor, 221...OLED unit, 223...OLED panel, 225...OLED driving circuit, 229...power supply unit, 230...left display unit board, 231...left I / F unit, 233...receiving unit, 235...DP 6-axis sensor, 237...DP magnetic sensor, 239...temperature sensor , 241...OLED unit, 243...OLED panel, 245...OLED driving circuit, 249...power supply unit, 251...right optical system, 252...left optical system, 261...half mirror, 281...half mirror, 300...control device, 310...CO control unit, 311...processor, 312...memory, 313...non-volatile memory, 321...GNSS, 322...CO camera, 323...CO 6-axis sensor, 324...CO magnetic sensor, 325...CO illuminance sensor, 326...audio output unit, 327...CO display unit, 32 8...touch panel, 329...battery, 330...communication unit, 331...I / F unit, 510...right speaker unit, 515...detector, 520, 521, 522, 523, 524, 525...right speaker, 530...left speaker unit, 535...detector, 541, 542, 543, 544, 545...left speaker, 600...HMD, 610...headband unit, 620...image display unit, 630...arm unit, 710...right speaker unit, 730...left speaker unit, 741, 742, 743, 744...speakers.

Claims

1. A main body that is attached to the user's head and includes a plurality of speakers; A control unit that controls the plurality of speakers; a detector for detecting a reflected wave of a sound wave output from at least one of the plurality of speakers; And, The plurality of speakers include: A first speaker disposed near the position of the user's ear; The first speaker is disposed at a different position from the first speaker, and the first speaker is positioned so that the first speaker can be a second speaker disposed at a position farther away than the speaker, The control unit controls the first speaker and the second speaker to have different audio settings. This will output the audio. The control unit causes at least one of the plurality of speakers to output a sound wave, and then estimating the position of the user's ear based on the time it takes for the detector to detect the reflected wave; The first speaker and the second speaker are connected to each other based on the estimated position of the user's ears.

2. An audio output device.

2. The control unit sets the sound pressure of the first speaker higher than the sound pressure of the second speaker. The audio output device according to claim 1 .

3. The control unit is configured to control a frequency of the sound output from the first speaker to a frequency of the sound output from the second speaker. The frequency of the sound output from the first speaker and the second speaker is set to be lower than the frequency of the sound output from the 3. The audio output device according to claim 1, further comprising a speaker for audio setting.

4. The main body has a eyeglass shape, The plurality of speakers are attached to the temples of the eyeglass-shaped main body by extensions of the temples. The audio output device according to claim 1 , which is disposed along an extension direction.

5. A main body that is attached to the user's head and has a plurality of speakers; A control unit that controls the plurality of speakers; A detector for detecting reflected waves of sound waves output from a preset speaker; A storage unit that stores information on the distance from a preset speaker to each speaker; Preparation, The plurality of speakers include: A first speaker disposed near the position of the user's ear; The first speaker is disposed at a different position from the first speaker, and the first speaker is positioned so that the first speaker can be a second speaker disposed at a position farther away than the speaker, The control unit controls the first speaker and the second speaker to have different audio settings. This will output the audio. The main body has a eyeglass shape, The plurality of speakers are attached to the temples of the eyeglass-shaped main body by extensions of the temples. are arranged along the stretch direction, The control unit causes at least one of the plurality of speakers to output a sound wave, and then estimating the position of the user's ear based on the time it takes for the detector to detect the reflected wave; Based on the estimated position of the user's ear and the information on the distance, The speaker disposed rearward in the extension direction of the temple is determined. An audio output device that does not output audio from the speaker.

Images