Display system, display device, and display device control method

Abstract

A display system, a display device, and a control method for a display device are provided, wherein the display device is operated using power supplied from an information processing device. In a display system (1) having an HMD (100) and a control device (300), the control device has a battery (329) and a CO control unit (310) for supplying power to the HMD (100), and the HMD (100) has: an image display unit (20) for displaying an image included in a reproduction signal received from the control device; a sound processing unit (147), a right earphone (32), and a left earphone (34) for outputting a sound included in the reproduction signal; a detection unit having a plurality of sensors; and a power supply control unit (126) for supplying the power supplied from the control device (300) to a supply destination. When the power consumption of the HMD is greater than the power supplied by the control device, the image display unit, the sound output unit, and some of the plurality of sensors are stopped.

Description

Technical Field

[0001] The present invention relates to a display system, a display device and a control method of the display device. Background Art

[0002] Conventionally, there is known a system in which an information processing device and a display device are connected to each other and power is supplied from the information processing device to the display device.

[0003] For example, the display system of Patent Document 1 includes: a small display device that stores monitor required voltage information and monitor current consumption information as EDID information; and a host device that obtains the EDID information from the small display device. The host device controls the output voltage of the host device based on the monitor required voltage information and monitor current consumption information obtained from the small display device as EDID information, and the power supply voltage value and power supply current consumption value detected by the detection unit.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2002-41188

[0005] However, if the power supplied from the host device to the small display device is insufficient, the small display device cannot be used, and the only option is to select a host device that can supply enough power to operate the small display device. Summary of the invention

[0006] One way to solve the above-mentioned problem is a display system comprising: an information processing device, comprising a first supply unit for supplying a first power, and an output unit for outputting a reproduction signal including an image and a sound; and a display device, comprising a first sensor, a display unit for displaying the image, a sound output unit for outputting the sound, and a second supply unit, wherein the second supply unit supplies the first power to the first sensor, the display unit, and the sound output unit, and when the second power consumed by the display device is greater than the first power, the display unit, the sound output unit, and the first sensor stop part of their operations, or the display unit, the sound output unit, and the first sensor operate in such a way that the second power is less than the first power.

[0007] One way to solve the above-mentioned problem is a display system comprising: an information processing device, comprising a first supply unit for supplying a first power, and an output unit for outputting a reproduction signal including an image; and a display device, comprising a first display unit for displaying the image, a second display unit for displaying the image and different from the first display unit, and a second supply unit for supplying the first power to the first display unit and the second display unit, wherein when the second power consumed by the display device is greater than the first power, at least one of the first display unit and the second display unit does not display the image.

[0008] One way to solve the above-mentioned problem is a display device comprising: a sensor; a display unit that displays an image; a sound output unit that outputs sound; a supply unit that supplies a first power to the display unit, the sound output unit, and the sensor; and a control unit that controls the supply unit so that a second power consumed by the display unit, the sound output unit, and the sensor becomes less than the first power.

[0009] One way to solve the above-mentioned problem is a display device comprising: a first display unit, which displays an image; a second display unit, which displays the image and is different from the first display unit; a supply unit, which supplies the first power to the first display unit and the second display unit; and a control unit, which controls the first display unit and the second display unit, and the control unit prevents at least one of the first display unit and the second display unit from displaying the image when the second power consumed by the first display unit and the second display unit is greater than the first power.

[0010] One way to solve the above-mentioned problem is a control method for a display device, wherein the display device has: a sensor; a display unit that displays an image; and a sound output unit that outputs a sound. In the control method for the display device, a first power is supplied to the sensor, the display unit, and the sound output unit, and the first power supplied to the display unit, the sound output unit, and the sensor is controlled so that a second power consumed by the display device is less than the first power.

[0011] One way to solve the above-mentioned problem is a control method of a display device, wherein the display device has a first display unit and a second display unit. In the control method of the display device, a first power is supplied to the first display unit and the second display unit. When the second power consumed by the first display unit and the second display unit is greater than the first power, at least one of the first display unit and the second display unit does not display an image. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a diagram showing a schematic structure of a display system.

[0013] Figure 2 It is a top view of the main parts showing the optical system structure of the image display unit.

[0014] Figure 3 is a block diagram of the display system.

[0015] Figure 4 is a block diagram of the control device.

[0016] Figure 5 This is a diagram showing an example of a power consumption meter.

[0017] Figure 6 This is a flowchart showing the operation of the HMD.

[0018] Figure 7 This is a flowchart showing the operation of the control device.

[0019] Figure 8 This is a flowchart showing the operation of the HMD.

[0020] Fig. 9 This is a flowchart showing the operation of the control device.

[0021] Description of symbols

[0022] 1: Display system; 10: Connecting device; 11A: Connector; 11D: Connector; 20: Image display unit; 21: Right holding unit; 22: Right display unit; 23: Left holding unit; 24: Left display unit; 26: Right light guide plate; 27: Front frame; 28: Left light guide plate; 30: Earphone; 32: Right earphone; 34: Left earphone; 36: Audio connector; 40: Connection cable; 46: USB cable; 61: DP external camera; 63: Microphone; 64: Distance sensor; 65 : DP illumination sensor; 67: LED indicator; 100: HMD; 110: I / F unit; 120: DP control unit; 122: sensor control unit; 126: power control unit; 130: non-volatile storage unit; 135: power consumption meter; 140: operation unit; 145: connection unit; 147: sound processing unit; 201: image signal; 210: right display unit substrate; 211: right I / F unit; 213: receiving unit; 215: EEPROM; 217: temperature sensor sensor; 221: OLED unit; 223: OLED panel; 225: OLED driving circuit; 229: power supply unit; 230: left display unit substrate; 231: left I / F unit; 233: receiving unit; 235: DP six-axis sensor; 237: DP magnetic sensor; 239: temperature sensor; 241: OLED unit; 243: OLED panel; 245: OLED driving circuit; 249: power supply unit; 251: right optical system; 252: left optical system; 261: half-reflecting mirror; 281: half-reflecting mirror; 300: control device; 310: CO control unit; 311: processor; 312: memory; 313: non-volatile memory; 321: GNSS; 322: CO camera; 323: CO six-axis sensor; 324: CO magnetic sensor; 325: CO illuminance sensor; 326: sound output unit; 327: CO display unit; 328: touch panel; 329: battery; 330: communication unit; 331: I / F unit. DETAILED DESCRIPTION

[0023] 1. Display system structure

[0024] Figure 1 It is a diagram showing a schematic configuration of the display system 1 .

[0025] The display system 1 includes an HMD 100 corresponding to a display device and a control device 300 corresponding to an information processing device. The HMD 100 is a head mounted display device having an image display unit 20 mounted on the head of a user U to allow the user to see images or videos. HMD is an abbreviation of Head Mounted Display.

[0026] The HMD 100 includes a connection device 10 connected to the image display unit 20. The connection device 10 functions as an interface for connecting the HMD 100 and a device different from the HMD 100. In the display system 1, the control device 300 is connected to the connection device 10.

[0027] In the following description and drawings, for convenience of description, a prefix DP is added to the names of several functional units constituting the HMD 100 , and a prefix CO is added to the names of several functional units constituting the control device 300 .

[0028] The control device 300 has a display screen for displaying characters and images and a touch panel 328 that functions as an operation unit for detecting touch operations and press operations, and is a terminal device of a portable size, such as a smartphone. The control device 300 may also be a desktop personal computer, a notebook personal computer, a tablet personal computer, etc.

[0029] The connection device 10 has a connector 11A and a connector 11D on a box-shaped housing. The image display unit 20 is connected to the connector 11A via a connection cable 40, and the control device 300 is connected to the connector 11D via a USB cable 46. Thus, the image display unit 20 and the control device 300 are connected to each other so that data can be sent and received. For example, the control device 300 outputs image data and sound data for the image display unit 20 to display an image. For example, the image display unit 20 sends detection data of various sensors possessed by the image display unit 20 to the control device 300 as described later. The control device 300 can also supply power to the image display unit 20. USB is the abbreviation of Universal Serial Bus.

[0030] The structure of connecting the connection device 10 and the control device 300 using the USB cable 46 is only an example, and the specific connection method of the connection device 10 and the control device 300 is not limited. For example, other types of cables can be used for wired connection, and the connection can also be made via wireless communication. For example, in the structure in which the USB cable 46 is connected to the connector 11D of the USB-TypeC standard, a 20-volt direct current can be supplied through the USB cable 46, and as a function of the alternative mode of the USB-TypeC, image data of the HDMI standard can be transmitted. HDMI is a registered trademark.

[0031] The image display unit 20 includes a right display unit 22 , a left display unit 24 , a right light guide plate 26 , and a left light guide plate 28 on a main body including a right holding unit 21 , a left holding unit 23 , and a front frame 27 .

[0032] The right holding portion 21 and the left holding portion 23 extend rearward from both ends of the front frame 27 to hold the image display unit 20 on the head of the user U. The right holding portion 21 is connected to the end ER of the front frame 27 located on the right side of the user U, and the left holding portion 23 is connected to the end EL of the front frame 27 located on the left side of the user U.

[0033] The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is located in front of the right eye of the user U when the image display unit 20 is worn, so that the right eye can see the image. The left light guide plate 28 is located in front of the left eye of the user U when the image display unit 20 is worn, so that the left eye can see the image. The right light guide plate 26 and the left light guide plate 28 are optical parts formed of a light-transmitting resin or the like, and guide the image light output from the right display unit 22 and the left display unit 24 to the eyes of the user U. The right light guide plate 26 and the left light guide plate 28 are, for example, prisms.

[0034] The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other, and when the user U wears the image display unit 20, the connection position corresponds to the brow of the user U. The front frame 27 may have a nose pad that abuts against the nose of the user U when the image display unit 20 is worn, or may be a structure in which a belt is connected to the right holding portion 21 and the left holding portion 23, and the image display unit 20 is held on the head of the user U by the belt.

[0035] The right display unit 22 and the left display unit 24 are modules obtained by unitizing an optical unit and a peripheral circuit. The right display unit 22 displays an image through a right light guide plate 26, and the left display unit 24 displays an image through a left light guide plate 28. The right display unit 22 is provided on the right holding unit 21, and the left display unit 24 is provided on the left holding unit 23.

[0036] The image light guided by the right light guide plate 26 and the external light transmitted through the right light guide plate 26 enter the right eye of the user U. Similarly, the image light guided by the left light guide plate 28 and the external light transmitted through the left light guide plate 28 enter the left eye. The image light from the right light guide plate 26 and the left light guide plate 28 and the external light transmitted through the right light guide plate 26 and the left light guide plate 28 enter the eyes of the user U. Thus, the user U sees the image displayed on the image display unit 20 and the external scene transmitted through the right light guide plate 26 and the left light guide plate 28 in a superimposed manner.

[0037] The front frame 27 is provided with a DP illuminance sensor 65. The DP illuminance sensor 65 is a sensor that receives external light from the front of the user U wearing the image display unit 20. The DP illuminance sensor 65 can detect the illuminance and light amount of the external light that passes through the right light guide plate 26 and the left light guide plate 28 and enters the eyes of the user U.

[0038] The DP outside camera 61 is set in the front frame 27 at a position that does not block the external light passing through the right light guide plate 26 and the left light guide plate 28. The DP outside camera 61 is a digital camera having an imaging element such as a CCD or a CMOS and a camera lens, etc., and can be a single-eye camera or a stereo camera. The field of view of the DP outside camera 61 includes at least a part of the range of the outside scenery seen by the user U wearing the image display unit 20 through the right light guide plate 26 and the left light guide plate 28. The DP outside camera 61 can be a wide-angle camera or a camera that can capture the entire outside scenery seen by the user U wearing the image display unit 20. CCD is the abbreviation of Charge Coupled Device, and CMOS is the abbreviation of Complementary Metal Oxide Semiconductor.

[0039] An LED indicator 67 that lights up during the operation of the DP outside camera 61 is disposed on the front frame 27 .

[0040] The front frame 27 is provided with a distance sensor 64, which detects the distance of the measurement object in a preset measurement direction. The distance sensor 64 is, for example, a light reflection distance sensor using an LED or a laser diode, an infrared depth sensor, an ultrasonic distance sensor, or a laser rangefinder. The distance sensor 64 may be a distance detection unit that combines image detection and sound detection, or a device that processes an image obtained by stereoscopic photography of a camera to detect the distance. The measurement direction of the distance sensor 64 is, for example, the direction of the outside view that the user U sees through the right light guide plate 26 and the left light guide plate 28.

[0041] The right display unit 22 and the left display unit 24 are connected to the connection device 10 through the connection cable 40. The connection cable 40 has an audio connector 36. The earphone 30 having the right earphone 32 and the left earphone 34 constituting stereo earphones and the microphone 63 is connected to the audio connector 36. The right earphone 32 and the left earphone 34 output sound according to the sound signal output from the connection device 10. The microphone 63 collects the sound and outputs the sound signal to the connection device 10.

[0042] 2. Structure of the optical system of the image display unit

[0043] Figure 2 FIG. 2 is a top view showing the main parts of the optical system structure of the image display unit 20. Figure 2 For the convenience of explanation, the left eye LE and the right eye RE of the user U are shown.

[0044] The right display unit 22 and the left display unit 24 are configured to be bilaterally symmetrical, for example.

[0045] As a structure for allowing the right eye RE to see an image, the right display unit 22 includes an OLED unit 221 that emits image light and a right optical system 251 that guides the image light L emitted by the OLED unit 221 to the right light guide plate 26. OLED is an abbreviation of Organic Light Emitting Diode.

[0046] The OLED unit 221 includes an OLED panel 223 and an OLED driving circuit 225 for driving the OLED panel 223. The OLED panel 223 is, for example, a self-luminous display panel that is provided with light-emitting elements that emit R, G, and B color lights, respectively. The OLED driving circuit 225 drives the OLED panel 223 according to the control of the DP control unit 120. The OLED driving circuit 225 is, for example, mounted on a substrate (not shown) fixed to the back of the OLED panel 223, and a Figure 3 A temperature sensor 217 is shown.

[0047] The right optical system 251 converts the image light L emitted from the OLED panel 223 into parallel light beams through a collimating lens, and makes them incident on the right light guide plate 26. Inside the right light guide plate 26, the image light L is reflected by multiple reflection surfaces, and is reflected by the half mirror 261 located in front of the right eye RE and emitted from the right light guide plate 26 toward the right eye RE.

[0048] As a structure for allowing the left eye LE to see an image, the left display unit 24 includes an OLED unit 241 that emits image light and a left optical system 252 that guides the image light L emitted by the OLED unit 241 to the left light guide plate 28 .

[0049] The OLED unit 241 includes an OLED panel 243 and an OLED driving circuit 245 for driving the OLED panel 243. The OLED panel 243 is, for example, a self-luminous display panel that is provided with light-emitting elements that emit R, G, and B color lights, respectively. The OLED driving circuit 245 drives the OLED panel 243 in accordance with the control of the DP control unit 120. The OLED driving circuit 245 is, for example, mounted on a substrate (not shown) fixed to the back of the OLED panel 243, and a Figure 3 A temperature sensor 239 is shown.

[0050] The left optical system 252 converts the image light L emitted from the OLED panel 243 into parallel light beams through a collimating lens, and makes them incident on the left light guide plate 28. Inside the left light guide plate 28, the image light L is reflected by multiple reflection surfaces, and is reflected by the half mirror 281 located in front of the left eye LE and emitted from the left light guide plate 28 toward the left eye LE.

[0051] The HMD 100 functions as a transmissive display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 enter the right eye RE of the user U. The image light L reflected by the half mirror 281 and the external light OL transmitted through the half mirror 281 enter the left eye LE. The HMD 100 allows the image light L of the image processed internally to overlap with the external light OL and enter the eye of the user U. Therefore, the user U can see the outside scene through the right light guide plate 26 and the left light guide plate 28, and can see the image based on the image light L overlapping with the outside scene. The half mirrors 261 and 281 are image extraction units that reflect the image lights outputted from the right display unit 22 and the left display unit 24, respectively, to extract images, and constitute the display unit.

[0052] 3.HMD control system

[0053] Figure 3 is a block diagram of the display system 1 , and particularly shows the structure of the HMD 100 in detail.

[0054] In the image display unit 20, the right display unit 22 has a right display unit substrate 210. A right I / F unit 211 connected to the connection cable 40, a receiving unit 213 receiving data input from the connection device 10 via the right I / F unit 211, and an EEPROM 215 are mounted on the right display unit substrate 210. The right I / F unit 211 connects the receiving unit 213, the EEPROM 215, the temperature sensor 217, the DP outside camera 61, the distance sensor 64, the DP illumination sensor 65, and the LED indicator 67 to the connection device 10. The receiving unit 213 connects the OLED unit 221 to the connection device 10.

[0055] The left display unit 24 has a left display unit substrate 230. The left display unit substrate 230 has a left I / F unit 231 connected to the connection cable 40 and a receiving unit 233 that receives data input from the connection device 10 via the left I / F unit 231. The left display unit substrate 230 has a DP six-axis sensor 235 and a DP magnetic sensor 237 mounted thereon.

[0056] The left I / F unit 231 connects the receiving unit 233 , the DP six-axis sensor 235 , the DP magnetic sensor 237 , and the temperature sensor 239 to the connection device 10 . The receiving unit 233 connects the OLED unit 241 to the connection device 10 .

[0057] EEPROM is an abbreviation of Electrically Erasable Programmable Read-Only Memory. In addition, the receiving unit 213 and the receiving unit 233 are sometimes described as Rx 213 and Rx 233, respectively.

[0058] The EEPROM 215 stores various data in a nonvolatile manner. For example, the EEPROM 215 stores data related to the light emission characteristics and display characteristics of the OLED units 221 and 241 included in the image display unit 20, data related to the characteristics of the sensors included in the right display unit 22 or the left display unit 24, etc. Specifically, the EEPROM 215 stores parameters related to the gamma correction of the OLED units 221 and 241, data for compensating the detection values ​​of the temperature sensors 217 and 239, etc. in a manner that the DP control unit 120 can read.

[0059] The DP external camera 61 performs photography according to the signal inputted via the right I / F unit 211, and outputs the photographed image data to the right I / F unit 211. The DP illumination sensor 65 receives external light and outputs a detection value corresponding to the amount of light received or the intensity of light received. The LED indicator 67 lights up according to the control signal or driving current inputted via the right I / F unit 211.

[0060] The temperature sensor 217 detects the temperature of the OLED unit 221 and outputs a voltage value or a resistance value corresponding to the detected temperature as a detection value.

[0061] The distance sensor 64 outputs a signal indicating a detection result of the detected distance to the connection device 10 via the right I / F unit 211 .

[0062] The receiving unit 213 receives the image data for display transmitted from the connection device 10 via the right I / F unit 211, and outputs the image data to the OLED unit 221. The OLED unit 221 displays an image based on the image data transmitted from the connection device 10.

[0063] The receiving unit 233 receives the image data for display transmitted from the connection device 10 via the left I / F unit 231, and outputs the image data to the OLED unit 241. The OLED units 221 and 241 display images based on the image data transmitted from the connection device 10.

[0064] The DP six-axis sensor 235 is a motion sensor having a three-axis acceleration sensor and a three-axis gyro sensor. The DP magnetic sensor 237 is, for example, a three-axis geomagnetic sensor. The DP six-axis sensor 235 and the DP magnetic sensor 237 may be an IMU in which the above-mentioned sensors are modularized, or the DP six-axis sensor 235 and the DP magnetic sensor 237 may be an integrated module. IMU is the abbreviation of Inertial Measurement Unit. The temperature sensor 239 detects the temperature of the OLED unit 241. The DP six-axis sensor 235, the DP magnetic sensor 237 and the temperature sensor 239 respectively output the detection values ​​to the connection device 10.

[0065] Each part of the image display unit 20 operates using the power supplied from the connection device 10 through the connection cable 40. The image display unit 20 has a power supply unit 229 in the right display unit 22 and a power supply unit 249 in the left display unit 24. The power supply unit 229 distributes the power supplied from the connection device 10 via the connection cable 40 and supplies it to each part of the right display unit 22 including the right display unit substrate 210. The power supply unit 249 distributes the power supplied from the connection device 10 via the connection cable 40 and supplies it to each part of the left display unit 24 including the left display unit substrate 230. The power supply units 229 and 249 may also have a conversion circuit for converting voltages, etc. The power supply units 229 and 249 together with the power supply control unit 126 correspond to the second supply unit.

[0066] The connection device 10 includes an I / F unit 110 , a DP control unit 120 , a sensor control unit 122 , a display control unit 124 , a power control unit 126 , a nonvolatile storage unit 130 , an operation unit 140 , a connection unit 145 , and a sound processing unit 147 .

[0067] The I / F unit 110 includes a connector 11D and an interface circuit that executes a communication protocol based on various communication standards. The I / F unit 110 is, for example, an interface substrate on which the connector 11D and the interface circuit are mounted. The I / F unit 110 may include an external storage device, a memory card interface that can connect to a storage medium, or the like, or may be composed of a wireless communication interface.

[0068] The DP control unit 120 has a processor such as a CPU or a microcomputer, and controls each part of the connection device 10 by executing a program through the processor. The DP control unit 120 may also have a RAM constituting a work area of ​​the processor. RAM is an abbreviation of Random Access Memory.

[0069] The DP control unit 120 is connected to a nonvolatile storage unit 130, an operation unit 140, a connection unit 145, and a sound processing unit 147. The nonvolatile storage unit 130 is a ROM that nonvolatilely stores programs and data executed by the DP control unit 120. ROM is an abbreviation of Read Only Memory.

[0070] The sensor control unit 122 operates the sensor group of the image display unit 20. The sensor group includes the DP external camera 61, the distance sensor 64, the DP illumination sensor 65, the temperature sensor 217, the DP six-axis sensor 235, the DP magnetic sensor 237, and the temperature sensor 239. The sensor control unit 122 sets and initializes the sampling period of each sensor according to the control of the DP control unit 120, and performs power-on to each sensor, transmission of control data, acquisition of detection values, etc. according to the sampling period of each sensor.

[0071] The sensor control unit 122 outputs detection data indicating detection values ​​and detection results of each sensor to the I / F unit 110 at a preset timing. Here, the image data captured by the DP outside camera 61 is referred to as detection data in the same manner as the detection values ​​and detection results of other sensors.

[0072] The sensor control unit 122 may also include an A / D converter that converts an analog signal into digital data. In this case, the sensor control unit 122 converts the analog signal of the detection value or detection result obtained from the sensor of the image display unit 20 into detection data and outputs it. The sensor control unit 122 may also obtain the digital data of the detection value or detection result from the sensor of the image display unit 20, perform data format conversion, adjust output timing, etc., and output the detection data to the I / F unit 110.

[0073] Through the operation of the sensor control unit 122 , the control device 300 connected to the I / F unit 110 can obtain the detection values ​​of the sensors of the HMD 100 and the captured image data of the DP outside camera 61 .

[0074] The sensor control unit 122 may output the result of the calculation processing based on the detection values ​​of the above-mentioned sensors as detection data. For example, the sensor control unit 122 may also uniformly process the detection values ​​or detection results of multiple sensors and function as a so-called sensor fusion processing unit. In this case, the sensor control unit 122 may generate detection data of a virtual sensor that is not included in each sensor of the image display unit 20 through sensor fusion. For example, the sensor control unit 122 may also output trajectory data representing the trajectory of the movement of the image display unit 20, coordinate data representing the position of the image display unit 20 in three-dimensional space, and direction data representing the direction of the image display unit 20 as detection data. Here, the coordinate data may be data representing relative coordinates based on the position of the connection device 10, or data representing a position relative to a reference position set in the space where the image display unit 20 exists. The direction data may be data representing a direction based on the position and direction of the connection device 10, or data representing a direction relative to a reference position set in the space where the image display unit 20 exists.

[0075] The sensor control unit 122 executes a communication protocol with the control device 300 connected via the USB cable 46 and the connector 11D, and outputs detection data.

[0076] The display control unit 124 performs various processes for causing the image display unit 20 to display an image based on the image data or display data contained in the reproduction signal input to the I / F unit 110. In the present embodiment, the image data is transmitted in the alternative mode of the USB-TypeC through the connector 11D composed of a USB-TypeC connector. The display control unit 124 performs various processes such as frame capture, resolution conversion, scaling, intermediate frame generation, frame rate conversion, etc. The display control unit 124 outputs the image data corresponding to the OLED units 221 and 241 to the connection unit 145. The image data input to the connection unit 145 is transmitted from the connector 11A as an image signal 201 to the right I / F unit 211 and the left I / F unit 231. The display control unit 124 adjusts and changes the display state of the image display unit 20 according to the display control data input to the I / F unit 110.

[0077] At least one of the sensor control unit 122 and the display control unit 124 may also be implemented by the cooperation of software and hardware by executing a program by a processor. That is, the sensor control unit 122 and the display control unit 124 are composed of a processor, and the above-mentioned actions are performed by executing a program. In this example, the sensor control unit 122 and the display control unit 124 may also be implemented by executing a program by a processor constituting the DP control unit 120. In other words, the DP control unit 120, the display control unit 124, and the sensor control unit 122 may also function as the DP control unit 120, the display control unit 124, and the sensor control unit 122 by executing a program by a processor. Here, the processor may be rephrased as a computer. The sensor control unit 122 and the display control unit 124 may have a working memory for data processing, or may use the memory of the DP control unit 120 for processing.

[0078] The display control unit 124 and the sensor control unit 122 may also be formed by programmed hardware such as DSP, FPGA, etc. The sensor control unit 122 and the display control unit 124 may also be combined to form a SoC-FPGA. DSP is the abbreviation of Digital Signal Processor. FPGA is the abbreviation of Field Programmable Gate Array. SoC is the abbreviation of System-on-a-Chip.

[0079] The power control unit 126 is a circuit that is connected to the connector 11D and supplies power to each unit of the connection device 10 and the image display unit 20 based on the power supplied from the connector 11D.

[0080] The power supply control unit 126 together with the power supply units 229 and 249 corresponds to a second supply unit.

[0081] The operation unit 140 detects an operation of a switch or the like included in the connection device 10 , and outputs data indicating the operation content to the DP control unit 120 .

[0082] The sound processing unit 147 generates a sound signal according to the sound data input from the DP control unit 120. The sound data also includes the sound data included in the reproduction signal input from the control device 300. In addition, the sound processing unit 147 has an amplifier, which amplifies the generated sound signal and outputs it to the connection unit 145. The sound signal is output from the connection unit 145 to the right earphone 32 and the left earphone 34 via the audio connector 36. The sound processing unit 147 generates sound data of the sound collected by the microphone 63 and outputs it to the DP control unit 120. The sound data output from the sound processing unit 147 can be processed by the sensor control unit 122 in the same way as the detection data of the sensor possessed by the image display unit 20.

[0083] 4. Structure of control device

[0084] Figure 4 3 is a block diagram of the control device 300 .

[0085] The control device 300 has a CO control unit 310. The CO control unit 310 has a processor 311, a memory 312, and a non-volatile memory 313. The processor 311 is composed of a CPU, a microcomputer, a DSP, etc., and controls each part of the control device 300 by executing a program. The memory 312 forms a work area of ​​the processor 311. The non-volatile memory 313 is composed of a semiconductor storage function unit, etc., and stores the program executed by the processor 311 and various data processed by the processor 311 in a non-volatile manner. For example, the non-volatile memory 313 stores an operating system as a basic control program executed by the processor 311, and an application program that works on the operating system. The non-volatile memory 313 stores data processed when executing the application program and data of the processing result. The CO control unit 310 can also be a SoC integrating the processor 311, the memory 312, and the non-volatile memory 313.

[0086] The CO control unit 310 is connected to the GNSS 321 , the CO camera 322 , the CO six-axis sensor 323 , the CO magnetic sensor 324 , the CO illuminance sensor 325 , the sound output unit 326 , the CO display unit 327 , the battery 329 , the communication unit 330 , and the I / F unit 331 .

[0087] The GNSS 321 performs positioning using a satellite positioning system, and outputs the position of the control device 300 to the CO control unit 310. GNSS is an abbreviation of Global Navigation Satellite System.

[0088] The CO camera 322 is a digital camera provided on the main body of the control device 300, and is arranged adjacent to the touch panel 328, for example, to capture images in the direction opposite to the touch panel 328. The CO camera 322 captures images under the control of the CO control unit 310, and outputs captured image data to the CO control unit 310.

[0089] The CO six-axis sensor 323 is a motion sensor having a three-axis acceleration sensor and a three-axis gyroscope sensor, and outputs detection data indicating the detection value to the CO control unit 310. The CO magnetic sensor 324 is, for example, a three-axis geomagnetic sensor, and outputs detection data indicating the detection value to the CO control unit 310. The CO six-axis sensor 323 and the CO magnetic sensor 324 may be an IMU in which the above sensors are modularized, or the CO six-axis sensor 323 and the CO magnetic sensor 324 may be an integrated module.

[0090] The CO illuminance sensor 325 receives external light, and outputs detection data indicating a detection value corresponding to the amount of received light or the intensity of received light to the CO control unit 310 .

[0091] The sound output unit 326 includes a speaker, and outputs sound from the speaker according to the control of the CO control unit 310. The sound output unit 326 may include an amplifier that amplifies the sound signal output by the CO control unit 310 and outputs it to the speaker. In the case where the CO control unit 310 is configured to output digital sound data, the sound output unit 326 may include a D / A converter that converts the digital sound data into an analog sound signal.

[0092] The CO display unit 327 includes a touch panel 328 , and displays characters or images on the touch panel 328 in accordance with the control of the CO control unit 310 .

[0093] The battery 329 is a secondary battery built in the main body of the control device 300 , and supplies power to each unit of the control device 300 , and also supplies power to the connected HMD 100 .

[0094] When the HMD 100 is connected to the I / F unit 331, the CO control unit 310 negotiates with the HMD 100 and determines the power to be supplied to the HMD 100. The negotiation is a process of setting a power role, setting the amount of power to be transferred, etc. The power role is a power source that serves as a supply source for supplying power or a power receiver that receives power from a power source.

[0095] The CO control unit 310 supplies the power determined by negotiation to the HMD 100 via the I / F unit 331 .

[0096] The battery 329 and the CO control unit 310 correspond to a first supply unit.

[0097] The communication unit 330 corresponds to a wireless communication protocol such as Bluetooth and Wi-Fi, and performs wireless communication with an external device of the display system 1. Bluetooth and Wi-Fi are registered trademarks. The communication unit 330 may also be a structure for performing mobile data communication using a mobile communication network such as LTE or a fifth-generation mobile communication system. LTE is a registered trademark.

[0098] The I / F unit 331 includes a connector (not shown) for connecting a data communication cable, and an interface circuit that uses the connector to execute communication protocols that comply with various communication standards. For example, the I / F unit 331 includes a connector and an interface circuit that comply with the USB standard, transmits and receives data through the USB cable 46, and supplies power to the HMD 100.

[0099] The nonvolatile memory 313 stores content data.

[0100] The CO control unit 310 reproduces the content data and generates a reproduction signal including video and audio. The CO control unit 310 transmits the generated reproduction signal to the HMD 100 via the I / F unit 331 .

[0101] 5. Operation when the control device 300 is connected to the HMD 100

[0102] When the control device 300 is connected to the connection device 10 , the DP control unit 120 negotiates with the control device 300 and acquires information on the power that the control device 300 can supply to the HMD 100 .

[0103] When the HMD 100 is not equipped with a battery, the control device 300 supplies the HMD 100 with power required for the control device 300 to execute negotiation when connected to the HMD 100. When the negotiation with the HMD 100 is completed, the control device 300 supplies the required power according to the operation of the HMD 100.

[0104] When acquiring information on the power that can be supplied from the control device 300 , the DP control unit 120 compares the power indicated by the acquired information with the power consumption of the HMD 100 .

[0105] When the power consumption of the HMD 100 is greater than the power that can be supplied by the control device 300, the DP control unit 120 stops the operation of at least part of the image display unit 20, the sound output unit, and the sensor group. In addition, when the power consumption of the HMD 100 is greater than the power that can be supplied by the control device 300, the DP control unit 120 switches the HMD 100 to a low power consumption setting.

[0106] The image display unit 20 includes a right display unit 22 and a left display unit 24 .

[0107] The sound output unit includes a sound processing unit 147 , a right earphone 32 , and a left earphone 34 .

[0108] The sensor group is a detection unit mounted on the HMD 100, and includes a DP external camera 61, a distance sensor 64, a DP illumination sensor 65, a temperature sensor 217, a DP six-axis sensor 235, a DP magnetic sensor 237, and a temperature sensor 239. Hereinafter, each sensor constituting the image display unit 20, the sound output unit, and the sensor group is referred to as a functional unit.

[0109] Figure 5 It is a diagram showing an example of the power consumption table 135 .

[0110] The nonvolatile storage unit 130 stores Figure 5 The power consumption table 135 is shown.

[0111] The power consumption table 135 is a table in which identification information for identifying a functional unit and power consumption of each functional unit are registered in association with each other.

[0112] First, a case where the DP control unit 120 stops the operation of at least part of the image display unit 20 , the sound output unit, and the sensor group will be described.

[0113] When the power consumption of the HMD 100 is greater than the power that the control device 300 can supply, the DP control unit 120 selects the function unit with the largest power consumption by referring to the power consumption table 135. The DP control unit 120 determines whether the power consumption of the HMD 100 will become less than the power that the control device 300 can supply by not operating the selected function unit.

[0114] When the power consumption of the HMD 100 becomes less than the power that the control device 300 can supply, the DP control unit 120 controls the power control unit 126 so that the selected functional unit is not supplied with power and sets the selected functional unit not to operate.

[0115] In addition, when the power consumption of the HMD 100 does not become less than the power that the control device 300 can supply even if the function unit with the largest power consumption is set not to be operated, the DP control unit 120 refers to the power consumption table 135 to select the function unit that will not be operated next. The DP control unit 120 refers to the power consumption table 135 to select the function unit with the second largest power consumption. When the DP control unit 120 selects the function units with the first and second largest power consumption, it determines whether the power consumption of the HMD 100 will become less than the power that the control device 300 can supply by not operating these selected function units. When the power consumption of the HMD 100 becomes less than the power that the control device 300 can supply, the DP control unit 120 controls the power supply control unit 126 so that the selected function unit is not supplied with power, and sets the selected function unit not to operate.

[0116] The DP control unit 120 selects the third largest power consumption functional unit when the power consumption of the HMD 100 is not less than the power that the control device 300 can supply. Thereafter, the DP control unit 120 repeats the same process until the power consumption of the HMD 100 is less than the power that the control device 300 can supply.

[0117] In addition, the DP control unit 120 can calculate the difference between the power consumption of the HMD 100 and the power that can be supplied by the control device 300, and select the functional unit that is not to be operated based on the calculated difference. The DP control unit 120 refers to the power consumption table 135, selects a functional unit or a combination of functional units whose power consumption is greater than the calculated difference, controls the power supply control unit 126 so that the selected functional unit is not supplied with power, and sets the selected functional unit to be inoperable.

[0118] In addition, the DP control unit 120 may also select the function unit to stop the action according to the priority when the priority is set for the function unit. For example, the priority of the right display unit 22 of the image display unit 20 is set to the highest. In the following, it is assumed that the priority is set in the order of the DP external camera 61, the sound output unit, and the sensor group. In this case, the DP control unit 120 selects the function unit with the first priority according to the set priority. The DP control unit 120 subtracts the power consumption of the selected function unit from the power consumption of the HMD 100. When the power after the subtraction becomes less than the power that the control device 300 can supply, the DP control unit 120 controls the power control unit 126 so that the function unit with the first priority is not supplied with power, and sets the selected function unit not to operate. In this embodiment, the setting of the priority is a case where the priority is to stop the action, but the setting of the priority can also set the priority order of the function unit that does not want to stop the action. In this case, the DP control unit 120 selects the function unit to stop the action in order from low to high priority. In addition, the priority setting may be not the priority of the functional unit to stop the operation, but the priority of the functional unit to change the operation parameter.

[0119] In addition, when the power obtained by subtracting the power consumption of the selected functional unit from the power consumption of the HMD 100 is greater than the power that the control device 300 can supply, the DP control unit 120 selects the functional unit with the second priority. The DP control unit 120 subtracts the power consumption of the functional units with the first and second priorities from the power consumption of the HMD 100. When the power consumption after the subtraction becomes less than the power that the control device 300 can supply, the DP control unit 120 controls the power supply control unit 126 so that power is not supplied to the functional units with the first and second priorities, and sets the selected functional units not to operate. Thereafter, the DP control unit 120 repeats the same processing until the power consumption of the HMD 100 becomes less than the power that the control device 300 can supply.

[0120] In addition, when an application installed in the control device 300 is already started, the DP control unit 120 does not need to stop the operation of the functional unit of the HMD 100 required for the execution of the application. Hereinafter, the application is simply referred to as an application.

[0121] For example, it is assumed that a music playback application is activated in the control device 300, and sound is output from the right earphone 32 and the left earphone 34 of the HMD 100. In this case, the DP control unit 120 is set so that the operation of the sound output unit is not stopped even if the power consumption of the sound output unit is large and the priority is set high. In addition, it is assumed that a map application of the control device 300 is activated. In this case, the DP control unit 120 may not stop the operation of the DP six-axis sensor 235 included in the sensor group even if the power consumption of the DP six-axis sensor 235 is large and the priority is set high.

[0122] Next, a case will be described where the DP control unit 120 switches the HMD 100 to a low power consumption setting so as not to exceed the power that the control device 300 can supply when the power consumption of the HMD 100 is greater than the power that the control device 300 can supply.

[0123] For example, the DP control unit 120 may also cause the display control unit 124 to change at least one of the refresh rate and the brightness of the image displayed on the right display unit 22 and the left display unit 24 based on the power consumption of the HMD 100 and the power that the control device 300 can supply. The DP control unit 120 changes at least one of the refresh rate and the brightness of the image based on a preset threshold. For example, the DP control unit 120 obtains the difference between the power that the control device 300 can supply and the power consumption of the HMD 100, and compares the obtained power difference with the threshold. When the obtained power difference is less than the first threshold, the DP control unit 120 instructs the display control unit 124 to change at least one of the refresh rate and the brightness of the image displayed on the right display unit 22 and the left display unit 24. In addition, when the obtained power difference is less than the second threshold, the DP control unit 120 instructs the display control unit 124 to change the refresh rate and the brightness of the image displayed on the right display unit 22 and the left display unit 24. The second threshold is set to a value larger than the first threshold.

[0124] The display control unit 124 controls the right display unit 22 and the left display unit 24 in accordance with the control of the DP control unit 120 so that the refresh rate of the images displayed on the right display unit 22 and the left display unit 24 is lower than the power consumption when the control device 300 can supply the HMD 100.

[0125] In addition, the display control unit 124 controls the right display unit 22 and the left display unit 24 according to the control of the DP control unit 120 so that the brightness of the images displayed on the right display unit 22 and the left display unit 24 is lower than the power consumption that the control device 300 can supply to the HMD 100.

[0126] In addition, the DP control unit 120 may also reduce the refresh rate or brightness of either the right display unit 22 or the left display unit 24. In addition, the DP control unit 120 may also be set so that either the right display unit 22 or the left display unit 24 does not display an image. In addition, the DP control unit 120 may also be set so that either the refresh rate or brightness of either the right display unit 22 or the left display unit 24 does not display an image on the other.

[0127] In addition, the DP control unit 120 may instruct the sensor control unit 122 to set the sampling frequency of at least a part of the sensors included in the sensor group to be lower than the power consumption that the control device 300 can supply to the HMD 100. For example, the DP control unit 120 instructs the sensor control unit 122 to set the sensor with a low sampling frequency based on the difference between the power that the control device 300 can supply and the power consumption of the HMD 100 and the preset priority. The sensor control unit 122 controls the right display unit 22 and the left display unit 24 so that the sampling frequency of the sensor designated by the DP control unit 120 is lower than the power consumption that the control device 300 can supply to the HMD 100.

[0128] In addition, the DP control unit 120 may also cause the sensor control unit 122 to change at least one of the shooting resolution and the number of shots per unit time of the DP outside camera 61 based on the power consumption of the HMD 100 and the power that can be supplied by the control device 300. The DP control unit 120 causes the sensor control unit 122 to change at least one of the shooting resolution and the number of shots per unit time of the DP outside camera 61 based on a preset threshold value.

[0129] When the difference between the power that the control device 300 can supply and the power consumption of the HMD 100 is equal to or smaller than a threshold value, the DP control unit 120 instructs the sensor control unit 122 to change at least one of the imaging resolution of the DP outside camera 61 and the number of imaging times per unit time.

[0130] Furthermore, when the difference between the power that the control device 300 can supply and the power consumption of the HMD 100 is greater than a preset threshold, the DP control unit 120 instructs the sensor control unit 122 to change the imaging resolution and the number of imaging times per unit time of the DP outside camera 61 .

[0131] The sensor control unit 122 controls the DP outside camera 61 according to the control of the DP control unit 120 so that the imaging resolution of the DP outside camera 61 is lower than the power consumption when the control device 300 can supply the HMD 100 .

[0132] In addition, the sensor control unit 122 controls the DP outside camera 61 according to the control of the DP control unit 120 so that the number of shots per unit time of the DP outside camera 61 is lower than the power consumption when the control device 300 can supply the HMD 100 .

[0133] In addition, the DP control unit 120 can reduce the sound output to the sound output unit based on the power consumption of the HMD 100 and the power that can be supplied by the control device 300.

[0134] The DP control unit 120 may also calculate the difference between the power that the control device 300 can supply and the power consumption of the HMD 100, compare the calculated difference with a preset threshold value, and determine the sound pressure of the sound to be output by the sound output unit. Since the power consumed by the amplifier is large, the power consumption of the HMD 100 can be reduced by reducing the sound pressure of the sound output by the sound output unit. In addition, the DP control unit 120 may also stop the sound output unit from outputting the sound when the calculated difference between the power that the control device 300 can supply and the power consumption of the HMD 100 is greater than the preset threshold value.

[0135] Alternatively, the operation mode of the HMD 100 may be set in advance, and the operation mode for operating the HMD 100 may be determined based on the difference between the power consumption of the HMD 100 and the power that can be supplied by the control device 300 .

[0136] For example, the HMD 100 has three operation modes, namely, a first operation mode, a second operation mode, and a third operation mode as operation modes.

[0137] The DP control unit 120 changes the operation mode to any one of the first operation mode, the second operation mode, and the third operation mode according to the power supplied from the control device 300 .

[0138] The first operation mode is a mode in which the right display unit 22 and the left display unit 24 of the image display unit 20, the DP outside camera 61, the sound processing unit 147, the right earphone 32 and the left earphone 34, and the sensor group are all able to operate.

[0139] The second operation mode is a mode in which power consumption is reduced compared to the first operation mode.

[0140] When the second operation mode is selected, the DP control unit 120 operates only one of the right display unit 22 and the left display unit 24. In addition, when the second operation mode is selected, the DP control unit 120 controls the sensor control unit 122 to reduce the shooting frame rate of the DP outside camera 61 compared to the first operation mode. In addition, the DP control unit 120 controls the sensor control unit 122 to reduce the sampling frequency of the sensor group compared to the first operation mode.

[0141] The third operation mode is a mode in which power consumption is reduced compared with the second operation mode.

[0142] When the third operation mode is selected, the DP control unit 120 stops the operation of either the DP outside camera 61 or the sensor group.

[0143] Furthermore, when the third operation mode is selected, the DP control unit 120 lowers at least one of the refresh rate and the brightness of the right display unit 22 and the left display unit 24 compared to the first operation mode.

[0144] Figure 6 : is a flowchart showing the operation of the HMD 100 during negotiation.

[0145] The DP control unit 120 detects the connection of the control device 300 to the I / F unit 110 (step S1). For example, when a voltage monitoring unit (not shown) provided in the I / F unit 110 detects VBUS, the DP control unit 120 detects the connection to the I / F unit 110. When the DP control unit 120 does not detect the connection (step S1 / No), it does not transfer to the steps after step S2.

[0146] When the DP control unit 120 detects connection (step S1 / Yes), it determines whether a connection request is received from the connected control device 300 (step S2). When the DP control unit 120 does not receive a connection request (step S2 / No), it does not proceed to step S3 and subsequent steps.

[0147] When receiving the connection request from the control device 300 (step S2 / Yes), the DP control unit 120 notifies the control device 300 of the power consumption when the HMD 100 is operated in the first operation mode in which the power consumption of the HMD 100 is not limited (step S3 ).

[0148] Next, the DP control unit 120 determines whether or not a notification of the power that the control device 300 can supply to the HMD 100 , that is, the supply power has been received (step S4 ).

[0149] When the DP control unit 120 has not received the notification of power supply from the control device 300 (step S4 / No), the process does not proceed to step S5 and subsequent steps.

[0150] When the DP control unit 120 receives the notification of the power supply from the control device 300 (step S4 / yes), it determines that the power consumption of HMD100 is an operating mode below the notified power supply (step S5). When there are multiple operating modes below the notified power supply, the DP control unit 120 selects the operating mode below the power supply notified from the control device 300 and consuming the most power. In addition, the DP control unit 120 may set a priority for each operating mode and select the operating mode with the highest priority among the operating modes below the notified power supply. In addition, when there is an operating mode selected at the last connection with the control device 300 among the operating modes below the notified power supply, the DP control unit 120 may select the operating mode selected at the last connection.

[0151] When determining that the operating mode is equal to or less than the notified supply power, the DP control unit 120 notifies the control device 300 of the power consumption of the determined operating mode (step S6 ).

[0152] Next, the DP control unit 120 determines whether or not a negotiation completion notification has been received from the control device 300 (step S7). If the DP control unit 120 has not received a completion notification from the control device 300 (step S7 / No), the process does not proceed to the next step.

[0153] When receiving the completion notification from the control device 300 (step S7 / Yes), the DP control unit 120 transmits a reception notification notifying that the completion notification has been received to the control device 300 (step S8), and ends the processing flow.

[0154] Figure 7 This is a flowchart showing the operation of the control device 300 during negotiation.

[0155] The CO control unit 310 determines whether a connection is detected (step T1 ).

[0156] The CO control unit 310 detects that the control device 300 is connected to the I / F unit 331 (step T1). The CO control unit 310 detects the connection by detecting VBUS using a voltage monitoring unit (not shown) provided in the I / F unit 331, similarly to the HMD 100. When the CO control unit 310 does not detect the connection (step T1 / No), the processing after step T2 is not performed.

[0157] When the CO control unit 310 detects the connection (step T1 / Yes), it transmits a connection request to the HMD 100 which is the connection destination of the connection (step T2 ).

[0158] Next, the CO control unit 310 determines whether or not the notification of power consumption has been received from the HMD 100 (step T3). When the CO control unit 310 has not received the notification of power consumption from the HMD 100 (step T3 / No), the process does not proceed to step T4.

[0159] When receiving the notification of power consumption from the HMD 100 (step T3 ), the CO control unit 310 notifies the HMD 100 of the power supply that the control device 300 can supply to the HMD 100 (step T4 ).

[0160] Next, the CO control unit 310 determines whether or not the notification of power consumption is received from the HMD 100 (step T5). When the CO control unit 310 has not received the notification of power consumption from the HMD 100 (step T5 / No), the process does not proceed to step T6.

[0161] When receiving the notification of power consumption from the HMD 100 (step T5 / Yes), the CO control unit 310 transmits a completion notification to the HMD 100 (step T6). Thereafter, the CO control unit 310 determines whether a reception notification is received from the HMD 100 (step T7). When the CO control unit 310 does not receive a reception notification (step T7 / No), it waits until a reception notification is received.

[0162] When receiving the completion notification from the HMD 100 (step T7 / Yes), the CO control unit 310 ends the negotiation and starts supplying power to the HMD 100 .

[0163] Next, the operation after the negotiation will be described.

[0164] When the DP control unit 120 receives an operation to newly start a functional unit after the negotiation is completed, it determines whether the functional unit selected by the operation can be started. For example, the DP control unit 120 determines whether the selected functional unit can be operated by changing the operating mode of the HMD 100 to an operating mode with lower power consumption than the operating mode determined during the negotiation. The change of the operating mode includes the selection of the functional unit to stop the operation, the change of the operating parameters, etc. The change of the operating parameters refers to the change to the operating parameters that can further reduce the power consumption of the functional unit.

[0165] The DP control unit 120 determines whether the selected functional unit can be activated based on the power consumption of each operating mode, the power supplied from the control device 300, and the power consumption of the selected functional unit. For example, if the sum of the power consumption of the newly activated functional unit and the power consumption of the changed operating mode is equal to or less than the power consumption of the current operating mode before the newly activated functional unit, the DP control unit 120 determines that the selected functional unit can be activated.

[0166] In addition, when the DP control unit 120 cannot start the new functional unit by changing the working mode, it determines whether the new functional unit can be started by stopping other functional units in operation. The other functional units to be stopped may be one or more. The DP control unit 120 selects other functional units that are in operation and can stop the operation. The DP control unit 120 compares the power consumption of the selected other functional units with the power consumption of the newly started functional unit to determine whether the new functional unit can be started. When the DP control unit 120 determines that the power consumption of the selected other functional units is greater than the power consumption of the newly started functional unit and the newly started functional unit can be started, the DP control unit 120 stops the operation of the selected other functional units and starts the newly started functional unit. In addition, when the power consumption of the newly started functional unit is greater than the power consumption of other functional units, the DP control unit 120 may determine whether there is a functional unit that can be further stopped.

[0167] Furthermore, the DP control unit 120 may determine whether a new functional unit can be activated by changing the operation of another functional unit in operation to an operation with low power consumption.

[0168] When the power consumption of other function units in operation is greater than the power consumption of the newly activated function unit, the DP control unit 120 determines that the function unit can be newly activated by stopping the operation of the function unit in operation. In this case, the DP control unit 120 sends information about other function units that are candidates for stopping the operation to the control device 300.

[0169] When receiving information about other functional units that are candidates for stopping operation from the HMD 100, the CO control unit 310 displays the received information about other functional units on the CO display unit 327. In addition, the CO control unit 310 displays guidance on the CO display unit 327 asking whether the operation of other functional units can be stopped.

[0170] When receiving an operation of permitting the stop of the operation of other functional units by operating the touch panel 328 , the CO control unit 310 transmits a permission notification permitting the stop of the operation to the HMD 100 .

[0171] When receiving the notification of permission to stop the operation of other functional units from the control device 300, the DP control unit 120 stops the operation of other functional units and activates the selected functional unit.

[0172] Figure 8 This is a flowchart showing the operation of the DP control unit 120 when an operation to activate a functional unit mounted on the HMD 100 is accepted.

[0173] First, the DP control unit 120 determines whether an operation to activate a function unit is accepted (step S11). If the DP control unit 120 does not accept an operation to activate a function unit (step S11 / No), the DP control unit 120 does not proceed to the determination of step S12 until an operation to activate a function unit is accepted.

[0174] When the DP control unit 120 receives the operation of activating the function unit (step S11 / Yes), it refers to Figure 5 The power consumption table 135 shown acquires the power consumption when the function unit selected by the operation is operated (step S12).

[0175] The DP control unit 120 determines whether the total power obtained by adding the power consumption of the selected functional unit to the power consumption of the current operation mode of the HMD 100 exceeds the power supplied by the control device 300 (step S13). When the total power does not exceed the supplied power (step S13 / No), the DP control unit 120 activates the selected functional unit (step S22).

[0176] When the total power exceeds the supplied power (step S13 / Yes), the DP control unit 120 determines whether the total power can be suppressed to be equal to or less than the supplied power by changing the operation mode of the HMD 100 to another mode (step S14 ).

[0177] When determining that the total power can be suppressed to be less than the supplied power by changing the operation mode (step S14 / Yes), the DP control unit 120 transmits a request to change the operation mode of the HMD 100 to the control device 300 (step S15 ).

[0178] Next, the DP control unit 120 determines whether or not a notification of permission to change the operation mode has been received from the control device 300 (step S16).

[0179] When the DP control unit 120 does not permit the change of the operation mode (step S16 / No), the operation mode is not changed, and the selected functional unit is not activated, and the flow is terminated.

[0180] If the change of the operation mode is permitted (step S16 / Yes), the DP control unit 120 switches the operation mode to the operation mode determined in step S14 to be equal to or less than the supply power (step S17). Then, the DP control unit 120 activates the selected functional unit (step S22).

[0181] In addition, when the DP control unit 120 determines that the change in the working mode cannot be handled (step S14 / No), it can stop the operation and determine whether there are other functional units whose total power becomes less than the supplied power by stopping the operation (step S18). When there are no other functional units (step S18 / No), the DP control unit 120 does not start the selected functional unit and ends the processing flow.

[0182] In addition, if there are other functional units (step S18 / Yes), the DP control unit 120 transmits information about the other functional units that can stop operation to the control device 300 (step S19). Then, if the DP control unit 120 receives a rejection notification for rejecting the stop of the operation of the other functional units from the control device 300 (step S20 / No), the DP control unit 120 does not stop the operation of the other functional units, but ends the process.

[0183] Furthermore, when receiving a permission notification permitting the stopping of the operation of other functional units from the control device 300 (step S20 / Yes), the DP control unit 120 stops the operation of other functional units (step S21) and activates the selected functional unit (step S22).

[0184] Fig. 9 It is a flowchart showing the operation of the control device 300 .

[0185] The CO control unit 310 determines whether or not a request to change the operation mode is received from the HMD 100 (step T11). When the CO control unit 310 does not receive a request to change the operation mode from the HMD 100 (step T11 / No), the process proceeds to determination at step T16.

[0186] When the CO control unit 310 receives the change request of the operation mode from the HMD 100 (step T11 / Yes), it causes the CO display unit 327 to display the change request of the operation mode of the HMD 100 (step T12). When the CO control unit 310 receives the operation of permitting the change of the operation mode of the HMD 100 (step T13 / Yes), it transmits a permission notification of permitting the change of the operation mode to the HMD 100 (step T14). When the CO control unit 310 receives the operation of rejecting the change of the operation mode of the HMD 100 (step T13 / No), it transmits a rejection notification of rejecting the change of the operation mode to the HMD 100 (step T15).

[0187] Next, the CO control unit 310 determines whether information of other functional units to stop operation is received from the HMD 100 (step T16). When the CO control unit 310 does not receive information of other functional units (step T16 / No), the determination returns to step T11.

[0188] In addition, when the CO control unit 310 receives information about other functional units that are to be stopped from the HMD 100 (step T16 / Yes), the CO control unit 310 causes the touch panel 328 to display the information about other functional units received from the HMD 100 and a guide to confirm whether the operation of the other functional units can be stopped (step T17). Next, the CO control unit 310 determines whether an operation to permit the operation to be stopped of the other functional units has been accepted (step T18). When the CO control unit 310 receives an operation to permit the operation to be stopped of the other functional units (step T18 / Yes), the CO control unit 310 transmits a permission notification to permit the operation to be stopped of the other functional units to the HMD 100 (step T19). When the CO control unit 310 receives an operation to reject the operation to be stopped of the other functional units (step T18 / No), the CO control unit 310 transmits a rejection notification to reject the operation to be stopped of the other functional units to the HMD 100 (step T20).

[0189] As described above, the display system 1 includes the HMD 100 that displays an image and the control device 300 connected to the HMD 100 .

[0190] The control device 300 includes a first supply unit and a first output unit.

[0191] The first supply unit is constituted by a battery 329 that supplies power to the HMD 100 .

[0192] The first output unit is constituted by an I / F unit 331 that outputs a reproduction signal including at least one of an image and an audio.

[0193] The HMD 100 includes an image display unit 20 , a sound output unit, a detection unit, and a second supply unit.

[0194] The image display unit 20 displays an image included in the reproduction signal received from the control device 300 .

[0195] The sound output unit is composed of the sound processing unit 147, the right earphone 32, and the left earphone 34, and outputs the sound included in the reproduction signal.

[0196] The detection unit is composed of a sensor group, which includes a DP external camera 61 , a distance sensor 64 , a DP illumination sensor 65 , a temperature sensor 217 , a DP six-axis sensor 235 , a DP magnetic sensor 237 , and a temperature sensor 239 .

[0197] In addition, the HMD 100 includes a power control unit 126 , and power supply units 229 and 249 that supply the power supplied from the control device 300 to a supply destination.

[0198] The display system 1 stops the operation of part of the display unit, the sound output unit, and the detection unit when the power consumption of the HMD 100 is greater than the power supplied by the control device 300. In addition, the HMD 100 operates the image display unit 20, the sound output unit, and the detection unit so that the power consumption of the HMD 100 is less than the power supplied from the control device 300.

[0199] Therefore, even if the power supplied from the control device 300 is smaller than the power consumption of the HMD 100 , the power consumption of the HMD 100 can be reduced, and the HMD 100 can be operated using the power supplied from the control device 300 .

[0200] In addition, the image display unit 20 includes a right display unit 22 as a first display unit and a left display unit 24 as a second display unit.

[0201] When the power supplied from the control device 300 is smaller than the power consumption of the HMD 100 , the display system 1 does not display an image on at least one of the right display unit 22 and the left display unit 24 .

[0202] Therefore, even if the power supplied from the control device 300 is smaller than the power consumption of the HMD 100, the power consumption of the HMD 100 can be reduced by setting the display unit for displaying images to one of the right display unit 22 and the left display unit 24. Therefore, the HMD 100 can be operated using the power supplied from the control device 300.

[0203] When the power supplied by the control device 300 is smaller than the power consumption of the HMD 100 , the display system 1 stops the operation of at least one of the plurality of sensors.

[0204] Furthermore, the display system 1 reduces the sampling frequency of at least one of the plurality of sensors compared to a case where the power supplied from the control device 300 is greater than the power consumption of the HMD 100 .

[0205] Therefore, the power consumption of the HMD 100 can be reduced, and the HMD 100 can be operated using the power supplied from the control device 300 .

[0206] The sensor group includes a DP outer camera 61.

[0207] When the power supplied by the control device 300 is less than the power consumption of the HMD 100, the display system 1 reduces at least one of the shooting resolution and the number of shots per unit time of the DP external camera 61 compared to when the power supplied by the control device 300 is greater than the power consumption of the HMD 100.

[0208] Therefore, the power consumption of the HMD 100 can be reduced, and the HMD 100 can be operated using the power supplied from the control device 300 .

[0209] In addition, the display system 1 stops the sound output to the sound output unit when the power supplied by the control device 300 is less than the power consumption of the HMD 100. In addition, the display system 1 reduces the sound output from the sound output unit when the power supplied by the control device 300 is greater than the power consumption of the HMD 100.

[0210] Therefore, the power consumption of the HMD 100 can be reduced, and the HMD 100 can be operated using the power supplied from the control device 300 .

[0211] When the power supplied by the control device 300 is lower than the power consumption of the HMD 100 , the display system 1 reduces at least one of the refresh rate of the image display unit 20 and the brightness of the display unit compared to when the power supplied by the control device 300 is higher than the power consumption of the HMD 100 .

[0212] Therefore, the power consumption of the HMD 100 can be reduced, and the HMD 100 can be operated using the power supplied from the control device 300 .

[0213] The present invention is not limited to the configurations described in the above embodiments, and can be implemented in various forms within the scope not departing from the gist of the invention.

[0214] For example, in the above embodiment, the DP control unit 120 of the HDM 100 determines the supply destination of the supplied power. As another method, the CO control unit 310 of the control device 300 may select a functional unit to stop the operation or limit the function based on the power consumption of the HDM 100 and the power that the control device 300 can supply.

[0215] In addition, the display system 1 is illustrated as having a structure of the HMD 100 as a head-mounted display device, but the present invention is not limited to this, and various display devices can be used. For example, the image display unit 20 can be replaced by an image display unit of other forms such as an image display unit worn like a hat, as long as there is a display unit that displays an image corresponding to the left eye of the user U and a display unit that displays an image corresponding to the right eye of the user U. In addition, the display device of the present invention can be configured as a head-mounted display mounted on a means of transportation such as a car or an airplane. In addition, for example, it can also be configured as a head-mounted display built into a body protective device such as a helmet. In this case, the part that locates the position relative to the body of the user U and the part that locates the part can be set as the wearing part.

[0216] The HMD 100 is an example of a display device to which the present invention is applied, and is not limited to Figure 3 The structure shown. For example, in the above-mentioned embodiment, the structure in which the image display unit 20 and the connecting device 10 are separated is described as an example, but it may also be a structure in which the connecting device 10 and the image display unit 20 are formed as a whole and worn on the head of the user U. In addition, the structure of the optical system of the image display unit 20 is arbitrary. For example, an optical component that is located in front of the eyes of the user U and overlaps part or all of the field of vision of the user U may be used. Alternatively, an optical system that uses a scanning method to form image light by scanning a laser or the like may be used. Alternatively, it is not limited to guiding the image light inside the optical component, and it may only have the function of guiding the image light by refracting and / or reflecting it toward the eyes of the user U.

[0217] In addition, as a display device, a liquid crystal monitor or a liquid crystal television that displays an image on a liquid crystal display panel may be used. A display device having a plasma display panel or an organic EL display panel may also be used. In this case, the display panel is equivalent to the display unit of the present invention. Furthermore, as a display device, a projector that projects image light onto a screen or the like may also be used.

[0218] In addition, for example, Figure 3 In the HMD 100 shown, the connection device 10 can also be formed by using a USB-TypeC connector, a USB-TypeC controller, and a USB hub. In this case, the DP external camera 61 and other sensors can also be connected to the USB hub. In addition, as a controller for controlling the display of the right display unit 22 and the left display unit 24 in the image display unit 20, the FPGA that outputs display data to the right display unit 22 and the left display unit 24 can also be configured on any one of the right display unit 22 and the left display unit 24. In this case, the connection device 10 can also have a bridge controller connecting the USB-TypeC controller and the FPGA. In addition, in the image display unit 20, a structure in which the DP six-axis sensor 235, the DP magnetic sensor 237, the EEPROM 215, etc. are installed on the same substrate as the FPGA can also be adopted. The configuration of other sensors can also be appropriately changed. For example, the distance sensor 64 and the DP illuminance sensor 65 can also be configured in a position suitable for measurement or detection, and are configured to be connected to the FPGA and the USB-TypeC controller.

[0219] In addition, there is no limitation on the specific specifications of the display device including the OLED units 221 and 241. For example, the OLED units 221 and 241 may have a common structure.

[0220] Figure 3 , Figure 4At least a part of each functional block shown may be implemented by hardware, or may be implemented by cooperation of hardware and software, and is not limited to the structure in which independent hardware resources are arranged as shown in the figure.

[0221] in addition, Figure 6 and Figure 8 The flowchart showing the processing of the HMD 100 is divided according to the main processing contents for easy understanding of the processing of the DP control unit 120. Figure 6 and Figure 8 The division method or name of the processing unit shown in the flowchart is not limited. In addition, the processing of the DP control unit 120 can be divided into more processing units according to the processing content, or one processing unit can be divided to include more processing. In addition, the processing order of the above-mentioned flowchart is not limited to the example shown in the figure.

[0222] in addition, Figure 7 and Fig. 9 The flowchart showing the processing of the control device 300 is divided according to the main processing contents for easy understanding of the processing of the CO control unit 310. Figure 7 and Fig. 9 The division method or name of the processing unit shown in the flowchart is not limited. In addition, the processing of the CO control unit 310 can be divided into more processing units according to the processing content, or one processing unit can be divided to include more processing. In addition, the processing order of the above-mentioned flowchart is not limited to the example shown in the figure.

[0223] In addition, when the control method of the display device is implemented using a computer included in the HMD 100, the program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the program. The recording medium can use a magnetic or optical recording medium or a semiconductor storage function unit. In addition, the above-mentioned recording medium can also be an internal storage device included in the server device, that is, a non-volatile storage device such as RAM, ROM, HDD, etc.

Claims

1. A display system, wherein: The display system has: An information processing device including a first supply unit for supplying a first power, and an output unit for outputting a reproduction signal including an image and a sound; and A display device including a functional unit and a second supply unit, wherein the functional unit includes a display unit for displaying the image, a sound output unit for outputting the sound, and a first sensor, and the second supply unit supplies the first power to the functional unit. When the second power consumed by the display device is greater than the first power, the functional unit stops a part of its operation; or, when the functional unit operates in a manner such that the second power is less than the first power, the order of the functional units whose operations are to be stopped is determined according to the difference between the second power and the first power, the priority set for the functional unit, and the startup state of the application installed in the information processing device executed by the operation of the functional unit.

2. The display system according to claim 1, wherein: The display unit includes a first display unit that displays the image, and a second display unit that displays the image and is different from the first display unit. When the second power consumed by the display device is larger than the first power, at least one of the first display unit and the second display unit does not display the image.

3. The display system according to claim 1, wherein: The functional unit further includes a second sensor different from the first sensor, When the second power is greater than the first power, at least one of the first sensor and the second sensor stops operating, or at least one of the first sensor and the second sensor reduces a sampling frequency compared to a case where the first power is greater than the second power.

4. The display system according to claim 1, wherein: The first sensor is a camera, When the second power is greater than the first power, at least one of an imaging resolution of the camera and a number of imaging times per unit time is reduced compared to a case where the first power is greater than the second power.

5. The display system according to claim 1, wherein: When the second power is greater than the first power, the sound output unit stops outputting the sound, or the sound output unit reduces the output sound compared to the sound output when the first power is greater than the second power.

6. The display system according to claim 1, wherein: When the second power is greater than the first power, at least one of a refresh rate of the display unit and a brightness of the display unit is reduced compared to a case where the first power is greater than the second power.

7. A display device, wherein: The display device is connected to the information processing device, receives a first power supply from the information processing device, and comprises: a functional unit including a sensor, a display unit for displaying an image, and a sound output unit for outputting a sound; a supply unit that supplies the first power to the functional unit; and a control unit that controls the supply unit and the functional unit, When the second power consumed by the functional unit is greater than the first power, the control unit stops a part of the operation of the functional unit, or, when the control unit causes the functional unit to operate in a manner in which the second power is less than the first power, the order of the functional units whose operations are to be stopped is determined according to the difference between the second power and the first power, the priority set for the functional unit, and the startup state of the application installed in the information processing device that is executed through the operation of the functional unit.

8. The display device according to claim 7, wherein: The display unit has: a first display unit that displays an image; and a second display unit which displays the image and is different from the first display unit, The control unit causes at least one of the first display unit and the second display unit not to display the image when the second power is greater than the first power.

9. A method for controlling a display device, wherein the display device is connected to an information processing device, receives a first power supply from the information processing device, and has a functional unit, wherein the functional unit includes: a sensor; a display unit that displays an image; and a sound output unit that outputs a sound, wherein in the method for controlling the display device, supplying the first power to the functional unit, When controlling the functional unit so as to stop a part of the operation of the functional unit when the second power consumed by the display device is greater than the first power, or to cause the functional unit to operate in a manner in which the second power is less than the first power, the order of the functional units whose operations are to be stopped is determined according to the difference between the second power and the first power, the priority set for the functional unit, and the startup state of an application installed in the information processing device that is executed through the operation of the functional unit.

10. The control method of the display device according to claim 9, wherein: The display unit includes a first display unit and a second display unit. In the control method of the display device, When the second power is greater than the first power, at least one of the first display unit and the second display unit does not display an image.

Images