Display device, display control method, program, display system

Abstract

To provide a technique which allows a display device to stop outputting a video and restrict transmission of operation information.SOLUTION: A display device 20 which displays a video received from a terminal device 40 and transmits operation information on the video to the terminal device includes: a communication unit which receives a video from the terminal device by wireless communication; an output unit which outputs the video received by the communication unit to display means; an operation receiving unit which acquires operation information on the video; and a display component control unit which displays a first display component together with the video, the first display component for receiving an operation that executes processing to stop outputting the video and processing to restrict transmission of the acquired operation information to the terminal device.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display device, a display control method, a program, and a display system.

Background Art

[0002] A technology is known in which a terminal device such as a smartphone or a PC (Personal Computer) and a display device such as a display or a projector perform wireless communication, and the terminal device wirelessly outputs video and audio to the display device. A technology is also known in which, through wireless communication, the display device transmits not only video and audio but also a touch signal obtained when a user touches the display device to the terminal device, thereby enabling operation of the terminal device from the display device.

[0003] A technology has been devised for concealing information that is not desired to be displayed on the display device when the terminal device causes the display device to display video (see, for example, Patent Document 1). Patent Document 1 discloses a technology in which, in response to a user's non-display operation, the terminal device temporarily stops video output.

Summary of the Invention

Problem to be Solved by the Invention

[0004] However, conventional technologies have a problem in that the display device side does not perform restrictions on stopping video output and transmitting operation position information. For example, if a user moves away from a display that is displaying video even for a short period of time, or moves while holding the display, there arises a security risk such as leakage of information from the video.

[0005] Furthermore, when a user moves while holding the display device, the user's hand may unintentionally touch the touch panel, causing the display device to transmit a touch signal to the terminal device, resulting in erroneous operation of the terminal device.

[0006] Therefore, conventionally, users had to operate the terminal device to stop video transmission while on the move, or carry the display device without touching its touch panel.

[0007] In view of the above problems, the present invention aims to provide a technology that allows the display device to stop displaying video and restrict the transmission of operation position information. [Means for solving the problem]

[0008] In view of the above problems, the present invention provides a display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, comprising: a communication unit that receives video from the terminal device by wireless communication; an output unit that outputs the video received by the communication unit to a display means; and operation information for the video Operation position information obtained based on operation of the display means of the display device An operation receiving unit that receives an operation to acquire the operation information, a process to stop the output of the video, and a display component control unit that receives an operation to perform both of the above and the process to restrict the transmission of the acquired operation information to the terminal device, and displays a first display component together with the video, The system has the following characteristics: When the operation reception unit acquires the operation position information for the first display component, the output unit stops outputting the video; the display component control unit erases the first display component and displays a second display component that resumes outputting the video; the operation reception unit restricts the transmission of the operation position information to the terminal device for any display component other than the second display component; even when the output unit has stopped outputting the video, the communication unit continues to receive video from the terminal device via wireless communication; and when the operation reception unit acquires the operation position information for the second display component, the output unit outputs the video that it has been continuously receiving. It is characterized by the following: [Effects of the Invention]

[0009] This technology allows the display device to stop displaying video and restrict the transmission of operation position information. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram illustrates the general operation of a communication system in which a display device outputs video and audio displayed by a terminal device. [Figure 2] This is an example of a system configuration diagram for a communication system. [Figure 3] This is an example of a block diagram showing the hardware configuration of a display device according to one embodiment. [Figure 4] This is an example of a hardware configuration diagram for a terminal device. [Figure 5] This is an example of a functional block diagram that explains the functions of a display device by dividing them into blocks. [Figure 6] This diagram illustrates the display layers that a display device uses for screen display. [Figure 7] This is a diagram explaining the function of the screen mute icon. [Figure 8] This is an example of a diagram showing the operation of the display device, the transitions of the icon layer, and the source video display layer in chronological order. [Figure 9] This is an example of a state transition diagram illustrating the transitions between display states in Figure 8. [Figure 10] This is an example diagram showing the time-series transitions of the display device's operation, icon layer, and source video display layer when wireless communication is disconnected while the screen is muted. [Figure 11] This is an example of a state transition diagram illustrating the transitions between the display states in Figure 10. [Figure 12] This is an example of a display state transition diagram showing how screen mute is immediately released if wireless communication is interrupted while screen mute is active. [Figure 13] This figure shows an example of an error message displayed on the screen while the screen is muted. [Figure 14] This figure shows an example configuration of a source video display layer in a display device that simultaneously displays multiple video feeds. [Figure 15] This is an example of a diagram showing the time-series transitions of the display device's operation, icon layer, and source video display layer when the display device can communicate wirelessly with multiple terminal devices. [Figure 16] This figure shows an example configuration of a source video display layer in a display device that simultaneously displays multiple video feeds. [Figure 17] This is an example of a diagram showing the time-series transitions of the display device's operation, icon layer, and source video display layer when the display device can communicate wirelessly with multiple terminal devices. [Figure 18] This figure shows an example configuration of a source video display layer in a display device that simultaneously displays multiple video feeds. [Figure 19] This figure shows another example of a display device configuration. MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, as an example of a mode for carrying out the present invention, a communication system and a display control method performed by the communication system will be described with reference to the drawings.

[0012] <Outline of Processing> FIG. 1 is a diagram illustrating the schematic operation of the communication system 100 in which the display device 20 outputs video and audio displayed by the terminal device 40. As shown in FIG. 1(a), the terminal device 40 and the display device 20 start wireless communication. The terminal device 40 displays video on its display and transmits the displayed video to the display device 20. Since the display device 20 displays the video on the display panel 102, the user can view the video on a relatively large screen.

[0013] When the terminal device 40 starts wireless communication with the display device 20, the display device 20 of the present embodiment causes the terminal device 40 to display the screen mute icon 15 (an example of a first display component) during video output. The screen mute icon 15 is a display component that allows a user to stop displaying the video 202 from the terminal device 40, mute audio from the terminal device 40, and disable the touch panel with one touch (a single operation).

[0014] FIG. 1(b) shows an example of the screen of the display device 20 after the screen mute icon 15 is pressed. The terminal device 40 continues to display video on its own display, while the display device 20 displays the screen during screen mute 203. The screen during screen mute 203 only needs to be a screen that makes the video difficult to visually recognize. Further, even if the user touches the display panel 102 of the display device 20 on the screen during screen mute 203, the display device 20 does not accept the touch input.

[0015] However, the display device 20 can accept touch operations on the screen mute release icon 16 (an example of a second display component) displayed on the screen mute screen 203. The screen mute release icon 16 is a display component for releasing the screen mute.

[0016] Thus, the display device 20 of this embodiment displays a screen mute icon 15 that stops the output of video and audio received by the display device 20 and the transmission of touch signals with a single touch. As a result, the user can hide the video and audio and disable touch signals to the terminal device 40 with a single operation.

[0017] In a portable display device 20, it is assumed that users will move the display device 20 to various locations, change the connected terminal device 40, and project images in diverse environments. In such environments, if the user leaves their seat or moves, the ability to disable video and audio output and touch operation all at once can improve both operability and security, providing a display device 20 that is easy to use in diverse environments. Conventionally, when a user leaves their seat or moves for a short time, it was cumbersome for the user to operate the terminal device 40 to stop the transmission of video and audio.

[0018] Furthermore, conventionally, if a user operated the terminal device 40 to stop the transmission of video and audio (disconnecting wireless communication), it was necessary to reconnect with the display device 20. Since the terminal device 40 can also be operated by touching the display device 20, and there are cases where the user has the terminal device 40 in their bag or clothing pocket, it was inconvenient for the user to have to operate the terminal device 40. For this reason, users tended to hesitate to stop the video and audio by operating the terminal device 40, especially when leaving their seat or moving for a short time. In this embodiment, the display device 20 can continue wireless communication with the terminal device 40 even while the screen muted screen 203 is displayed, so the display device 20 can resume outputting video 202 and audio from the terminal device without the need for reconnection.

[0019] <About Terminology> Display components are various parts that make up the screen. Display components are markers that are the target of touch operation. In this embodiment, icons are used as examples of display components, but display components may also be buttons, operation menus, symbols, or graphics. Icons are symbols that represent things using simple pictures.

[0020] The display means can be any device that has the function of displaying video from a terminal device. Examples of display means include a display or the projection function of a projector. The video can be a still image or a moving image.

[0021] Operation information is information acquired based on operations on the display means of the display device 20, and is, for example, operation position information. Operation position information is obtained by converting position information detected on the display panel 102 of the display device 20 into position information for the terminal device 40 to accept operations. Operation position information is, for example, the X,Y coordinates of the image or screen displayed by the terminal device 40. In addition to operation position information, operation information may also include information indicating whether or not an operation was performed on the display device 20, or information indicating that an operation was performed to press another button on the display device 20.

[0022] Screen muting refers to reducing the visibility of at least the video transmitted from the terminal device 40 to the extent that the user cannot see it. Screen muting may also stop the output of audio from the terminal device.

[0023] The restriction on the transmission of operation position information means that even if a user touches the touch panel and the touch panel detects the touch, the display device 20 does not transmit the information externally.

[0024] <Example System Configuration> Referring to Figure 2, an example of the system configuration of the communication system 100 will be explained. Figure 2 shows the system configuration diagram of the communication system 100. The terminal device 40 and the display device 20 can communicate wirelessly. The wireless communication method is, for example, Wi-Fi Direct (registered trademark) or Bluetooth (registered trademark), but there are no particular restrictions on the communication method. Also, the communication method does not have to be one-to-one communication (P2P), but may be communication via a Wi-Fi router (access point). Alternatively, the terminal device 40 and the display device 20 may communicate via the same server device. In this case, the terminal device 40 and the display device 20 are associated with the server device, and the server device transmits video 202 and audio from the terminal device 40 to the display device 20, and transmits touch signals from the display device 20 to the terminal device 40.

[0025] Furthermore, communication standards such as Miracast, Google Cast, and Airplay (all registered trademarks; the names of registered trademarks are omitted hereafter) are known to enable wireless communication between the terminal device 40 and the display device 20. The terminal device 40 and the display device 20 can communicate using these communication standards or improved communication methods. For example, Miracast is a display transmission technology developed by the Wi-Fi Alliance that uses one-to-one wireless communication. It can send images, audio, and video from a host to another wirelessly paired device using streaming technology. It is a technology that makes display displays, which are normally connected by dedicated cables, wireless, and is sometimes called wireless HDMI because it wirelessly implements the functions that HDMI (registered trademark) provides.

[0026] Furthermore, the terminal device 40 and the display device 20 may be connected via a wired video output interface such as HDMI (High Definition Multimedia Interface, HDMI is a registered trademark), DisplayPort (registered trademark), or Thunderbolt (registered trademark).

[0027] The terminal device 40 is, for example, a PC 40a (desktop PC, notebook PC, etc.), a smartphone 40b, or a tablet device used by the user. The terminal device 40 only needs to be able to run applications that support wireless communication and applications that play videos. The terminal device 40 is mainly portable by the user, but it may also be a stationary type. The terminal device 40 can have a general-purpose configuration, and no special hardware or installation of applications specific to this embodiment is required. However, if a dedicated application is installed on the display device 20 of this embodiment, user convenience will be further improved.

[0028] Furthermore, a device that transmits content (source) to be output, such as the terminal device 40, is called a source device. A source device does not refer to a specific device, but rather to the source of the content transmission. Similarly, a device that receives content to be output, such as the display device 20, is called a sink device. A sink device does not refer to a specific device, but rather to the destination of the content transmission. Therefore, the terminal device 40 can be a sink device, and the display device 20 can be a source device.

[0029] The display device 20 is, for example, a display 20a such as an LCD or OLED, a PC 20b (integrated display type), a television receiver 20c, a set-top box 20d, a projector 20e, or a wireless communication device 20f. The display device 20 only needs to have wireless communication functionality, but may also have display means. The display 20a, PC 20b, television receiver 20c, and projector 20e have display means, while the set-top box 20d and wireless communication device 20f do not.

[0030] The display device 20 (excluding the wireless communication device 20f) may have a built-in wireless communication function or an external wireless communication function. Since the wireless communication device 20f is the wireless communication function itself, it may be externally attached to the display device 20 excluding the wireless communication device 20f. For example, by connecting the wireless communication device 20f to the USB I / F of a display 20a, PC 20b, television receiver 20c, set-top box 20d, or projector 20e that does not have a built-in wireless communication function, these devices can communicate wirelessly with the terminal device 40. The state in which the wireless communication device 20f is connected to the display 20a, PC 20b, television receiver 20c, set-top box 20d, or projector 20e is sometimes referred to as a display system.

[0031] Examples of set-top boxes 20d include cable television receivers and satellite broadcasting receivers. Examples of wireless communication devices 20f include devices compatible with Miracast, Google Cast, and Airplay, as well as stick-type PCs.

[0032] The display device 20 may have a wireless communication application or function pre-installed, or it may be possible to install such an application. Some or all of the function may be implemented by hardware circuitry.

[0033] Furthermore, it is preferable that the display device 20 has a touch panel. The display device 20 wirelessly transmits touch signals from the user's touch to the terminal device 40, enabling the user to operate the terminal device 40 on the display device 20.

[0034] <Example Hardware Configuration> Referring to Figures 3 and 4, the hardware configuration of the display device 20 and the terminal device 40 will be described.

[0035] <<Display device>> Figure 3 is a block diagram showing the hardware configuration of a display device 20 according to one embodiment. The display device 20 shown in Figure 3 has a display panel 102 on the front of a housing (not shown) and is capable of displaying an image corresponding to a video signal on the display panel 102 by receiving a video signal from a terminal device 40 via wireless communication or a video signal conversion cable 110. The display device 20 has a built-in battery 108 and is a thin, portable display device 20. Therefore, the display device 20 can be carried together with the terminal device 40 and connected to the terminal device 40 when in use, making it usable as an extended display for the terminal device 40.

[0036] For example, the display device 20 can be used by sales representatives as an extended display for presentations at client sites. Alternatively, the display device 20 can be used as a dual display for office work in conjunction with a terminal device 40 such as a PC. Furthermore, the display device 20 can be connected to a smartphone used by an individual outside the office, allowing it to be used as an extended display for watching videos or for working from home.

[0037] As shown in Figure 3, the display device 20 includes a control unit 101, a display panel 102, a communication interface 103, a user interface 104, a speaker 109, a USB interface 105, a power supply unit 107, and a battery 108.

[0038] The control unit 101 controls the overall operation of the display device 20. For example, the control unit 101 is implemented by a control circuit such as an IC (Integrated Circuit) equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc.

[0039] The display panel 102 displays various images (videos, still images, etc.) corresponding to the video signal supplied from the control unit 101. For example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel can be used as the display panel 102. A touch sensor 102a is provided on the front of the display panel 102, superimposed on the display panel 102. The speaker 109 outputs various sounds corresponding to the audio signal supplied from the control unit 101.

[0040] Communication I / F 103 is an interface for wireless communication connection with terminal device 40 and for controlling wireless communication. Communication I / F 103 is wirelessly connected to terminal device 40 (e.g., a laptop computer, smartphone, etc.) and transmits and receives control signals, video signals, etc. (wireless communication) with terminal device 40. Wireless communication standards used by communication I / F 103 include, for example, Wi-Fi.

[0041] The hard keys 104 are an interface that accepts user input when operated by the user. For example, the display device 20 may have multiple buttons as hard keys 104, located on the side or front of the casing. These buttons may include, for example, a power button, a select button, a confirmation button, a back button, etc. However, the display device 20 may also have a screen mute key as a hard key 104, for example, a screen mute icon 15 implemented as a hard key. For example, the display device 20 can be turned on and off, and various settings such as brightness and contrast can be adjusted by operating the hard keys 104 through user operation.

[0042] The USB I / F 105 is an interface for connecting the terminal device 40. The USB I / F 105 has a USB port 105A (an example of a "connection means") that conforms to the USB standard, and a USB cable such as a video signal conversion cable 110 is connected to the USB port 105A. In this way, the USB I / F 105 is connected to the terminal device 40 via the USB cable. For example, when a terminal (e.g., a laptop computer, smartphone, etc.) is connected as the terminal device 40, the USB I / F 105 can transmit and receive control signals, video signals, power, etc. between the terminal and the USB I / F 105. Also, for example, when an external power supply is connected as the terminal device 40, the USB I / F 105 can receive power supplied from the external power supply.

[0043] The display device 20 is equipped with multiple USB I / F 105s, and each of the multiple USB I / F 105s can be connected to a terminal device 40. In the example shown in Figure 3, the display device 20 is equipped with two USB I / F 105s. However, the display device 20 is not limited to this and may be equipped with three or more USB I / F 105s.

[0044] The power supply unit 107 controls the supply of power for driving each part of the display device 20. For example, if power is not supplied from the terminal device 40 (external power supply, terminal, etc.), or if the power supplied from the terminal device 40 (external power supply, terminal, etc.) is insufficient, the power supply unit 107 can supply power stored in the battery 108 to each part of the display device 20. Also, for example, if power is supplied from the terminal device 40 (external power supply, terminal, etc.), the power supply unit 107 can supply the power supplied from the terminal device 40 to each part of the display device 20. Furthermore, for example, the power supply unit 107 can pass through power supplied from one terminal device 40 (external power supply, terminal, etc.) connected to one USB I / F 105 and supply it to another terminal device 40 (terminal, etc.) connected to another USB I / F 105.

[0045] Battery 108 stores power for driving the display device 20. Various types of rechargeable secondary batteries (e.g., lithium-ion batteries, lithium polymer batteries, etc.) are used for battery 108. The display device 20 (e.g., power supply unit 107) can charge battery 108 using power supplied from a terminal device 40 (external power supply, terminal, etc.) connected to the USB I / F 105.

[0046] <<Terminal device>> Figure 4 is a hardware configuration diagram of the terminal device 40. In Figure 4, a smartphone is used as an example of the terminal device 40. As shown in Figure 4, the terminal device 40 includes a CPU 401, ROM 402, RAM 403, EEPROM 404, CMOS sensor 405, image sensor I / F 406, acceleration / direction sensor 407, media I / F 409, and GPS receiver 411.

[0047] Of these, the CPU 401 controls the operation of the entire terminal device 40. The ROM 402 stores programs used to drive the CPU 401, such as the CPU 401 and the IPL. The RAM 403 is used as the work area for the CPU 401.

[0048] The EEPROM 404 reads or writes various data, such as programs for the terminal device 40, according to the control of the CPU 401.

[0049] The CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) and obtains image data according to the control of the CPU 401. The CMOS sensor may also be replaced with other imaging means such as a CCD (Charge Coupled Device) sensor.

[0050] The image sensor interface 406 is a circuit that controls the drive of the CMOS sensor 405. The acceleration and orientation sensor 407 is a variety of sensors, such as an electronic magnetic compass, gyrocompass, and acceleration sensor that detect the Earth's magnetic field.

[0051] The media interface 409 controls the reading or writing (storage) of data to or from the recording medium 408, such as flash memory. The GPS receiver 411 receives GPS signals from GPS satellites.

[0052] Furthermore, the terminal device 40 includes a long-range communication circuit 412, a CMOS sensor 413, an image sensor interface 414, a microphone 415, a speaker 416, an audio input / output interface 417, a display 418, an external device connection interface 419, a short-range communication circuit 420, an antenna 420a for the short-range communication circuit 420, and a touch panel 421.

[0053] Of these, the long-distance communication circuit 412 is a circuit that communicates with other devices via a wired or wireless network. The long-distance communication circuit 412 is capable of wireless communication such as Wi-Fi. The CMOS sensor 413 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 401.

[0054] The image sensor interface 414 is a circuit that controls the driving of the CMOS sensor 413. The microphone 415 is a built-in circuit that converts sound into electrical signals.

[0055] Speaker 416 is a built-in circuit that converts electrical signals into physical vibrations to produce sound such as music and speech. The sound input / output interface 417 is a circuit that processes the input and output of sound signals between the microphone 415 and speaker 416 according to the control of the CPU 401.

[0056] Display 418 is a type of display means, such as liquid crystal or organic EL (electroluminescence), that displays images of the subject and various icons. External device connection I / F 419 is an interface for connecting various external devices.

[0057] The short-range communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a type of input means that allows the user to operate the terminal device 40 by pressing the display 418.

[0058] Furthermore, the terminal device 40 is equipped with a bus line 410. The bus line 410 is an address bus, data bus, etc., for electrically connecting each component, such as the CPU 401 shown in Figure 4.

[0059] <About the features> Figure 5 is a functional block diagram illustrating the functions of the display device 20, divided into blocks. The functions of the display device 20 are mainly arranged in three layers, from bottom to top: the hardware control unit 30, the OS 25 (Operating System), and the wireless connection application 31. Although there are other applications that run on the OS 25, only the main applications are shown in this embodiment.

[0060] The hardware control unit 30 includes a touch control unit 26, a wireless control unit 27, an image and sound control unit 28, and a power supply control unit 29. Each of these functions of the hardware control unit 30 may be implemented, for example, by a device driver.

[0061] The touch control unit 26 converts the signal input from the touch sensor 102a into coordinates (X,Y coordinates with the upper left corner of the display panel 102 as the origin) and sends them to the OS 25. The touch control unit 26 may further convert the signal input into the touched area or the pressure contained in the signal input and send them to the OS 25.

[0062] The wireless control unit 27 processes the radio waves of the wireless communication received by the communication interface 103, performing filtering, amplification, demodulation, A / D conversion, etc., to convert them into digital signals. It then decapsulates them according to the communication protocol to extract the data. The wireless control unit 27 sends the data to the OS 25. This data consists of video and audio data from the terminal device 40. The wireless control unit 27 also acquires data from the OS 25, performs data encapsulation, D / A conversion, modulation, and amplification according to the communication protocol, and transmits radio waves of wireless communication from the communication interface 103. This data is the touch signal.

[0063] The image and sound control unit 28 converts the video data acquired from the OS 25 into a format compatible with the display panel 102, such as RGB, and stores it in video memory. The image and sound control unit 28 applies voltage to each cell of the display panel 102 according to the video data acquired from video memory. The image and sound control unit 28 also generates a differential signal using the audio data acquired from the OS 25 and outputs it to the positive and negative terminals of the speaker.

[0064] The power control unit 29 controls the power supply from the battery or from an external power source depending on the shutdown process from the OS 25 or the input status of the power switch or external power supply. If there is sufficient power from the external power source, the power control unit 29 charges the battery. The power control unit 29 manages the remaining power level of the battery. The power control unit 29 stops supplying power to the display panel 102 depending on the period of inactivity and resumes supplying power to the display panel 102 when operation is detected.

[0065] The wireless connection application 31 includes a display component control unit 21, an output unit 22, a communication unit 23, and an operation reception unit 24. Each function of the wireless connection application 31 is a function or means realized by the CPU of the control unit 101 executing instructions included in the wireless connection application 31.

[0066] The display component control unit 21 primarily controls the display of icons. The display component control unit 21 controls the display of icons in the icon layer (described later) according to the operation of the icons and the status of wireless communication with the terminal device 40. In addition to icons, the display component control unit 21 also displays various settings screens and other user interface elements.

[0067] The output unit 22 acquires video data and audio data (at least one) received by the wireless control unit 27 from the terminal device 40 from the OS 25 and controls the output of these by the image and audio control unit 28. The output unit 22 displays the video 202 from the terminal device 40 on the source video display layer described later and outputs the audio from the speaker 109. During screen mute, the output unit 22 displays the screen mute screen 203 on the source video display layer 210 instead of the video 202 from the terminal device 40.

[0068] The communication unit 23 sends the SSID and encryption key to the wireless control unit 27 and instructs it to start wireless communication with the terminal device 40, maintain communication, and end communication. The communication unit 23 can communicate with the terminal device 40 directly or can communicate with the terminal device 40 via a router (access point).

[0069] The operation reception unit 24 accepts touch operations on icons and touch operations on the video 202 from the terminal device 40. The operation reception unit 24 accepts an icon press if the touch coordinates are within the bounding rectangle of the icon, and accepts an operation to the terminal device 40 if the touch coordinates are not within the bounding rectangle of the icon but within the display range of the video. In addition, while the screen is muted, the operation reception unit 24 does not accept touches other than the screen mute release icon 16.

[0070] When operating the terminal device 40, the communication unit 23 transmits a touch signal to the terminal device 40. The touch signal is obtained by converting the touch coordinates on the display device 20 into coordinates on the display (image) of the terminal device 40. The converted coordinates are, for example, the ratio of the X coordinate to the width of the image on the display device 20, and the ratio of the Y coordinate to the height.

[0071] <Example of a display screen> Referring to Figure 6, an example of a screen displayed by the display device 20 communicating with the terminal device 40 will be explained. Figure 6 is a diagram illustrating the display layers used by the display device 20 for screen display. The display device 20 has two display layers.

[0072] 1. The source video display layer 210 is a display layer that displays video received from the terminal device 40. The source video display layer 210 can be approximately the same size as the display panel 102.

[0073] 2. Icon layer 220 is a display layer that displays icons related to wireless communication with the terminal device 40. Icon layer 220 may be approximately the same size as the display panel 102.

[0074] The source video display layer 210 and the icon layer 220 are superimposed on each other. Therefore, to prevent the icons from being hidden by the video, the icon layer 220 is displayed in front of the source video display layer 210 (the icon layer 220 is on the user's side). The video is drawn on the source video display layer 210 by the output unit 22, and the icons are drawn on the icon layer 220 by the display component control unit 21. The position of the icons on the icon layer 220 can be changed by the user.

[0075] A touch operation on the icon layer 220 is also a touch operation on the source video display layer 210, but if an icon is touched, the touch of the icon that is in front of the video 202 is detected. Touch operations on the source video display layer 210 other than icons are detected as touch operations on the video. For the sake of explanation, a touch operation on the source video display layer 210 will be referred to as a touch operation on the video 202 from the terminal device 40.

[0076] As shown in Figure 6(a), when wireless communication is not being performed with the terminal device 40, the connection waiting screen 208 is displayed on the source video display layer 210. The connection waiting screen 208 is a screen displayed by the display device 20 when it is powered on and started up. The connection waiting screen 208 is a screen that indicates that it is waiting for a wireless communication connection and displays connection information such as the SSID and encryption key.

[0077] When the terminal device 40 is not communicating wirelessly, the icon layer 220 displays the main unit settings icon 13. The main unit settings icon 13 is used to configure settings such as screen brightness, volume, and wireless communication settings (such as the access point's SSID and password).

[0078] As shown in Figure 6(b), when the display device 20 starts wireless communication with the terminal device 40, the video 202 displayed by the terminal device 40 is displayed on the source video display layer 210. The video 202 from the terminal device 40 is the entire screen that the terminal device 40 displays on the display 418. For example, the terminal device 40 may transmit only a part of the screen that it displays on the display 418.

[0079] When the terminal device 40 and the display device 20 are communicating wirelessly, various icons (connection-operable icons 10) for the user to adjust video and audio settings are displayed on the icon layer 220 during video output. The connection-operable icons 10 are icons that the user can use to adjust video and audio while the display device 20 is communicating wirelessly with the terminal device 40. The connection-operable icons 10 are, for example, as shown below, but in reality there may be more or fewer of them.

[0080] The summary icon 11 is an icon used to consolidate the connected, operable icons 10 into one. The display component control unit 21 can reduce the display area of ​​the connected, operable icons 10 to improve the visibility of the video 202 from the terminal device 40. When only the summary icon 11 is displayed, the summary icon 11 becomes semi-transparent.

[0081] The rotation icon 12 rotates the video 202 from the terminal device 40 by 90 degrees at a time.

[0082] • The main settings icon 13 is the same as the one shown above.

[0083] • The disconnection icon 14 is an icon for disconnecting wireless communication with the terminal device 40.

[0084] The screen mute icon 15 is used to mute the display device 20 (displaying the video, stopping audio output, and disabling touch operations other than the screen mute unmute icon 16). Further details are explained in Figure 7.

[0085] When the connected / operable icon 10 remains inactive for a certain period of time, only the summary icon 11 is displayed, and the summary icon 11 becomes semi-transparent. The display component control unit 21 can improve the visibility of the video 202 from the terminal device 40. When the user touches the semi-transparent summary icon 11, the entire connected / operable icon 10 is displayed. After a certain period of time has elapsed since the summary icon became semi-transparent, the summary icon 11 may be hidden again.

[0086] The summary icon 11 is semi-transparent, and even if the video 202 from the terminal device 40 is touched, the icon is not being operated, so the summary icon 11 remains semi-transparent. When the terminal device 40 receives a touch signal to the video 202 from the terminal device 40, it displays buttons such as pause, forward, and back on its own display 418. The display device 20 receives this video and can display the same buttons as the terminal device 40. The user can perform operations such as pause, forward, and back on the display device 20 that are possible on the terminal device 40.

[0087] Thus, touch operations on icons are used for processing operations on the display device 20. On the other hand, touch operations on the video 202 from the terminal device 40 are transmitted to the terminal device 40 during wireless communication as touch signals including the touch coordinates on the video. In this way, the icon layer 220 and the source video display layer 210 handle touch signals differently when a user touches them.

[0088] When no video is being output, the connected / operable icon 10 is not displayed. The connected / operable icon 10 is displayed when the display device 20 starts outputting video, and the connected / operable icon 10 is removed when the video output ends. This way, the user can be notified that the connected / operable icon 10 can be used to control the video.

[0089] Figure 7 is a diagram illustrating the function of the screen mute icon 15. The screen mute icon 15 is an icon that enables the screen mute function. When the screen mute icon 15 is pressed, the display device 20 performs the following three controls (screen mute function) simultaneously (in parallel). Figure 7(a) is the same as Figure 6(b), and Figure 7(b) shows the screen 203 while the screen is muted.

[0090] A. The output unit 22 changes the source video display layer 210 to a predetermined image (in this embodiment, a completely black display). This prevents the video 202 from the terminal device 40 from being visible to the user or people in the vicinity. The predetermined image can be anything that reduces the visibility of the video 202 from the terminal device 40, such as a mosaic screen, a completely white screen, a color bar for broadcast testing, or any wallpaper. The predetermined image can be a still image or a video.

[0091] B. The output unit 22 stops outputting audio from the terminal device 40 through the display device 20. This prevents the user and those nearby from hearing the audio from the terminal device 40. Only audio from the terminal device 40 is stopped from being output; sounds generated by the display device 20, such as tap sounds (click sounds), are not stopped.

[0092] C. The operation reception unit 24 disables touches to the video 202 from the terminal device 40. The operation reception unit 24 discards touch coordinates if the touch control unit 26 detects a touch signal to the display panel 102 and sends it to the operation reception unit 24, but the touch coordinates overlap with the video 202 from the terminal device 40. On the other hand, the operation reception unit 24 does not disable touch coordinates if the touch control unit 26 detects a touch signal to the display panel 102 and sends it to the operation reception unit 24, and the touch coordinates overlap with the screen mute release icon 16 described later (it accepts the operation to the screen mute release icon 16).

[0093] Furthermore, even if the screen mute icon 15 is pressed, the communication unit 23 maintains wireless communication with the terminal device 40. As a result, after the screen mute is restored, the output unit 22 can resume outputting video 202 and audio from the terminal device 40 without delay.

[0094] Furthermore, when the screen mute icon 15 is pressed, the display component control unit 21 displays only the screen mute release icon 16 on the icon layer 220. The screen mute release icon 16 is an icon that informs the user that the screen is muted and that the screen mute can be released by pressing the screen mute release icon 16.

[0095] Furthermore, pressing the screen mute icon 15 may cause the communication unit 23 to transmit a message to the terminal device 40 indicating that the screen is muted. In this case, a dedicated application is required on the terminal device 40, but the terminal device 40 can stop transmitting video and audio. Then, when the screen mute release icon 16 is pressed, the communication unit 23 transmits a message to the terminal device 40 indicating that the screen mute has been released, and the terminal device 40 can resume transmitting video and audio. While the screen is muted, the terminal device 40 and the display device 20 can reduce the power consumption that was previously consumed by wireless communication.

[0096] Alternatively, the communication unit 23 may instruct the wireless control unit 27 to disconnect wireless communication with the terminal device 40 simply by pressing the screen mute icon 15. The terminal device 40 is not notified that the screen is muted, but the terminal device 40 detects that the wireless communication has been disconnected. In this case, the terminal device 40 can reduce its power consumption, but it may require operation of the terminal device 40 to resume communication.

[0097] <Processing or operation flow> Next, referring to Figure 8, the states of the icon layer 220 and the source video display layer 210 will be explained in correspondence with the operation of the display device 20. Figure 8 is a diagram showing the transitions of the operation of the display device 20, the icon layer 220, and the source video display layer 210 in chronological order.

[0098] The icon layer 220 and the source video display layer 210 each have three display states. First, the icon layer 220 has the following three display states: (i) A state in which the standby operation icon 33 is displayed. The standby operation icon 33 displays only the icons of functions that can be operated while the wireless communication is in standby mode. For example, the main unit settings icon 13, which manages screen brightness, volume, wireless communication settings, etc., is displayed as the standby operation icon 33. (ii) A state in which the connected, operable icon 10 is displayed. The connected, operable icon 10 is an icon that allows operation during wireless communication. In this embodiment, the standby, operable icon 33 is displayed together with the connected, operable icon 10. (iii) The state in which the screen mute icon 16 is displayed. The screen mute icon 16 is an icon that is displayed when the screen is muted.

[0099] The source video display layer 210 has the following three display states: (i) The state in which the connection waiting screen 208 is displayed. The connection waiting screen 208 is a screen that is displayed from the time the display device 20 is started up until wireless communication is started. The connection waiting screen 208 displays that wireless communication is in standby mode and the method of connecting with the terminal device 40. (ii) The state in which the source device video screen is displayed. The source device video screen is a screen that displays the video 202 displayed by the terminal device 40 during wireless communication with the terminal device 40. Hereafter, the source device video screen will be referred to as the video 202 from the terminal device 40, etc. (iii) The state in which screen muting screen 203 is displayed. Screen muting screen 203 is the screen that is displayed when the screen is muted.

[0100] The display device 20 switches the states of the icon layer 220 and the source video display layer 210 to adapt to the operating state of the wireless connection application 31, as shown below.

[0101] S1: When the communication unit 23 is waiting for wireless communication, the display component control unit 21 displays a standby operation icon 33 on the icon layer 220, and the output unit 22 displays a connection waiting screen 208 on the source video display layer 210.

[0102] One example of how the terminal device 40 can initiate wireless communication with the display device 20 is by having the terminal device 40 capture a QR code (registered trademark) displayed by the display device 20. The QR code contains the IP address and password of the display device 20, allowing the terminal device 40 to connect to the display device 20. Alternatively, the display device 20 may connect to the terminal device 40.

[0103] S2: When the communication unit 23 initiates wireless communication with the terminal device 40, the display component control unit 21 displays a connected, operable icon 10 on the icon layer 220, and the output unit 22 displays the video 202 being displayed by the terminal device 40 on the source video display layer 210.

[0104] S3: If the user is using the display device 20, the procedure is the same as in step S2. The user can operate the terminal device 40 from the display device 20 by touching the video 202 from the terminal device 40.

[0105] S4: The user presses the screen mute icon 15. The operation reception unit 24 receives the press of the screen mute icon 15. The display component control unit 21 displays the screen mute release icon 16 on the icon layer 220, and the output unit 22 displays the screen mute screen 203 on the source video display layer 210. The operation reception unit 24 disables touch operations other than the screen mute release icon 16.

[0106] This prevents accidental operation because touch signals are not sent to the terminal device 40 even if the user moves with the display device 20, and reduces the risk of information from the video 202 being leaked while the user is away from their seat or moving. The communication unit 23 may cache the video while the screen 203 is muted. When the screen mute is released, the output unit 22 can display the video from immediately after it stopped.

[0107] S5: The user presses the screen mute release icon 16. The display component control unit 21 receives the press of the screen mute release icon 16. The display component control unit 21 displays the connected operation icon 10 on the icon layer 220, and the output unit 22 displays the last video 202 that the communication unit 23 had received even while the screen muted screen 203 was active on the source video display layer 210.

[0108] S6: If the user is using the display device 20, the procedure is the same as in step S2.

[0109] S7: The user disconnects the wireless connection (by pressing the disconnect icon 14). The display component control unit 21 receives the press of the disconnect icon 14. The display component control unit 21 displays the standby operation icon 33 on the icon layer 220, and the output unit 22 displays the connection standby screen 208 on the source video display layer 210.

[0110] Figure 9 is a state transition diagram illustrating the transitions in the display states shown in Figure 8. The step numbers in Figure 9 generally correspond to the steps in Figure 8. Figure 9(a) shows the transitions in the display states of the icon layer 220, and Figure 9(b) shows the transitions in the display states of the source video display layer 210. In both cases, the display states transition according to the transition conditions explained in Figure 8.

[0111] <<Disconnection of wireless communication while screen muted>> Next, with reference to Figure 10, the display state when wireless communication is disconnected during screen muting will be explained. Figure 10 is a diagram showing the operation of the display device 20, the icon layer 220, and the transition of the source video display layer 210 in chronological order when wireless communication is disconnected during screen muting. The processing in steps S1 to S4 may be the same as in Figure 8.

[0112] S21: While the screen was muted, the user operated the terminal device 40 to disconnect the wireless communication. The reason for disconnecting the wireless communication is not specified; the wireless communication may be disconnected if at least one of the display device 20 and the terminal device 40 is unable to receive radio waves due to a decrease in radio wave strength such as distance or interference.

[0113] When wireless communication is disconnected, the display component control unit 21 continues to display the screen mute release icon 16 on the icon layer 220, and the output unit 22 continues to display the screen muted screen 203 on the source video display layer 210. However, if the screen mute release icon 16 is pressed during wireless communication, the video 202 from the terminal device is displayed.

[0114] If wireless communication with the terminal device 40 is disconnected while the screen is muted, the display device 20 continues to display the screen muted screen 203, thereby concealing from those nearby whether or not the disconnection process has been performed, and thus providing a higher level of security protection.

[0115] S22: The user presses the screen mute release icon 16. The display component control unit 21 receives the press of the screen mute release icon 16. The display component control unit 21 displays the standby operation icon 33 on the icon layer 220, and the output unit 22 displays the connection standby screen 208 on the source video display layer 210.

[0116] S23: The display state during connection waiting can be the same as in step S1.

[0117] Figure 11 is a state transition diagram illustrating the transitions in the display states shown in Figure 10. The step numbers in Figure 11 generally correspond to the steps in Figure 10. Figure 11(a) shows the transitions in the display states of the icon layer 220, and Figure 11(b) shows the transitions in the display states of the source video display layer 210. In both cases, the display states transition according to the transition conditions explained in Figure 10.

[0118] However, even if the wireless communication is interrupted while the screen mute screen 203 is displayed, the screen mute screen 203 continues to be displayed. Therefore, the display state of the screen mute screen 203 can be either wireless communication in progress or wireless communication has been disconnected. For this reason, the screen mute screen 203 transitions as follows.

[0119] As explained in Figure 10, if wireless communication is interrupted while the screen mute screen 203 is displayed, the screen mute release icon 16 and the screen mute screen 203 will continue to be displayed (S21).

[0120] If the screen mute release icon 16 is pressed during wireless communication, the display component control unit 21 clears the screen mute release icon 16 and displays the connected operation-ready icon 10. Similarly, the output unit 22 switches the screen mute screen 203 to the video 202 from the terminal device 40 (S22).

[0121] If the screen mute release icon 16 is pressed after the wireless communication is disconnected, the display component control unit 21 clears the screen mute release icon 16 and displays the standby operation icon 33. Similarly, the output unit 22 switches the screen mute screen 203 to the connection standby screen 208 (S23).

[0122] Furthermore, instead of controlling the display device 20 as shown in Figures 10 and 11, the display device 20 may, considering the case where the user forgets to disconnect while the screen is muted, immediately release the screen mute if the wireless communication with the terminal device 40 is disconnected while the screen is muted.

[0123] Figure 12 shows the display state transitions when the wireless communication is disconnected while the screen is muted, and the screen mute is immediately released. In Figure 12(a), if the wireless communication is disconnected while the screen is muted (while the screen mute release icon 16 is displayed), the standby operation icon 33 is displayed (S21). In Figure 12(b), if the wireless communication is disconnected while the screen is muted (while the screen mute screen 203 is displayed), the connection waiting screen 208 is displayed.

[0124] Furthermore, as shown in Figure 12(a), if wireless communication is not disconnected while the screen is muted (while the screen mute release icon 16 is displayed), and the screen mute release icon 16 is pressed, the connected, operable icon 10 is displayed (S22). As shown in Figure 12(b), if wireless communication is not disconnected while the screen is muted (while the screen mute screen 203 is displayed), and the screen mute release icon 16 is pressed, the video 202 of the terminal device 40 is displayed.

[0125] <<Displaying error message>> As shown in Figure 13, if wireless communication is disconnected while the screen is muted, the output unit 22 may display an error message. Figure 13 shows the error message 209 displayed on the screen muted screen 203. In Figure 13, the error message 209 displays "Wireless communication has been disconnected." This allows the user to realize that wireless communication has been disconnected, even if the screen muted screen 203 remains displayed while wireless communication is disconnected, as in the process shown in Figure 10.

[0126] In this case, in step S21 of Figure 10, the output unit 22 should continue to display the screen muting screen 203 and overlay the error message 209 on the source video display layer 210.

[0127] <When a display device communicates wirelessly with multiple terminal devices> A single display device 20 can communicate wirelessly with multiple terminal devices 40 in parallel. In this case, the display device 20 divides the source video display layer 210 into multiple areas and displays multiple videos in each area.

[0128] Figure 14 shows an example configuration of the source video display layer 210 in which the display device 20 displays multiple videos simultaneously. As shown in Figure 14(a), the display device 20 communicates wirelessly with two terminal devices 40A and 40B. The display device 20 displays video 202A from terminal device 40A in the left area 50A of the source video display layer 210, and video 202B from terminal device 40B in the right area 50B. The size of the two areas 50A and 50B, and which terminal device 40's video is displayed in which area, can be controlled by the user. In addition, only one icon 10 indicating that the device is connected and operable is displayed in the icon layer 220.

[0129] As shown in Figure 14(b), even when the display device 20 is displaying multiple videos in a split screen, when the user presses the screen mute icon 15, the output unit 22 displays the screen mute screen 203 on the source video display layer 210 all at once.

[0130] By incorporating a function that switches to a screen mute screen 203 for the video from multiple terminal devices 40 at once, the output of video and audio can be stopped with a single operation, even if the user is displaying video from multiple terminal devices 40 on the display device 20.

[0131] Furthermore, when the display device 20 is wirelessly communicating with multiple terminal devices 40, it is preferable that the connected operation icon 10 has an audio mute button. The audio mute button accepts the request to stop the output of audio from both terminal devices 40A and 40B, or from one of them. The user can output only the audio from the selected terminal device 40. If the audio mute button is not pressed, the display device 20 can also output all audio from multiple terminal devices 40 simultaneously.

[0132] Figure 15 is a diagram showing the operation of the display device 20, the icon layer 220, and the transitions of the source video display layer 210 in chronological order when the display device 20 can communicate wirelessly with multiple terminal devices 40. Display state A of the source video display layer 210 is the display state of area 50A, and display state B of the source video display layer 210 is the display state of area 50B.

[0133] As explained in Figure 14, since areas 50A and 50B are switched to the screen muted screen 203 in a single operation, the processing flow of steps S31 to S37 is the same as in Figure 8. In other words, Figure 15 is simply Figure 8 with the addition of display state B of the source video display layer 210.

[0134] Furthermore, the display device 20 is not limited to displaying only two videos 202A and 202B from the terminal device 40; it can simultaneously display three or more videos.

[0135] <<Display of screen muting screen by region>> Instead of switching multiple areas 50 to the screen muted screen 203 in a single operation, the display device 20 may change only the area 50 selected by the user to the screen muted screen 203.

[0136] Figure 16 shows an example configuration of the source video display layer 210 in which the display device 20 displays multiple videos simultaneously. Figure 16(a) is the same as Figure 14(a). As shown in Figure 16(b), when the user presses the screen mute icon 15, the output unit 22 switches only area 50A of the source video display layer 210 to the screen muted screen 203. In addition, the display component control unit 21 switches the connected operationable icon 10 in area 50A to the screen mute release icon 16 and displays a new connected operationable icon 10 in area 50B. This is to enable screen muting of area 50B.

[0137] In this case, the user first selects either area 50A or 50B to switch to the screen muted screen 203, and then presses the screen mute icon 15. The area selected by the user is indicated by its outline being highlighted. In this way, the user can hide only the desired image.

[0138] Alternatively, the selection of area 50 may occur after pressing the screen mute icon 15. When the user presses the screen mute icon 15, they are prompted to touch either area 50A or 50B, and the user selects area 50A to switch to the screen mute screen 203. Alternatively, when the user presses the screen mute icon 15, thumbnails of areas 50A and 50B may be displayed, and the user may select the area to switch to the screen mute screen 203.

[0139] Figure 17 is a diagram showing the operation of the display device 20, the icon layer 220, and the transitions of the source video display layer 210 in chronological order when the display device 20 can communicate wirelessly with multiple terminal devices 40. Display state A of the source video display layer 210 is the display state of 50A, and display state B of the source video display layer 210 is the display state of 50B. First, steps S41 to S43 can be the same as in Figure 15.

[0140] S44: The user selects the area to switch to on the screen muted screen 203. The operation reception unit 24 accepts the selection. Here, 50A is assumed to be selected. The display states A and B of the icon layer 220 and the source video display layer 210 remain as they were in step S43.

[0141] S45: The user presses the screen mute icon 15. The operation reception unit 24 receives the press of the screen mute icon 15. The display component control unit 21 displays the screen mute release icon 16 in area 50A of the icon layer 220 and the connected operation enabled icon 10 in area 50B of the icon layer 220. The output unit 22 displays the screen muted screen 203 in area 50A of the source video display layer 210, and then displays the video from the terminal device 40 in area 50B of the source video display layer 210.

[0142] S46: The user presses the screen mute release icon 16. The operation reception unit 24 receives the press of the screen mute release icon 16. The display component control unit 21 erases the screen mute release icon 16 in area 50A, erases the connected operationable icon 10 in area 50B, and displays the connected operationable icon 10 in area 50A of the icon layer 220. The output unit 22 displays the video 202A displayed by the terminal device 40 in area 50A of the source video display layer 210, and then displays the video 202B displayed by the terminal device 40 in area 50B.

[0143] S47: If the user is using the display device 20, the procedure is the same as in step S43 (step S3 in Figure 8).

[0144] S48: The user disconnects the wireless communication (by pressing the disconnect icon 14). The operation reception unit 24 receives the press of the disconnect icon 14. The display component control unit 21 displays the standby operation-ready icon 33 on the icon layer 220. The output unit 22 displays the connection waiting screen 208 on the source video display layer 210. Before the start of wireless communication, the source video display layer 210 is not divided into multiple areas 50A and 50B.

[0145] <<Display of icons indicating connected / operable areas>> To prevent the user from having to select area 50A to switch to the screen muted screen 203, the display component control unit 21 may display a connected, operable icon 10 for each of areas 50A and 50B.

[0146] Figure 18 shows an example configuration of a source video display layer 210 in which the display device 20 displays multiple videos simultaneously. In Figure 18(a), the connected and operable icon 10 is displayed in both area 50A and area 50B. Figure 18(b) can be the same as Figure 16(b). In Figure 18(a), the user can switch only any area 50A to the screen mute screen 203 by operating the connected and operable icon 10 of the area 50A that they want to switch to the screen mute screen 203, without having to select area 50A by tapping or other means.

[0147] <Configuration example when the display device is a projector> Although the display device 20 of this embodiment is described as having a touch panel, the display device 20 is not limited to having a touch panel.

[0148] Figure 19 shows another example of the display device 20 configuration. In Figure 19, a projector 501 is installed on the top edge of a regular whiteboard 502. This projector 501 corresponds to the display device 20. A regular whiteboard 502 is not a flat panel display integrated with a touch panel, but a whiteboard on which the user directly writes with a marker. Note that the whiteboard can also be a blackboard, as long as it is a flat surface large enough to project an image.

[0149] The projector 501 has an ultra-short-throw optical system and can project a low-distortion image onto the whiteboard 502 from a distance of about 10 cm. This image is transmitted from a terminal device 40 connected wirelessly or via a wired connection.

[0150] The user writes on the whiteboard 502 using a dedicated electronic pen 503. The electronic pen 503 has a light-emitting part, for example at its tip, which switches on and emits light when the user presses it against the whiteboard 502 to write. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eye. The projector 501 has a camera that captures an image of the light-emitting part and analyzes the image to determine the direction of the electronic pen 503. In addition, the electronic pen 503 emits sound waves along with the light, and the projector 501 calculates the distance based on the arrival time of the sound waves. The projector 501 can determine the position of the electronic pen 503 based on its direction and distance. The handwritten data is drawn (projected) based on the position of the electronic pen 503.

[0151] The projector 501 projects an operational icon 10 while connected, so when the user presses the screen mute icon 15 with the electronic pen 503, the projector 501 identifies the pressed screen mute icon 15 based on the position of the electronic pen 503 and the ON signal of the switch.

[0152] Therefore, just as when the display device 20 has a display, the projector 501 can switch to the screen muted screen 203 and display the screen mute unmute icon 16.

[0153] <Other application examples> Although the best mode for carrying out the present invention has been described above using examples, the present invention is not limited in any way to these examples, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0154] For example, in this embodiment, the screen mute icon 15 and the screen mute unmute icon 16 were described as soft keys displayed on the touch panel. However, similar functions can also be implemented using hard keys. The display device 20 is assumed to have predetermined hard keys. The hard keys perform the same function as the screen mute icon 15 with the first press and the same function as the screen mute unmute icon 16 with the second press. Even if operation by hard keys is possible, the screen mute icon 15 and the screen mute unmute icon 16 may or may not be displayed on the display panel 102.

[0155] Furthermore, in this embodiment, the output of video and audio is stopped by pressing the screen mute icon 15, but the output of video only may be stopped, or the output of audio only may be stopped.

[0156] Furthermore, although this embodiment mentions a portable display device 20, the display device 20 may also be a stationary type. In a conference room or similar setting, each user can display the video from their terminal device 40 on a large display device 20. Examples of large display devices 20 include electronic whiteboards and digital signage.

[0157] Furthermore, the terminal device 40 and the display device 20 may be the same model. For example, if both the terminal device 40 and the display device 20 are electronic whiteboards, they can share their screens to conduct a remote conference, and both can be muted by muting the screen of one device.

[0158] Furthermore, the display device 20 may display the elapsed time since the screen muting screen 203 was displayed while the screen muting screen 203 is being displayed. The display device 20 may also change the screen muting screen 203 over time. This can prompt the user to return to video 202. Alternatively, for example, the display device 20 may change the display to a full-screen black, full-screen red, full-screen blue, or full-screen white. This allows the user to get an approximate idea of ​​the elapsed time.

[0159] Furthermore, if a certain period of time has elapsed while the screen muted screen 203 is displayed, the communication unit 23 may disconnect the wireless communication. This reduces power consumption when the user becomes engrossed in another task while the screen muted screen 203 is displayed. In this case, the screen muted screen 203 may remain as the screen muted screen 203, or it may transition to the connection waiting screen 208.

[0160] Furthermore, the configuration examples shown in Figure 4 and other figures are divided according to their main functions to facilitate understanding of the processing performed by the display device 20. The present invention is not limited by the way the processing units are divided or the names of those units. The processing of the display device 20 can be further divided into many more processing units depending on the processing content. It can also be divided so that one processing unit includes even more processing.

[0161] Furthermore, each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0162] Embodiments of the present invention bring about significant improvements in the capabilities and functionality of computers. These improvements enable users to utilize computers that provide more efficient and robust interaction with tables, which are methods for storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of a more efficient, powerful, and robust user interface. Such a user interface provides better interaction between humans and machines.

[0163] <Additional notes regarding claims> [Claim 1] A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, A display device. [Claim 2] The display device according to claim 1, characterized in that the operation information for the video is operation position information obtained based on an operation on the display means of the display device. [Claim 3] The display component control unit does not display the first display component when the video is not being output. The display device according to claim 1, characterized in that the first display component is displayed when the output unit starts outputting the video, and the first display component is erased when the output unit stops outputting the video. [Claim 4] When the operation reception unit acquires the operation position information for the first display component, The output unit stops outputting the video, The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The display device according to claim 1 or 2, characterized in that the operation reception unit restricts the transmission of the operation position information to the terminal device to any part other than the second display component. [Claim 5] Even when the output unit has stopped outputting the video, The communication unit continues to receive video from the terminal device via wireless communication. When the operation reception unit acquires the operation position information for the second display component, The display device according to claim 3, characterized in that the output unit outputs the video that was being continuously received. [Claim 6] When the output unit has stopped outputting the video, it displays a predetermined image. If the predetermined image is being displayed and the transmission of the operation position information to the terminal device for components other than the second display component is restricted, then, if wireless communication with the terminal device is disconnected, The display device according to claim 3, characterized in that the output unit continues to display the predetermined image. [Claim 7] When wireless communication with the terminal device is disconnected, and the operation reception unit acquires operation position information for the second display component, The display device according to claim 5, characterized in that the output unit displays a screen indicating that it is waiting for a wireless connection with the terminal device. [Claim 8] The display device according to claim 5 or 6, characterized in that the output unit displays a message indicating that wireless communication has been disconnected, along with the display of the predetermined image. [Claim 9] When the output unit has stopped outputting the video, it displays a predetermined image. If the predetermined image is being displayed and the transmission of the operation position information to the terminal device for components other than the second display component is restricted, then, if wireless communication with the terminal device is disconnected, The display device according to claim 3, characterized in that the output unit displays a screen indicating that it is waiting for a wireless connection with the terminal device. [Claim 10] The aforementioned communication unit receives video from multiple terminal devices via wireless communication. When the output unit is outputting multiple video images, and the operation reception unit acquires operation position information for the first display component, The display device according to any one of claims 1 to 8, characterized in that the output unit stops the output of multiple video images. [Claim 11] The aforementioned communication unit receives video from multiple terminal devices via wireless communication. When the output unit is outputting multiple video images, and the operation reception unit acquires operation position information for the first display component, The display device according to any one of claims 1 to 8, characterized in that the output unit stops outputting only the video selected by the operation reception unit. [Claim 12] The display device according to claim 10, characterized in that the output unit stops outputting video, erases the first display component related to the video whose output has been stopped, and displays the first display component for the video whose output has not been stopped. [Claim 13] The communication unit receives voice from the terminal device via wireless communication. When the operation reception unit acquires operation position information for the first display component, The display device according to any one of claims 3 to 11, characterized in that the output unit stops outputting the video and the audio. [Explanation of symbols]

[0164] 20 Display device 40 Terminal devices 100 Communication Systems [Prior art documents] [Patent Documents]

[0165] [Patent Document 1] Japanese Patent Publication No. 2005-345543

Claims

1. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation position information obtained based on an operation on the display means of the display device as operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, When the operation reception unit acquires the operation position information for the first display component, the output unit stops outputting the video. The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The operation reception unit restricts the transmission of the operation position information to the terminal device to any component other than the second display component. Even when the output unit has stopped outputting the video, the communication unit continues to receive video from the terminal device via wireless communication. A display device characterized in that, when the operation reception unit acquires the operation position information for the second display component, the output unit outputs the video that was being continuously received.

2. The display component control unit does not display the first display component when the video is not being output. The display device according to claim 1, characterized in that it displays the first display component in response to the output unit starting to output the video, and erases the first display component in response to the output unit ending to outputting the video.

3. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation position information obtained based on an operation on the display means of the display device as operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, When the operation reception unit acquires the operation position information for the first display component, the output unit stops outputting the video. The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The operation reception unit restricts the transmission of the operation position information to the terminal device to any component other than the second display component. When the output unit has stopped outputting the video, it displays a predetermined image, and when the predetermined image is being displayed and the transmission of the operation position information to the terminal device to all other components except the second display component is restricted, if wireless communication with the terminal device is disconnected, The output unit is characterized by continuing to display the predetermined image.

4. When wireless communication with the terminal device is disconnected, and the operation reception unit acquires the operation position information for the second display component, The display device according to claim 3, characterized in that the output unit displays a screen indicating that it is waiting for a wireless connection with the terminal device.

5. The display device according to claim 3, characterized in that the output unit displays a message indicating that wireless communication has been disconnected, along with the display of the predetermined image.

6. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation position information obtained based on an operation on the display means of the display device as operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, When the operation reception unit acquires the operation position information for the first display component, the output unit stops outputting the video. The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The operation reception unit restricts the transmission of the operation position information to the terminal device to any component other than the second display component. When the output unit has stopped outputting the video, it displays a predetermined image. If the predetermined image is being displayed and the transmission of the operation position information to the terminal device for components other than the second display component is restricted, then, if wireless communication with the terminal device is disconnected, The display device is characterized in that the output unit displays a screen indicating that it is waiting for a wireless connection with the terminal device.

7. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, The aforementioned communication unit receives video from multiple terminal devices via wireless communication. When the output unit is outputting multiple video images, and the operation reception unit acquires operation position information for the first display component, The output unit is characterized by stopping the output of multiple video images.

8. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, The aforementioned communication unit receives video from multiple terminal devices via wireless communication. When the output unit is outputting multiple video images, and the operation reception unit acquires operation position information for the first display component, The output unit is characterized in that it stops outputting only the video selected by the operation reception unit.

9. The display device according to claim 7, characterized in that the output unit stops outputting video, erases the first display component related to the video whose output has been stopped, and displays the first display component for the video whose output has not been stopped.

10. The communication unit receives voice from the terminal device via wireless communication. When the operation reception unit acquires the operation position information for the first display component, The display device according to claim 3, characterized in that the output unit stops the output of the video and the audio.

11. A display control method performed by a display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, The steps include receiving video from the aforementioned terminal device via wireless communication, The steps include outputting the received video to a display means, The steps include: acquiring operation position information obtained based on an operation on the display means of the display device as operation information for the aforementioned video; A step of displaying a first display component together with the video, which accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, When the operation position information for the first display component is obtained, the step of stopping the output of the video is to The steps include: erasing the first display component and displaying a second display component that resumes outputting the video; A step of restricting the transmission of the operation position information to the terminal device for components other than the second display component, If the output of the aforementioned video is stopped, a predetermined image is displayed; if the predetermined image is being displayed and the transmission of the operation position information to the terminal device other than the second display component is restricted, the wireless communication with the terminal device is disconnected, the display of the predetermined image is continued. A display control method having [a specific feature].

12. A display device that displays video received from a terminal device and transmits operation information for the video to the terminal device, A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation position information obtained based on an operation on the display means of the display device as operation information for the aforementioned video, A display component control unit is configured to display a first display component together with the video, which accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device. When the operation reception unit acquires the operation position information for the first display component, the output unit stops outputting the video. The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The operation reception unit restricts the transmission of the operation position information to the terminal device to any component other than the second display component. When the output unit has stopped outputting the video, it displays a predetermined image, and when the predetermined image is being displayed and the transmission of the operation position information to the terminal device to all other components except the second display component is restricted, if wireless communication with the terminal device is disconnected, The output unit is a program for continuing to display the predetermined image.

13. A display system comprising a wireless communication device that displays video received from a terminal device and transmits operation information for the video to the terminal device, and a display means for displaying the video, The aforementioned wireless communication device is A communication unit that receives video from the aforementioned terminal device via wireless communication, The output unit outputs the video received by the communication unit to the display means, An operation reception unit that acquires operation position information obtained based on operations on the display means as operation information for the aforementioned video, A display component control unit that displays a first display component together with the video, which accepts an operation to perform both the process of stopping the output of the video and the process of restricting the transmission of the acquired operation information to the terminal device, It has, When the operation reception unit acquires the operation position information for the first display component, The output unit stops outputting the video, The display component control unit erases the first display component and displays a second display component that resumes outputting the video. The operation reception unit restricts the transmission of the operation position information to the terminal device to any component other than the second display component. When the output unit has stopped outputting the video, it displays a predetermined image, and when the predetermined image is being displayed and the transmission of the operation position information to the terminal device to all other components except the second display component is restricted, if wireless communication with the terminal device is disconnected, The output unit continues to display the predetermined image. The aforementioned display means is A display system that displays the video output by the aforementioned output unit.

Images