Display device, method for controlling the same and terminal device

Abstract

A display device, a method for controlling the same, and a terminal device are disclosed. The terminal device transmits aspect ratio information of the terminal device to the display device, and the display device overlays the screen displayed on the terminal device onto the screen displayed on the display device based on the received aspect ratio information and displays them together.

Description

Technology Field

[0001] The present invention relates to a display device capable of intuitively minimizing time delay occurring during a mirroring (screen casting) service and a method for controlling the same, and to a display device and a terminal device that performs a mirroring service. Background Technology

[0003] A display device is a device equipped with the function of providing video that a user can view. Users can watch broadcasts through a display device. The display device provides a broadcast selected by the user from among the broadcast signals transmitted by a broadcasting station, and this broadcast video is displayed on the screen. A TV is a representative example of a display device.

[0004] Meanwhile, with the development of wired and wireless communication networks, digital devices that display screens and output data that users can visually perceive can be interconnected through wired and wireless communication networks.

[0005] In particular, terminal devices such as smartphones can perform mirroring (screen casting) services with display devices such as TVs. Mirroring service is a technology in which the screen of a terminal device is transmitted to a display device, and the display device displays the screen of the terminal device. Related background technology is disclosed in Korean Patent Publication No. 10-2018-0109340. The problem to be solved

[0006] The objective of the present invention is to provide a display device capable of minimizing time delay occurring during a mirroring (screen casting) service and a control method thereof, and a terminal device that performs a mirroring service and a display device.

[0007] In addition, the objective of the present invention is to provide a display device capable of providing a mirroring service without time delay even on a terminal device having a novel aspect ratio, a control method thereof, and a terminal device that performs a mirroring service with the display device.

[0008] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0010] According to one embodiment of the present invention, a terminal device transmits aspect ratio information of the terminal device to a display device, and the display device overlays the screen displayed on the terminal device onto the screen displayed on the display device based on the received aspect ratio information and displays them together.

[0011] A terminal device according to one embodiment of the present invention is a terminal device that performs a mirroring service with a display device, and comprises a communication unit that communicates with the display device based on a first protocol, a display unit, and a processor that controls the communication unit and the display unit, wherein the processor controls the communication unit to transmit aspect ratio information of the display to the display device.

[0012] A display device according to one embodiment of the present invention is a display device that performs a mirroring service with a terminal device, and comprises a communication unit that communicates with the terminal device based on a first protocol, a display unit, and a processor that controls the communication unit and the display unit, wherein the communication unit receives screen ratio information of the display unit from the terminal device, and the processor controls the display unit to overlay the screen of the terminal device on the screen displayed on the display unit based on the screen ratio information and display them together.

[0013] A control method for a display device according to an embodiment of the present invention is a control method for a display device that performs a mirroring service with a terminal device, comprising: a communication unit receiving screen ratio information of the display unit from the terminal device; and a processor controlling the display unit to display the screen of the terminal device together by overlaying it on the screen displayed on the display unit based on the screen ratio information. Effects of the invention

[0015] According to the present invention, time delay occurring during a mirroring (screen casting) service can be minimized.

[0016] In addition, according to the present invention, a mirroring service can be provided without time delay even on a terminal device having a novel aspect ratio.

[0017] In addition to the effects described above, the specific effects of the present invention are described together with the specific details for implementing the invention below. Brief explanation of the drawing

[0019] FIG. 1 is a diagram illustrating the schematic configuration of a digital device according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the schematic configuration of a network system according to one embodiment of the present invention. Figure 3 is a flowchart illustrating the capability negotiation of WFD in which RTSP is used. FIG. 4 is a diagram illustrating the schematic configuration of a display device according to one embodiment of the present invention. FIG. 5 is a diagram illustrating the schematic configuration of a terminal device according to one embodiment of the present invention. FIG. 6 is a diagram illustrating the operation flowchart of a network system according to one embodiment of the present invention. FIGS. 7 and FIGS. 8 are drawings illustrating an example of an overlay mirroring service according to an embodiment of the present invention. Specific details for implementing the invention

[0020] The aforementioned objectives, features, and advantages are described in detail below with reference to the attached drawings, thereby enabling those skilled in the art to easily implement the technical concept of the present invention. In describing the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions would unnecessarily obscure the essence of the invention. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0021] Although terms such as "first," "second," etc., are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used merely to distinguish one component from another, and unless specifically stated otherwise, the first component may also be the second component.

[0022] In addition, where it is stated that one component is "connected," "combined," or "connected" to another component, it should be understood that while the components may be directly connected or connected to each other, another component may be "interposed" between each component, or each component may be "connected," "combined," or "connected" through another component.

[0023] Throughout the specification, unless specifically stated otherwise, each component may be singular or plural.

[0024] As used in this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “composed” or “comprising” should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may be omitted or additional components or steps may be included.

[0025] Throughout the specification, "A and / or B" means A, B, or A and B unless specifically stated otherwise, and "C to D" means C or more and D or less unless specifically stated otherwise.

[0026] Meanwhile, the display device described in this specification is, for example, an intelligent display device that adds computer support functions to broadcast reception functions. While faithful to broadcast reception functions, it also includes internet functions and can be equipped with interfaces that are more convenient to use, such as a handwriting input device, a touch screen, or a spatial remote control. Furthermore, by supporting wired or wireless internet functions, it can be connected to the internet and computers to perform functions such as email, web browsing, banking, or games. A standardized general-purpose OS may be used for these various functions.

[0027] Additionally, the mobile terminal devices described in this specification may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), etc.

[0028] However, those skilled in the art will readily understand that the configuration according to the embodiments described in this specification may also be applied to fixed terminal devices such as digital display devices, desktop computers, and digital signage, except in cases where it is applicable only to mobile terminal devices.

[0029] Hereinafter, a display device and a control method thereof according to some embodiments of the present invention will be described.

[0031] FIG. 1 is a diagram illustrating the schematic configuration of a digital device according to one embodiment of the present invention.

[0032] Referring to FIG. 1, the digital device (100) may include a broadcast receiver (110), an external device interface unit (120), a storage unit (130), a user interface unit (140), a display unit (150), an audio output unit (160), a power supply unit (170), and a control unit (180).

[0033] Here, the broadcast receiver (110) may include a tuner (111), a demodulator (112), and a network interface (113). However, depending on the case, the broadcast receiver (110) may be equipped with a tuner (111) and a demodulator (112) but may not include a network interface (113), and vice versa.

[0034] Additionally, the broadcast receiver (110) may include a multiplexer, although not illustrated. In this case, the multiplexer may multiplex the signal demodulated by the demodulator (112) and the signal received through the network interface (113). Furthermore, the broadcast receiver (110) may include a demultiplexer, although also not illustrated. The demultiplexer may demultiplex the multiplexed signal or demultiplex the signal demodulated by the demodulator (112) or the signal received through the network interface (113).

[0035] The tuner (111) tunes a specific RF (Radio Frequency) broadcast signal. The specific RF broadcast signal corresponds to a channel selected by the user or any previously stored channels. Additionally, the tuner (111) converts the RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

[0036] For example, the tuner (111) converts an RF broadcast signal, which is a digital broadcast signal, into a digital IF signal (DIF), and converts an RF broadcast signal, which is an analog broadcast signal, into an analog baseband video or audio signal (CVBS / SIF). That is, the tuner (111) can process both digital broadcast signals and analog broadcast signals. The analog baseband video or audio signal (CVBS / SIF) output from the tuner (111) can be directly input to the control unit (180).

[0037] Additionally, the tuner (111) can receive RF broadcast signals of a single carrier or multiple carriers. Meanwhile, the tuner (111) can sequentially tune and receive RF broadcast signals of all stored broadcast channels through a channel memory function and convert them into intermediate frequency signals or baseband signals.

[0038] The demodulation unit (112) can receive and demodulate a digital IF signal converted by the tuner (111) and perform channel decoding, etc. To do this, the demodulation unit (112) may be equipped with a trellis decoder, a de-interleaver, a Reed-Solomon decoder, etc., or may be equipped with a convolution decoder, a de-interleaver, and a Reed-Solomon decoder, etc.

[0039] The demodulation unit (112) can output a stream signal (TS) after performing demodulation and channel decoding. At this time, the stream signal may be a signal in which a video signal, an audio signal, or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 TS (Transport Stream) in which a video signal of the MPEG-2 standard, an audio signal of the Dolby AC-3 standard, etc. are multiplexed.

[0040] The stream signal output from the demodulation unit (112) can be input to the control unit (180). The control unit (180) controls demultiplexing, video / audio signal processing, etc., and can control the output of video through the display unit (150) and the output of audio through the audio output unit (160).

[0041] The external device interface section (120) provides an interfacing environment between the digital device (100) and various external devices.

[0042] The external device interface unit (120) can be connected via wired or wireless connection to external devices such as a DVD (Digital Versatile Disk), Blu-ray, game device, camera, camcorder, computer (laptop), tablet PC, smartphone, Bluetooth device, cloud, etc. The external device interface unit (120) transmits a signal containing data such as images, video, and audio input through the connected external device to the control unit (180). The control unit (180) can control the output of the processed images, video, audio, etc. data signals to the connected external device. To this end, the external device interface unit (120) may further include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

[0043] The A / V input / output unit may include a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), a DVI (Digital Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, etc., so as to input video and audio signals from an external device into a digital device (100).

[0044] The wireless communication unit can perform short-range wireless communication with other digital devices. The digital device (100) can be network-connected with other digital devices according to communication protocols such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra Wideband), ZigBee, and DLNA (Digital Living Network Alliance).

[0045] Additionally, the external device interface section (120) is connected to the set-top box (STB) through at least one of the various terminals described above and can perform input / output operations with the set-top box (STB).

[0046] Meanwhile, the external device interface unit (120) can receive an application or an application list within an adjacent external device and transmit it to the control unit (180) or storage unit (130).

[0047] The network interface section (113) provides an interface for connecting the digital device (100) to a wired / wireless network. The network interface section (113) may be equipped with an Ethernet terminal or the like for connection to a wired network, and may use communication standards such as WLAN (Wireless LAN), Wi-Fi, Wibro (Wireless broadband), WiMAX (World Interoperability for Microwave Access), and HSDPA (High Speed ​​Downlink Packet Access) for connection to a wireless network.

[0048] The network interface unit (113) can transmit and receive data with other digital devices through a connected network or another network linked to the connected network. In particular, it can transmit some of the content data stored in the digital device (100) to a selected digital device among other digital devices that are pre-registered.

[0049] Meanwhile, the network interface unit (113) can access a specific web page through a connected network or another network linked to the connected network. That is, it can access a specific web page through the network and transmit or receive data with the corresponding server. In addition, it can receive content or data provided by a content provider or network operator. That is, it can receive content such as movies, advertisements, games, VOD, broadcast signals, and related information provided by a content provider or network provider through the network. It can also receive firmware update information and update files provided by a network operator. Furthermore, it can transmit data to the Internet or to a content provider or network operator.

[0050] The network interface unit (113) can select and receive a desired application among the applications open through the network.

[0051] The storage unit (130) can store programs for each signal processing and control within the control unit (180), and can store signal-processed video, audio, or data signals.

[0052] The storage unit (130) can temporarily store video, audio, or data signals input from the external device interface unit (120) or the network interface unit (113). The storage unit (130) can store information regarding a predetermined broadcast channel through a channel memory function.

[0053] The storage unit (130) can store an application or a list of applications input from an external device interface unit (120) or a network interface unit (330).

[0054] The storage unit (130) may store various platforms described later.

[0055] The storage unit (130) may include at least one type of storage medium among, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), RAM, ROM (EEPROM, etc.). The digital device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the storage unit (130) and provide them to the user.

[0056] Meanwhile, the storage unit (130) may be included within the control unit (180) and may be provided separately from the control unit (180).

[0057] The user input interface unit (140) transmits a signal input by the user to the control unit (180) or transmits a signal from the control unit (180) to the user.

[0058] For example, the user input interface unit (140) is connected to the remote control device (190) according to various communication methods, such as RF communication and infrared (IR) communication, and can receive and process control signals transmitted from the remote control device (190), such as power on / off, channel selection, and screen setting, or transmit control signals from the control unit (180) to the remote control device (500). In addition, the user input interface unit (140) can transmit control signals input from local keys (not shown), such as power key, channel key, volume key, and setting value, to the control unit (180).

[0059] The user input interface unit (140) can transmit a control signal input from a sensing unit (not shown) that senses a user's gesture to a control unit (180), or transmit a signal from the control unit (180) to the sensing unit (not shown). Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, a motion sensor, etc.

[0060] The control unit (180) can demultiplex a stream input through the tuner (111), demodulator (112), or external device interface unit (120), or process the demultiplexed signals to generate and output a signal for video or audio output.

[0061] The image signal processed by the control unit (180) can be input to the display unit (150) and displayed as an image corresponding to the image signal. Additionally, the image signal processed by the control unit (180) can be input to an external output device through the external device interface unit (120).

[0062] The voice signal processed by the control unit (180) can be output as audio to the audio output unit (160). Additionally, the voice signal processed by the control unit (180) can be input to an external output device through the external device interface unit (120).

[0063] Meanwhile, the control unit (180) may include a demultiplexer, an image processing unit, etc.

[0064] The control unit (180) can control the overall operation of the digital device (100). For example, the control unit (180) can control the tuner (111) to tune the RF broadcast corresponding to a channel selected by the user or a previously stored channel.

[0065] The control unit (180) can control the digital device (100) by means of user commands or internal programs input through the user input interface unit (140). In particular, it can connect to a network to download an application or a list of applications desired by the user into the digital device (100).

[0066] For example, the control unit (180) controls the tuner (111) so that the signal of the selected channel is input according to a predetermined channel selection command received through the user input interface unit (140). Then, it processes the video, audio, or data signal of the selected channel. The control unit (180) controls the user-selected channel information, etc., so that it can be output through the display unit (150) or audio output unit (160) together with the processed video or audio signal.

[0067] As another example, the control unit (180) controls the video signal or audio signal from an external device to be output through the display unit (150) or audio output unit (160) in accordance with an external device video playback command received through the user input interface unit (140).

[0068] Meanwhile, the control unit (180) can control the display unit (150) to display an image. For example, it can control the display unit (150) to display a broadcast image input through the tuner (111), an external input image input through the external device interface unit (120), an image input through the network interface unit, or an image stored in the storage unit (130). At this time, the image displayed on the display unit (150) may be a still image or a video, and may be a 2D image or a 3D image.

[0069] Additionally, the control unit (180) can control the display unit (150) to play content. At this time, the content may be content stored in the digital device (100), received broadcast content, or external input content input from the outside. The content may be at least one of broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0070] Meanwhile, the control unit (180) can control the display of applications or a list of applications that can be downloaded from an internal or external network of the digital device (100) when entering the application view item.

[0071] The control unit (180) can control the installation and execution of applications downloaded from an external network, along with various user interfaces. Additionally, the control unit (180) can control the display unit (150) to display an image related to an application executed by the user's selection.

[0072] Meanwhile, although not shown in the drawing, it is also possible to further provide a channel browsing processing unit that generates a thumbnail image corresponding to a channel signal or an external input signal.

[0073] The channel browsing processing unit receives a stream signal (TS) output from the demodulation unit (320) or a stream signal output from the external device interface unit (120), and can generate a thumbnail image by extracting an image from the input stream signal. The generated thumbnail image can be input to the control unit (180) as is or encoded. Additionally, the generated thumbnail image can be encoded in a stream form and input to the control unit (180). The control unit (180) can display a thumbnail list containing multiple thumbnail images on the display unit (150) using the input thumbnail image. Meanwhile, the thumbnail images within this thumbnail list can be updated sequentially or simultaneously. Accordingly, the user can easily identify the content of multiple broadcast channels.

[0074] The display unit (150) converts the video signal, data signal, OSD signal processed by the control unit (180) or the video signal, data signal, etc. received from the external device interface unit (120) into R, G, and B signals, respectively, to generate a driving signal.

[0075] The display unit (150) may be a PDP display unit, an LCD display unit, an OLED display unit, a flexible display unit, a 3D display unit, etc.

[0076] Meanwhile, the display unit (150) is configured as a touch screen and can be used as an input device in addition to an output device.

[0077] The audio output unit (160) receives a voice-processed signal from the control unit (180), for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs it as voice. The voice output unit (160) can be implemented as a speaker of various types.

[0078] Meanwhile, to detect a user's gesture, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the digital device (100) as described above. A signal detected by the sensing unit (not shown) may be transmitted to a control unit (3180) through a user input interface unit (140).

[0079] Meanwhile, a shooting unit (not shown) for shooting the user may be further provided. Image information captured by the shooting unit (not shown) may be input to the control unit (180).

[0080] The control unit (180) may detect a user's gesture by each or in combination with an image captured from a shooting unit (not shown) or a signal detected from a sensing unit (not shown).

[0081] The power supply unit (170) supplies power to the components of the digital device (100). In particular, it can supply power to a control unit (180) which can be implemented in the form of a System on Chip (SoC), a display unit (150) for displaying images, and an audio output unit (160) for audio output.

[0082] To this end, the power supply unit (170) may be equipped with a converter (not shown) that converts AC power into DC power. Meanwhile, if the display unit (150) is implemented as a liquid crystal panel having a plurality of backlight lamps, it may further be equipped with an inverter (not shown) capable of Pulse Width Modulation (PWM) operation for brightness variation or dimming driving.

[0083] The digital device (100) may be a digital broadcast receiver capable of processing fixed or mobile ATSC or DVB digital broadcast signals.

[0084] In addition, the digital device (100) may omit some of the illustrated configurations as needed, or conversely, include additional configurations that are not illustrated. Meanwhile, unlike the above description, the digital device (100) may not be equipped with a tuner and a demodulator, and may receive and play content through a network interface or an external device interface.

[0085] Meanwhile, among the digital devices (100), a terminal device such as a smartphone and a display device such as a TV can perform a mirroring (screen casting) service through linkage. In particular, the mirroring service may include an overlay mirroring service in which a screen displayed on the display device and a screen displayed on the terminal device are overlaid together on the display device.

[0086] When performing a conventional overlay mirroring service, the display device transmits the screen displayed on the terminal device to the display device as is, calculates the aspect ratio of the received screen (i.e., the ratio of the horizontal length to the vertical length of the terminal device's screen), and can display the terminal device's screen together with the display device's screen based on the calculated aspect ratio. The process of calculating the aspect ratio is performed through a background operation. For example, the display device calculates the aspect ratio by calculating the horizontal and vertical lengths of the black areas (i.e., empty areas where image data does not exist) present on the terminal device's screen.

[0087] However, conventional overlay mirroring services may experience time delays during the process in which the display device calculates the aspect ratio. Additionally, when a new terminal device with an unknown aspect ratio performs the overlay mirroring service, a blank space may appear on the terminal device's screen during the overlay process due to the lack of a definition for the new aspect ratio.

[0088] Hereinafter, a display device and a control method according to an embodiment of the present invention will be described with reference to the contents described above.

[0090] FIG. 2 is a diagram illustrating the schematic configuration of a network system according to one embodiment of the present invention.

[0091] Referring to FIG. 2, the network system may be a small-scale network system installed in a home and includes a display device (210) and a terminal device (220).

[0092] The display device (210) may be, for example, a TV and displays video to the user. For example, it communicates with an external server via the internet and displays video received from the external server.

[0093] The terminal device (220) may be, for example, a smartphone, which is a mobile terminal device, and is connected to the display device (210) via communication. A software program may be installed on the terminal device (220), the installed software program is executed, and the execution screen of the software program is displayed to the user.

[0094] Meanwhile, the display device (210) and the terminal device (220) can perform a mirroring service. That is, a screen displayed on the terminal device (220) is transmitted to the display device (210), and the display device (210) can display the screen of the received terminal device (220) to the user. In particular, the display device (210) can display the screen of the terminal device (220) together with the screen it intends to provide (overlay mirroring).

[0095] Meanwhile, a mirroring service can be performed based on the first protocol, and the display device (210) and the terminal device (220) can be connected via WiFi. In particular, the first protocol may be RTSP (Real-Time Streaming Protocol).

[0096] RTSP is a communication protocol for transmitting and receiving voice or video in real time. It is a network control protocol designed for use in communication systems to control streaming media servers. RTSP is used to establish media sessions between endpoints and to control the established sessions. Clients of the media server facilitate real-time media streaming control by issuing VHS-style commands such as play, record, and pause.

[0097] Compared to the HTTP protocol, the RTSP protocol has similarities in syntax and operation, but it differs in that RTSP defines control sequences useful for controlling multimedia playback. Additionally, it differs in that while HTTP is stateless, RTSP is stateful.

[0098] Figure 3 is a flowchart illustrating the capability negotiation of WFD in which RTSP is used.

[0099] The terminal device (220) (i.e., WFD Source or Primary Sink) can transmit an RTSP M1 request message to the display device (210) (i.e., WFD Sink or Second Sink) to initiate an RSTP procedure and WFD capability negotiation (step S302).

[0100] The RTSP M1 request message may include an RTSP OPTIONS request to determine the set of RTSP methods supported by the display device (210) (i.e., WFD Sink).

[0101] A display device (210) that receives an RTSP M1 request message can send an RTSP M1 response message, in which the RTSP methods it supports are listed, to a terminal device (220) (step S304).

[0102] Subsequently, the display device (210) can transmit an RTSP M2 request message to determine the set of RTSP methods supported by the terminal device (220) (step S306).

[0103] When an RTSP M2 request message is received, the terminal device (220) can send an RTSP M2 response message, in which the RTSP methods it supports are listed, to the display device (210) (step S308).

[0104] And, the terminal device (220) can transmit an RTSP M3 request message (RTSP GET_PARAMETER request message) specifying a list of WFD capabilities to be known to the display device (210) (step S310).

[0105] When an RTSP M3 request message is received, the display device (210) can transmit an RTSP M3 response message (RTSP GET_PARAMETER response message) to the terminal device (220) (step S312).

[0106] After that, the terminal device (220) can determine the optimal set of parameters to be used during the WFD session based on the RTSP M3 response message and transmit an RTSP M4 request message (RTSP SET_PARAMETER request message) containing the determined set of parameters to the display device (210) (step S314).

[0107] A display device (210) that receives an RTSP M4 request message can transmit an RTSP M4 response message (RTSP SET_PARAMETER response message) to a terminal device (220) (step S316).

[0109] Hereinafter, the display device (210) and the mobile terminal device (220) will be described in more detail.

[0110] FIG. 4 is a diagram illustrating the schematic configuration of a display device (210) according to one embodiment of the present invention.

[0111] Referring to FIG. 4, the display device (210) includes a first communication unit (211), a first display unit (212), a first memory (213), and a first processor (214).

[0112] Below, the function of each component will be explained in detail.

[0113] The first communication unit (211) is connected to a plurality of external servers and terminal devices (220) for communication. At this time, the communication unit (211) may include a mobile communication module for communication with a plurality of external servers and a short-range communication module for communication with a terminal device (220).

[0114] A mobile communication module can perform communication using technical standards or communication methods for mobile communication. Here, technical standards or communication methods may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G networks, etc.

[0115] A short-range communication module is for short-range communication and may include at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0116] The first display unit (212) may be composed of a liquid crystal display unit (LCD), a light-emitting diode display unit (LED), an organic light-emitting diode display unit (OLED), etc., and performs the function of displaying images or videos to the user.

[0117] The first memory (213) may be a volatile and / or non-volatile memory and stores instructions or data related to at least one other component of the display device (210). The first memory (213) may store a software program for performing a mirroring service.

[0118] The first processor (214) may include one or more of a central processing unit, an application processor, or a communication processor. The first processor (214) may execute operations or data processing regarding the control and / or communication of at least one other component of the display device (210), and may execute said instructions. In particular, the first processor (214) may execute said software program.

[0120] FIG. 5 is a diagram illustrating the schematic configuration of a terminal device (220) according to one embodiment of the present invention.

[0121] Referring to FIG. 5, the terminal device (220) includes a second communication unit (221), a second display unit (222), a sensing unit (223), a second memory (224), and a second processor (225).

[0122] Below, the function of each component will be explained in detail.

[0123] The second communication unit (221) is connected to the display device (210) for communication. At this time, the communication unit (211) may include a short-range communication module for communication with the display device (210). Of course, the second communication unit (221) may be connected to another digital device for communication, and may include a mobile communication module for communication with another digital device.

[0124] The second display unit (222) may be composed of a liquid crystal display (LCD), a light-emitting diode display (LED), an organic light-emitting diode display (OLED), etc. The second display unit (222) performs the function of displaying images or videos to a user. Additionally, the second display unit (222) may be a touch display unit capable of receiving touch signals.

[0125] The sensing unit (223) obtains internal information and surrounding environment information of the terminal device (220) using various sensors.

[0126] For example, the sensing unit (223) includes a proximity sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, etc.

[0127] The second memory (224) may be a volatile and / or non-volatile memory and stores instructions or data related to at least one other component of the terminal device (220). The second memory (224) may store a software program for performing a mirroring service.

[0128] It stores data related to multiple software programs.

[0129] The second processor (225) may include one or more of a central processing unit, an application processor, or a communication processor. The second processor (225) may execute operations or data processing regarding the control and / or communication of at least one other component of the terminal device (220), and may execute the instructions.

[0130] Hereinafter, the operation of the terminal device (220) controlling the display device (210) will be explained in more detail with reference to the drawings below.

[0132] FIG. 6 is a diagram illustrating the operation flowchart of a network system according to one embodiment of the present invention.

[0133] The network system includes a display device (210) and a terminal device (220) as described above. At this time, the steps shown on the left correspond to a control method of the terminal device (220), and the steps shown on the right correspond to a control method of the display device (210).

[0134] Below, the process performed at each stage is explained in more detail.

[0135] In step (S602), a mirroring service is requested by the user.

[0136] In this case, the mirroring service can be performed based on RTSP.

[0137] In step (S604), the second communication unit (221) attempts to establish a connection according to RTSP. At this time, the first protocol may be RTSP.

[0138] In step (S606), the terminal device (220) and the display device (210) perform WFD capability negotiation. This is as illustrated in FIG. 3.

[0139] Meanwhile, within the WFD capability negotiation, the second communication unit (221) can transmit aspect ratio information of the screen displayed on the second display unit (222) to the first communication unit (212).

[0140] Here, the aspect ratio information of the screen may mean the ratio of the length information in the horizontal direction (first direction) of the screen displayed on at least a part of the second display unit (220) to the length information in the vertical direction (second direction).

[0141] That is, the second display unit (220) has a physical size, and the screen may be displayed on the entire second display unit (220) or on a part of the second display unit (220). The aspect ratio of the screen may be determined according to the user's settings or based on the default settings of the software program implementing the screen. Accordingly, the second processor (225) can calculate the aspect ratio of the second display unit (220) according to the above-mentioned settings.

[0142] For example, the aspect ratio of the screen can be 4:3, 18:9, 16:9, etc.

[0143] Meanwhile, the second processor (225) knows in advance the screen information of the software program displayed on the second display unit (221). Therefore, the terminal device (220) can transmit the aspect ratio information of the screen to the display device (210) without any additional procedures.

[0144] According to one embodiment of the present invention, the second communication unit (221) can transmit screen ratio information to the first communication unit (211) using an RTSP M4 request message.

[0145] The RTSP M4 request message may be a message used to transmit additional information over the RTSP. Accordingly, the terminal device (220) can transmit screen ratio information to the display device (210) through the RTSP M4 request message.

[0146] In step (S608), the terminal device (220) and the display device (210) complete a communication connection for the mirroring service.

[0147] In step (S610), the first processor (214) can overlay and display together the screen displayed on the second display unit (222) on the screen displayed on the first display unit (212) based on the received screen ratio information.

[0148] In summary, conventional overlay mirroring services may experience time delays during the process in which the display device calculates the aspect ratio, and when a new terminal device with an unknown aspect ratio performs the overlay mirroring service, blank spaces may occur on the screen during the overlay process.

[0149] However, in the case of the present invention, the terminal device (220) transmits known aspect ratio information of the screen to the display device (210), and the display device (210) can display screens by overlaying them using the received aspect ratio information.

[0150] Therefore, since the display device (210) does not need to calculate the aspect ratio of the screen, no time delay occurs. In addition, even if a new terminal device (220) is overlaid and displayed on the display device (210), no blank space occurs on the screen of the overlaid terminal device (220).

[0151] Meanwhile, although not illustrated in FIG. 6, there may be cases where the aspect ratio of the screen displayed on the terminal device (220) changes. In this case, the terminal device (220) can also transmit the changed aspect ratio of the screen to the display device (210). That is, the second processor (225) can control the second communication unit (221) to transmit the aspect ratio information of the screen to the display device (210) whenever the aspect ratio of the screen changes. Based on the aspect ratio information of the changed screen, the display device (210) can overlay the screen displayed on the second display unit (222) on the screen displayed on the first display unit (212) and display them together.

[0152] At this time, a change in the aspect ratio can be detected through an accelerometer included in the sensing unit (223). That is, the second processor (225) determines whether the terminal device (220) is positioned horizontally or vertically based on information detected by the accelerometer, and if the terminal device (220) changes its position from horizontal to vertical or the terminal device (220) changes its position from vertical to horizontal, it can determine that the aspect ratio has changed.

[0153] FIG. 7 is a diagram illustrating the situation described above. At this time, it is assumed that the physical screen ratio of the second display unit (222) is 18:9 based on the landscape mode.

[0154] Referring to FIG. 7(a), at the first time point, when the terminal device (220) is positioned in portrait mode and the screen of the second display unit (222) is displayed on the entire second display unit (222), the terminal device (220) can transmit information with a ratio of 9:18 to the display device (210).

[0155] And, referring to FIG. 7(b), at a second time point after the first time point, when the terminal device (220) is positioned in landscape mode and the screen of the second display unit (222) is displayed on the entire second display unit (222), the terminal device (220) transmits information with an 18:9 aspect ratio to the display device (210). In this case, the display device (210) can change the orientation of the screen of the second display unit (222) displayed on the first display unit (212) based on the aspect ratio information of the changed screen.

[0156] Meanwhile, an image may be displayed on the second display device (222) of the terminal device (220). At this time, the image is a concept that includes video. In this case, the terminal device (220) may further transmit image ratio information to the display device (210). Also, the terminal device (220) may further transmit image ratio information to the display device (210) whenever the image ratio changes. At this time, the image ratio information may refer to the ratio of the length information in the horizontal direction (first direction) of the image to the length information in the vertical direction (second direction).

[0157] FIG. 8 is a diagram illustrating the situation described above. At this time, it is assumed that the physical screen ratio of the second display unit (222) is 18:9 based on the landscape mode.

[0158] Referring to FIG. 8(a), at the first time point, the terminal device (220) is positioned in portrait mode, and the image is displayed in a 16:9 aspect ratio within the second display unit (222), and the upper and lower areas of the image are displayed as black areas. In this case, the terminal device (220) transmits data related to the image excluding the black areas, along with aspect ratio information of the displayed image, to the display device (210), and the display device (210) can display only the image displayed on the terminal device (220) as an overlay.

[0159] Referring to FIG. 8(b), at the second time point, the terminal device (220) is positioned in landscape mode, and the image is displayed in a 16:9 aspect ratio within the second display unit (222), and the left and right areas of the image are displayed as black areas. Even in this case, the terminal device (220) transmits data related to the image excluding the black areas, along with aspect ratio information of the displayed image, to the display device (210), and the display device (210) can display only the image displayed on the terminal device (220) as an overlay.

[0160] In summary, by providing screen ratio information to the display device (210), the time delay occurring during the mirroring service can be minimized. Additionally, by providing image ratio information along with screen ratio information to the display device (210), the mirroring service can be provided without time delay even on a terminal device having a new screen ratio.

[0162] Furthermore, although it has been described that all components constituting an embodiment of the present invention are combined or operate as a single unit, the present invention is not necessarily limited to such an embodiment, and within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Additionally, while all components may each be implemented as a single independent piece of hardware, some or all of the components may be selectively combined to be implemented as a computer program having a program module that performs some or all of the combined functions on one or more pieces of hardware. The codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. An embodiment of the present invention may be implemented by storing such a computer-readable storage medium and reading and executing it by a computer. As a storage medium for the computer program, it includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording element. Furthermore, a computer program implementing an embodiment of the present invention includes a program module that is transmitted in real time through an external device.

[0163] Although the present invention has been described above with reference to the illustrated drawings, the present invention is not limited by the embodiments and drawings disclosed in this specification, and it is obvious that various modifications can be made by a person skilled in the art within the scope of the technical concept of the present invention. Furthermore, even if the effects of the configuration of the present invention were not explicitly described while explaining the embodiments of the present invention above, it is natural to acknowledge that the effects predictable by said configuration should also be recognized.

Claims

Claim 1 A terminal device that performs an overlay mirroring service with a display device, comprising: a communication unit connected to the display device based on a first protocol; an accelerometer; a display unit; and a processor that controls the communication unit and the display unit, wherein the processor controls the communication unit to transmit information regarding the physical aspect ratio of the display unit to the display device before the communication connection for the overlay mirroring service is completed, so that the screen of the display unit is overlaid and displayed on a part of the screen of the display device at the aspect ratio, and during the process of performing the overlay mirroring service after the communication connection for the overlay mirroring service is completed, the processor determines whether the display mode of the terminal device is a horizontal mode located in the horizontal direction or a vertical mode located in the vertical direction based on information detected by the accelerometer, and if the display mode is changed from one of the horizontal and vertical modes to the other, determines that the aspect ratio has been changed, and controls the communication unit to transmit information regarding the changed aspect ratio to the display device, thereby changing the aspect ratio of the screen of the display unit that is overlaid and displayed on the display device. Claim 2 A terminal device according to claim 1, wherein the first protocol is RTSP (real-time streaming protocol), and the information of the aspect ratio is transmitted to the display device through an M4 request message of the RTSP. Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 A terminal device according to claim 1, wherein the processor controls the communication unit to further transmit image ratio information to the display device when an image is displayed on the screen of the display unit. Claim 7 A terminal device according to claim 6, wherein the processor controls the communication unit to transmit the aspect ratio information of the image to the display device whenever the aspect ratio of the image changes. Claim 8 A terminal device according to claim 6, wherein the display unit displays an image and a black area located adjacent to the image together, and the processor controls the communication unit to transmit data related to the image excluding the black area to the display device, so that only the image is displayed on the display device overlaid with a part of the screen of the display device. Claim 9 delete Claim 10 A display device that performs an overlay mirroring service with a terminal device, comprising: a communication unit connected to the terminal device based on a first protocol; a display unit; and a processor that controls the communication unit and the display unit; wherein, before the communication connection for the overlay mirroring service is completed, the communication unit receives information on the physical aspect ratio of the terminal device, and during the process of performing the overlay mirroring service after the communication connection for the overlay mirroring service is completed, the processor controls the display unit to display the screen of the terminal device overlayed at the aspect ratio on a part of the screen of the display unit based on the information on the aspect ratio, the communication unit receives information on the changed aspect ratio of the terminal device, and the processor controls the display unit to change the aspect ratio of the screen of the terminal device displayed overlayed on the display unit based on the information on the changed aspect ratio, wherein the information on the changed aspect ratio is generated when it is determined that the display mode of the terminal device has changed according to information detected by the accelerometer of the terminal device, and the display mode includes a horizontal mode located in the horizontal direction and a vertical mode located in the vertical direction. Claim 11 In paragraph 10, the first protocol is RTSP (real-time streaming protocol), and the display device receives information on the aspect ratio from the terminal device through an M4 request message of the RTSP. Claim 12 A display device according to claim 10, wherein the communication unit further receives aspect ratio information of an image displayed on the screen of the terminal device, and the processor further controls the display unit to display the screen of the terminal device by overlaying it on a part of the screen of the display unit based on the aspect ratio information of the image. Claim 13 A display device according to claim 12, wherein the communication unit receives data related to the image excluding the black area among the image and black area displayed on the screen of the terminal device, the black area is located adjacent to the image, and the processor controls the display unit to display only the image by overlaying it with a part of the screen of the display unit based on ratio information of the image. Claim 14 A method for controlling a terminal device and a display device that performs an overlay mirroring service, comprising: a step in which a communication unit receives information on the physical aspect ratio of the terminal device before a communication connection for the overlay mirroring service is completed; and a step in which a processor performs the overlay mirroring service based on the information on the aspect ratio after a communication connection for the overlay mirroring service is completed; wherein the step of performing the overlay mirroring service comprises: a step in which the processor controls the display unit to display the screen of the terminal device overlayed at the aspect ratio on a part of the screen of the display device based on the information on the aspect ratio; and a step in which the processor controls the display device to change the aspect ratio of the screen of the terminal device displayed overlayed on the display device based on information on the changed aspect ratio of the terminal device received from the communication unit; wherein the information on the changed aspect ratio is generated when it is determined that the display mode of the terminal device has changed according to information detected by an accelerometer of the terminal device, and the display mode includes a horizontal mode located in the horizontal direction and a vertical mode located in the vertical direction.

Images