Display device and split-screen playing method

Abstract

The invention discloses a display device and a split-screen playing method, and the method comprises the steps: obtaining a target window corresponding to a current focus when an operation of switching the focus of a window is received, and the target window is a first window or a second window; and playing the audio data associated with the target window. According to the method, when the media data pushed by the terminal is received, the signal source channel does not need to be switched, the playing process of the current signal source is not changed, the two paths of data are synchronously displayed in the split-screen playing mode, synchronous playing of multiple paths of videos is achieved, and the user experience is improved.

Description

[0001] This invention patent application is a divisional application of the Chinese invention patent application with application date of March 30, 2022, application number 202280026627.7, and application name “A split-screen playback method and display device”. Technical Field

[0002] The present application relates to the technical field of display devices, and in particular to a display device and a split-screen playback method. Background Art

[0003] In some application scenarios, users can project screens to display devices through terminal devices such as smartphones and PCs, thereby projecting media data such as images, audio and video in the terminal devices to the display devices for large-screen playback. Since the current display devices only support single-channel video playback, when the display device is playing the signal of physical channel A, if it receives media data pushed by the terminal, it will automatically switch to physical channel B corresponding to the media data. For example, the display device is currently displaying the game interface corresponding to the output signal of the game console. At this time, if it receives video data from another channel of screen projection, since the two signals are mutually exclusive, it generally exits the game interface by default and automatically starts playing the video data, causing the user to have to terminate the game operation. Summary of the invention

[0004] The display device provided in the first aspect includes:

[0005] A display, for displaying a user interface;

[0006] A communicator, used for communication connection with a terminal device;

[0007] A sound player or an interface for connecting an external amplifier to play audio;

[0008] The controller is configured to execute:

[0009] When receiving the media data pushed by the terminal device, generating and displaying the first window and the second window;

[0010] The video data of the current signal channel is displayed in the first window, and the media data is played in the second window.

[0011] The split-screen playback method provided in the second aspect includes:

[0012] When receiving the media data pushed by the terminal device, generating and displaying the first window and the second window;

[0013] The video data of the current signal channel is displayed in the first window, and the media data is played in the second window. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be accessed in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 shows a usage scenario of a display device according to some embodiments;

[0016] Figure 2 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0017] Figure 3 The interface display effect A of the first window and the second window is exemplarily shown;

[0018] Figure 4 The interface display effect B of the first window and the second window is exemplarily shown;

[0019] FIG5( a) exemplarily shows a schematic diagram of a first operation guide page;

[0020] FIG5( b) exemplarily shows a schematic diagram of a second operation guide page;

[0021] FIG5( c) exemplarily shows a schematic diagram of a third operation guide page;

[0022] Figure 6 The interface display effect C when the media data playback ends is exemplified;

[0023] FIG. 7( a ) exemplarily shows an interface display effect D when the number of second windows is 2;

[0024] FIG. 7( b ) is a schematic diagram showing an exemplary interface change when the second window A in FIG. 7( a ) is destroyed;

[0025] Figure 8 A flowchart of a split-screen playback method is exemplarily shown;

[0026] Fig. 9 A flowchart of another split-screen playback method is exemplified;

[0027] Fig.10a This is a structural schematic diagram of a cross-network screen projection system in an embodiment of the present application;

[0028] Fig.10b This is a schematic diagram of the cross-network screen casting operation process in the embodiment of the present application;

[0029] Fig.10cThis is a connection signaling diagram relayed by a server in an embodiment of the present application;

[0030] Fig.11a This is a structural diagram of another cross-network screen projection system in an embodiment of the present application;

[0031] Fig.11b This is a connection signaling diagram of a direct connection in an embodiment of the present application;

[0032] Fig.12 This is a schematic diagram of the mobile terminal cross-network screen projection connection structure in an embodiment of the present application;

[0033] Fig.13 This is a schematic diagram of the cross-network screen projection connection structure of the display device in the embodiment of the present application;

[0034] Fig.14 A flowchart of a mirroring screen projection method according to some embodiments is exemplarily shown in FIG.

[0035] Fig.15 A flow chart of a mirroring screen projection method according to some further embodiments is exemplarily shown in FIG. DETAILED DESCRIPTION

[0036] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0037] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0038] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0039] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0040] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0041] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0042] The control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user can input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200.

[0043] In some embodiments, a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0044] In some embodiments, the display device may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0045] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0046] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide the display device 200 with various contents and interactions.

[0047] The control device 100 can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, thereby playing the role of an interactive intermediary between the user and the display device 200.

[0048] like Figure 2 As shown, the display device 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , an external device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface 280 .

[0049] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0050] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0051] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0052] The user interface may be used to receive a control signal from the control device 100 .

[0053] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0054] The external device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0055] The controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0056] The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. In some embodiments, the user can input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0057] In an exemplary implementation, the terminal device can push media data to the display device. For example, the user can use the screen projection function in some video apps to project the film source of interest to the display device for large-screen playback; or the terminal device and the display device establish a connection through certain protocols, such as the DLNA (Digital Living Network Alliance) protocol. When the user plays media data in the terminal device, he can select a device for playing the media data from the device list, and the display device that supports the screen projection function will be identified and automatically added to the device list. After the user selects the display device from the device list, the media data can be projected from the terminal device to the display device for playback. When pushing media data by screen projection, it is generally necessary for the terminal device and the display device to be connected to the same Wifi network, and the terminal is not limited to the way of pushing media data to the display device. Of course, in other technologies, the screen projection method can also be performed across networks. The specific method of screen projection is not limited in this application.

[0058] In an exemplary implementation, the media data described in the present application includes, but is not limited to, one or more multimedia forms of data in multiple formats such as text, image, audio, and video.

[0059] In an exemplary implementation, if the media data is text, the text information can be displayed in the interface of the display device. If the amount of text information is large and cannot be fully displayed on a single page, display operations such as scrolling and page turning can also be performed.

[0060] In an exemplary implementation, if the media data is an image, it may be in a static image, or a dynamic image (such as a GIF format animation), and the display device may load and display the image in an interface.

[0061] In an exemplary implementation, if the media data is audio, a voice control can be generated and displayed on the interface of the display device. After the user clicks the voice control, the display device can play the audio through its built-in speaker or external amplifier. The user can also trigger the voice control to perform operations such as pause.

[0062] In an exemplary implementation, if the media data is a video, the video needs to be loaded and displayed on the interface of the display device. The video can be a video source in a video application, a network video source in a browser, or a local video stored in a terminal device.

[0063] It should be noted that after receiving the media data pushed by the terminal, the display device can adaptively display and play the media data according to the specific format and category of the media data, which is not limited to the embodiments of the present application.

[0064] Given that the current display device needs to switch the signal channel, stop playing the video signal of the current signal channel, and start displaying the media data after receiving the media data pushed by the terminal device, the user has to interrupt the video viewing of the original signal source. In response to this, the present application provides a split-screen playback mode, so that after the display device receives the media data pushed by the terminal, it does not need to switch the signal channel, but synchronously plays the media data and the video data of the current signal source in different windows in a split-screen playback mode. This article first describes the implementation of split-screen playback from the UI perspective.

[0065] In an exemplary implementation, when the display device receives the media data pushed by the terminal device, it generates a first window and a second window, and can display the first window and the second window in a split-screen playback mode on the interface according to the split-screen scaling ratio and the relative position distribution between the two windows. Figure 3 As shown, the first window is on the left side of the middle and the second window is on the right side of the middle. The two windows are displayed on the split-screen playback interface according to a certain split-screen scaling ratio. The left side of the middle refers to the center in the vertical direction and the left side in the horizontal direction; the right side of the middle refers to the center in the vertical direction and the right side in the horizontal direction.

[0066] In an exemplary implementation, the split-screen scaling ratio may be preset. For example, in the horizontal direction, the first window and the second window are scaled and adjusted according to a preset ratio of P = width W1 of the first window / width W2 of the second window, thereby allocating the horizontal proportions of the two windows to the screen.

[0067] In an exemplary implementation, the split screen scaling ratio can also be planned based on factors such as screen size, the size of the video to be loaded in the dual windows, and the screen orientation type, so that the scaled sizes of the two windows are more adaptable to and match the screen interface. The screen orientation type includes landscape and portrait. If the aspect ratio of the window is greater than or equal to 1, it is a landscape window; if the aspect ratio of the window is less than 1, it is a portrait window. Figure 4 As shown, the second window displays media data in the vertical screen mode, and the P value can be adaptively increased. The setting method of the split screen scaling ratio is not limited.

[0068] In an exemplary implementation, video data of the current signal channel is displayed in a first window, and media data is loaded and played in a second window. The two windows are displayed in parallel in a split-screen playback interface to achieve synchronous playback of two channels of data.

[0069] In an exemplary implementation, the signal source currently played by the display device is not limited, for example, it can be a set-top box, a game console, a DVD, etc. connected via an HDMI interface, an electronic device connected via a VGA (Video Graphics Array) interface, video data in a storage device connected via a USB interface, etc. Each signal source has a corresponding signal channel, such as HDMI, VGA, AV, USB, LAN, and optical fiber interfaces, etc. The type of signal channel is subject to the actual use of the display device and is not limited in the embodiments of the present application.

[0070] Figure 3 and Figure 4 The dual split-screen playback mode is shown to achieve synchronous playback of two-channel data. In an exemplary implementation, the display device may not be limited to two-channel split-screen playback. For example, if the display device is pushed multiple media data in sequence, and the media data received by the display device is M channels, M is greater than 1, then M second windows can be generated, plus the first window, a total of M+1 windows, to achieve multi-channel data split-screen playback.

[0071] In an exemplary implementation, the first window and the second window are displayed in different layers, the first window is displayed in the Video layer, and the second window can be displayed in the OSD (On-Screen Display) layer. The Video layer can be displayed according to the highest resolution supported by the display device. For example, for a 4K TV, the Video layer resolution is 3840*2160. The OSD layer generally defaults to 2K display, that is, the corresponding resolution is 1920*1080. When M is greater than 1, the size of M+1 windows and their layout in the split-screen playback interface can be planned based on factors such as the M value, resolution, size of each video channel, aspect ratio / screen orientation type of each video channel, etc.

[0072] In an exemplary implementation, the user can switch the focus between the first window and the second window, such as for FIG. 5 or Figure 6, the user clicks the "left button" of the remote control to focus on the first window, and then clicks the "right button" of the remote control to switch the focus to the second window, that is, the focus can be transferred and switched by operating the left or right button of the remote control according to the current focus, and the window with the current focus is the target window. It should be noted that the operation of switching the window focus is not limited, for example, the focus can be adjusted by inputting voice commands, touching the screen or moving the remote control.

[0073] In an exemplary implementation, since the number of windows in the split-screen playback interface is at least two, and the display device only supports single-channel audio output, if the media data has audio, such as media data in the form of pure audio, audio and video, image + background audio, text + background audio, graphic + background audio, etc., it will conflict with the audio data of the signal source in the first window.

[0074] In this regard, in an exemplary implementation, only the audio data associated with the target window that currently has focus is played. For example, if the current focus is on the first window, only the audio data of the current signal channel is played, and the audio of the media data is not played; if the current focus is on the second window, only the audio of the media data is played, and the audio data of the current signal channel is not played. In an exemplary implementation, when the first window and the second window are initially displayed, the focus can be set to the first window by default, that is, the audio of the current signal channel is played by default when the split-screen playback is just started. Subsequent users can choose whether to switch focus as desired, and when switching focus, identify and locate the target window to which the focus points, and play the audio associated with the target window in a single channel. In this way, the audio can be switched according to the user's wishes, and playback conflicts between multiple audio channels can be avoided.

[0075] In other exemplary embodiments, when the display device is connected to an external audio output device, the audio data corresponding to the window with current focus can be played through the local speaker, while the audio data corresponding to another window is played through the external audio output device.

[0076] Figure 3 and Figure 4 In the example, the first window and the second window are both thumbnail windows after zoom adjustment. In an exemplary implementation, for a target window with focus, the user can switch the target window from thumbnail mode to full-screen mode by inputting a first preset operation to the target window. The first preset operation is, for example, pressing the "Confirm / OK" key on the remote control. After the user selects and maintains the focus, and continues to click the confirmation key of the remote control, the target window is automatically switched to full-screen display, and other non-target windows will be covered by the target window. The first preset operation is not limited to the settings of this embodiment.

[0077] In an exemplary implementation, for the target window that is switched to full-screen display, the user can enter a second preset operation to exit its full-screen mode. For example, when the media data is currently displayed in full screen, the user clicks the "return key" on the remote control to switch the full-screen window of the media data to the second window in the original thumbnail mode. At this time, the screen interface returns to the mode of synchronously playing multiple data in the split screen of the first window and the second window, and because the opening and exiting of the full-screen mode of the second window is only a change in the window display mode, and the total number of windows on the split-screen playback interface remains unchanged, the size and position layout of each window on the split-screen playback interface remain unchanged, that is, they are consistent with before switching to full screen. The second preset operation is not limited to the settings of this embodiment.

[0078] In an exemplary implementation, in order to guide the user to operate correctly in the multi-window split-screen playback mode, before displaying the first window and the second window, the following may be displayed: Figure 5(a) to Figure 5(c) The operation guide page. Figure 5(a) shows a first operation guide page, which displays the first operation prompt information, such as "Operation 1: Press the left and right keys on the remote control to switch the window focus, and the TV plays the sound of the current focus window". In order to guide the user more vividly and accurately, a first guide map can also be set in the first operation guide page.

[0079] In an exemplary implementation, referring to FIG5(a), a skip control and a next control may also be provided in the first operation guide page. If the user clicks the skip control, the first operation guide page shown in FIG5(a) is closed, and the second operation guide page shown in FIG5(b) is no longer displayed. If the user clicks the next control, the page switches to the second operation guide page shown in FIG5(b), and the second operation guide page displays a second operation prompt message, such as "Operation 2: Press the confirmation button on the remote control to display the current focus window in full screen". In order to guide the user more vividly and accurately, a second guide map may also be provided in the second operation guide page.

[0080] In an exemplary implementation, referring to FIG5(b), a next control and a return control may also be set in the second operation guide page. If the user clicks the return control, the user returns to the first operation guide page shown in FIG5(a). If the user clicks the next control, the user switches to the third operation guide page shown in FIG5(c), and the third operation guide page displays a third operation prompt message, such as "Operation three: Press the return key of the remote control to exit the full screen mode". In order to guide the user more vividly and accurately, a third guide map may also be set in the third operation guide page.

[0081] In an exemplary implementation, referring to FIG5(c), a close control and a return control may also be provided in the third operation guide page. When the user clicks the close control, the third operation guide page is closed, the current operation guide process is ended, and the first window and the second window are displayed. When the user clicks the return control, the page returns to the second operation guide page shown in FIG5(b).

[0082] In an exemplary implementation, a third control may also be set in the third operation guidance page, namely, "Do not show again" in FIG. 5(c). When the user clicks the third control in the third operation guidance page, the third operation guidance page is closed. When the current signal source is played subsequently, if the media data pushed by the terminal device is received, the first operation guidance page to the third operation guidance page will no longer be displayed. Instead, the first window and the second window will be directly displayed in split screen. That is, after the first operation guidance, if the user clicks "Do not show again", the operation guidance process will never be executed subsequently.

[0083] In an exemplary implementation, Figure 5(a) to Figure 5(c) The three operation guide pages shown can be turned by a specified operation, such as clicking the next control and the return control according to the above embodiment; for a display device that supports a touch screen, the operation guide page can also be switched by sliding the page left or right; or by pressing the left or right key of the remote control to switch the operation guide page. The switching method between the operation guide pages is not limited to that described in the embodiment of the present application.

[0084] In an exemplary implementation, referring to Figure 3 and Figure 4 , an exit control can be set on the second window, and the user can stop playing the media data at any time by clicking the exit control on the second window. It should be noted that the operation of stopping the playing of the media data is not limited. Alternatively, the media data can also be played naturally. For example, for media data in a video format with a total playing time of 30 minutes, when it is detected that the playing process of the media data reaches the 30th minute (i.e., the end point / playing end time node), the playing of the media data ends.

[0085] In an exemplary implementation, when it is detected that the playback process of the media data reaches the end naturally, or an operation of stopping the playback of the media data input by the user is received, Figure 6 As shown, the second window needs to be destroyed, and then the first window is switched to display the video of the current signal source in full screen mode, and the audio output channel plays the audio of the current signal source.

[0086] In an exemplary implementation, before destroying the second window, if the focus is on the second window, after destroying the second window, the audio output channel needs to be switched to playing the audio of the current signal source, and the audio of the media data needs to be stopped to ensure that the audio and video of the media data are played synchronously.

[0087] In an exemplary implementation, if the number of second windows M is greater than 1, for example, when M=2, the second window A and the second window B are included, and the second window A is used to play the media data A, and the second window B is used to play the media data B, as shown in FIG7(a). At this time, the interface layout is that the first window is at the top, the second window A is at the lower left, and the second window B is at the lower right. When the media data A is played to the end, or the user inputs an operation to stop playing the media data A, the second window A is destroyed, as shown in FIG7(b). At this time, only the first window and the second window B remain in the interface. Since the total number of windows has changed, the layout of the split-screen playback interface can be adaptively adjusted, and the first window is changed to the left side of the middle and the second window B is changed to the right side of the middle.

[0088] That is to say, when the number of second windows exceeds one, each time a second window is destroyed, the layout of the split-screen playback interface can be adaptively adjusted according to the total number of windows after decrement, as well as the size, split-screen scaling ratio and relative position of each window, until all M second windows are destroyed, the first window is switched to full-screen mode, and the audio output channel only outputs and plays the audio of the current signal source, and stops playing the audio of any media data. It should be noted that the total number of windows in the split-screen playback interface, the size (scaling ratio) of each window and the interface layout can be adaptively set according to the actual application, and this application is not limited. The UI shown in the embodiments and drawings of this application is only exemplary, and the actual design and application shall prevail.

[0089] With reference to the above descriptions of various embodiments from the UI perspective, in an exemplary implementation, Figure 8 A split-screen playback method is shown, and the execution subject of the method is a controller 250 in a display device. The controller 250 can control the display to display a UI interface, and the UI interface can display data such as video, text, and images. It can also control the built-in sound player (i.e., speaker) or external power amplifier (such as audio, etc.) of the display device to play audio. Specifically, the method includes the following program steps:

[0090] Step S101: upon receiving media data pushed by a terminal device, generating a first window and a second window.

[0091] Step S102, controlling the display to display the first window and the second window respectively in a split-screen playback mode on the interface according to the split-screen zoom ratio and the relative position distribution between the two windows.

[0092] Step S103: display the video data of the current signal channel in the first window, and play the media data in the second window.

[0093] Step S104, detecting whether an operation of switching the window focus is received. If an operation of switching the window focus is received, executing step S105, otherwise, continuing to execute step S104.

[0094] Step S105, obtaining the target window corresponding to the current focus. The user can adjust the focus to switch between the first window and the second window by means of a remote control, voice or touch screen, and the window currently having the focus is the target window.

[0095] Step S106, control the sound player or external amplifier to play the audio data associated with the target window. When the window focus is transferred and switched, the interface layout of the split-screen playback window remains unchanged, and only switches to the audio data associated with the target window corresponding to the output focus, for example, when the target window is the first window, only the audio of the current signal source is played; when the target window is the second window, only the audio of the media data is played.

[0096] Step S107, detecting whether a first preset operation on the target window is received. If the first preset operation is received, executing step S108; otherwise, maintaining the current playing state and continuing to execute step S107.

[0097] Step S108, switching the target window to full screen mode.

[0098] Step S109, detecting whether a second preset operation is received. If the second preset operation is received, executing step S10; otherwise, maintaining the state of full-screen display of the target window, and continuing to execute step S109.

[0099] Step S110, switching the target window from full screen mode to thumbnail window mode. The target window exits full screen mode and switches back to the original thumbnail window mode. After the switch, the interface resumes the split screen playback mode, and the size and position layout of the first window and the second window remain unchanged, which is consistent with before the target window switches to full screen mode.

[0100] Step S111, detecting whether the stop condition of the media data is met. The stop condition includes that the playing process of the media data reaches the end, and receiving an operation to stop playing the media data. If the stop condition of the media data is met, step S112 is executed, otherwise, the process returns to step S103.

[0101] Step S112, destroying the second window, controlling the first window to display the video data of the current signal channel in full screen mode, and controlling the sound player or external amplifier to play the audio data of the current signal channel. That is, when the stop condition of the media data is met, the split-screen playback mode is exited and the playback state before receiving the media data is restored.

[0102] In an exemplary implementation, the split-screen playback mode may have scenario requirements. For example, a user wants to play a game strategy in split-screen while playing a game. However, when watching cable TV, if the user receives media data related to the game strategy pushed by the terminal, the user would rather maintain the current state of playing the cable TV independently instead of watching the media data. Therefore, whether the split-screen playback mode needs to be turned on can be determined based on playback scenario factors such as the type of signal channel and the image mode output by the signal channel.

[0103] Based on the previous embodiment, in an exemplary implementation, Fig. 9 Another split-screen playback method is shown, the method comprising the following steps:

[0104] Step S201: upon receiving media data pushed by a terminal device, detecting the type of the current signal channel.

[0105] Step S202: Check whether the channel list includes the type of the current signal channel.

[0106] In an exemplary implementation, the channel list includes preset signal channel types that support split-screen playback mode. For example, if the user sets the split-screen playback mode to be supported in the game mode, the HDMI connected to the game console can be added to the channel list. If the type of the current signal channel is included in the channel list, step S203 is executed, otherwise step S205 is executed.

[0107] Step S203, detect whether the image mode output by the current signal channel is a preset mode. The preset mode is an image mode specified by the user that allows split-screen playback, such as game mode, TV mode, etc. If it is a preset mode, execute step S204, otherwise execute step S205.

[0108] Step S204, starting the split-screen playback mode, generating a first window and a second window, and controlling the display to display the first window and the second window according to the split-screen zoom ratio and the relative position distribution between the two windows.

[0109] Step S205: do not generate the first window and the second window, control the display to display the video data of the current signal channel in full screen, and control the sound player or the external power amplifier to play the audio data of the current signal channel.

[0110] Among them, step S202 and step S203 detect the signal channel type and the image mode respectively, and determine whether the current playback scene supports the split-screen playback mode through these two judgment conditions. In this embodiment, the split-screen playback mode is started when the two judgment conditions are met at the same time. If any of the judgment conditions is not met, the split-screen playback mode is not started, that is, the first window and the second window are not generated and displayed, but the audio and video of the current signal source are played independently on a single screen.

[0111] In an exemplary implementation, the split-screen playback mode can be started when any one of the two judgment conditions is met. For example, as long as it is detected that the current signal channel is HDMI, the split-screen playback process is executed regardless of the image mode output by HDMI; for another example, if it is detected that the image mode of the current interface is a game mode, and the game mode is not output through HDMI, but comes from a game application installed on the display device, the split-screen playback operation can also be started. It should be noted that the judgment conditions for starting the split-screen playback mode can be set according to the actual application scenario, and are not limited to those described in the embodiments of the present application. After executing step S204, execute step S206.

[0112] Step S206: display the video data of the current signal channel in the first window, and play the media data in the second window.

[0113] Step S207, detecting whether an operation of switching the window focus is received. If an operation of switching the window focus is received, executing step S208, otherwise, continuing to execute step S207.

[0114] Step S208, obtaining the target window corresponding to the current focus. The user can adjust the focus to switch between the first window and the second window by means of a remote control, voice or touch screen, and the window currently having the focus is the target window.

[0115] Step S209, control the sound player or external amplifier to play the audio data associated with the target window. When the window focus is transferred and switched, the interface layout of the split-screen playback window remains unchanged, and only switches to the audio data associated with the target window corresponding to the output focus, for example, when the target window is the first window, only the audio of the current signal source is played; when the target window is the second window, only the audio of the media data is played.

[0116] Step S210, detecting whether a first preset operation on the target window is received. If the first preset operation is received, executing step S211, otherwise maintaining the current playing state.

[0117] Step S211, switching the target window to full screen mode.

[0118] Step S212, detecting whether a second preset operation is received. If the second preset operation is received, executing step S213, otherwise maintaining the state of full-screen display of the target window.

[0119] Step S213, switching the target window from the full screen mode to the original thumbnail window mode. After the switching, the size and position layout of the first window and the second window remain unchanged, which is consistent with before the target window is switched to the full screen mode.

[0120] Step S214, detecting whether the stop condition of the media data is met. The stop condition includes that the playing process of the media data reaches the end, and receiving an operation to stop playing the media data. If the stop condition of the media data is met, step S215 is executed, otherwise, the process returns to step S206.

[0121] Step S215, destroy the second window, control the first window to display the video data of the current signal channel in full screen mode, and control the sound player or external amplifier to play the audio data of the current signal channel. That is, when the stop condition of the media data is met, exit the split-screen playback mode and restore the playback state before receiving the media data.

[0122] In this specification, the same or similar parts among the display device embodiments, UI embodiments and method embodiments may be referenced to each other, and the related contents will not be repeated here.

[0123] It can be known from the above technical scheme that when the display device receives the media data pushed by the terminal device, the split-screen display is started, and the first window and the second window are first generated. The first window and the second window are simultaneously displayed in the split-screen playback interface, and then the video data of the current signal channel is loaded and displayed in the first window, and the media data is synchronously played in the second window. For example, the current signal channel is a game console connected to the display device through HDMI, and the media data is a game strategy video. Then, the game operation interface is loaded and displayed in the first window, and the game strategy video is synchronously played in the second window. When receiving the media data pushed by the terminal, the present application does not need to switch the signal source channel, nor does it change the playback process of the current signal source. The split-screen playback mode synchronously displays two or even multiple channels of data to achieve synchronous playback of multiple channels of video. In addition, the present application can also switch the focus to operate a certain channel of data separately, such as switching audio, switching full-screen / thumbnail window display mode, etc., to enhance the user interaction experience, and can also determine whether to start the split-screen playback mode according to the adaptability of the actual scene.

[0124] In some embodiments, the display device is used as a device for receiving screen projection, and the mobile device is used as a device for sending screen projection, and the two may not be in the same network environment.

[0125] In order to cast screens across networks, Fig.10aAs shown, in some embodiments of the present application, a cross-network screen projection system is provided, including a mobile terminal 300, a display device 200 and a server 400; wherein the mobile terminal 300 includes a display unit 330, a communication unit and a processor 380; the display device 200 includes a display 275, a communicator 220 and a controller 250.

[0126] In actual applications, users can start the proxy screen casting service through the mobile terminal 300, that is, start a DMR (Digital Media Renderer) service for screen casting reception, which can receive and play media files pushed by DMC (Digital Media Controller). DMR is the main functional module of DLNA, which is based on the UPnP (Universal Plug and Play) protocol. SSDP (Simple Service Discovery Protocol) is the device discovery protocol defined by UPnP, and SOAP (Simple Object Access Protocol) is the control protocol defined by UPnP, based on HTTP. The mobile terminal 300 communicates with the display device 200 through the proxy screen casting service.

[0127] Among them, the proxy screen projection service can be implemented through an application installed in the mobile terminal 300, or in the form of a system application or system service, background service, etc.

[0128] If the proxy screen projection service is implemented by an application installed in the mobile terminal 300, the application is a self-owned APP, such as Fig.10b , Fig.10cAs shown, when the display device 200 is turned on and the login account has been entered, the display device 200 starts the receiving service to receive the command message and screen projection information sent by the server 400. The application is a self-owned APP. The self-owned APP is opened, and the user logs in to the account on the self-owned APP. The login account of the self-owned APP is the same as the login account of the display device 200. At this time, the server 400 establishes a communication connection with the self-owned APP; the proxy screen projection service is started, which can be through starting the DMR function of DLNA. When started, the IP bound to SSDP and HTTP can be 0.0.0.0; at this time, a third-party APP is started, and the third-party APP supports the DMC function in DLNA, that is, it can search for DMS (Digital Media Server, digital media server), and specify the DMR that can play the media file to play it or control the upload and download of multimedia files to the DMS device; when the user selects the movie to be pushed and turns on the screen projection service, the user can view his own APP in the list of screen projection devices. By selecting this push, the user can realize the screen projection connection between the third-party APP and his own APP. At this time, the third-party APP can also initiate control instructions to his own APP.

[0129] After the own APP receives the screen projection instruction sent by the third-party APP, it can send the screen projection information to the display device 200 with which a communication connection has been established. After receiving the screen projection instruction, the display device 200 parses the screen projection information, except for the media connection, media name, etc. or action information such as pausing playback in the instruction, and performs corresponding media playback or control on the display device 200.

[0130] In some embodiments, the received screen projection instruction can be processed by its own APP, or the screen projection information can be passed to the display device 200 without processing.

[0131] When binding 0.0.0.0, it is equivalent to binding all IPV4 addresses of the mobile terminal 300, and data packets from all network cards can be received. No matter whether the third-party APP is bound to the WiFi network or mobile data network of the mobile terminal, it is in the same local area network as the own APP. In some embodiments, the bound IP can be a loopback address such as 127.0.0.1, etc., which can establish a binding relationship for the same network communication.

[0132] The mobile terminal 300 communicates with the display device 200 through a proxy screen projection service. The communication connection can be achieved through server transit, or through direct connection without transit, etc. The binding method between the mobile terminal 300 and the display device 200 can be the same account binding, PIN code binding, MAC code binding of the display device 200, or unique identifier binding, etc. Through binding, the mobile terminal 300 knows which display device to project the screen to. The mobile terminal 300 communicates with the display device 200 through a proxy screen projection service to transmit the projection information. The communication connection can be directly connected without transit, and the cross-network connection between the two can be achieved by both the mobile terminal 300 and the display device 200 having a public IP address. If the mobile terminal 300 is directly connected to the display device 200, such as Fig.11a As shown, in some embodiments of the present application, a cross-network screen projection system is provided, including a mobile terminal 300 and a display device 200; wherein the mobile terminal 300 includes a display unit 330, a communication unit and a processor 380; the display device 200 includes a display 275, a communicator 220 and a controller 250.

[0133] like Fig.11b As shown, when the display device 200 is turned on and the login account has been entered, the display device 200 starts the receiving service for receiving the command message and screen projection information sent by the mobile terminal 300. If the application is a self-owned APP, the self-owned APP is opened, and the user logs in to the account on the self-owned APP. The login account of the self-owned APP is the same as the login account of the display device 200. At this time, the display device 200 establishes a communication connection with the self-owned APP; the proxy screen projection service is started, which can be through starting the DMR function of DLNA. When started, the IP bound to SSDP and HTTP can be 0.0.0.0; at this time, a third-party APP is started, and the third-party APP supports the DMC function in DLNA, that is, it can search for media files on the DMS, and specify the DMR that can play the media file to play or control the multimedia file to be uploaded and uploaded to the DMS device; when the user selects the movie that needs to be pushed and starts the screen projection service, the self-owned APP can be viewed in the list of its screen projection devices. The user realizes the screen projection connection between the third-party APP and the self-owned APP by selecting the push item. At this time, the third-party APP can also initiate control instructions to the self-owned APP.

[0134] After the own APP receives the screen projection instruction sent by the third-party APP, it can send the screen projection information to the display device 200 with which a communication connection has been established. After receiving the screen projection instruction, the display device 200 parses the screen projection information, except for the media connection, media name, etc. or action information such as pausing playback in the instruction, and performs corresponding media playback or control on the display device 200.

[0135] In the present application, the mobile terminal 300 starts a DMR service for screen projection reception in the same local area network. This service does not execute the playback after successfully receiving the command, but only provides a proxy for the remote target playback device. A channel for cross-network communication is established between this proxy service and the display device 200. After receiving the instruction, the proxy service either processes the instruction or does not process it, and forwards the necessary information therein to the real executor display device 200 through the communication channel to complete the execution of the instruction.

[0136] Based on the above cross-network screen projection system, Fig.12 In some embodiments of the present application, a mobile terminal 300 is also provided, including a display unit 330, a communication unit and a processor 380; the communication unit is configured to establish a communication connection with the display device; the processor 380 is configured to start a proxy screen projection service, and the proxy screen projection service sends a request to establish a communication connection to the display device through the communication unit; receives a screen projection instruction input by a user in a third-party application, and the screen projection instruction points to the proxy screen projection service; in response to the screen projection instruction, the proxy screen projection service establishes a screen projection connection with the third-party application, and obtains screen projection information; and sends the screen projection information to the display device through the proxy screen projection service.

[0137] Among them, in the step of starting the proxy screen projection service, the processor is further configured to: the proxy screen projection service starts the function of receiving screen projection requests; the proxy screen projection service and the third-party application are bound to the same local area network; if the proxy screen projection service is selected in the projection device of the third-party application, the proxy screen projection service receives the screen projection instructions entered by the user in the third-party application.

[0138] like Fig.13 In some embodiments of the present application, a display device 200 is further provided, including a display 275, a communicator 220 and a controller 250. The communicator 220 is configured to establish a communication connection with a mobile terminal; the controller 250 is configured to: receive a communication connection establishment request sent by the mobile terminal; in response to the communication connection request, receive the screen projection information and broadcast control instructions sent by the proxy screen projection service, and parse the screen projection information. Parse the video link and video name information in the screen projection information.

[0139] In some embodiments of the present application, a cross-network screen casting method is also provided, which is applied to a cross-network screen casting system, wherein the cross-network screen casting system includes a mobile terminal 300 and a display device 200, and the method includes the following steps: the mobile terminal 300 starts a proxy screen casting service, and the proxy screen casting service sends a communication connection establishment request to the display device through the communication unit; the display device receives the communication connection request 200, and establishes a communication connection with the mobile terminal according to the communication connection request; the proxy screen casting service obtains the screen casting instruction input by the user in a third-party application; the proxy screen casting service establishes a screen casting connection with the third-party application, and obtains the screen casting information; the mobile terminal 300 sends the screen casting information to the display device through the proxy screen casting service. The display device 200 parses the screen casting information and presents the screen casting video.

[0140] In addition, as a large-screen device, the display device can provide users with a better viewing experience. In the related technology, users can project the content on the mobile phone to the display device, so that the content on the mobile phone can be viewed on the display device, making the viewed image clearer. However, the screen projection technology is highly dependent on the system. Terminals of different systems need to use different screen projection technologies to achieve mirror projection with display devices. It is impossible for multiple systems to use the same screen projection technology to complete the mirror projection operation. For example, a mobile phone with an Android system type uses miracast technology to complete mirror projection, and a mobile phone with an IOS system type uses airplay technology to complete mirror projection.

[0141] Before introducing in detail a mirror screen projection method provided by an embodiment of the present application, the UPNP protocol (Universal Plug and Play) is first introduced. The technical basis of DLNA (DIGITAL LIVING NETWORK ALLIANCE) is the UPNP protocol (Universal Plug and Play). The workflow of the UPNP protocol is briefly introduced below: Step 1: First, addressing, which indicates that the device obtains a network address. Step 2: Execute the device discovery process, which is a control point (a type of device of UPNP device) to find devices on the entire network. At the same time, the device also needs to prompt its own existence. The discovery process can obtain some information about the device, such as the device type, the device's globally unique identifier, and the address of the device description. Step 3: Execute the description process to obtain detailed information in the description file. In this step, you can understand the detailed capabilities of the device. Step 4: Execute the control step, such as sending push information and responding to push information. Step 5: Execute events. Events are similar to the observer mode. The observer is notified of any state changes of the device. Step 6: Display steps. As a supplement to control steps and events, you can see the information and status of the device.

[0142] Throughout the process, different devices play different roles and implement different functions. UPNP devices are DMR (Digital Media Renderer), DMS (Digital Media Server) and DMC (Digital Media Control Point). DMR devices can support media playback and control, DMS devices can support media acquisition, recording, storage and output functions, and DMC devices can control UPNP devices. According to the role definition and the entire workflow, the UPNP protocol can implement the media push function. Media push means that after discovering the DMR device, the DMC pushes the media resources on the DMC side to the DMR side. It should be noted that the media resources are not the content of screen mirroring. The DMR obtains the media file stream from the DMS side for playback. The DMC can control the playback of the media on the DMR side, and the media playback status on the DMR side can be transmitted back to the DMC side.

[0143] For example, a commonly used function is that the TV acts as a DMR, and the mobile phone acts as a DMC and DMS. The mobile phone first discovers the TV, and then pushes the media resources in the mobile phone to the TV for display.

[0144] The following is a detailed description of a mirror screen projection method of an embodiment of the present application, which can realize mirror screen projection based on the UPNP protocol for real-time video streaming transmission, and does not limit the type of terminal system using this method, such as Fig.14 As shown, including:

[0145] S301, the terminal is turned on, the DMC and DMS functions are started, and the terminal joins the multicast group of the UPNP protocol.

[0146] S302: Determine an online device through the multicast group.

[0147] S303: Control the display interface to display the screen mirroring control of the online device. In the embodiment of the present application, the terminal acts as a DMC and a DMS, and starts the DMC and DMS functions.

[0148] The display device is turned on, the DMR function is started, the device service capability is created, and the multicast group of the UPNP protocol is joined. In the embodiment of the present application, the display device plays the role of DMR and starts the DMR function. The creation of device service capability refers to the creation of DMR service capability.

[0149] Exemplarily, the multicast group address of the UPNP protocol may be 239.255.255.250. In the embodiment of the present application, the terminal may select a device that needs to be mirrored, that is, an online device.

[0150] In the embodiment of the present application, the online device is determined automatically or manually.

[0151] First, we will introduce how to automatically determine the online device.

[0152] In some embodiments, the display device sends an online message after joining the multicast group.

[0153] The terminal receives the online message of the display device and analyzes the device service capability of the display device;

[0154] If the display device has a DMR capability, it is determined that the display device is an online device.

[0155] In some embodiments, the online message includes a local network address of the display device. The terminal determines, based on the local network address of the display device, that the device corresponding to the local network address of the display device is the online device.

[0156] In the embodiment of the present application, the display device can only realize mirroring screen projection with the terminal when it has DMR capability. In this embodiment, when receiving the online message of the display device, the display device acts as an online device that can be mirrored with the terminal.

[0157] In some embodiments, the terminal receives an operation of searching for a display device and sends search information to a multicast group.

[0158] The display device receives the search information sent by the terminal to the multicast group; determines whether the display device has DMR capability according to the search information; if the display device has DMR capability, sends a search success message to the terminal. The terminal receives the search success message and determines that the display device is an online device.

[0159] Exemplarily, the user can enter the name of the display device in the search box, and the device service capability of the display device is parsed. In this embodiment, when it is parsed that the display device has DMR capability, the display device is used as an online device that can be mirrored with the terminal.

[0160] In some embodiments, the search information includes a local network address of the terminal. The display device sends a search success message to the terminal according to the local network address of the terminal.

[0161] The above two methods determine the online device automatically or manually.

[0162] In some embodiments, after determining the online device, the terminal controls the display interface to display the screen mirroring control of the online device. S304: receiving an instruction to select the screen mirroring control, sending the video stream data obtained by real-time screen recording to the display device. The display device receives the video stream data, decodes and plays it.

[0163] In some embodiments, the step of receiving an instruction to select a screen mirroring control and sending video stream data obtained by real-time screen recording is as follows: Fig.15 As shown, including:

[0164] S401, the terminal receives an instruction to select a screen mirroring control, and sends a push message to a display device, wherein the push message includes a cache address. Exemplarily, a screen mirroring control may be displayed on a user interface of the terminal, and a user may touch a position corresponding to the screen mirroring control on the screen to generate an instruction to select the screen mirroring control.

[0165] In the embodiment of the present application, the cache corresponding to the cache address is generated when an instruction to select the screen mirroring control is received. When a push message is sent to the display device, the cache address of the cache is carried.

[0166] The display device receives a push message sent by the terminal, parses the push message, obtains a cache address, and sends a video acquisition request to the terminal, wherein the video acquisition request includes the cache address.

[0167] In some embodiments, the display device may determine whether to send a video acquisition request to the terminal based on actual conditions. If the display device does not have the conditions for mirroring the screen with the terminal, the video acquisition request may not be sent to the terminal.

[0168] S402, the terminal receives a video acquisition request, starts recording the screen to obtain video stream data, and stores the video stream data in a cache corresponding to a cache address. In the embodiment of the present application, the video stream data obtained by recording the screen is the content displayed in real time by the terminal.

[0169] S403: The terminal reads the video stream data in the buffer and sends it to the display device. It should be noted that as long as the video stream data exists in the buffer, the video stream data will be sent to the display device.

[0170] In some embodiments, the terminal receives a first instruction to stop mirroring the screen, stops recording the screen, and stops reading video stream data from the cache and sending it to the display device.

[0171] In some embodiments, the display device receives a second instruction to stop mirroring and screen projection, and stops decoding and playing the video stream data.

[0172] In some embodiments, the display device receives an instruction to pause decoding and playing the video stream data, and pauses decoding and playing the video stream data.

[0173] In some embodiments, the display device receives an instruction to re-decode and play the video stream data, and continues to decode and play the video stream data. In the embodiment of the present application, the instruction to re-decode and play the video stream data is received after pausing to decode and play the video stream data.

[0174] In order to ensure that real-time video streams can be transmitted, the embodiments of the present application make the following improvements to the data format of sending a video acquisition request to a terminal, and the data format of sending video stream data to a display device after the terminal receives the video acquisition request, that is, the data format of request data and reply data.

[0175] The UPNP protocol is for video streams transmitted by the HTTP protocol. The display device sends a request to obtain the video, and the terminal receives the request and responds and transmits the corresponding video stream. The specific interaction format is as follows:

[0176] (1) UPNP protocol transfer local file format:

[0177] Request data format:

[0178] GET / XXXX.mp4 HTTP / 1.1

[0179] Host:XXXXXX

[0180] Connection:close

[0181] Range:bytes=0

[0182] User-Agent:hsp / 1.0(Linix:Android 6.0)

[0183] Reply data format:

[0184] HTTP / 1.1 206 Partial Content

[0185] Content-Type: video / mp4

[0186] Content-Length:3234061

[0187] Accept-Ranges:bytes

[0188] Content-Ranges:bytes 0-3234060 / 3234061

[0189] TransferMode.DLNA.ORG:Streaming

[0190] ContentFeatures.DLNA.ORG:

[0191] DLNA.ORG_PN=AVC_MP4_BL_CIF15_AAC_520; DLNA.ORG_OP=01: DLNA.ORG_CI=1; DLNA.ORG_FLAGS=015000000000000000000000000000

[0192] Connection:close

[0193] Date:Tue,09Mar 2021 05:42:26GMT

[0194] (2) UPNP protocol transmission real-time video stream format:

[0195] Request data format:

[0196] GET / cacdbd8e6fed4e2a06a5a32a3ced76021988f44f HTTP / 1.1

[0197] Host:192.168.1.125:8081

[0198] Connection:close

[0199] Range: bytes = 0-

[0200] User-Agent:hsp / 1.0 (Linux; Android 6.0)

[0201] Reply data format:

[0202] HTTP / 1.1 200OK

[0203] Content-Type:video / vnd.dlna.mpeg-tts

[0204] transferMode.dlna.org:Streaming

[0205] contentFeatures.dlna.org:DLNA.ORG_CI=1;DLNA.ORG_FLAGS=013000000000000000000000000000000

[0206] Connection:Keep-Alive

[0207] Date:Tue,09Mar 2021 07:54:23GMT

[0208] Cache-Control:no-store,no-cache,must-revalidate

[0209] Transfer-Encoding: chunked

[0210] The UPNP protocol transmits real-time video stream format. Please note the following:

[0211] The cache address must be unique, so MD5 encryption is used to generate a unique ID to mark this video stream. When generating MD5 for a local file, the file path is generally used as a seed. However, there is no unified method for real-time video streams. The timestamp of the video stream can be used as a seed to generate an MD5 value. The timestamp is unique, and the generated MD5 value is unique, ensuring the uniqueness of the cache address of the video stream.

[0212] Since the video stream has no length, the Content-Length, Accept-Ranges, and Content-Range fields should no longer be responded to.

[0213] The Content-Type field is filled in according to the transmission video stream format. For example, if the video stream is TS encapsulated, you can fill in video / vnd.dlna.mpeg-tts.

[0214] DLNA.ORG_FLAGS = 0130000000000000000000000000000000: The UPNP protocol stipulates that each number is an octal number, the upper 8 bits are valid values, and the lower 24 bits are reserved values. The upper 8 bits are converted to binary format: 0000,0001,0011; among which 24-bit is tm-s (Streaming Mode Flag). If DLNA1.5 version is supported (20-bit is dlna-v1.5-flag), this value must be set to 1, that is, Streaming Mode is supported. Once the terminal has data, it must be transmitted to the display device quickly enough.

[0215] The Connection field should be Keep-Alive to reduce the time consumed by sending HTTP requests to establish a TCP connection, allowing the display device and terminal to continue to transmit content through this connection.

[0216] Cache-Control:no-store,no-cache,must-revalidate: Requires the client to revalidate the data on the server for each request.

[0217] Transfer-Encoding: chunked: HTTP is specified as a persistent connection, but due to the characteristics of real-time video streaming, length and range (the unit for defining range requests) cannot be calculated, and data must be transmitted in chunked mode. If Transfer-Encoding: chunked is specified, the content entity needs to be packaged into a block for transmission.

[0218] In the above embodiment, a mirroring screen projection method, display device and terminal are provided. Some embodiments of the present application can use the UPNP protocol to enable terminals of different systems to use the same screen projection technology to complete the screen projection operation. The method includes: turning on the display device, starting the DMR function, creating device service capabilities, and joining the multicast group of the UPNP protocol, so that the terminal determines the online device according to the multicast group, controls the display interface to display the screen mirroring control of the online device, and sends the video stream data obtained by real-time screen recording when the terminal receives an instruction to select the screen mirroring control corresponding to the online device; the display device receives the video stream data, and decodes and plays it.

[0219] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. In a specific implementation, the present invention also provides a computer storage medium, which can store a program. When the computer storage medium is located in the display device 200, the program may include the split-screen playback method that the aforementioned controller 250 is configured to execute when it is executed. Among them, the computer storage medium can be a disk, an optical disk, a read-only memory (English: Read-Only Memory, referred to as ROM) or a random access memory (English: Random Access Memory, referred to as RAM), etc.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0221] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display, for displaying a user interface; A communicator, used for communication connection with a terminal device; A sound player or an interface for connecting an external amplifier to play audio; The controller is configured to execute: Determine that the image mode output by the current signal channel is the game mode; Start the split-screen playback mode, generate and display the first window and the second window; The video data of the current signal channel is displayed in the first window, and the media data is played in the second window; the media data is pushed to the display device by the terminal device through screen projection.

2. The display device according to claim 1, characterized in that The controller is further configured to execute: If it is determined that the image mode output by the current signal channel is not the game mode, the split-screen mode is not started, and the display is controlled to display the video data of the current signal channel in full screen.

3. The display device according to claim 1, characterized in that Before the controller determines that the image mode output by the current signal channel is the game mode, the controller is further configured to execute: The display device is set to support split-screen playback mode in game mode.

4. The display device according to claim 1 or 3, characterized in that: The current signal channel is a signal channel corresponding to a game console to which the display device is connected via an HDMI interface, and the media data is a game strategy video; The controller is further configured to execute: The game operation interface is loaded and displayed in the first window, and the game strategy video is synchronously played in the second window.

5. The display device according to claim 1 or 3, characterized in that: The controller is further configured to execute: When it is detected that the playback process of the media data has reached an end, or an operation to terminate the playback of the media data is received, the second window is destroyed, the first window is controlled to display the video data of the current signal channel in full-screen mode, and the sound player or the external power amplifier is controlled to play the audio data of the current signal channel.

6. The display device according to claim 1 or 3, characterized in that: The controller is further configured to execute: When the display device is connected to an external power amplifier, the sound player is controlled to play audio data corresponding to the window where the current focus is located, and the external power amplifier is controlled to play audio data corresponding to another window; the other window represents the window other than the window where the current focus is located among the first window and the second window.

7. The display device according to claim 1 or 3, characterized in that: The controller is further configured to execute: When receiving a first preset operation on the target window, switching the target window to a full screen mode; And, after the target window is switched to the full screen mode, if a second preset operation is received, the target window is switched from the full screen mode to the thumbnail window mode, and the size and position layout of the first window and the second window remain unchanged after the switching.

8. The display device according to claim 1 or 3, characterized in that: After generating the first window and the second window, the controller is further configured to execute: According to the split-screen zoom ratio and the relative position distribution between the two windows, the first window and the second window are respectively displayed in the split-screen playback mode on the split-screen playback interface.

9. The display device according to claim 8, characterized in that When the number of the second windows is greater than 1, the controller is further configured to execute: After each time a second window is destroyed, the split-screen scaling ratio and the relative position distribution are adjusted according to the decreasing total number of windows until all second windows are destroyed, the first window is switched to full-screen mode, and the sound player or the external power amplifier is controlled to play the audio data of the current signal channel, and the audio data of any media data is stopped from being played.

10. A split-screen playback method, characterized in that: The method comprises: Determine that the image mode output by the current signal channel is the game mode; Start the split-screen playback mode, generate and display the first window and the second window; The video data of the current signal channel is displayed in the first window, and the media data is played in the second window; the media data is pushed to the display device by the terminal device through screen projection.

Images