A TV screen sharing collaborative control method, terminal device and storage medium

Abstract

The present invention is applicable to the technical field of content sharing, and provides a TV screen sharing collaborative control method, terminal device and storage medium, the method comprising: obtaining RGB compressed data input from a signal output terminal, encoding the RGB compressed data, and obtaining a general real-time video stream; caching the real-time video stream based on a slice duration τ, and obtaining a plurality of video clips; scanning DLNA supporting devices in a local area network, monitoring sharing requests of DLNA supporting devices, and obtaining a plurality of DLNA clients; after obtaining the sharing request, continuously pushing the video clip to the DLNA client, and pushing the named list to the DLNA client based on a preset update duration, so that the DLNA client plays the video clip based on the information in the named list. It can be flexibly set in various data transfer platforms with computing capabilities to realize screen sharing of low-performance devices, with flexible configuration, low latency and good display effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of content sharing, and in particular relates to a television screen sharing collaborative control method, terminal equipment and storage medium. Background Art

[0002] There are many brands of smart TVs, mobile phones, tablets, smart screens and other products. Different brands and alliances use different systems, and the types of content sharing protocols they use are huge. There are many compatibility issues in the collaborative transmission of content between these devices.

[0003] DLNA technology came into being as a content collaboration method. DLNA stands for Digital Living Network Alliance, an industry standard protocol for sharing digital content within a local area network, which aims to achieve content collaboration and interconnection between different local devices. Based on the DLNA protocol, smartphones, tablets and other devices can share multimedia content such as videos and photos through a local area network. For most smart TVs, especially early smart TVs, the DLNA protocol is built-in to receive and play multimedia data on shared devices. Therefore, there is still value in further improving and developing the DLNA protocol to further expand the applicability and compatibility of screen projection.

[0004] One of the limitations of DLNA technology is that it is difficult to transmit and play videos based on screen mirroring due to the restrictions of its development protocol. It needs to cooperate with third-party software such as Serviio Pro and LeTV Cast to push screen content in a disguised auxiliary manner in conjunction with the DLNA protocol to indirectly achieve screen mirroring. However, the above-mentioned screen casting method still has limitations, such as involving more non-open source private protocols or hardware devices, poor video playback stability, and prone to screen frustration. Therefore, the existing TV screen sharing method needs to be further improved. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a TV screen sharing collaborative control method, which aims to solve the problems of poor playback stability and many private protocols that are difficult to bypass in the existing screen sharing method based on the DLNA protocol.

[0006] The embodiment of the present application is implemented by providing a TV screen sharing collaborative control method, the method comprising: Acquire RGB compressed data input from the signal output terminal, where the RGB compressed data is data obtained after collecting and processing RGB information of the screen to be shared; Encoding the RGB compressed data to obtain a universal real-time video stream; Acquire cache space, cache the real-time video stream based on the slice duration τ, obtain a number of video segments, name the video segments, and record the names and naming timestamp information in a naming list; Scan the DLNA supported devices in the local area network, monitor the sharing requests of the DLNA supported devices, and obtain several DLNA clients; After the sharing request is obtained, the video clip is continuously pushed to the DLNA client, and the named list is pushed to the DLNA client based on a preset update duration, so that the DLNA client plays the video clip based on the information in the named list.

[0007] Another object of an embodiment of the present application is to provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the TV screen sharing collaborative control method as described above.

[0008] Another object of an embodiment of the present application is to provide a terminal device, which includes a processor, and when the terminal device is running, the processor executes the steps of the TV screen sharing collaborative control method as described above.

[0009] A TV screen sharing collaborative control method provided in an embodiment of the present application has the outstanding advantage that it can be flexibly set in various data transfer platforms with computing capabilities to realize screen sharing of low-performance devices; by acquiring screen images and packaging their caches into the form of video files that can be transmitted based on DLNA, a wider range of screen sharing based on the traditional DLNA protocol can be realized; at the same time, the present application can utilize the platform's own encoding and decoding capabilities to reduce the performance occupancy of the source signal end, reduce the transmission delay caused by insufficient hardware performance and possible screen stuttering and tearing, and has flexible configuration, thereby improving the display effect of existing screen mirroring. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 An application environment diagram of a TV screen sharing collaborative control method provided in an embodiment of the present application; Figure 2 A flowchart of a TV screen sharing collaborative control method provided in an embodiment of the present application; Figure 3 A schematic diagram of time slicing of a video clip provided in an embodiment of the present application; Figure 4 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0012] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first unit or module from another unit or module. For example, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script without departing from the scope of this application.

[0013] Figure 1 The application environment diagram of the TV screen sharing collaborative control method provided in the embodiment of the present application is as follows: Figure 1 As shown, in this application environment, it includes a video signal output terminal 110, a signal processing terminal 120 and a display terminal 130.

[0014] The video signal output terminal 110 and the display terminal 130 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a television, etc., but are not limited thereto.

[0015] The signal processing end 120 can be a network storage NAS with computing power for personal home use, a dedicated hardware integration device, a single-chip device, a soft router, a personal computer, etc., but is not limited thereto.

[0016] The video signal output terminal 110, the signal processing terminal 120, and the display terminal 130 may all be connected via a wired or wireless network, and this application does not impose any limitation thereto.

[0017] like Figure 2 As shown, in one embodiment, a TV screen sharing collaborative control method is proposed. This embodiment applies this method to the above Figure 1 A method for collaborative control of TV screen sharing may include the following steps: Step S10, obtaining RGB compressed data input from the signal output terminal, wherein the RGB compressed data is data obtained after collecting and processing RGB information of the screen to be shared.

[0018] In this embodiment, considering that video encoding and decoding requires high computational effort, it is difficult for the CPU of early mobile phones and other devices with poor performance to simultaneously maintain the data transmission protocol, obtain screen content, and transcode the video. Therefore, only preliminary image collection and processing is required at the signal output end to reduce the dependence on hardware performance, thereby alleviating frame drops and high latency. The RGB information of the shared screen can be collected by calling the system's built-in API.

[0019] Since the amount of original RGB data is large (each pixel R, G, and B in the RGB24 format takes up 1 byte each, and the higher the number of pixels, the greater the bandwidth required), it is preferred that the signal output end compress the original RGB data. RGB compressed data can refer to a common video format that has been initially compressed, such as data converted from RGB data to YUV420P format, or data compressed after partially discarding pixels after sampling, etc., to balance the performance pressure of transcoding on the hardware performance of the video signal output end and the performance pressure caused by the amount of data transmission.

[0020] Step S20, encoding the RGB compressed data to obtain a general real-time video stream.

[0021] In this embodiment, after the data input from the signal output end is obtained, the data can be transcoded using a dedicated video encoding chip. The general real-time video stream can refer to a format such as H.264, H.265, etc. that can support real-time video stream transmission.

[0022] Step S30, obtain cache space, cache the real-time video stream based on the slice duration τ, obtain a number of video clips, name the video clips, and record the naming names and naming timestamp information in a naming list.

[0023] In this embodiment, although the cache layer may increase the delay, in the application scenario of screen mirroring video playback, the delay is not sensitive, but the requirement for fluency is higher, so the cache can improve the fluency of playback. The τ value can be dynamically adjusted according to the network conditions or actual needs, and can be set to hundreds of milliseconds, seconds, tens of seconds, etc.

[0024] Step S40, scanning DLNA supporting devices in the local area network, monitoring sharing requests of DLNA supporting devices, and obtaining a number of DLNA clients.

[0025] In this embodiment, all devices in a wired or wireless network can be set as DLNA clients. Since the dedicated relay terminal has stronger performance, it can send synchronization signals of the same content to one or more devices to achieve synchronous collaborative sharing control of different televisions.

[0026] Step S50, after obtaining the sharing request, the video clip is continuously pushed to the DLNA client, and the named list is pushed to the DLNA client based on a preset update duration, so that the DLNA client plays the video clip based on the information in the named list.

[0027] In this embodiment, each video player can play the received video clips uninterruptedly based on the time sequence or naming order or the time sequence in the named list.

[0028] It can be seen from the above method that the outstanding advantage is that it can be flexibly set in various data transfer platforms with computing capabilities to realize screen sharing of low-performance devices, and can push the output to multiple TVs at the same time for collaborative sharing; by obtaining the screen image and packaging it into a video file that can be transmitted based on DLNA, a wider range of screen sharing based on the traditional DLNA protocol can be achieved; at the same time, the present application can utilize the platform's own encoding and decoding capabilities to reduce the performance occupancy of the source signal end, reduce the transmission delay caused by insufficient hardware performance and possible screen tearing, and improve the display effect of existing screen mirroring.

[0029] In a preferred embodiment, the method for obtaining the RGB compressed data is: Obtain system permissions for collecting screen data, collect screen RGB information, and obtain initial RGB data; Get the idle bandwidth of the transmission bandwidth : ; Adjust the compression rate of the RGB initial data based on the idle bandwidth And the frame rate of the screen , get RGB compressed data: ; ; in, is the total bandwidth, is the idle bandwidth, T is the monitoring window duration, is the time slice at time t, is the idle bandwidth at time t, is the compression algorithm efficiency coefficient, is the base frame rate, is the maximum frame rate, is the single frame data volume, is the sensitivity parameter.

[0030] In an embodiment of the present application, considering that the data of the signal source may be transmitted by wire or wirelessly, it is necessary to compress the data in the wireless case, and the frame rate and clarity can be dynamically balanced based on the above compression method. For example, in one test, the idle bandwidth is 8.0Mbps, the data compression rate = 0.89, and the frame rate = 60.0fps; the idle bandwidth is 5.0Mbps, the data compression rate = 0.56, and the frame rate = 52.5fps. The basic frame rate can be preset to 30 frames per second, and the maximum frame rate supported by the device can be set to 60 frames per second. Exponential smoothing is used to adjust the frame rate, and the transition is natural. Compression can reduce the amount of data based on color space downsampling. The total bandwidth can refer to the total bandwidth of the channel transmitted from the signal source end to the signal processing end.

[0031] In a preferred embodiment, the method of caching the real-time video stream based on the slice duration τ to obtain a plurality of video segments is as follows: Get the preset duration of the slice ; Set the first frame contained in the i-th video clip to , whose timestamp is ; Get the video stream The timestamp of each frame , where k is the number of frames contained in the i-th video clip; Get additional delay : Fill the delay at the end of the video frame of the i-th video clip , get the i-th video clip.

[0032] In the embodiments of the present application, Figure 3 As shown, the horizontal line with an arrow is the timeline. Assuming that the duration of each slice is m, the dots on the timeline are several frames and the timestamp of the frame. In the traditional slicing method, the slice duration is fixed, and the initial picture of the video during continuous playback is the first frame of the video. Therefore, when the first frame of the picture played continuously in the traditional method is played in real time, the picture frame will change the timestamp that should appear and be displayed on the screen in advance. When displayed at a low frame rate, the picture has an obvious sense of frustration. Therefore, this application adopts a dynamically changing slice duration to solve the above problems and make the picture more coherent. Figure 3 In the above method, we know that the actual slice duration of each video segment is , the duration needs to be supplemented at the end of the video, and the length of the clip changes dynamically.

[0033] In a preferred embodiment, the method further comprises: Set the total cache space value and warning threshold; When the cache space reaches a threshold, the video segments generated earlier are dynamically deleted based on the generation time of the video segments.

[0034] In the embodiment of the present application, a cache threshold is set for the cache space to limit the capacity to prevent memory overflow. Space is saved by dynamically deleting content that has been transmitted and played.

[0035] In a preferred embodiment, a method of scanning DLNA supporting devices in a local area network, monitoring sharing requests of DLNA supporting devices, and obtaining a plurality of DLNA clients is as follows: Send matching broadcast requests to all devices in the LAN based on UDP to query DLNA supported devices; Wait for the return result of the DLNA supporting device, set the supporting device that returns a correct response as a DLNA client, and record its information in the available device list; Subscribe to and monitor UPnP events, wait for sharing requests from DLNA-supported devices, and set the sender of the sharing request as a DLNA client.

[0036] In an embodiment of the present application, a method for performing active and passive queries at the same time is provided to speed up the matching process. The SSDP protocol in UPnP can be used to discover services in the network. By sending an SSDP broadcast request, the device can find other devices that support DLNA. The matching broadcast request may refer to an M-SEARCH request, and the device that supports DLNA will respond with a NOTIFY message to inform it of its existence and the services it provides. The IP address and other information of the supporting device that returns the response result will be recorded in the list of available devices for the user to select. DLNA devices can communicate through the HTTP protocol and SOAP messages, and can parse and listen to SOAP requests to obtain sharing information for screen sharing.

[0037] In a preferred embodiment, the method of pushing the naming list to the DLNA client based on a preset update duration so that the DLNA client plays the naming results one by one based on the order of the naming time in the naming list is: Obtaining timestamp information of each video clip in the named list; Preloading the video clips in the named list; Based on the chronological order of the timestamps of the video clips, the video clips are played continuously and uninterruptedly.

[0038] In the embodiment of the present application, the AVTransport service of UPnP can be used to control the client to play different media, or the video clip can be pushed to the device and then freely controlled to play. There are many ways to achieve this, which will not be repeated here. Through the above-mentioned control method based on the named list, more flexible video playback control can be performed.

[0039] Figure 4 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1 The signal processing end 120 in FIG. Figure 4 As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program, which, when executed by the processor, enables the processor to implement a TV screen sharing collaborative control method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to execute a TV screen sharing collaborative control method. The display screen of the computer device may be a liquid crystal display or an electronic ink display, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0040] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0041] In one embodiment, the TV screen sharing collaborative control method provided in the present application can be implemented in the form of a computer program. The computer program can be Figure 4 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the TV screen sharing collaborative control device. The computer program composed of various program modules enables the processor to execute the steps of the TV screen sharing collaborative control method of each embodiment of the present application described in this specification.

[0042] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the TV screen sharing collaborative control method as described above.

[0043] In the embodiment of the present application, please refer to the above for the description of the above-mentioned TV screen sharing collaborative control method, which will not be repeated here.

[0044] In the embodiment of the present application, the program run by the method stored in the storage medium of the embodiment of the present application can be flexibly set in various data transfer platforms with computing capabilities to realize screen sharing of low-performance devices, and can push the output to multiple TVs at the same time for collaborative sharing; by obtaining the screen image and packaging it into a video file that can be transmitted based on DLNA, a wider range of screen sharing based on the traditional DLNA protocol can be achieved; at the same time, the present application can utilize the platform's own encoding and decoding capabilities to reduce the performance occupancy of the source signal end, reduce the transmission delay caused by insufficient hardware performance and possible screen tearing, and improve the display effect of the existing screen mirroring.

[0045] In one embodiment, a terminal device is proposed, wherein the terminal device includes a memory and a processor, and when the terminal device is running, the processor executes the steps of the television screen sharing collaborative control method as described above.

[0046] In an embodiment of the present application, a device may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the processor executes the steps of the TV screen sharing collaborative control method as described above. For the description of the TV screen sharing collaborative control method, please refer to the above text and will not be repeated here.

[0047] In an embodiment of the present application, the device may include an independent hardware video processing chip. When in use, the device can be connected to a mobile phone, etc., so as to realize screen sharing of low-performance devices, and can push the output to multiple TVs at the same time for collaborative sharing; by obtaining the screen image and packaging it into a video file that can be transmitted based on DLNA, a wider range of screen sharing based on the traditional DLNA protocol can be achieved; at the same time, the present application can utilize the platform's own encoding and decoding capabilities to reduce the performance occupancy of the source signal end, reduce the transmission delay caused by insufficient hardware performance and possible screen stuttering and tearing, and improve the display effect of the existing screen mirroring.

[0048] In one embodiment, the terminal device is connected to the signal output terminal in a wired manner to obtain the RGB compressed data.

[0049] In the embodiments of the present application, the wired connection may refer to a connection method such as an Ethernet cable or a USB cable, thereby significantly improving the reliability and stability of the transmission.

[0050] It should be understood that, although each step in the flow chart of each embodiment of the present application is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0051] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0052] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A TV screen sharing collaborative control method, characterized in that: The method comprises: Acquire RGB compressed data input from the signal output terminal, where the RGB compressed data is data obtained after collecting and processing RGB information of the screen to be shared; Encoding the RGB compressed data to obtain a universal real-time video stream; Acquire cache space, cache the real-time video stream based on the slice duration τ, obtain a number of video segments, name the video segments, and record the names and naming timestamp information in a naming list; Scan the DLNA supported devices in the local area network, listen to the sharing requests of the DLNA supported devices, and obtain several DLNA clients; After the sharing request is obtained, the video clip is continuously pushed to the DLNA client, and the named list is pushed to the DLNA client based on a preset update duration, so that the DLNA client plays the video clip based on the information in the named list.

2. A TV screen sharing collaborative control method according to claim 1, characterized in that: The method for obtaining the RGB compressed data is: Obtain system permissions for collecting screen data, collect screen RGB information, and obtain initial RGB data; Get the idle bandwidth of the transmission bandwidth : ； Adjust the compression rate of the RGB initial data based on the idle bandwidth And the frame rate of the screen , get RGB compressed data: ； ； in, is the total bandwidth, is the idle bandwidth, T is the monitoring window duration, is the time slice at time t, is the idle bandwidth at time t, is the compression algorithm efficiency coefficient, is the base frame rate, is the maximum frame rate, is the single frame data volume, is the sensitivity parameter.

3. A TV screen sharing collaborative control method according to claim 1, characterized in that: The method of caching the real-time video stream based on the slice duration τ to obtain several video clips is as follows: Get the preset duration of the slice ; Set the first frame contained in the i-th video clip to , whose timestamp is ; Get the video stream The timestamp of each frame , where k is the number of frames contained in the i-th video clip; Get additional delay : Fill the delay at the end of the video frame of the i-th video clip , get the i-th video clip.

4. A TV screen sharing collaborative control method according to claim 1, characterized in that: The method further comprises: Set the total cache space value and warning threshold; When the cache space reaches a threshold, the video segments generated earlier are dynamically deleted based on the generation time of the video segments.

5. A TV screen sharing collaborative control method according to claim 1, characterized in that: The method of scanning DLNA-supported devices in the local area network, monitoring the sharing requests of DLNA-supported devices, and obtaining several DLNA clients is as follows: Send matching broadcast requests to all devices in the LAN based on UDP to query DLNA supported devices; Wait for the return result of the DLNA supporting device, set the supporting device that returns a correct response as a DLNA client, and record its information in the available device list; Subscribe to and monitor UPnP events, wait for sharing requests from DLNA-supported devices, and set the sender of the sharing request as a DLNA client.

6. A TV screen sharing collaborative control method according to claim 1, characterized in that: The method of pushing the naming list to the DLNA client based on the preset update duration so that the DLNA client plays the naming results one by one based on the order of the naming time in the naming list is: Obtaining timestamp information of each video clip in the named list; Preloading the video clips in the named list; Based on the chronological order of the timestamps of the video clips, the video clips are played continuously and uninterruptedly.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the TV screen sharing collaborative control method described in any one of claims 1 to 6.

8. A terminal device, characterized in that: The terminal device includes a memory and a processor. When the terminal device is running, the processor executes the steps of the TV screen sharing collaborative control method described in any one of claims 1 to 6.

9. A terminal device according to claim 8, characterized in that: The terminal device is connected to the signal output end in a wired manner to obtain the RGB compressed data.

Images