Display device and screen projection self-adaption method

Abstract

The invention relates to a display device and a screen projection self-adaption method. The display equipment comprises a communicator which is configured to receive a link to be subjected to screen projection sent by the screen projection equipment; the controller is configured to set a first flag bit under the condition that a target streaming media transmission protocol is analyzed through the to-be-projected screen link; the target streaming media transmission protocol carries a target playlist file; carrying out decapsulation processing based on the target playlist file, and if a data head format of a target media stream is analyzed to be a target media format and a data packet of the target media stream comprises data of a plurality of target audio and video encapsulation formats, assigning a first numerical value to the target audio and video encapsulation format of the target media stream; the assignment of the target media format is a second numerical value; and when a data format decision is made, if the first flag bit is true, the first numerical value and the second numerical value of the target media stream are both greater than a preset numerical value, and the first numerical value is smaller than the second numerical value, determining that the target media stream is in a video format.

Description

Technical Field

[0001] This disclosure relates to the field of electronic technology, and in particular to a display device and a screen projection adaptive method. Background Technology

[0002] Casting content from mobile phones or PCs to a large TV screen has become a highly popular feature among viewers. This technology achieves seamless connectivity between small and large screens, fully meeting users' needs for displaying video content on a large screen. Currently, major smart TV and smart TV box brands on the market all support mobile phone screen casting, providing users with great convenience.

[0003] In related technologies, the main scenarios for screen mirroring from mobile phones or PCs include single-channel and multi-channel mirroring. Single-channel mirroring involves mirroring a single phone or PC; multi-channel mirroring involves mirroring multiple phones or PCs, including mixed mirroring or simultaneous video viewing and mirroring. When mirroring from a PC or phone, due to the wide variety of websites and applications on PCs and phones, some mirroring links may not conform to standards. In such cases, the player will recognize the non-standard mirroring link as an image (e.g., it may use the HLS protocol, but the video stream contains an image header with formats like PNG, JPG, GIF, or TIFF; the video format will likely be interpreted as an image, leading to playback failure) and display a message indicating that playback is not supported. Therefore, how to achieve video mirroring when non-standard mirroring links prevent video playback is a pressing issue that needs to be addressed. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a display device and a screen casting adaptive method, which can enable video casting when the screen casting link does not meet the standard and the video cannot be played.

[0005] In a first aspect, this disclosure provides a display device, comprising:

[0006] The communicator is configured as follows:

[0007] Receive the screen-sharing link sent by the screen-sharing device;

[0008] The controller is configured as follows:

[0009] If the target streaming media transmission protocol is resolved through the link to be cast to the screen, a first flag is set; the first flag is used to identify that the target streaming media transmission protocol is resolved based on the link to be cast to the screen; the target streaming media transmission protocol carries a target playlist file;

[0010] Based on the target playlist file, decapsulation processing is performed. If the data header format of the target media stream is parsed to be the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value; if the target media format is an image format, the target media format is assigned a second value.

[0011] When making a data format decision, if the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

[0012] As an optional implementation of this disclosure, the controller is further configured to:

[0013] When the target streaming media transmission protocol is resolved through the link to be projected, setting the first flag bit includes:

[0014] Decode the protocol of the link to be projected to determine whether the target streaming media transmission protocol is parsed.

[0015] If the target streaming media transmission protocol is parsed, then the first flag bit is set.

[0016] As an optional implementation of this disclosure, the target playlist file includes multiple media resource links, and the controller is specifically configured as follows:

[0017] The process of decapsulating the target playlist file includes:

[0018] Based on the multiple media resource links included in the target playlist file, download multiple media stream files to obtain the target media stream;

[0019] The target media stream is parsed to determine its data format.

[0020] As an optional implementation of this disclosure, the controller is further specifically configured as follows:

[0021] The step of parsing the target media stream and determining its data format includes:

[0022] The target media stream is parsed to obtain header information and data packet information;

[0023] The data format in the header information is compared with multiple preset media formats to determine the data format corresponding to the header information;

[0024] If parsing the data packet information yields data in multiple target audio / video encapsulation formats, then the data format corresponding to the data packet information is determined to be the target audio / video encapsulation format.

[0025] As an optional implementation of this disclosure, the controller is further specifically configured to:

[0026] The step of comparing the data format in the header information with multiple preset media formats to determine the data format in the header information includes:

[0027] The data format in the header information is sorted by similarity to the plurality of preset media formats;

[0028] The preset media format with the highest similarity is determined as the target media format of the data header information.

[0029] As an optional implementation of this disclosure, after decapsulating the target playlist file, the process further includes:

[0030] If the data header format of the target media stream is not the target media format, or if the data packet of the target media stream does not contain data of the target audio / video encapsulation format, then the data format of the target media stream is determined to be a video format based on the first flag bit, the first value, and the second value.

[0031] As an optional implementation of this disclosure, the controller is further configured to:

[0032] After determining that the target media stream is in video format, the method further includes:

[0033] A decoder is created based on the video format, and the target media stream is demultiplexed to obtain the video stream and audio stream;

[0034] The video and audio streams are decoded and rendered, and the decoded media data is sent to the player for playback.

[0035] Secondly, a screen projection adaptive method is provided, the method comprising:

[0036] Receive the screen-sharing link sent by the screen-sharing device;

[0037] If the target streaming media transmission protocol is resolved through the link to be cast to the screen, a first flag is set; the first flag is used to identify that the target streaming media transmission protocol is resolved based on the link to be cast to the screen; the target streaming media transmission protocol carries a target playlist file;

[0038] Based on the target playlist file, decapsulation processing is performed. If the data header format of the target media stream is parsed to be the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value; if the target media format is an image format, the target media format is assigned a second value.

[0039] When making a data format decision, if the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

[0040] As an optional implementation of this disclosure, setting a first flag bit when the target streaming media transmission protocol is resolved through the link to be projected includes:

[0041] Decode the protocol of the link to be projected to determine whether the target streaming media transmission protocol is parsed.

[0042] If the target streaming media transmission protocol is parsed, then the first flag bit is set.

[0043] As an optional implementation of this disclosure, the step of decapsulating the target playlist file includes:

[0044] Based on the multiple media resource links included in the target playlist file, download multiple media stream files to obtain the target media stream;

[0045] The target media stream is parsed to determine its data format.

[0046] As an optional implementation of this disclosure, the step of parsing the target media stream and determining the data format of the target media stream includes:

[0047] The target media stream is parsed to obtain header information and data packet information;

[0048] The data format in the header information is compared with multiple preset media formats to determine the data format corresponding to the header information;

[0049] If parsing the data packet information yields data in multiple target audio / video encapsulation formats, then the data format corresponding to the data packet information is determined to be the target audio / video encapsulation format.

[0050] As an optional implementation of this disclosure, the step of comparing the data format in the header information with multiple preset media formats to determine the data format in the header information includes:

[0051] The data format in the header information is sorted by similarity to the plurality of preset media formats;

[0052] The preset media format with the highest similarity is determined as the target media format of the data header information.

[0053] As an optional implementation of this disclosure, after decapsulating the target playlist file, the process further includes:

[0054] If the data header format of the target media stream is not the target media format, or if the data packet of the target media stream does not contain data of the target audio / video encapsulation format, then the data format of the target media stream is determined to be a video format based on the first flag bit, the first value, and the second value.

[0055] As an optional implementation of this disclosure, after determining that the target media stream is in video format, the method further includes:

[0056] A decoder is created based on the video format, and the target media stream is demultiplexed to obtain the video stream and audio stream;

[0057] The video and audio streams are decoded and rendered, and the decoded media data is sent to the player for playback.

[0058] Thirdly, a computer-readable storage medium is provided, comprising: storing a computer program on the computer-readable storage medium, wherein when the computer program is executed by a processor, it implements the screen projection adaptive method as described in the first aspect.

[0059] The technical solution provided in this disclosure has the following advantages compared with the prior art: It receives a screen-casting link sent by a screen-casting device; when a target streaming media transmission protocol is parsed through the screen-casting link, a first flag is set, wherein the first flag is used to identify that a target streaming media transmission protocol is parsed based on the screen-casting link; the target streaming media transmission protocol carries a target playlist file; based on the target playlist file, decapsulation processing is performed; if the parsed header format of the target media stream is the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value; wherein, if the target media format is an image format, the target media format is assigned a second value; when making a data format decision, if the first flag is true, and both the first and second values ​​of the target media stream are greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format. When a screen mirroring link is received from a screen mirroring device, the protocol is parsed to resolve the link. If the target streaming media transmission protocol is found, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the playlist file is decapsulated. If the target media stream's header format is an image format, and the target media stream contains data in multiple target audio and video encapsulation formats, but its header format is an image format, this indicates that the target media stream link is misleading. Therefore, the target media stream's format is corrected by assigning the target audio and video encapsulation format a first value and the target media format a second value. That is, when the correction conditions are met: the first flag is true, the first and second values ​​of the target media stream are both greater than preset values, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video mirroring can be achieved when the screen mirroring link does not meet the standard and the video cannot be played. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0061] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0062] Figure 1AThis is a schematic diagram illustrating one of the application scenarios of the screen projection adaptive method in one or more embodiments of this disclosure;

[0063] Figure 1B This is a second schematic diagram illustrating an application scenario of the screen projection adaptive method of one or more embodiments of this disclosure;

[0064] Figure 1C This is a third illustration of an application scenario of the screen projection adaptive method in one or more embodiments of this disclosure;

[0065] Figure 2A This is a hardware configuration block diagram of a display device according to one or more embodiments of the present disclosure;

[0066] Figure 2B This is one of the software configuration block diagrams of a display device according to one or more embodiments of the present disclosure;

[0067] Figure 2C This is a second software configuration block diagram of a display device according to one or more embodiments of the present disclosure;

[0068] Figure 2D This is a schematic diagram showing the icon control interface of an application included in a smart device according to one or more embodiments of the present disclosure;

[0069] Figure 3 This is one of the flowcharts illustrating an adaptive screen projection method provided in this embodiment of the disclosure;

[0070] Figure 4 This is a second flowchart illustrating an adaptive screen projection method provided in this embodiment of the present disclosure.

[0071] Figure 5 The third flowchart illustrates a screen projection adaptive method provided in this embodiment of the present disclosure;

[0072] Figure 6 The fourth flowchart illustrates a screen projection adaptive method provided in this embodiment of the present disclosure;

[0073] Figure 7 The fifth flowchart illustrates a screen projection adaptive method provided in this embodiment of the present disclosure;

[0074] Figure 8 This is a sixth flowchart illustrating an adaptive screen projection method provided in this embodiment of the present disclosure. Detailed Implementation

[0075] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0076] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0077] The terms "first," "second," "third," etc., used in this disclosure, in the specification, claims, and accompanying drawings are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0078] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0079] Terminology Explanation:

[0080] Protocol decoding typically refers to the process of parsing the protocol used by data packets in network communication. This process involves identifying and understanding the format, structure, and various fields and information contained in the data packets. The steps of protocol decoding include: 1. Identifying the protocol type: First, it is necessary to determine which protocol the data packet uses. This is usually done by examining the packet header information, for example, by analyzing specific fields in the Ethernet header, IP header, or other protocol headers. 2. Parsing the protocol header: Each protocol has its specific header format, containing information such as source address, destination address, length, and protocol type. Parsing this header information is key to understanding the content of the data packets. 3. Processing the protocol payload: Following the header information is the protocol payload, which contains the actual data being transmitted. The format and content of the payload will vary depending on the protocol.

[0081] Decapsulation, often simply called demultiplexing, refers to the process of separating a received composite data stream (such as video, audio, and other data) into its original, independent data streams. In video streaming processing: During video encoding and transmission, video and audio data are typically encapsulated in a container format, such as MPEG-TS, MP4, or AVI. Decapsulation separates these encapsulated data streams into individual video and audio streams for further processing or playback. In web streaming services, such as HLS or DASH, video content is divided into multiple small segments, each potentially containing video, audio, and metadata. Decapsulation is used to extract individual data streams from these segments.

[0082] A decoder is a device or software used to convert encoded data back to its original form. Some common decoder types include: (1) Video decoders: used to decode compressed video data streams into displayable images. Common video decoder formats include H.264 / AVC, H.265 / HEVC, VP9, ​​etc. (2) Audio decoders: used to decode compressed audio data streams into playable sounds. Common audio decoder formats include MP3, AAC, FLAC, etc. (3) MPEG-TS decoders: used to decode MPEG transport stream (TS) format data, which is commonly used in satellite, cable television, and terrestrial broadcast television systems.

[0083] When casting to electronic devices such as mobile phones or computers, due to the large number of websites and applications on these devices, some casting links may not conform to the standard. When the display device receives such a non-standard casting link, it may be unable to recognize it or may incorrectly recognize it as another format (such as an image format), resulting in video playback failure.

[0084] To address the aforementioned shortcomings, this embodiment of the disclosure receives a screen-casting link sent by a screen-casting device. If a target streaming media transmission protocol is found through the screen-casting link, a first flag is set. This first flag identifies that a target streaming media transmission protocol has been found based on the screen-casting link. The target streaming media transmission protocol carries a target playlist file. Based on the target playlist file, decapsulation processing is performed. If the data header format of the target media stream is the target media format, and the data packet of the target media stream includes data of multiple target audio / video encapsulation formats, then the target audio / video encapsulation format of the target media stream is assigned a first value. If the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, and both the first and second values ​​of the target media stream are greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format. When a screen mirroring link is received from a screen mirroring device, the protocol is parsed to resolve the link. If the target streaming media transmission protocol is found, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the playlist file is decapsulated. If the target media stream's header format is an image format, and the target media stream contains data in multiple target audio and video encapsulation formats, but its header format is an image format, this indicates that the target media stream link is misleading. Therefore, the target media stream's format is corrected by assigning the target audio and video encapsulation format a first value and the target media format a second value. That is, when the correction conditions are met: the first flag is true, the first and second values ​​of the target media stream are both greater than preset values, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video mirroring can be achieved when the screen mirroring link does not meet the standard and the video cannot be played.

[0085] For example, such as Figure 1A As shown, Figure 1A This is a schematic diagram illustrating an application scenario of the adaptive screen casting method provided in this disclosure. Taking a smart TV as an example, the application layer scenario of the adaptive screen casting method is explained. For example, screen casting can be performed on a display device such as a mobile phone or PC (Personal Computer) based on the DLNA (Digital Living Network Alliance) protocol or other protocols. In this disclosure embodiment, refer to... Figure 1AAs shown, a user uses a mobile terminal 300 to project their screen onto a display device 200. Furthermore, after projection, the display device 200 can be controlled via a control device 100 or the mobile terminal 300. The projection process includes two scenarios: single-channel projection and multi-channel projection. (Refer to...) Figure 1B As shown, Figure 1B This is a schematic diagram illustrating a scenario where the adaptive screen mirroring method provided in this disclosure is applied to a single-channel screen mirroring scenario. For example, a mobile phone or PC performs single-channel screen mirroring on a display device, and the display device plays the mirrored image through a monitor. (Refer to...) Figure 1C As shown, Figure 1C This is a schematic diagram illustrating a scenario in which the adaptive screen casting method provided in this embodiment is applied to a multi-channel screen casting scenario. For example, multiple mobile phones or multiple PCs can perform multi-channel screen casting on a display device. This can be a mixed screen casting of mobile phones and PCs, or screen casting while watching a movie, or a mixed screen casting of mobile phones and PCs while playing a video locally on the display device.

[0086] The following explanation uses DLNA screen mirroring as an example. DLNA screen mirroring is a wireless screen mirroring technology based on the DLNA protocol, allowing users to wirelessly project multimedia content, such as photos, videos, and music, from devices like mobile phones, computers, and tablets onto a TV or large screen for a large-screen viewing experience. The DLNA protocol is a file-based protocol that supports the encapsulation and transmission of various types of multimedia content, including audio, video, and images. It allows multimedia content from one device to be projected onto other devices, such as TVs and speakers, within the same Wi-Fi network, achieving wireless connection and screen mirroring display. When a user uses DLNA screen mirroring on a mobile phone or PC, the player receives the playback request from the application, parses the transmitted URL, and creates different decoders based on the parsed video and audio formats for video decoding and playback.

[0087] The screen projection adaptive method provided in this disclosure can be implemented based on a display device, or a functional module or entity within the display device. The display device can be a smart TV, personal computer (PC), server, mobile phone, tablet computer, laptop computer, mainframe computer, etc., and this disclosure does not specifically limit its application to these devices.

[0088] For example, Figure 2A This is a hardware configuration block diagram of a display device according to one or more embodiments of the present disclosure. Figure 2AAs shown, the display device includes at least one of the following: a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to nth interface for input / output. The display 260 can be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and can also be a projection device and a projection screen. The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG audio and video data signals, from multiple wireless or wired broadcast television signals. The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device can establish the transmission and reception of control signals and data signals with the server or local control device through the communicator 220. The detector 230 is used to collect signals from the external environment or to interact with the outside world. The controller 250 and the tuner / demodulator 210 can be located in different separate devices, that is, the tuner / demodulator 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0089] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device. The user can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0090] Figure 2B This is a schematic diagram of the software configuration of a display device according to one or more embodiments of the present disclosure, such as... Figure 2B As shown, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Library Layer"), and the kernel layer.

[0091] In some embodiments, the software framework is described in conjunction with the application scenarios of the screen projection adaptive method. Figure 2C This is a second schematic diagram of the software configuration of a display device according to one or more embodiments of the present disclosure, referring to... Figure 2C As shown, the mobile and TV terminals control and transmit video streams via the DLNA protocol. When the screen mirroring protocol receives the URL link from the mobile terminal, it creates a player in the Java interface layer and a read thread in the native implementation layer to perform protocol decoding. Based on the parsed video and audio formats, it creates different decoders to demultiplex the media streams to obtain the video and audio streams. The extracted video and audio streams are then decoded and rendered, and the decoded media data is sent to the player for playback.

[0092] Figure 2D This is a schematic diagram showing the icon control interface of an application included in a display device according to one or more embodiments of the present disclosure, such as... Figure 2D As shown, the application layer contains at least one application whose corresponding icon control can be displayed on the screen, such as: live TV application icon control, video-on-demand (VOD) application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV applications can provide live television from different signal sources. Video-on-demand (VOD) applications can provide video from different storage sources. Unlike live TV applications, video-on-demand provides video display from certain storage sources. Media center applications can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0093] To illustrate this solution in more detail, the following will use examples to illustrate it. Figure 3 , Figure 4 , Figure 5 , Figure 6 To explain, it is understandable that although 3, Figure 4 , Figure 5 , Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 3 , Figure 4 , Figure 5 , Figure 6At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps. The aim is to achieve the screen projection adaptive method provided in the embodiments of this disclosure.

[0094] like Figure 3 As shown, the method specifically includes the following steps:

[0095] S31. Receive the screen mirroring link sent by the screen mirroring device.

[0096] The links to be projected include, but are not limited to, URLs (Uniform Resource Locators). A URL is an address on the internet used to identify a resource. It allows users to access specific web pages, files, images, videos, and other resources over the network.

[0097] Specifically, the display device's communicator receives the projection URL sent by the projection device.

[0098] For example, the smart TV's communicator receives a screen mirroring URL sent by a smartphone.

[0099] S32. If the target streaming media transmission protocol is resolved through the link to be projected, set the first flag bit.

[0100] The first flag bit is used to identify the target streaming media transmission protocol resolved from the link to be projected; the target streaming media transmission protocol carries a target playlist file. The target playlist file is an M3U8 file. M3U8 is a playlist file format used in the HLS protocol, which contains index information of audio and video streams. HLS divides video content into a series of shorter media files (usually TS or MP4 format), each file containing a portion of the video stream. The M3U8 file lists the URLs of these media files. M3U8 is a text-based playlist file format, commonly used in web streaming services. The M3U8 file contains a series of media resource links that point to segmented media files, typically video or audio files.

[0101] Specifically, when the display device's communicator receives the projection URL from the projection device, it first decodes the protocol. If it identifies HLS or DASH as a streaming media network transmission protocol, it sets the `hls_dash` flag to true. HLS (HTTP Live Streaming) is a streaming media network transmission protocol developed by Apple. It is primarily used to transmit real-time or pre-recorded audio and video content via HTTP. HLS allows clients to play media streams without downloading the entire file, thus enabling streaming playback. DASH (Dynamic Adaptive Streaming over HTTP) is an adaptive streaming media transmission protocol that allows the video stream to dynamically adjust its bitrate based on network performance to ensure smooth playback. DASH achieves adaptive bitrate streaming by cutting video content into multiple short segments with different bitrates and using an MPD (Media Presentation Description) file. The MPD file describes the different representations and segmentation information of the video.

[0102] For example, for the HLS protocol, a preliminary judgment is usually made by checking if the URL ends with .m3u8; then pattern matching is performed to try to find string patterns specific to HLS playlists, such as the initial #EXTM3U tag, or HLS-specific tags like #EXT-X-STREAM-INF, #EXTINF, etc. These tags are defined in the HLS protocol and are used to describe stream information and segment durations, etc. If they match, it is considered to be an HLS protocol.

[0103] S33. Based on the target playlist file, perform decapsulation processing. If the data header format of the target media stream is parsed as the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then assign the target audio and video encapsulation format of the target media stream to the first value.

[0104] The target media format is an image format, and the target media format is assigned a second value. The first value can be set according to the actual application scenario, and there is no specific restriction on the first value here. The target audio and video container format is MPEG-TS. MPEG-TS (Moving Picture Experts Group-Transport Stream) is a container format mainly used for broadcasting and streaming video and audio data. MPEG-TS can multiplex multiple video streams, audio streams, and other data streams (such as subtitles or data broadcasts) into a single data stream. An MPEG-TS data stream consists of fixed-size packets, typically 188 bytes.

[0105] Specifically, since an M3U8 file contains a series of media resource links that point to segmented media files, typically video or audio files, the corresponding media files are downloaded via these links. After obtaining the segmented media files, they are then decapsulated. During format parsing, if the header format of the target media stream is an image format, and the data packets of the target media stream include multiple MPEG-TS format encapsulated data, then the MPEG-TS format is assigned a larger value.

[0106] For example, in HLS or DASH, video content is divided into multiple small segments, each of which may contain video, audio, and metadata. Decapsulation is used to extract individual data streams from these segments. If the parsed video stream contains an image header (such as PNG, JPG, GIF, TIFF), assuming it's in PNG format, the PNG format is assigned a value of 90. The parsed data will then conform to multiple MPEG-TS encapsulation formats, i.e., 10 MPEG-TS data packets are parsed. In this case, the target audio and video encapsulation format of the target media stream is assigned a value of 80.

[0107] S34. When making a data format decision, if the first flag bit is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

[0108] The preset values ​​can be set according to the actual application scenario, and no specific restrictions are imposed here.

[0109] Specifically, when making a video format decision, if the first flag is true and the conditions are met: both the first and second values ​​of the target media stream are greater than preset values, and the first value is less than the second value, then the target media stream is determined to be a video format.

[0110] In some embodiments, when the player receives the casting URL sent by the application, it first decodes the protocol. If it detects that the protocol is HLS or DASH, it sets an hls_dash flag to true. During format parsing, in demuxmpegts, if the video stream contains header information with images (such as png, jpg, gif, tiff), for example, the first 4 bytes are "89 50 4E". "47" is the header of the PNG file. If the parsed data contains multiple MPEGTS strings (each 188 bytes starting with "47" is counted as 1, and the count is accumulated based on the number of MPEGTS string header strings), then assigning a larger MPEGTS string value indicates that the actual format of the data packet is a video stream. During format parsing, if the MPEGTS string score and the score for a certain image format (such as PNG) are both relatively large during probe, and the image format score is larger than the video format score, there is a possibility of link misdirection. When making a video format decision, if the HLS_Dash flag is true and the above conditions are met, then the actual format corresponding to the link is a video format, not an image format.

[0111] Through a progressively deeper intelligent analysis process, the system can adaptively identify and detect the true video format. With this mechanism integrated, users can experience a significant improvement in their viewing experience, with noticeable enhancements in both video casting success rate and playback smoothness.

[0112] In this embodiment, a screen-casting link sent by a screen-casting device is received. If a target streaming media transmission protocol is parsed through the screen-casting link, a first flag is set. The first flag is used to identify that a target streaming media transmission protocol is parsed based on the screen-casting link. The target streaming media transmission protocol carries a target playlist file. Based on the target playlist file, decapsulation processing is performed. If the data header format of the parsed target media stream is the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value. If the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, and both the first and second values ​​of the target media stream are greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format. When a screen mirroring link is received from a screen mirroring device, the protocol is parsed to resolve the link. If the target streaming media transmission protocol is found, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the playlist file is decapsulated. If the target media stream's header format is an image format, and the target media stream contains data in multiple target audio and video encapsulation formats, but its header format is an image format, this indicates that the target media stream link is misleading. Therefore, the target media stream's format is corrected by assigning the target audio and video encapsulation format a first value and the target media format a second value. That is, when the correction conditions are met: the first flag is true, the first and second values ​​of the target media stream are both greater than preset values, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video mirroring can be achieved when the screen mirroring link does not meet the standard and the video cannot be played.

[0113] Figure 4 This is a flowchart illustrating another screen projection adaptive method provided in this embodiment. Figure 3 Further expansion and optimization based on this. Optionally, after executing the above step S34 (when making a data format decision, if the first flag bit is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format), the following steps S45-S46 can also be executed:

[0114] S41. Receive the screen mirroring link sent by the screen mirroring device.

[0115] S42. If the target streaming media transmission protocol is resolved through the link to be projected, set the first flag bit.

[0116] The first flag is used to identify the target streaming media transmission protocol resolved based on the link to be cast to the screen; the target streaming media transmission protocol carries a target playlist file.

[0117] S43. Based on the target playlist file, perform decapsulation processing. If the data header format of the target media stream is parsed to be the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then assign the target audio and video encapsulation format of the target media stream to the first value.

[0118] The target media format is an image format, and the target media format is assigned a second numerical value.

[0119] S44. When making a data format decision, if the first flag bit is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

[0120] It should be noted that the specific implementation methods of steps S41-S44 are the same as those of steps S31-S34, and will not be repeated here.

[0121] S45. Create a decoder according to the video format, demultiplex the target media stream, and obtain the video stream and audio stream.

[0122] S46. Decode and render the video and audio streams, and send the decoded media data to the player for playback.

[0123] Specifically, a decoder is created according to a preset video format (such as mp4, flv, etc.), the media stream is demultiplexed to obtain the video stream and audio stream, the extracted video stream and audio stream are decoded and rendered, and the decoded media data is sent to the player for playback.

[0124] In this embodiment, a screen-casting link sent by a screen-casting device is received. If a target streaming media transmission protocol is parsed through the screen-casting link, a first flag is set. The first flag is used to identify that a target streaming media transmission protocol is parsed based on the screen-casting link. The target streaming media transmission protocol carries a target playlist file. Based on the target playlist file, decapsulation processing is performed. If the data header format of the parsed target media stream is the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value. If the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, and both the first and second values ​​of the target media stream are greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format. When a screen mirroring link is received from a screen mirroring device, the protocol is parsed to resolve the link. If the target streaming media transmission protocol is found, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the playlist file is decapsulated. If the target media stream's header format is an image format, and the target media stream contains data in multiple target audio and video encapsulation formats, but its header format is an image format, this indicates that the target media stream link is misleading. Therefore, the target media stream's format is corrected by assigning the target audio and video encapsulation format a first value and the target media format a second value. That is, when the correction conditions are met: the first flag is true, the first and second values ​​of the target media stream are both greater than preset values, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video mirroring can be achieved when the screen mirroring link does not meet the standard and the video cannot be played.

[0125] Figure 5 This is a flowchart illustrating another screen projection adaptive method provided in this embodiment. Figure 3 This is further expanded and optimized based on the existing model. Optionally, this embodiment mainly describes the process of step S32 (setting the first flag bit when the target streaming media transmission protocol is resolved through the link to be projected).

[0126] S51. Decode the protocol of the link to be projected and determine whether the target streaming media transmission protocol is parsed.

[0127] S52. If the target streaming media transmission protocol is parsed, then set the first flag bit.

[0128] Specifically, the system determines whether the target streaming media protocol is parsed based on the preset suffix. If the target streaming media transmission protocol is parsed, the first flag is set.

[0129] For example, for the HLS protocol, a preliminary judgment is usually made by checking if the URL ends with .m3u8; then pattern matching is performed to try to find string patterns specific to HLS playlists, such as the initial #EXTM3U tag, or HLS-specific tags like #EXT-X-STREAM-INF, #EXTINF, etc. These tags are defined in the HLS protocol and are used to describe stream information and segment durations, etc. If they match, it is considered to be an HLS protocol.

[0130] Figure 6 This is a flowchart illustrating another screen projection adaptive method provided in this embodiment. Figure 3 This is further expanded and optimized based on the existing model. Optionally, this embodiment mainly describes the process of step S33 (decapsulation processing based on the target playlist file). The target playlist file includes multiple media resource links.

[0131] S61. Based on the multiple media resource links included in the target playlist file, download multiple media stream files to obtain the target media stream.

[0132] S62. Perform format parsing on the target media stream to determine the data format of the target media stream.

[0133] The target media stream can be any one of the multiple media stream files.

[0134] Specifically, since an M3U8 file contains a series of media resource links that point to segmented media files, usually video or audio files, the corresponding media files are downloaded through the media resource links provided by M3U8. After obtaining the segmented media files, the format of these media files is parsed to determine the data format of the target media stream.

[0135] Figure 7 This is a flowchart illustrating another screen projection adaptive method provided in this embodiment. Figure 6 This is further expanded and optimized based on the existing methods. Optionally, this embodiment mainly describes the process of step S62 (parse the target media stream to determine its data format).

[0136] S71. Parse the target media stream to obtain header information and data packet information.

[0137] Specifically, format parsing of the target media stream aims to extract useful information from the raw media data, such as video frames, audio samples, and metadata. By reading the header information of the media file, the container format, duration, encoding format, and bitrate of the media stream can be obtained. For segmented or indexed media formats, the index information is parsed to determine the position of each part in the media stream. The video and audio streams are separated from the container format, and sometimes subtitle or data streams may also be included. The container format of the media stream is, for example, MPEG-TS, MP4, FLV, MKV, etc.

[0138] S72. Compare the data format in the data header information with multiple preset media formats to determine the data format corresponding to the data header information.

[0139] The preset media formats include audio, video, and image formats. For example, audio formats include, but are not limited to, wav, mp3, aac, flac, alac, and ac3. Video formats include, but are not limited to, mp4, avi, mkv, flv, mpeg-1, and wmv. Image formats include, but are not limited to, jpeg, png, jpg, gif, tiff, raw, and bmp.

[0140] Specifically, the data format in the parsed header information is compared with all audio, video, and image formats to determine the corresponding data format.

[0141] S73. If multiple target audio and video encapsulation formats are obtained by parsing the data packet information, then the data format corresponding to the data packet information is determined to be the target audio and video encapsulation format.

[0142] For example, if parsing the data packet information yields multiple MPEGTS data, i.e. multiple data starting with "47" and with a byte length of 188, then the data format corresponding to the data packet information is determined to be the MPEGTS encapsulation format.

[0143] In this embodiment, when a screen-casting link is received from a screen-casting device, the protocol is parsed by parsing the link. If a target streaming media transmission protocol is parsed, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the target playlist file is decapsulated. If the header format of the target media stream is an image format and the target media stream contains data in multiple target audio and video encapsulation formats, and since the header format of the target media stream is an image format, it indicates that the link of the target media stream is misleading. Therefore, the format of the target media stream is corrected by assigning the target audio and video encapsulation format of the target media stream to a first value and the target media format to a second value. That is, when the correction conditions are met: the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video casting can be achieved when the screen-casting link does not meet the standard and the video cannot be played.

[0144] Figure 8 This is a flowchart illustrating another screen projection adaptive method provided in this embodiment. Figure 7 This is further expanded and optimized based on the existing model. Optionally, this embodiment mainly describes the process of step S72 (comparing the data format in the data header information with multiple preset media formats to determine the data format corresponding to the data header information).

[0145] S81. Sort the data formats in the header information by similarity with the multiple preset media formats.

[0146] S82. Determine the preset media format with the highest similarity as the target media format of the data header information.

[0147] Specifically, the data format in the parsed header information is compared with all audio, video, and image formats, and scored according to the similarity. For example, since the first 4 bytes of the PNG header are "8950 4E 47", the PNG format is scored as 100 points when the first 4 bytes are "89 50 4E 47". And so on, other media formats are scored according to the similarity, thus determining the media format with the highest similarity as the target media format of the header information.

[0148] In this embodiment, when a screen-casting link is received from a screen-casting device, the protocol is parsed by parsing the link. If a target streaming media transmission protocol is parsed, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the target playlist file is decapsulated. If the header format of the target media stream is an image format and the target media stream contains data in multiple target audio and video encapsulation formats, and since the header format of the target media stream is an image format, it indicates that the link of the target media stream is misleading. Therefore, the format of the target media stream is corrected by assigning the target audio and video encapsulation format of the target media stream to a first value and the target media format to a second value. That is, when the correction conditions are met: the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video casting can be achieved when the screen-casting link does not meet the standard and the video cannot be played.

[0149] In some embodiments, after performing step S33 (decapsulation processing based on the target playlist file), the following steps may also be performed:

[0150] If the data header format of the target media stream is not the target media format, or if the data packet of the target media stream does not contain data of the target audio / video encapsulation format, then the data format of the target media stream is determined to be a video format based on the first flag bit, the first value, and the second value.

[0151] Specifically, if the header format of the target media stream is not the target media format, or if the data packet of the target media stream does not contain data of the target audio / video encapsulation format, then the data format of the target media stream is determined to be a video format based on the first flag bit, the first value, and the second value.

[0152] In this embodiment, a screen-casting link sent by a screen-casting device is received. If a target streaming media transmission protocol is parsed through the screen-casting link, a first flag is set. The first flag is used to identify that a target streaming media transmission protocol is parsed based on the screen-casting link. The target streaming media transmission protocol carries a target playlist file. Based on the target playlist file, decapsulation processing is performed. If the data header format of the parsed target media stream is the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value. If the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, and both the first and second values ​​of the target media stream are greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format. When a screen mirroring link is received from a screen mirroring device, the protocol is parsed to resolve the link. If the target streaming media transmission protocol is found, a first flag is set to further determine whether the link supports playback. Since the target streaming media transmission protocol carries a target playlist file, the playlist file is decapsulated. If the target media stream's header format is an image format, and the target media stream contains data in multiple target audio and video encapsulation formats, but its header format is an image format, this indicates that the target media stream link is misleading. Therefore, the target media stream's format is corrected by assigning the target audio and video encapsulation format a first value and the target media format a second value. That is, when the correction conditions are met: the first flag is true, the first and second values ​​of the target media stream are both greater than preset values, and the first value is less than the second value, the target media stream is determined to be a video format. Thus, video mirroring can be achieved when the screen mirroring link does not meet the standard and the video cannot be played.

[0153] This disclosure also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions implement the various processes of the methods provided in any of the above embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.

[0154] The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0155] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments above is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of the embodiments suitable for specific application considerations.

Claims

1. A display device, characterized in that, include: The communicator is configured as follows: Receive the screen-sharing link sent by the screen-sharing device; The controller is configured as follows: When the target streaming media transmission protocol is resolved through the link to be cast to the screen, a first flag is set; the first flag is used to identify that the target streaming media transmission protocol is resolved based on the link to be cast to the screen; the target streaming media transmission protocol carries a target playlist file; Based on the target playlist file, decapsulation processing is performed. If the data header format of the target media stream is parsed to be the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value; if the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

2. The display device according to claim 1, characterized in that, The controller is also configured to: When the target streaming media transmission protocol is resolved through the link to be projected, setting the first flag bit includes: Decode the protocol of the link to be projected to determine whether the target streaming media transmission protocol is parsed. If the target streaming media transmission protocol is parsed, then the first flag bit is set.

3. The display device according to claim 2, characterized in that, The target playlist file includes multiple media resource links, and the controller is specifically configured as follows: The process of decapsulating the target playlist file includes: Based on the multiple media resource links included in the target playlist file, download multiple media stream files to obtain the target media stream; The target media stream is parsed to determine its data format.

4. The display device according to claim 3, characterized in that, The controller is further specifically configured as follows: The step of parsing the target media stream and determining its data format includes: The target media stream is parsed to obtain header information and data packet information; The data format in the header information is compared with multiple preset media formats to determine the data format corresponding to the header information; If parsing the data packet information yields data in multiple target audio / video encapsulation formats, then the data format corresponding to the data packet information is determined to be the target audio / video encapsulation format.

5. The display device according to claim 4, characterized in that, The controller is further specifically configured as follows: The step of comparing the data format in the header information with multiple preset media formats to determine the data format in the header information includes: The data format in the header information is sorted by similarity to the plurality of preset media formats; The preset media format with the highest similarity is determined as the target media format of the data header information.

6. The display device according to claim 1, characterized in that, The controller is also configured to: After decapsulating the target playlist file, the process further includes: If the data header format of the target media stream is not the target media format, or if the data packet of the target media stream does not contain data of the target audio / video encapsulation format, then the data format of the target media stream is determined to be a video format based on the first flag bit, the first value, and the second value.

7. The display device according to claim 1, characterized in that, The controller is also configured to: After determining that the target media stream is in video format, the method further includes: A decoder is created based on the video format, and the target media stream is demultiplexed to obtain the video stream and audio stream; The video and audio streams are decoded and rendered, and the decoded media data is sent to the player for playback.

8. A screen projection adaptive method, characterized in that, The method includes: Receive the screen-sharing link sent by the screen-sharing device; When the target streaming media transmission protocol is resolved through the link to be cast to the screen, a first flag is set; the first flag is used to identify that the target streaming media transmission protocol is resolved based on the link to be cast to the screen; the target streaming media transmission protocol carries a target playlist file; Based on the target playlist file, decapsulation processing is performed. If the data header format of the target media stream is parsed to be the target media format, and the data packet of the target media stream includes data of multiple target audio and video encapsulation formats, then the target audio and video encapsulation format of the target media stream is assigned a first value; if the target media format is an image format, the target media format is assigned a second value. When making a data format decision, if the first flag is true, the first value and the second value of the target media stream are both greater than a preset value, and the first value is less than the second value, then the target media stream is determined to be a video format.

9. The method according to claim 8, characterized in that, When the target streaming media transmission protocol is resolved through the link to be projected, setting the first flag bit includes: Decode the protocol of the link to be projected to determine whether the target streaming media transmission protocol is parsed. If the target streaming media transmission protocol is parsed, then the first flag bit is set.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the screen projection adaptive method as described in any one of claims 8-9.

Images