URL projection method and device

Abstract

This application provides a URL casting method and apparatus. The casting method includes: a source device determining a first bitstream and download indication information based on the playback progress during casting switching; the first bitstream being a bitstream downloaded from a media server before casting switching; the download indication information indicating the start frame of a second bitstream; the second bitstream being a bitstream that the destination device will download from the media server; and the start frame of the second bitstream being related to the end frame of the first bitstream; the source device sending the download indication information and the first bitstream to the destination device; the destination device downloading the second bitstream from the media server; and the destination device playing video based on either the first bitstream or the second bitstream. This application can improve the casting switching response speed and prevent video playback stuttering.

Description

[0001] This application claims priority to Chinese patent application filed on March 13, 2020, with application number 202010177999.2 and entitled "URL projection method and apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to screen mirroring technology, and more particularly to a URL screen mirroring method and apparatus. Background Technology

[0003] With the widespread adoption of smart mobile devices and the rapid development of internet video services, the demand for playing videos on smartphones, tablets, and other smart devices, and then projecting them onto large-screen devices such as TVs and set-top boxes for viewing, is increasing. Screen mirroring based on the Digital Living Network Alliance (DLNA) protocol, as a typical Uniform Resource Locator (URL) screen mirroring technology, is designed to meet this need. In DLNA screen mirroring, the source device (SourcePlayer) sends the video's URL address to the destination device (DestPlayer). The DestPlayer then requests the content corresponding to the URL address from the media server and plays it.

[0004] However, when using the URL casting technology mentioned above, in order to ensure the response speed of casting switching, DestPlayer will start playing the video from the beginning after casting switching, and cannot keep in sync with the playback progress of SourcePlayer. This will affect the video playback efficiency on the one hand, and the user experience on the other. Summary of the Invention

[0005] This application provides a URL casting method and apparatus that can improve the casting switching response speed and prevent video playback stuttering.

[0006] In a first aspect, this application provides a URL casting method, comprising: receiving a first bitstream sent by a source device, the first bitstream being a bitstream downloaded by the source device from a media server before casting switching; receiving download instruction information sent by the source device; downloading a second bitstream from the media server, the start frame of the second bitstream being indicated by the download instruction information, the start frame of the second bitstream being related to the end frame of the first bitstream; and playing a video based on the first bitstream or the second bitstream.

[0007] During screen mirroring switching, the source device sends the cached bitstream to the destination device. On one hand, the cached bitstream is transmitted via a local area network (LAN). Leveraging the advantages of the LAN, including higher bandwidth and better Quality of Service (QoS), fast and stable transmission of the cached bitstream can be achieved. On the other hand, the destination device receives the cached bitstream from the source device while simultaneously downloading the bitstream after the cached bitstream from the media server. This allows for quick playback of the video based on the existing bitstream, improving the screen mirroring switching response speed, and also ensures that sufficient bitstream is stored during video playback to prevent video playback stuttering.

[0008] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that the source device is playing when the screen projection is switched.

[0009] The destination device cannot decode the image corresponding to Fc based solely on the information of the image frame Fc that is playing at the time of screen mirroring switch. It also needs the keyframe Fk corresponding to Fc (Fk is the keyframe that precedes Fc and is closest to Fc). Only with the information from Fk can the image corresponding to Fc be completely decoded. The same applies to image frames after Fc. Therefore, even if Fk has already played at the time of screen mirroring switch, to ensure the integrity of the first bitstream information, the keyframe Fk corresponding to Fc must be sent to the destination device. Thus, the starting frame of the first bitstream can be Fk. This reduces the amount of data transmitted between the source and destination devices while providing sufficient image information to ensure effective image decoding.

[0010] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that the source device has already downloaded when the screen projection is switched.

[0011] In this case, the source device will send the entire bitstream of all image frames that have been downloaded from the media server but have not yet been played to the destination device, i.e., the bitstream of image frames from Fc to Fd. In this embodiment, as much of the bitstream already downloaded by the source device as possible is transmitted to the destination device via the local area network, thus reducing the amount of data in the second bitstream that the destination device downloads from the media server.

[0012] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, and the last image frame that the source device has finished downloading during screen mirroring switching is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0013] To ensure that the destination device can achieve screen mirroring time synchronization and to meet the storage requirements of media data, the first bitstream sent from the source device to the destination device can end at the last image frame Fe of the media segment to which Fc belongs.

[0014] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0015] Typically, the starting frame of the bitstream downloaded by the source device from the media server is the first image frame of the first media segment of the video. In this embodiment, the starting frame of the bitstream (second bitstream) to be downloaded by the destination device from the media server is the first image frame of the media segment to which the next image frame of the first bitstream belongs. This ensures that the second bitstream downloaded by the destination device from the server is sequentially connected to the first bitstream received from the source device, and further ensures screen mirroring time synchronization.

[0016] For example, when the ending frame of the first stream is the last image frame Fd that the source device has already downloaded when switching screens, considering the media segmentation processing on the media server, the starting frame of the second stream can be the first image frame of the media segment to which the next image frame Fn of Fd belongs. In this case, the end part of the first stream and the beginning part of the second stream may overlap. When the ending frame of the first stream is the last image frame Fe of the media segment to which Fc belongs, considering the media segmentation processing on the media server, the starting frame of the second stream can be the first image frame of the media segment to which the next image frame of Fe belongs. Since the ending frame of the first stream is the last image frame Fe of the media segment to which the image frame being played before the screen switch belongs, the next image frame of Fe is the first image frame of the adjacent media segment, and this first image frame is the starting frame of the second stream. Therefore, the first stream and the second stream will not overlap. In this way, the first and second streams can be kept in a continuous manner and avoid overlapping as much as possible, thus reducing data transmission redundancy. However, even if there is overlap, this overlap is still part of the media segmentation, and this part of redundant transmission is allowed under the premise of ensuring screen projection time synchronization.

[0017] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, the last image frame that the source device has finished downloading during screen mirroring switching is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0018] The source device removes parts that may overlap with the second bitstream from its own cached bitstream, and then transmits the remaining bitstream to the destination device via the local area network. This reduces the amount of data in the second bitstream that the destination device downloads from the media server.

[0019] In one possible implementation, playing the video according to the first bitstream or the second bitstream specifically includes: determining the first image frame to be played after the screen casting switch from the first bitstream or the second bitstream, and starting to play the video from the first image frame.

[0020] In one possible implementation, before playing the video according to the first bitstream or the second bitstream, the method further includes: receiving playback time indication information sent by the source device, the playback time indication information being used to indicate the image frame Fc being played by the source device when the screen casting is switched; the method of playing the video according to the first bitstream or the second bitstream includes: when the data volume of the first bitstream is greater than a set threshold, determining the first image frame to be played as Fc from the first bitstream according to the playback time indication information, and playing the video from Fc to the end frame of the first bitstream according to the first bitstream, and playing the video starting from Fm according to the second bitstream, where Fm is the next image frame after the end frame of the first bitstream.

[0021] Receiving the first stream and downloading the second stream from the media server are two independent processes for the destination device, and therefore can occur simultaneously. Since the first stream precedes the second stream, the destination device can start playback from the first stream. During the reception of the first and second streams, if the amount of data received from the first stream is sufficient for playback, the destination device can start playing the video based on the playback progress when switching from screen mirroring, thus improving the screen mirroring response speed. After the first stream has finished playing, the destination device has also accumulated enough data from the second stream and can continue playing the video based on the second stream, ensuring smooth video playback.

[0022] In one possible implementation, the receiving source device sends a first bitstream, which includes receiving the first bitstream sent by the source device when the total data volume of the first bitstream is not less than a set threshold.

[0023] In one possible implementation, the method further includes: when the total data volume of the first bitstream is less than the set threshold, receiving the data volume indication information sent by the source device; downloading the third bitstream from the media server, wherein the starting frame of the third bitstream is the first image frame of the media segment to which the image frame Fc being played by the source device at the time of screen casting switch.

[0024] In one possible implementation, the method further includes: when the total data volume of the first bitstream is less than the set threshold, receiving the data volume indication information and the first bitstream sent by the source device; downloading the fourth bitstream from the media server, wherein the last image frame of the first bitstream is Fd, the next image frame of Fd is Fn, and the starting frame of the fourth bitstream is the first frame of the media segment to which Fn belongs.

[0025] If, during screen mirroring switching, the amount of cached data downloaded by the source device from the media server is small, less than a set threshold, even if sent to the destination device, the destination device would still need to download almost the entire stream from the media server, or the video might not be playable based on the first stream, or the bandwidth consumption and latency costs of transmitting the first stream might exceed the costs of directly downloading from the media server. In this case, the source device can send a data volume indication message to the destination device. This message notifies the device that the total data volume of the first stream is less than the set threshold. In this situation, the source device can choose to send the first stream to the destination device or choose not to send it.

[0026] In one possible implementation, before receiving the first bitstream sent by the source device, the method further includes: receiving control information sent by the source device, the control information including the encapsulation format of the first bitstream; receiving the first bitstream sent by the source device includes: receiving format data sent by the source device, and decapsulating the format data according to the encapsulation format to obtain the first bitstream.

[0027] The first bitstream may include image frames spanning multiple media segments. The source device can encapsulate the first bitstream into a single media segment, such as a TS or FMP4 segment, to simplify the organization of transmitted data. Optionally, the source device can also divide the first bitstream into multiple media segments for transmission; this application does not specifically limit this. The encapsulation object in this application is the video bitstream downloaded and cached from the media server. This encapsulation process can improve the performance of screen mirroring time synchronization. It does not involve image encoding and decoding, thereby reducing the complexity of the algorithm and the hardware requirements.

[0028] In one possible implementation, after the video starts playing, the current playback status (including playback position) is periodically reported to the source device. If the user adjusts the playback progress by dragging, it can also be reported to the source device.

[0029] In one possible implementation, a new message is defined for transmitting media information and a first bitstream. This message may include a uniform, fixed-length header, a message type and response code to carry an identifier, and an optional message body.

[0030] Secondly, this application provides a URL casting method, comprising: determining a first bitstream and download indication information based on the playback progress at the time of casting switch, wherein the first bitstream is a bitstream downloaded from a media server before casting switch, the download indication information is used to indicate the start frame of a second bitstream, the second bitstream is a bitstream that the destination device wants to download from the media server, and the start frame of the second bitstream is related to the end frame of the first bitstream; and sending the download indication information and the first bitstream to the destination device.

[0031] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that is being played when the screen casting is switched.

[0032] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that has been downloaded when the screen casting is switched.

[0033] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, and the last image frame that has been downloaded during the screen mirroring switch is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0034] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0035] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, the last image frame that has been downloaded during the screen mirroring switch is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0036] In one possible implementation, before sending the download instruction information and the first bitstream to the destination device, the method further includes: sending playback time instruction information to the destination device, the playback time instruction information being used to indicate the image frame Fc being played when the screen mirroring is switched.

[0037] In one possible implementation, the method further includes: when the total data volume of the first bitstream is less than a set threshold, sending a data volume indication message to the destination device, the data volume indication message being used to notify that the total data volume of the first bitstream is less than the set threshold.

[0038] In one possible implementation, before sending the download instruction information and the first bitstream to the destination device, the method further includes: encapsulating the first bitstream according to a set encapsulation format to obtain format data; and sending control information and the format data to the destination device, wherein the control information includes the encapsulation format.

[0039] In one possible implementation, when the source device initiates screen mirroring, it may include extended parameters in the screen mirroring switching request in addition to the URL. These extended parameters indicate that the source device supports the URL screen mirroring method provided in this application. If the destination device supports the URL screen mirroring method provided in this application, it can recognize the extended parameters in the screen mirroring request.

[0040] In one possible implementation, a new message is defined for transmitting media information and a first bitstream. This message may include a uniform, fixed-length header, a message type and response code to carry an identifier, and an optional message body.

[0041] Thirdly, this application provides a screen mirroring device, comprising: a receiving module for receiving a first bitstream sent by a source device, the first bitstream being a bitstream downloaded by the source device from a media server before screen mirroring is switched; the receiving module is further configured to receive download instruction information sent by the source device, and download a second bitstream from the media server according to the download instruction information, wherein the start frame of the second bitstream is indicated by the download instruction information, and the start frame of the second bitstream is related to the end frame of the first bitstream; and a playback module for playing a video according to the first bitstream or the second bitstream.

[0042] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that the source device is playing when the screen projection is switched.

[0043] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that the source device has already downloaded when the screen projection is switched.

[0044] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, and the last image frame that the source device has finished downloading during screen mirroring switching is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0045] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0046] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, the last image frame that the source device has finished downloading during screen mirroring switching is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0047] In one possible implementation, the playback module is specifically used to determine the first image frame to be played after the screen casting switch from the first bitstream or the second bitstream, and to start playing the video from the first image frame.

[0048] In one possible implementation, the receiving module is further configured to receive playback time indication information sent by the source device, the playback time indication information being used to indicate the image frame Fc being played by the source device when the screen casting is switched; the playback module is further configured to, when the data volume of the first bitstream is greater than a set threshold, determine the first image frame to be played as Fc from the first bitstream according to the playback time indication information, and play the video from Fc to the end frame of the first bitstream according to the first bitstream, and play the video starting from Fm according to the second bitstream, where Fm is the next image frame after the end frame of the first bitstream.

[0049] In one possible implementation, the receiving module is specifically used to receive the first bitstream sent by the source device when the total data volume of the first bitstream is not less than a set threshold.

[0050] In one possible implementation, the receiving module is further configured to receive the data volume indication information sent by the source device when the total data volume of the first bitstream is less than the set threshold; and download the third bitstream from the media server, wherein the starting frame of the third bitstream is the first image frame of the media segment to which the image frame Fc being played by the source device at the time of screen casting switch.

[0051] In one possible implementation, the receiving module is further configured to receive the data volume indication information and the first bitstream sent by the source device when the total data volume of the first bitstream is less than the set threshold; download the fourth bitstream from the media server, wherein the last image frame of the first bitstream is Fd, the next image frame of Fd is Fn, and the starting frame of the fourth bitstream is the first frame of the media segment to which Fn belongs.

[0052] In one possible implementation, the receiving module is further configured to receive control information sent by the source device, the control information including the encapsulation format of the first bitstream; and to receive format data sent by the source device; the device further includes a decoding module, configured to decapsulate the format data according to the encapsulation format to obtain the first bitstream.

[0053] Fourthly, this application provides a screen mirroring device, comprising: a processing module, configured to determine a first bitstream and download indication information based on the playback progress during screen mirroring switching, wherein the first bitstream is a bitstream downloaded from a media server before screen mirroring switching, and the download indication information is used to indicate the start frame of a second bitstream, wherein the second bitstream is a bitstream that the destination device wants to download from the media server, and the start frame of the second bitstream is related to the end frame of the first bitstream; and a sending module, configured to send the download indication information and the first bitstream to the destination device.

[0054] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that is being played when the screen casting is switched.

[0055] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that has been downloaded when the screen casting is switched.

[0056] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, and the last image frame that has been downloaded during the screen mirroring switch is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0057] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0058] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, the last image frame that has been downloaded during the screen mirroring switch is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0059] In one possible implementation, the sending module is further configured to send playback time indication information to the destination device, the playback time indication information being used to indicate the image frame Fc being played during screen mirroring switching.

[0060] In one possible implementation, the sending module is further configured to send data volume indication information to the destination device when the total data volume of the first bitstream is less than a set threshold. The data volume indication information is used to notify that the total data volume of the first bitstream is less than the set threshold.

[0061] In one possible implementation, the device further includes: an encapsulation module for encapsulating the first bitstream according to a set encapsulation format to obtain format data; and a transmission module specifically for sending control information and the format data to the destination device, wherein the control information includes the encapsulation format.

[0062] Fifthly, this application provides a screen projection device, including: a processor and a transmission interface; the processor is configured to invoke program instructions stored in a memory to implement the method as described in any one of the first to second aspects above.

[0063] In a sixth aspect, this application provides a computer-readable storage medium, characterized in that it includes a computer program, which, when executed on a computer or processor, causes the computer or processor to perform the method of any one of the first or second aspects described above.

[0064] In a seventh aspect, this application provides a computer program, characterized in that, when the computer program is executed by a computer or processor, it is used to perform the method of any one of the first or second aspects described above. Attached Figure Description

[0065] Figure 1 An exemplary schematic diagram of a screen mirroring scenario is shown;

[0066] Figure 2 An exemplary structural schematic diagram of device 200 is shown;

[0067] Figure 3 This is a flowchart of Embodiment 1 of the URL projection method of this application;

[0068] Figure 4 An exemplary sequence diagram of image frames is shown;

[0069] Figure 5 An exemplary sequence diagram of image frames is shown;

[0070] Figure 6 An exemplary sequence diagram of image frames is shown;

[0071] Figure 7 This is a flowchart of Embodiment 2 of the URL projection method of this application;

[0072] Figure 8 This paper illustrates a possible embodiment of the URL projection method of this application;

[0073] Figure 9 This is a schematic diagram of the structure of Embodiment 1 of the projection device of this application;

[0074] Figure 10 This is a schematic diagram of the structure of Embodiment 2 of the projection device of this application. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0077] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0078] Definitions of terms used in this application:

[0079] URL casting: This method involves the casting initiator providing the video's URL to the casting receiver, which then retrieves the video stream from the media server and plays it. URL casting differs from mirror casting, which transmits the casting initiator's screen data to the casting receiver in real-time, synchronously displaying the initiator's screen on the receiving end.

[0080] DLNA casting is a typical URL casting technology. The casting process includes: the user playing a video on a terminal device (such as a mobile phone or tablet) and clicking the casting button in the media player to initiate casting. The terminal device is the casting initiator. The casting initiator sends the video's URL address to the casting receiver using the DLNA protocol. Large-screen devices such as TVs and set-top boxes are the casting receivers. Simultaneously, the casting initiator uses the seek command in the DLNA protocol to indicate the current playback position to the casting receiver. The casting receiver requests the content corresponding to the URL address from the media server and plays it.

[0081] The screen mirroring termination process includes: the user terminating the mirroring on either the initiating or receiving end via a button or key press. The receiving end stops playing the video. If the media player on the initiating end has not exited, the initiating end requests content from the media server and continues playback.

[0082] The sender initiates the URL casting process. When casting is initiated, the sender provides the video URL address to the receiver. Typically, devices such as mobile phones, tablets, and computers can initiate URL casting through their running media players.

[0083] Receiver: The receiving end for URL-based screen mirroring. During screen mirroring, the Receiver retrieves the video stream from the media server based on the URL address provided by the Sender. Typically, devices such as TVs, set-top boxes, audio / video (AV) amplifiers, Chromecast, and computers can all receive screen mirrors through their built-in receivers.

[0084] Media Server: A server that provides video streams. The Receiver requests the corresponding video stream from the Server using the video's URL address. Currently, the Server primarily uses streaming media protocols such as Hypertext Transfer Protocol Live Streaming (HLS) and Fragmented MP4 (FMP4) to provide media services.

[0085] Initial cache data (Cache): Before starting playback, the device playing the video needs to download and cache a video stream from the media server. Usually, at least 2 to 3 seconds of video stream needs to be cached, otherwise stuttering may occur when playing the video.

[0086] Screen mirroring switching: When screen mirroring is started or stopped, the device playing the video switches from one end to the other. When screen mirroring is started, the sender will generally stop playing, and the video will switch to being played by the receiver. When screen mirroring is stopped, the receiver will stop playing; if the sender's video player has not stopped, the video will switch back to being played by the sender.

[0087] Destination device (DestPlayer): The device that should play the video after screen mirroring is switched. When screen mirroring is started, video playback switches from Sender to Receiver, at which point the Receiver is the DestPlayer; when screen mirroring is terminated, the video should switch back from Receiver to Sender, with the Sender being the DestPlayer.

[0088] Source device (SourcePlayer): The end that plays the video before screen mirroring is switched. Before screen mirroring is started, the Sender is SourcePlayer; before screen mirroring is terminated, the Receiver is SourcePlayer.

[0089] Screen mirroring switching response speed: Also known as screen mirroring response speed, response speed, etc. This refers to the speed at which the video switches to DestPlayer for playback after the user initiates screen mirroring.

[0090] Mirroring time synchronization: When switching mirroring, DestPlayer can start playing from the position of the video that SourcePlayer is currently playing.

[0091] Keyframes: In video compression, each frame represents a still image. A keyframe is an I-frame, which uses full-frame compression encoding, meaning its entire image information is compressed and encoded. During decoding, the complete image can be reconstructed using only the data from the I-frame. I-frames describe details of the image background and moving subjects; they do not require reference to other image frames and belong to intra-frame compression. In contrast to keyframes, ordinary image frames belong to inter-frame compression. During decoding, the information from the corresponding keyframe is superimposed with the residual information of the current frame to obtain the final image.

[0092] Figure 1 An exemplary diagram of a URL casting scenario is shown, such as... Figure 1 As shown, in this scenario, the source device sends the URL of the video to the destination device, and the destination device requests the content corresponding to the URL from the media server and plays it. However, URL casting has always had two problems: (1) The casting switching response speed is not fast enough, and the video cannot be quickly switched to the destination device for playback. (2) It is difficult to ensure casting time synchronization while taking into account the casting switching response speed, and the destination device cannot start playing from the position of the video that is being played on the source device.

[0093] Current optimization efforts mostly focus on reducing the time overhead of each step in the screen mirroring switching process, or pre-starting the target device. While this can alleviate the problem to some extent, it does not fundamentally solve it.

[0094] On the one hand, improving the response speed of screen mirroring switching is quite difficult. For online video services, the playback device needs to download and cache a video stream from the media server before it can start playback, which usually requires at least 2-3 seconds of buffering. Due to unpredictable network jitter, the initial buffered data cannot be too small, otherwise playback will easily stutter. When the bandwidth capacity and bandwidth demand are not significantly different, the initial buffered data will require at least 2-3 seconds. Correspondingly, when switching screens, the destination device needs 2-3 seconds before it can start playing. The contradiction between bandwidth demand and bandwidth capacity has always existed, and video is a major bandwidth consumer. High-definition, ultra-high-definition, 4K, 8K, Virtual Reality (VR) / Augmented Reality (AR), etc., often quickly fill up the bandwidth added by users. In addition, there are initial download overheads when downloading the stream from the media server, such as the initial negotiation of the distribution network (e.g., Content Delivery Network (CDN) + Peer-to-Peer (P2P)). While media service providers optimize performance as much as possible, overhead always exists. It typically takes hundreds of milliseconds and is closely related to the media service provider's Quality of Service (QoS). Therefore, without more efficient methods, this problem cannot be fundamentally solved during screen mirroring switching. Another reason is the difficulty in providing screen mirroring time synchronization. Typically, for a device to play video at time T, it must start downloading from the first frame of the media segment containing time T. For example, assuming a media segment is 5 seconds long, the source device initiates a screen mirroring switch at the 14th second. Upon receiving the screen mirroring switch request, the destination device seeks to the 3rd segment, corresponding to a time range of 10-15 seconds. To achieve time synchronization, the destination device, in addition to the necessary initial buffer (2-3 seconds after 14 seconds), also needs to download the already played data from the 10th to 13th seconds, leading to a longer waiting time.

[0095] Therefore, until a more effective method is found, related technologies are forced to choose between screen mirroring switching response speed and screen mirroring time synchronization: either sacrifice screen mirroring switching response speed to improve screen mirroring time synchronization, or sacrifice screen mirroring time synchronization to improve screen mirroring switching response speed. Currently, most technologies choose the latter, i.e., improving screen mirroring switching response speed, while always starting video playback from the beginning during screen mirroring switching.

[0096] This application proposes a URL-based screen mirroring method to address the aforementioned issues. For example... Figure 1 As shown, the sender in the URL projection method can also be called user equipment (UE), which can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; it can also be deployed on water (such as ships); and it can also be deployed in the air (such as airplanes, balloons, and satellites). The source device can be a mobile phone, tablet, wearable device with wireless communication capabilities (such as a smartwatch), location tracker with positioning capabilities, computer with wireless transceiver capabilities, VR device, AR device, wireless device in industrial control, wireless device in self-driving, wireless device in remote medical care, wireless device in smart grid, wireless device in transportation safety, wireless device in smart city, wireless device in smart home, etc., and this application does not limit it.

[0097] The destination device in the URL casting method can be a smart TV, TV box, projection screen, etc., and this application does not limit it.

[0098] The network between the source device and the destination device can be a communication network that supports short-range communication technologies, such as a communication network that supports Wireless-Fidelity (WIFI) technology; or a communication network that supports Bluetooth technology; or a communication network that supports Near Field Communication (NFC) technology; etc.; or the communication network can also be a communication network that supports fourth-generation (4G) access technologies, such as Long Term Evolution (LTE) access technology; or the communication network can also be a communication network that supports fifth-generation (5G) access technologies, such as New Radio (NR) access technology; or the communication network can also be a communication network that supports third-generation (3G) access technologies, such as Universal Mobile Telecommunications System (UMTS) access technology; or the communication network can also be a communication network that supports multiple wireless technologies, such as a communication network that supports LTE and NR technologies; or the communication network can also be applicable to future-oriented communication technologies, which are not specifically limited in this application.

[0099] Figure 2 An exemplary structural schematic diagram of device 200 is shown. Device 200 can be used as either the source device or the destination device described above. Figure 2 As shown, device 200 includes: an application processor 201, a microcontroller unit (MCU) 202, a memory 203, a modem 204, a radio frequency (RF) module 205, a wireless-fidelity (Wi-Fi) module 206, a Bluetooth module 207, a sensor 208, an input / output (I / O) device 209, a positioning module 210, and other components. These components can communicate via one or more communication buses or signal lines. The aforementioned communication bus or signal line can be the CAN bus provided in this application. Those skilled in the art will understand that device 200 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0100] The following is combined with Figure 2 A detailed description of each component of equipment 200 is provided below:

[0101] The application processor 201 is the control center of the device 200, connecting various components of the device 200 via various interfaces and buses. In some embodiments, the processor 201 may include one or more processing units.

[0102] The memory 203 stores computer programs, such as Figure 2 The operating system 211 and application program 212 are shown. The application processor 201 is configured to execute computer programs stored in memory 203 to implement the functions defined by those programs. For example, the application processor 201 executes the operating system 211 to implement various functions of the operating system on device 200. Memory 203 also stores other data besides the computer programs, such as data generated during the operation of the operating system 211 and application program 212. Memory 203 is a non-volatile storage medium, generally including main memory and secondary storage. Main memory includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), or cache. Secondary storage includes, but is not limited to, flash memory, hard disk, optical disk, Universal Serial Bus (USB) disk, etc. Computer programs are typically stored on secondary storage, and the processor loads the program from secondary storage into main memory before executing it.

[0103] The memory 203 can be independent and connected to the application processor 201 via a bus; the memory 203 can also be integrated with the application processor 201 into a chip subsystem.

[0104] MCU 202 is a coprocessor used to acquire and process data from sensor 208. MCU 202 has lower processing power and power consumption than application processor 201, but features an "always-on" characteristic, allowing it to continuously collect and process sensor data while application processor 201 is in sleep mode, ensuring normal sensor operation with extremely low power consumption. In one embodiment, MCU 202 can be a sensor hub chip. Sensor 208 may include a light sensor and a motion sensor. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of display 2091 according to the ambient light level, and the proximity sensor can turn off the display power when device 200 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. Sensor 208 may also include other sensors such as a gyroscope, barometer, hygrometer, thermometer, and infrared sensor, which will not be described in detail here. The MCU 202 and sensor 208 can be integrated onto the same chip or are separate components connected via a bus.

[0105] The modem 204 and the radio frequency (RF) module 205 constitute the communication subsystem of device 200, used to implement the main functions of the wireless communication standard protocol. The modem 204 is used for encoding / decoding, signal modulation / demodulation, equalization, etc. The RF module 205 is used for receiving and transmitting wireless signals, and includes, but is not limited to, an antenna, at least one amplifier, a coupler, a duplexer, etc. The RF module 205 works with the modem 204 to implement wireless communication functions. The modem 204 can be a standalone chip or integrated with other chips or circuits to form a system-on-a-chip (SoC) or integrated circuit. These chips or integrated circuits can be used in all devices that implement wireless communication functions, including: mobile phones, computers, laptops, tablets, routers, wearable devices, automobiles, home appliances, etc.

[0106] Device 200 can also use Wi-Fi module 206, Bluetooth module 207, etc., for wireless communication. Wi-Fi module 206 provides network access to device 200 conforming to Wi-Fi related standard protocols. Device 200 can access a Wi-Fi access point through Wi-Fi module 206, thereby accessing the Internet. In some other embodiments, Wi-Fi module 206 can also act as a Wi-Fi wireless access point, providing Wi-Fi network access to other devices. Bluetooth module 207 enables short-range communication between device 200 and other devices (such as mobile phones, smartwatches, etc.). In this embodiment, Wi-Fi module 206 can be an integrated circuit or a Wi-Fi chip, and Bluetooth module 207 can be an integrated circuit or a Bluetooth chip.

[0107] The positioning module 210 is used to determine the geographical location of the device 200. It is understood that the positioning module 210 may specifically be a receiver for a global positioning system (GPS), such as the BeiDou Navigation Satellite System or the Russian GLONASS system.

[0108] The Wi-Fi module 206, Bluetooth module 207, and positioning module 210 can each be a separate chip or integrated circuit, or they can be integrated together. For example, in one embodiment, the Wi-Fi module 206, Bluetooth module 207, and positioning module 210 can be integrated onto the same chip. In another embodiment, the Wi-Fi module 206, Bluetooth module 207, positioning module 210, and MCU 202 can also be integrated into the same chip.

[0109] Input / output devices 209 include, but are not limited to: display 2091, touch screen 2092, and audio circuit 2093, etc.

[0110] The touchscreen 2092 can collect touch events from the user of device 200 on or near it (such as user actions using a finger, stylus, or any suitable object on or near the touchscreen 2092) and send the collected touch events to other devices (such as the application processor 201). User actions near the touchscreen 2092 can be termed hover touch; through hover touch, the user can select, move, or drag objects (such as icons) without directly touching the touchscreen 2092. Furthermore, the touchscreen 2092 can be implemented using various types of touchscreens, including resistive, capacitive, infrared, and surface acoustic wave.

[0111] A display (also called a screen) 2091 is used to display information input by the user or information shown to the user. The display can be configured using a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other similar methods. A touchscreen 2092 can be placed over the display 2091. When the touchscreen 2092 detects a touch event, it transmits the information to the application processor 201 to determine the type of touch event. The application processor 201 then provides corresponding visual output on the display 2091 based on the type of touch event. Although... Figure 2 In this embodiment, the touchscreen 2092 and the display 2091 are two separate components for implementing the input and output functions of the device 200. However, in some embodiments, the touchscreen 2092 and the display 2091 can be integrated to achieve the input and output functions of the device 200. Furthermore, the touchscreen 2092 and the display 2091 can be configured as a full-panel display on the front of the device 200 to achieve a borderless structure.

[0112] Audio circuit 2093, speaker 2094, and microphone 2095 provide an audio interface between the user and device 200. Audio circuit 2093 converts received audio data into electrical signals and transmits them to speaker 2094, where speaker 2094 converts them into sound signals for output. On the other hand, microphone 2095 converts collected sound signals into electrical signals, which are received by audio circuit 2093, converted into audio data, and then transmitted to another device via modem 204 and radio frequency module 205, or output to memory 203 for further processing.

[0113] In addition, device 200 may also have fingerprint recognition functionality. For example, a fingerprint sensor can be configured on the back of device 200 (e.g., below the rear camera) or on the front of device 200 (e.g., below touchscreen 2092). Alternatively, a fingerprint sensor can be integrated into touchscreen 2092 to implement fingerprint recognition; that is, the fingerprint sensor can be integrated with touchscreen 2092 to achieve fingerprint recognition functionality for device 200. In this case, the fingerprint sensor is configured within touchscreen 2092, and may be part of touchscreen 2092 or configured in other ways. The main component of the fingerprint sensor in this embodiment is a fingerprint sensor, which can employ any type of sensing technology, including but not limited to optical, capacitive, piezoelectric, or ultrasonic sensing technologies.

[0114] Furthermore, the operating system 211 mounted on device 200 can provide... This application does not impose any restrictions on other operating systems, including those used in the embodiments of the present application.

[0115] With Taking device 200, which operates the system, as an example, device 200 can be logically divided into a hardware layer, an operating system 211, and an application layer. The hardware layer includes hardware resources such as the application processor 201, MCU 202, memory 203, modem 204, Wi-Fi module 206, sensor 208, and positioning module 210, as described above. The application layer includes one or more applications, such as application 212, which can be any type of application, such as a social application, e-commerce application, or browser. The operating system 211, as software middleware between the hardware layer and the application layer, is a computer program that manages and controls hardware and software resources.

[0116] In one embodiment, the operating system 211 includes a kernel, a hardware abstraction layer (HAL), libraries and runtime, and a framework. The kernel provides low-level system components and services, such as power management, memory management, thread management, and hardware drivers. Hardware drivers include Wi-Fi drivers, sensor drivers, and positioning module drivers. The hardware abstraction layer encapsulates the kernel drivers, providing interfaces to the framework and shielding them from low-level implementation details. The hardware abstraction layer runs in user space, while the kernel drivers run in kernel space.

[0117] Libraries and runtimes, also known as runtime libraries, provide the necessary library files and execution environment for executable programs at runtime. In one embodiment, libraries and runtimes include the Android runtime (ART), libraries, and scene package runtimes. ART is a virtual machine or virtual machine instance capable of converting application bytecode into machine code. Libraries are program libraries that provide support for executable programs at runtime, including browser engines (such as WebKit), script execution engines (such as JavaScript engines), and graphics processing engines. The scene package runtime is the runtime environment for scene packages, mainly including the page execution environment and the script execution environment. The page execution environment parses page code in HTML, CSS, and other formats by calling corresponding libraries, while the script execution environment parses and executes code or executable files implemented in scripting languages ​​such as JavaScript by calling corresponding function libraries.

[0118] The framework provides various basic public components and services for applications in the application layer, such as window management, location management, and so on. In one embodiment, the framework includes a geofencing service, a policy service, a notification manager, and so on.

[0119] The functions of each component of the operating system 211 described above can be implemented by the application processor 201 executing the program stored in the memory 203.

[0120] Those skilled in the art will understand that equipment 200 may include more Figure 2 Fewer or more components are shown. Figure 2 The device shown includes only components more relevant to the various implementations disclosed in this application.

[0121] Figure 3 This is a flowchart of an embodiment of the URL casting method of this application. This process 300 can be executed by a source device, a destination device, and a media server. Process 300 is described as a series of steps or operations; it should be understood that process 300 can be executed in various orders and / or occur simultaneously, and is not limited to... Figure 3 The execution order is shown. Figure 3 As shown, the method in this embodiment may include:

[0122] Step 301: The source device determines the first bitstream and download instruction information based on the playback progress during screen mirroring switching.

[0123] It should be noted that in this application, Sender refers to the initiating end of URL casting, such as a mobile phone, tablet, or computer, while Receiver refers to the receiving end, such as a TV, set-top box, AV amplifier, Chromecast, or computer. Typically, the URL casting process begins with the Sender sending a casting request to the Receiver. The Sender can include the URL address of the currently playing video in this request. The Receiver then downloads the corresponding video stream from the media server based on the URL address and plays it. Therefore, Sender and Receiver are fixed. The source device (SourcePlayer) refers to the end playing the video before the casting switch, and the destination device (DestPlayer) refers to the end playing the video after the casting switch. When casting is initiated, the Sender initiates the transfer of video playback rights to the Receiver; at this time, SourcePlayer is the Sender and DestPlayer is the Receiver. When casting is terminated, the Receiver returns video playback rights to the Sender; at this time, SourcePlayer is the Receiver and DestPlayer is the Sender. In other words, SourcePlayer and DestPlayer are relative terms, and the Sender and Receiver will switch depending on the different stages of screen casting.

[0124] Online media playback typically employs streaming media technology, which uses streaming transmission technology to continuously play media formats, such as audio, video, or multimedia files, over the network in real time. Streaming media technology compresses continuous media data and stores it as media segments on a media server. Each media segment consists of a bitstream corresponding to multiple image frames. During playback, the media server sequentially or in real time transmits video media segments to the user's terminal device. The terminal device downloads and plays the video simultaneously, without waiting for the entire video file to download completely. The terminal device creates a buffer, pre-downloading a segment of the video bitstream as a buffer before playback. When the actual network connection speed is lower than the playback speed, the player uses a small segment of the buffered bitstream for playback, preventing video stuttering.

[0125] Therefore, it can be seen that the source device is doing two things simultaneously while playing video: playback and download caching. To ensure the continuity of the video, the timing of the image frames being played by the source device at any given moment is usually earlier than the timing of the image frames being downloaded. In other words, for the same image frame, the time spent downloading and caching that image frame is earlier than the time spent playing that image frame. Figure 4 An exemplary sequence diagram of image frames is shown, such as... Figure 4 As shown, Fc is the image frame being played by the source device when the screen mirroring is switched, Fd is the last image frame that the source device has finished downloading when the screen mirroring is switched, Fk is the keyframe corresponding to Fc, and Fn is the next image frame of Fd. In this example, Fk, Fc, Fd, and Fn belong to the same media segment Si. Figure 5 An exemplary sequence diagram of image frames is shown, such as... Figure 5 As shown, Fc is the image frame being played by the source device when the screen mirroring is switched, Fd is the last image frame that the source device has finished downloading when the screen mirroring is switched, Fk is the keyframe corresponding to Fc, and Fn is the next image frame of Fd. In this example, Fk and Fc belong to the same media segment Si, and Fd and Fn belong to the same media segment Sj. Si and Sj are different media segments. Si and Sj can be adjacent or separated by one or more other media segments. This application does not make specific limitations in this regard. Figure 6 An exemplary sequence diagram of image frames is shown, such as... Figure 6As shown, Fc is the image frame being played by the source device during the screen mirroring switch, Fd is the last image frame that the source device has finished downloading during the screen mirroring switch, Fk is the keyframe corresponding to Fc, and Fn is the next image frame after Fd. In this example, Fk and Fc belong to the same media segment Si, Fd belongs to media segment Sl, and Fn belongs to media segment Sj. Si, Sl, and Sj are different media segments. Sl and Sj must be adjacent, while Si and Sl can be adjacent or separated by one or more other media segments. This application does not impose specific limitations on this. It should be noted that in the above example, Fk and Fc belong to the same media segment Si. It should be understood that Fk and Fc can also belong to different media segments. When a media segment is very small, it does not always contain a keyframe. In addition, Figures 4-6 The example illustrates the timing relationship of image frames already downloaded by the source device before screen mirroring, as well as the possible correspondence between image frames and media segments, but this does not limit the video bitstream involved in this application.

[0126] To ensure video continuity, frame number d (Fd) is later than frame number c (Fc), and it can be assumed that the image frames from Fc to Fd have not yet been played by the source device. The bitstream downloaded by the source device from the media server includes information about the image frames before Fc, as well as information about the image frames from Fc to Fd.

[0127] When switching screen mirroring, the source device is playing image frame Fc. Therefore, to ensure screen mirroring time synchronization, the destination device must start playing from image frame Fc after the screen mirroring switch is complete. The starting position of the video playback on the destination device can be determined using the seek command in the URL protocol. Optionally, this application can also use a new message, such as a time indication message, to indicate the image frame currently playing during the screen mirroring switch.

[0128] The destination device, starting playback from image frame Fc, undergoes the same process as the source device: before playback, it downloads a video stream from the media server as a buffer, uses this buffer to begin playback, and downloads the buffer for subsequent image frames while playing. It should be understood that for the destination device to start playback from image frame Fc, the buffer it downloads must include at least the video streams from Fc to Fd.

[0129] In this application, the source device can send its cached bitstream to the destination device via a communication network. Since URL casting typically occurs between two devices located on the same local area network (LAN), sending the bitstream from the source device to the destination device can be achieved through the LAN, which offers faster data transmission speeds and shorter processing times. Furthermore, to reduce the amount of data transmitted, the source device can send only the bitstream of image frames already downloaded from the media server but not yet played; this portion of the bitstream can be referred to as the first bitstream.

[0130] In addition to the first bitstream, the destination device also needs to buffer subsequent bitstreams to ensure normal video playback. This second bitstream can be called the second bitstream, which the destination device can download from the media server. In this application, the destination device cannot determine from which image frame the second bitstream begins. Therefore, the source device can determine a download indication information to indicate the starting frame of the second bitstream.

[0131] Theoretically, the starting frame of the first bitstream should be Fc. However, based on media encoding / decoding techniques and the role of keyframes, if Fc is not a keyframe, the destination device cannot decode the image corresponding to Fc using only the information from Fc. A corresponding keyframe Fk (Fk is the keyframe that precedes Fc and is closest to it) is also required to fully decode the image corresponding to Fc. The same applies to image frames after Fc. Therefore, even if Fk has already played during screen mirroring, to ensure the integrity of the first bitstream's information, the keyframe Fk corresponding to Fc must be sent to the destination device. Thus, the starting frame of the first bitstream can be Fk. As for image frames before Fk, since these image frames have already been played on the source device during screen mirroring, the destination device only needs to start playing from Fc to achieve screen mirroring time synchronization. Image frames played before Fc are unnecessary for the destination device. In summary, the starting frame of the first bitstream can be Fk, which reduces the amount of data transmitted between the source and destination devices while providing sufficient image information to ensure effective image decoding.

[0132] The end frame of the first stream can include two possible cases: (1) The end frame of the first stream is the last image frame Fd that the source device has finished downloading when switching the screen. In this case, it means that the source device will send the stream of all image frames that have been downloaded from the media server and have not yet been played to the destination device, that is, the stream of image frames from Fc to Fd. (2) The end frame of the first stream is the last image frame Fe of the media segment to which Fc belongs. According to the above description of streaming media technology, continuous media data is stored uniformly in the form of media segments after compression processing, and each media segment contains multiple image frames. In order to ensure that the destination can achieve screen casting time synchronization, and in order to meet the storage requirements of media data, the first stream sent by the source device to the destination device can end at Fe. However, there is also a prerequisite for this case: Fd and Fc belong to different media segments, so that Fe is earlier than Fd. Otherwise, if Fd and Fc belong to the same media segment, the source device will only download Fd when switching the screen, and Fd and Fe may not be the same image frame. Therefore, the source device cannot send Fe to the destination device.

[0133] Based on the video storage format, devices download video streams from media servers in units of media segments, downloading one segment at a time sequentially. Therefore, whether it's the source device or the destination device, the starting frame of the stream downloaded from the media server must be the first image frame of a certain media segment. Usually, the starting frame of the stream downloaded by the source device from the media server is the first image frame of the first media segment of the video, while the starting frame of the stream (second stream) to be downloaded by the destination device from the media server is related to the ending frame of the first stream. The starting frame of the second stream can be the first image frame of the media segment to which the next image frame of the ending frame of the first stream belongs. In case (1), the ending frame of the first stream is the last image frame Fd that the source device has already downloaded when switching the screen. In the ideal case, the second stream would start from the next image frame Fn of Fd. However, considering the media segment processing on the media server, the starting frame of the second stream can be the first image frame of the media segment to which Fn belongs. At this time, the ending part of the first stream and the beginning part of the second stream may overlap, for example, Figure 5 In the first stream, Fd and Fn belong to the same media segment Sj. The ending frame of the first stream is Fd, and the starting frame of the second stream is the first image frame of the media segment to which Fn belongs. This first image frame is either Fd or earlier than Fd, resulting in partial overlap between the first and second streams. For example... Figure 6In this case, Fd belongs to media segment Sl, and Fn belongs to media segment Sj. Sl and Sj are different media segments, and Fd is the previous image frame of Fn. Therefore, the ending frame Fd of the first bitstream is exactly the last image frame of media segment Sl, and Fn is exactly the first image frame of the adjacent media segment Sj. In this way, the first bitstream and the second bitstream will not overlap. For case (2), the ending frame of the first bitstream is the last image frame Fe of the media segment to which Fc belongs. In the ideal state, the second bitstream would start from the next image frame of Fe. However, considering the media segment processing on the media server, the starting frame of the second bitstream can be the first image frame of the media segment to which the next image frame of Fe belongs. Since the ending frame of the first bitstream is the last image frame Fe of the media segment, the next image frame of Fe is the first image frame of the adjacent media segment, and this first image frame is the starting frame of the second bitstream. Therefore, the first bitstream and the second bitstream will not overlap.

[0134] Leveraging the advantages of local area network (LAN) transmission, the source device in this application can transmit as much of its cached bitstream as possible to the destination device via the LAN, thus reducing the amount of data in the second bitstream that the destination device downloads from the media server. Therefore, the starting frame of the second bitstream can be the first image frame F1 of the media segment to which Fn belongs, and Fn is the image frame following the last image frame Fd that the source device has already downloaded when switching screens. This is the latest starting frame that the second bitstream can be set; if subsequent media segments begin downloading, screen synchronization will inevitably fail. Therefore, the ending frame of the first bitstream is related to the starting frame of the second bitstream, i.e., the ending frame of the first bitstream is the image frame F0 preceding F1. In this case, the first and second bitstreams will not overlap.

[0135] Step 302: The source device sends a download instruction to the destination device.

[0136] Step 303: The source device sends the first bitstream to the destination device.

[0137] Through the communication network between the source device and the destination device, the source device can send download instruction information and the first bit stream to the destination device.

[0138] The source device can send playback time indication information to the destination device. This playback time indication information is used to indicate the image frame being played when switching screen projections. The purpose of the source device sending playback time indication information is to inform the destination device of its own playback progress when switching screen projections, so that the destination device can determine from which frame to start playing the video based on the playback progress, thereby achieving screen projection time synchronization between the two sides.

[0139] In one possible implementation, the source device can encapsulate the first bitstream according to a set encapsulation format to obtain format data. The source device then sends this format data, along with control information, to the destination device. The control information includes the aforementioned encapsulation format. The image frames included in the first bitstream may span multiple media segments. The source device can encapsulate the first bitstream into a single media segment, such as a Transport Stream (TS) or FMP4 segment, to simplify the organization of transmitted data. Optionally, the source device can also divide the first bitstream into multiple media segments for transmission; this application does not specifically limit this. The encapsulation object in this application is the video bitstream downloaded and cached from the media server. This encapsulation process can improve the performance of screen mirroring time synchronization. It does not involve image encoding or decoding, thereby reducing the algorithm's complexity and hardware requirements.

[0140] Step 304: The destination device downloads the second stream from the media server.

[0141] The download instruction information indicates the start frame of the second bitstream, so the destination device downloads a bitstream of consecutive image frames starting from that start frame from the media server according to the download instruction information. As mentioned above, the start frame of the second bitstream has been determined to be the first image frame of a certain media segment, so the destination device can request the media server to start downloading from that media segment.

[0142] Optionally, the download instruction information may include information about the media segment to which the starting frame of the second bitstream belongs. The destination device can determine the media segment to which the starting frame belongs based on this information, and thus determine the starting frame.

[0143] Optionally, the download instruction information may include information about the start frame of the second stream. The destination device can directly determine the start frame based on this information, and then determine the media segment to which the start frame belongs.

[0144] This application does not specifically limit the information included in the above download instructions.

[0145] Step 305: The destination device plays the video according to the first bitstream or the second bitstream.

[0146] Receiving the first stream and downloading the second stream from the media server are two independent processes for the destination device, and therefore can occur simultaneously. Since the first stream precedes the second stream, the destination device can start playback from the first stream. During the reception of the first and second streams, if the amount of data received from the first stream is sufficient for playback, the destination device can start playing the video based on the playback progress when switching from screen mirroring, thus improving the screen mirroring response speed. After the first stream has finished playing, the destination device has also accumulated enough data from the second stream and can continue playing the video based on the second stream, ensuring smooth video playback.

[0147] In the above process, to achieve screen mirroring time synchronization, the destination device can determine the first image frame to be played after the screen mirroring switch from either the first or second bitstream, and start playing the video from the first image frame. The first image frame is the key to screen mirroring time synchronization. As mentioned above, in order to achieve screen mirroring time synchronization, the source device sends playback time indication information to the destination device to indicate the playback time of the image frame Fc at the time of screen mirroring switch. Based on this playback time indication information, the destination device can first determine that the first image frame to be played is Fc, then play the video from Fc to the end frame of the first bitstream according to the first bitstream, and play the video starting from Fm according to the second bitstream, where Fm is the image frame after the end frame of the first bitstream.

[0148] As described above, the end portion of the first bitstream and the beginning portion of the second bitstream may overlap, or they may not overlap at all. If they do not overlap, the destination device, after playing the first bitstream, directly starts playing the second bitstream. The end frame of the first bitstream is the previous frame of the start frame of the second bitstream, thus enabling seamless switching between the first and second bitstreams. If they overlap, the destination device can choose to play the first bitstream (from frame c to the end frame) and the second bitstream (starting from frame m) in the overlapping portion. It should be noted that the destination device can also choose to play the second bitstream in the overlapping portion (playing the first bitstream from frame c to the previous frame of the second bitstream, discarding the portion of the first bitstream from the start frame to the end frame of the second bitstream, and playing the second bitstream starting from its start frame). This application does not specifically limit this.

[0149] Steps 301-305 mainly describe the screen casting switching process when the total data volume of the first bitstream is greater than or equal to a set threshold. The set threshold is used to characterize whether the total data volume of the first bitstream is sufficient to support video playback. In one possible implementation, if the data volume of the cached bitstream downloaded by the source device from the media server is small, less than the set threshold, even if it is sent to the destination device, the destination device still needs to download almost the entire bitstream from the media server, or the video cannot be played based on the first bitstream, or the bandwidth consumption and transmission latency costs incurred in transmitting the first bitstream are higher than the costs incurred in directly downloading from the media server, then the source device can send a data volume indication message to the destination device. This data volume indication message is used to notify that the total data volume of the first bitstream is less than the set threshold. In this case, the source device can choose to send the first bitstream to the destination device or choose not to send the first bitstream. This process can be negotiated through high-level configuration or mutual interaction information, which will not be elaborated here.

[0150] When the source device sends the first bitstream to the destination device, the destination device can download the fourth bitstream from the media server. The starting frame of this fourth bitstream is the first frame of the media segment to which Fn belongs. That is, the second bitstream is downloaded starting from the first image frame of the media segment to which the next frame after the end frame of the first bitstream belongs.

[0151] If the source device does not send the first bitstream, the destination device can download the third bitstream from the media server. The starting frame of this third bitstream is the first image frame of the media segment to which the image frame Fc being played by the source device at the time of screen mirroring switch. Since there is no first bitstream, in order to ensure normal video playback and to achieve screen mirroring time synchronization, the destination device can download the second bitstream starting from the first image frame of the media segment to which Fc belongs.

[0152] During screen mirroring switching, the source device sends the cached bitstream to the destination device. On one hand, the cached bitstream is transmitted via a local area network (LAN). Leveraging the advantages of the LAN, including higher bandwidth and better QoS, fast and stable transmission of the cached bitstream can be achieved. On the other hand, the destination device receives the cached bitstream from the source device while simultaneously downloading the bitstream after the cached bitstream from the media server. This allows for quick playback of the video based on the existing bitstream, improving the screen mirroring switching response speed, and also ensures that sufficient bitstream is stored during video playback to prevent video playback stuttering.

[0153] It should be noted that the above method embodiments describe the process of switching screen mirroring, whereby the source device sends the first bitstream and download instruction information to the destination device, thereby realizing the screen mirroring switch. This process occurs when screen mirroring is started. The source device can be a mobile phone, tablet, computer, or other device acting as the sender, and the destination device can be a TV, set-top box, AV amplifier, Chromecast, computer, or other device acting as the receiver. This process also occurs when screen mirroring is terminated. Again, the source device can be a TV, set-top box, AV amplifier, Chromecast, computer, or other device acting as the receiver, and the destination device can be a mobile phone, tablet, computer, or other device acting as the sender. Therefore, the roles of sender and receiver can switch at different stages of screen mirroring, and this application does not specifically limit this.

[0154] Figure 7 This is a flowchart of Embodiment 2 of the URL casting method of this application. This process 700 can be executed by a source device, a destination device, and a media server. Process 700 is described as a series of steps or operations; it should be understood that process 700 can be executed in various orders and / or occur simultaneously, and is not limited to... Figure 7 The execution order is shown. Figure 7 As shown, the method in this embodiment may include:

[0155] Step 701: The user initiates screen mirroring via Sender.

[0156] This process can employ any technology that supports URL casting protocols, such as applications, video platforms, and player programs; this application does not impose any specific limitations on it.

[0157] Step 702: The sender sends a screen mirroring request to the receiver.

[0158] The screen mirroring request can include the URL of the video to be played, as well as seek commands, to inform the Receiver of the address of the currently playing video and the current playback progress.

[0159] Step 703: The Receiver sends a screen mirroring response to the Sender.

[0160] If the Receiver correctly receives the screen sharing request sent by the Sender, the Receiver can send an acknowledgment response (ACK); otherwise, the Receiver can send a negative acknowledgment response (NAK).

[0161] Step 704: Sender determines the first bitstream and download instruction information based on the current playback progress.

[0162] Step 704 can be referred to step 301 above, and will not be repeated here.

[0163] Step 705: The sender sends media information to the receiver.

[0164] The first bitstream determined by the sender includes the start and end frames of the first bitstream, as well as the encapsulation format of the first bitstream, and may also include the organization and storage method of the first bitstream. The sender can send all this information, along with the aforementioned download instruction information, as media information to the receiver. Optionally, the media information can be described using JavaScript Object Notation (JSON) or Extensible Markup Language (XML), etc. It should be understood that other information description methods can also be used for media information, and this application does not specifically limit them.

[0165] Step 706: The sender sends the first bitstream to the receiver.

[0166] Step 706 can be referred to step 303 above, and will not be repeated here.

[0167] Step 707: The Receiver sends a download request to the media server.

[0168] The download request may include the start frame of the second bitstream.

[0169] Step 708: The Receiver downloads the second stream from the media server.

[0170] Step 708 can be referred to step 304 above, and will not be repeated here.

[0171] Step 709: When the amount of data in the first bitstream is sufficient, the Receiver starts playing the video.

[0172] During the process of receiving the first and second bitstreams, if the amount of data received in the first bitstream is sufficient for playback, such as a 500ms video bitstream, the receiver can start playing the video based on the playback progress when switching from screen mirroring. This helps to improve the screen mirroring switching response speed.

[0173] Step 709 can be referred to step 305 above, and will not be repeated here.

[0174] Step 710: The user terminates screen mirroring via Sender.

[0175] This process can employ any technology that supports URL casting protocols, such as applications, video platforms, and player programs; this application does not impose any specific limitations on it.

[0176] Optionally, users can also terminate screen mirroring through the Receiver. Similarly, any application, video platform, player program, or other technology that supports URL screen mirroring protocols can be used. This application does not impose any specific limitations on this.

[0177] Step 711: The sender sends a request to the receiver to terminate screen mirroring.

[0178] Step 712: The Receiver sends a termination screen mirroring response to the Sender.

[0179] The response to terminate screen mirroring can include commands such as seek to inform the sender of the current video playback progress.

[0180] Step 713: The Receiver determines the first bitstream and download instruction information based on the current playback progress.

[0181] Step 713 can be referred to step 301 above, and will not be repeated here.

[0182] Step 714: The Receiver sends media information to the Sender.

[0183] Step 714 can be referred to step 705 above, and will not be repeated here.

[0184] Step 715: The Receiver sends the first bitstream to the Sender.

[0185] Step 715 can be referred to step 303 above, and will not be repeated here.

[0186] Step 716: Sender sends a download request to the media server.

[0187] The download request may include the start frame of the second bitstream.

[0188] Step 717: Sender downloads the second stream from the media server.

[0189] Step 717 can be referred to step 304 above, and will not be repeated here.

[0190] Step 718: When the amount of data in the first bitstream is sufficient, Sender starts playing the video.

[0191] During the process of receiving the first and second bitstreams, if the amount of data received from the first bitstream is sufficient for playback, the sender can start playing the video based on the playback progress of the first bitstream when switching from screen casting. This helps to improve the response speed of screen casting.

[0192] Step 718 can be referred to step 305 above, and will not be repeated here.

[0193] It should be noted that, Figure 3 and Figure 7 The illustrated embodiment represents the standard flow of the URL casting method provided in this application. It can serve as an extension to relevant casting protocols, i.e., an extension is made to the relevant casting protocols to implement the URL casting method provided in this application. It should be understood that abnormal situations specified in relevant protocols, or other processes not covered in the embodiments of this application, can also be supported and compatible with the URL casting method provided in this application.

[0194] For example, one or both of the sender and receiver may not support the URL casting method provided in this application. If the receiver does not support the URL casting method, the receiver can notify the sender of this situation through the casting response in step 703. If the sender does not support the URL casting method, then the sender will not send the first stream to the receiver. If either party does not support the URL casting method, the sender and receiver will execute the casting process in related technologies to achieve compatibility between the URL casting method and related casting methods.

[0195] For example, after the Receiver starts playing a video, it periodically reports the current playback status (including the playback position) to the Sender. If the user adjusts the playback progress by dragging, the Receiver can also report it to the Sender.

[0196] For example, if the Sender initiates screen mirroring and the Receiver has already started playing the video, and the user closes the player, application, etc., on the Sender, then the user cannot terminate the screen mirroring from the Sender. In this case, the user can terminate the screen mirroring through the Receiver, or they can terminate it by reopening the player, application, etc., on the Sender.

[0197] For example, the first stream can be actively sent from the source device to the destination device, or the destination device can request to retrieve it from the source device after receiving a screen mirroring switching request. This application does not impose specific limitations on this.

[0198] For example, the transmission of the first bitstream can use Transmission Control Protocol (TCP), Hypertext Transfer Protocol (HTTP), WebSocket, or various proprietary protocols, and this application does not make any specific limitations on this.

[0199] For example, the aforementioned related protocols may include URL casting protocols such as DLNA, AirPlay, and GoogleCast. The URL casting method provided in this application extends these protocols, requiring deployment on both the sender and receiver ends. Different extension schemes can be adopted for different protocols based on their characteristics to achieve compatibility extensions.

[0200] The following uses the WebSocket protocol as an example to describe the key points of transmitting the first bitstream in the URL casting method provided in this application. Although using TCP directly would be simpler, more and more media players are now choosing to run as web applications (WebApps) in browsers, and browsers do not have direct access to the standard TCP interface.

[0201] The Sender always acts as a WebSocket client, actively initiating WebSocket connections. The Receiver always acts as a WebSocket server, passively listening for connections. Ideally, the Receiver should dynamically select the WebSocket server port and report it to the Sender. This approach relies on the Receiver's ability to extend the casting response message to include the selected port when responding to the Sender's casting request. DLNA initiates casting requests via SOAPSetAVTransportURI, and the casting response is in XML format, allowing for compatibility extension. AirPlayer initiates casting requests via HTTP POST / play, and the casting response has no HTTP body; however, an HTTP body or HTTP header can be added for compatibility extension. GoogleCast is unique; after the Sender initiates a casting request via the SDK, both ends can establish a WebSocket connection and exchange messages via the SDK. This WebSocket connection can be directly reused to send the first stream. For cases like GoogleCast, it's not necessary to establish a WebSocket connection as described below; the WebSocket connection is considered established upon initiation of casting.

[0202] When the Sender initiates screen mirroring, it includes extended parameters in the mirroring request, indicating that the Sender supports this scheme. If the Receiver supports this scheme, it will be able to recognize the extended parameters in the mirroring request. Before responding to the Sender, the WebSocket Server is started first, and then a screen mirroring response is sent to the Sender, carrying the WebSocket Server Port. After the Sender recognizes the WebSocket Server Port in the Receiver's response, it initiates a WebSocket connection request. The WebSocket Server Port returned by the Receiver is valid throughout a complete screen mirroring process; the Receiver listens on this port continuously during this period, waiting for a connection. A complete screen mirroring process refers to the time range from when the Sender initiates the screen mirroring request until the user terminates the screen mirroring through the Sender or Receiver, and the reverse transmission of the first stream is completed (or a timeout occurs, or the Sender initiates a new screen mirroring request).

[0203] After the Receiver detects the termination of the screen mirroring request, it waits for the Sender to retrieve its cached first stream and starts a timer. When the first stream transmission is complete, the timer expires, or a new screen mirroring request is detected from the same Sender, the Receiver determines that the current screen mirroring process has ended, terminates the WebSocket Server, and the corresponding port becomes invalid. The Sender decides whether to maintain a long connection or a short connection. A long connection means that after the Sender completes the first stream transmission, it does not close the WebSocket connection. When screen mirroring terminates, it uses this connection to retrieve the reverse first stream, and then terminates the connection. A short connection means that after the Sender completes the first stream transmission, it immediately closes the WebSocket connection. When screen mirroring terminates, it re-establishes a WebSocket connection with the Receiver to retrieve the reverse first stream, and then terminates the connection.

[0204] This application defines new message transmission media information and a first bitstream. This message may have a uniform, fixed-length header carrying an identifier for the message type and a response code, followed by an optional message body. The following examples illustrate the functions of several new messages. It should be understood that these new messages are examples needed for extending the relevant protocols using the URL casting method provided in this application, but are not intended to be limiting.

[0205] PutCacheMeta: When screen mirroring is started, after the Sender and Receiver establish a WebSocket connection, the Sender sends a PutCacheMeta message, which carries media information in the message body.

[0206] PutCacheMedia: After receiving the ACK for PutCacheMeta from the Receiver, the Sender sends a PutCacheMedia message, which carries the first bitstream.

[0207] GetCacheMeta: When screen mirroring is terminated, the Sender sends a GetCacheMeta message, and the Receiver carries media information in the screen mirroring termination response.

[0208] GetCacheMedia: When screen mirroring ends, the Sender sends a GetCacheMedia message, and the Receiver replies with the first bitstream.

[0209] based on Figure 7 The flowchart shown, combined with the aforementioned new types of messages, Figure 8 This paper illustrates a possible embodiment of the URL casting method of this application. Figure 8 This is a flowchart of Embodiment 3 of the URL casting method of this application. This process 800 can be executed by a source device, a destination device, and a media server. Process 800 is described as a series of steps or operations; it should be understood that process 800 can be executed in various orders and / or occur simultaneously, and is not limited to... Figure 8 The execution order is shown. Figure 8 As shown, the method in this embodiment may include:

[0210] Step 801: The user initiates screen mirroring via Sender.

[0211] Step 802: The sender sends a screen mirroring request to the receiver.

[0212] Step 803: The Receiver sends a screen mirroring response to the Sender.

[0213] If the Receiver correctly receives the screen sharing request sent by the Sender, the Receiver can send an acknowledgment response (ACK); otherwise, the Receiver can send a negative acknowledgment response (NAK). The Receiver can include extended information such as the WebSocket Server Port in the ACK.

[0214] Step 804: The sender requests the receiver to establish a WebSocket connection.

[0215] Step 805: The Receiver sends a connection establishment response to the Sender.

[0216] If the Receiver successfully establishes a connection, it will reply with ACK; otherwise, it will reply with NAK.

[0217] Step 806: Sender determines the first bitstream and download instruction information based on the current playback progress.

[0218] Step 807: The sender sends a PutCacheMeta message to the receiver.

[0219] The PutCacheMeta message carries media information.

[0220] Step 808: The sender sends a PutCacheMedia message to the receiver.

[0221] The PutCacheMedia message carries the first bitstream.

[0222] Step 809: The Receiver sends a download request to the media server.

[0223] Step 810: The Receiver downloads the second bitstream from the media server.

[0224] Step 811: When the amount of data in the first bitstream is sufficient, the Receiver starts playing the video.

[0225] Step 812: The user terminates screen mirroring via Sender.

[0226] Step 813: The sender sends a request to the receiver to terminate the screen mirroring.

[0227] Step 814: The Receiver sends a termination screen mirroring response to the Sender.

[0228] Step 815: The Receiver determines the first bitstream and download instruction information based on the current playback progress.

[0229] Step 816: The sender sends a GetCacheMeta message to the receiver.

[0230] Step 817: The Receiver carries media information in the response message.

[0231] Step 818: The sender sends a GetCacheMedia message to the receiver.

[0232] Step 819: The Receiver sends the first bitstream to the Sender.

[0233] Step 820: Sender sends a download request to the media server.

[0234] Step 821: Sender downloads the second stream from the media server.

[0235] Step 822: When the amount of data in the first bitstream is sufficient, Sender starts playing the video.

[0236] The URL projection method of this application has been described above. The apparatus of this application is described below. The apparatus of this application includes a projection device applied to a source device and a projection device applied to a destination device. It should be understood that the projection device applied to the source device is the source device in the above method, which has any of the functions of the source device in the above method. The projection device applied to the destination device is the destination device in the above method, which has any of the functions of the destination device in the above method.

[0237] like Figure 9 As shown, the screen projection device applied to the destination device includes: a receiving module 901, a playback module 902, and a decoding module 903. The receiving module 901 is used to receive a first bitstream sent by the source device, which is a bitstream downloaded by the source device from a media server before screen projection switching; the receiving module 901 is also used to receive download instruction information sent by the source device, and download a second bitstream from the media server according to the download instruction information, wherein the start frame of the second bitstream is indicated by the download instruction information, and the start frame of the second bitstream is related to the end frame of the first bitstream; the playback module 902 is used to play video according to the first bitstream or the second bitstream.

[0238] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that the source device is playing when the screen projection is switched.

[0239] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that the source device has already downloaded when the screen projection is switched.

[0240] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, and the last image frame that the source device has finished downloading during screen mirroring switching is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0241] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0242] In one possible implementation, the image frame being played by the source device during screen mirroring switching is Fc, the last image frame that the source device has finished downloading during screen mirroring switching is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0243] In one possible implementation, the playback module 902 is specifically used to determine the first image frame to be played after the screen casting switch from the first bitstream or the second bitstream, and to start playing the video from the first image frame.

[0244] In one possible implementation, the receiving module 901 is further configured to receive playback time indication information sent by the source device, the playback time indication information being used to indicate the image frame Fc being played by the source device when the screen casting is switched; the playback module is further configured to, when the data volume of the first bitstream is greater than a set threshold, determine the first image frame to be played as Fc from the first bitstream according to the playback time indication information, and play the video from Fc to the end frame of the first bitstream according to the first bitstream, and play the video starting from Fm according to the second bitstream, where Fm is the next image frame after the end frame of the first bitstream.

[0245] In one possible implementation, the receiving module 901 is specifically used to receive the first bitstream sent by the source device when the total data volume of the first bitstream is not less than a set threshold.

[0246] In one possible implementation, the receiving module 901 is further configured to receive the data volume indication information sent by the source device when the total data volume of the first bitstream is less than the set threshold; download the third bitstream from the media server, wherein the starting frame of the third bitstream is the first image frame of the media segment to which the image frame Fc being played by the source device at the time of screen casting switch.

[0247] In one possible implementation, the receiving module 901 is further configured to receive the data volume indication information and the first bitstream sent by the source device when the total data volume of the first bitstream is less than the set threshold; download the fourth bitstream from the media server, wherein the last image frame of the first bitstream is Fd, the next image frame of Fd is Fn, and the starting frame of the fourth bitstream is the first frame of the media segment to which Fn belongs.

[0248] In one possible implementation, the receiving module 901 is further configured to receive control information sent by the source device, the control information including the encapsulation format of the first bitstream; receive format data sent by the source device; and the decoding module 903 is configured to decapsulate the format data according to the encapsulation format to obtain the first bitstream.

[0249] The apparatus of this embodiment can be used to perform Figures 3-8 The technical solutions of any of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0250] like Figure 10 As shown, a screen mirroring device applied to a destination device includes: a processing module 1001, a sending module 1002, and an encapsulation module 1003. The processing module 1001 is used to determine a first bitstream and download indication information based on the playback progress during screen mirroring switching. The first bitstream is the bitstream downloaded from the media server before screen mirroring switching. The download indication information is used to indicate the start frame of a second bitstream, which is the bitstream that the destination device needs to download from the media server. The start frame of the second bitstream is related to the end frame of the first bitstream. The sending module 1002 is used to send the download indication information and the first bitstream to the destination device.

[0251] In one possible implementation, the starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that is being played when the screen casting is switched.

[0252] In one possible implementation, the end frame of the first bitstream is the last image frame Fd that has been downloaded when the screen casting is switched.

[0253] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, and the last image frame that has been downloaded during the screen mirroring switch is Fd. Fd and Fc belong to different media segments, and each media segment contains multiple image frames. The end frame of the first bitstream is the last image frame of the media segment to which Fc belongs.

[0254] In one possible implementation, the starting frame of the second bitstream is related to the ending frame of the first bitstream, specifically including: the starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

[0255] In one possible implementation, the image frame playing during the screen mirroring switch is Fc, the last image frame that has been downloaded during the screen mirroring switch is Fd, the next image frame of Fd is Fn, Fn and Fc belong to different media segments, the starting frame of the second stream is the first image frame F1 of the media segment to which Fn belongs, and the ending frame of the first stream is the previous image frame F0 of F1.

[0256] In one possible implementation, the sending module 1002 is further configured to send playback time indication information to the destination device, the playback time indication information being used to indicate the image frame Fc being played during screen mirroring switching.

[0257] In one possible implementation, the sending module 1002 is further configured to send data volume indication information to the destination device when the total data volume of the first bitstream is less than a set threshold. The data volume indication information is used to notify that the total data volume of the first bitstream is less than the set threshold.

[0258] In one possible implementation, the encapsulation module 1003 is used to encapsulate the first bitstream according to a set encapsulation format to obtain format data; the sending module 1002 is specifically used to send control information and the format data to the destination device, wherein the control information includes the encapsulation format.

[0259] The apparatus of this embodiment can be used to perform Figures 3-8 The technical solutions of any of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0260] In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The steps of the method disclosed in this application can be directly implemented by a hardware encoding processor, or by a combination of hardware and software modules in the encoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and combines it with its hardware to complete the steps of the above method.

[0261] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0262] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0263] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0264] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0265] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A URL-based screen mirroring method, characterized in that, include: The first bitstream sent by the source device is at least one media segment downloaded by the source device from the media server one by one in the form of media segments via the streaming media protocol before the screen casting switch; During video playback based on the first bitstream, a second bitstream is downloaded according to the download instruction information sent by the source device. The second bitstream includes at least one media segment downloaded one by one from the media server via the streaming media protocol in units of media segments. After playing the video to image frame Fd based on the first bitstream, the video is then played from image frame Fn based on the second bitstream, where image frame Fn is the next image frame after image frame Fd. Wherein, the image frame Fd is located in media segment S1, the image frame Fn is located in media segment Sj, and the media segment S1 is adjacent to the media segment Sj.

2. The method according to claim 1, characterized in that, The streaming media protocol is the Hypertext Transfer Protocol Live Streaming (HLS) streaming media protocol.

3. The method according to claim 1, characterized in that, The download instruction information is used to indicate the position of the image frame Fn.

4. The method according to claim 1, characterized in that, The starting frame of the first bitstream is the key frame Fk corresponding to the image frame Fc that the source device is playing when the screen projection is switched.

5. The method according to any one of claims 1-4, characterized in that, The end frame of the first bitstream is the last image frame Fd that the source device has already downloaded when the screen projection is switched.

6. The method according to any one of claims 1-4, characterized in that, The start frame of the second bitstream is related to the end frame of the first bitstream, specifically including: The starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

7. The method according to claim 1, characterized in that, The playback of video based on the first bitstream specifically includes: The first image frame to be played after the screen casting switch is determined from the first bitstream, and the video is played starting from the first image frame.

8. The method according to claim 1, characterized in that, The first bitstream sent by the receiving source device includes: When the total data volume of the first bitstream is not less than a set threshold, the first bitstream sent by the source device is received.

9. The method according to any one of claims 1-4, characterized in that, Before receiving the first bitstream sent by the source device, the method further includes: Receive control information sent by the source device, the control information including the encapsulation format of the first bitstream; The receiving of the first bitstream sent by the source device includes: The system receives formatted data sent by the source device and decapsulates the formatted data according to the encapsulation format to obtain the first bitstream.

10. A URL projection method, characterized in that, include: The first bitstream and download instruction information are determined based on the playback progress during screen mirroring switching. The first bitstream is at least one media segment that is downloaded one by one from the media server via the streaming media protocol before screen mirroring switching. During video playback based on the first bitstream, the second bitstream is downloaded according to the download instruction information. The second bitstream is at least one media segment that the destination device needs to download one by one from the media server via the streaming media protocol. Send the download instruction information and the first bitstream to the destination device; After playing the video to image frame Fd based on the first bitstream, the video is then played from image frame Fn based on the second bitstream, where image frame Fn is the next image frame after image frame Fd. Wherein, the image frame Fd is located in media segment S1, the image frame Fn is located in media segment Sj, and the media segment S1 is adjacent to the media segment Sj.

11. The method according to claim 10, characterized in that, The streaming media protocol is the Hypertext Transfer Protocol Live Streaming (HLS) streaming media protocol.

12. The method according to claim 10, characterized in that, The download instruction information is used to indicate the position of the image frame Fn.

13. The method according to claim 10, characterized in that, The starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that is being played when the screen projection is switched.

14. The method according to any one of claims 10-13, characterized in that, The end frame of the first bitstream is the last image frame Fd that has been downloaded when the screen projection is switched.

15. The method according to any one of claims 10-13, characterized in that, The start frame of the second bitstream is related to the end frame of the first bitstream, specifically including: The starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

16. The method according to claim 10, characterized in that, Before sending the download instruction information and the first bitstream to the destination device, the method further includes: Send playback time indication information to the destination device. The playback time indication information is used to indicate the image frame Fc that is being played when the screen projection is switched.

17. The method according to claim 16, characterized in that, Also includes: When the total data volume of the first bitstream is less than a set threshold, a data volume indication message is sent to the destination device. The data volume indication message is used to notify that the total data volume of the first bitstream is less than the set threshold.

18. The method according to any one of claims 10-13, characterized in that, Before sending the download instruction information and the first bitstream to the destination device, the method further includes: The first bitstream is encapsulated according to the set encapsulation format to obtain format data; The control information and the formatted data are sent to the destination device, wherein the control information includes the encapsulation format.

19. A screen projection device, characterized in that, include: The receiving module is used to receive a first bitstream sent by the source device. The first bitstream is at least one media segment downloaded by the source device from the media server one by one in terms of media segments via the streaming media protocol before the screen projection is switched. The receiving module is further configured to download a second stream according to the download instruction information sent by the source device during video playback based on the first stream, wherein the second stream includes at least one media segment downloaded one by one from the media server in units of media segments via a streaming media protocol; The playback module is used to play video from image frame Fn based on the second bitstream after playing the first bitstream to image frame Fd, wherein image frame Fn is the next image frame after image frame Fd; Wherein, the image frame Fd is located in media segment S1, the image frame Fn is located in media segment Sj, and the media segment S1 is adjacent to the media segment Sj.

20. The apparatus according to claim 19, characterized in that, The streaming media protocol is the Hypertext Transfer Protocol Live Streaming (HLS) streaming media protocol.

21. The apparatus according to claim 19, characterized in that, The download instruction information is used to indicate the position of the image frame Fn.

22. The apparatus according to claim 19, characterized in that, The starting frame of the first bitstream is the key frame Fk corresponding to the image frame Fc that the source device is playing when the screen projection is switched.

23. The apparatus according to any one of claims 19-22, characterized in that, The end frame of the first bitstream is the last image frame Fd that the source device has already downloaded when the screen projection is switched.

24. The apparatus according to any one of claims 19-22, characterized in that, The start frame of the second bitstream is related to the end frame of the first bitstream, specifically including: The starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

25. The apparatus according to claim 19, characterized in that, The playback module is specifically used to determine the first image frame to be played after the screen casting switch from the first bitstream, and to start playing the video from the first image frame.

26. The apparatus according to claim 19, characterized in that, The receiving module is specifically used for: When the total data volume of the first bitstream is not less than a set threshold, the first bitstream sent by the source device is received.

27. The apparatus according to any one of claims 19-22, characterized in that, The receiving module is further configured to: Receive control information sent by the source device, the control information including the encapsulation format of the first bitstream; Receive formatted data sent by the source device; The device further includes a decoding module, used to decapsulate the format data according to the encapsulation format to obtain the first bitstream.

28. A screen projection device, characterized in that, include: The processing module is used to determine the first bitstream and download indication information based on the playback progress when switching the screen mirroring. The first bitstream is at least one media segment downloaded one by one from the media server via the streaming media protocol before the screen mirroring switch. The sending module is used to send the download instruction information and the first bitstream to the destination device; During the playback of video based on the first bitstream, the destination device downloads a second bitstream according to the download instruction information. The second bitstream is at least one media segment that the destination device needs to download one by one from the media server in media segments via the streaming media protocol. After the destination device plays the video to image frame Fd based on the first bitstream, it continues to play the video from image frame Fn based on the second bitstream, where image frame Fn is the next image frame after image frame Fd. Wherein, the image frame Fd is located in media segment S1, the image frame Fn is located in media segment Sj, and the media segment S1 is adjacent to the media segment Sj.

29. The apparatus according to claim 28, characterized in that, The streaming media protocol is the Hypertext Transfer Protocol Live Streaming (HLS) streaming media protocol.

30. The apparatus according to claim 28, characterized in that, The download instruction information is used to indicate the position of the image frame Fn.

31. The apparatus according to claim 28, characterized in that, The starting frame of the first bitstream is the keyframe Fk corresponding to the image frame Fc that is being played when the screen projection is switched.

32. The apparatus according to any one of claims 28-31, characterized in that, The end frame of the first bitstream is the last image frame Fd that has been downloaded when the screen projection is switched.

33. The apparatus according to any one of claims 28-31, characterized in that, The start frame of the second bitstream is related to the end frame of the first bitstream, specifically including: The starting frame of the second bitstream is the first image frame of the media segment to which the next image frame of the first bitstream belongs.

34. The apparatus according to claim 28, characterized in that, The sending module is also used to send playback time indication information to the destination device, the playback time indication information being used to indicate the image frame Fc being played when the screen projection is switched.

35. The apparatus according to claim 34, characterized in that, The sending module is further configured to send data volume indication information to the destination device when the total data volume of the first bitstream is less than a set threshold. The data volume indication information is used to notify that the total data volume of the first bitstream is less than the set threshold.

36. The apparatus according to any one of claims 28-31, characterized in that, It also includes an encapsulation module, used to encapsulate the first bitstream according to a set encapsulation format to obtain format data; The sending module is specifically used to send control information and the formatted data to the destination device, wherein the control information includes the encapsulation format.

37. A screen projection device, characterized in that, include: Processor and transmission interface; The processor is configured to invoke program instructions stored in memory to implement the method as described in any one of claims 1-9 or 10-18.

38. A computer-readable storage medium, characterized in that, Includes a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method of any one of claims 1-9 or 10-18.

Images