Screen projection method, device and equipment

Abstract

The embodiment of the invention provides a screen projection method, device and equipment. The method comprises the following steps: acquiring to-be-projected screen data; determining a first coding and decoding protocol of the to-be-projected screen data and a second coding and decoding protocol supported by first equipment; wherein the first coding and decoding protocol is different from the second coding and decoding protocol; based on the first coding and decoding protocol and the second coding and decoding protocol, converting the to-be-projected screen data into a format matched with the second coding and decoding protocol to obtain target projection screen data; and displaying the target screen projection data through the first device.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of communication technology, and in particular to a screen projection method, device, and equipment. Background Art

[0002] Different devices support different screen casting protocols. Apple devices support AirPlay, Android devices and Windows laptops support Miracast, and most online video apps use DLNA. However, if the receiving and sending ends of the screen casting support different protocols, screen casting will not be possible. Summary of the Invention

[0003] The embodiments of the present application provide a screen projection method, apparatus, and device. The technical solution of the embodiments of the present application is implemented as follows:

[0004] In a first aspect, an embodiment of the present application provides a screen projection method, the method comprising: obtaining data to be projected; determining a first codec protocol for the data to be projected and a second codec protocol supported by a first device; wherein the first codec protocol is different from the second codec protocol; based on the first codec protocol and the second codec protocol, converting the data to be projected into a format matching the second codec protocol to obtain target projection data; and displaying the target projection data through the first device.

[0005] In the second aspect, an embodiment of the present application provides a screen projection device, which includes: an acquisition module, configured to acquire the data to be projected; a determination module, configured to determine the first codec protocol of the data to be projected and the second codec protocol supported by the first device; a conversion module, configured to convert the data to be projected into a format matching the second codec protocol based on the first codec protocol and the second codec protocol to obtain target projection data; a sending module, configured to display the target projection data through the first device.

[0006] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it obtains data to be projected; determines a first codec protocol for the data to be projected and a second codec protocol supported by a first device; based on the first codec protocol and the second codec protocol, converts the data to be projected into a format that matches the second codec protocol to obtain target projection data; and displays the target projection data through the first device.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, obtains data to be projected; determines a first codec protocol for the data to be projected and a second codec protocol supported by a first device; based on the first codec protocol and the second codec protocol, converts the data to be projected into a format matching the second codec protocol to obtain target projection data; and displays the target projection data through the first device.

[0008] In the fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, realizes obtaining the data to be projected; determining a first codec protocol of the data to be projected and a second codec protocol supported by a first device; based on the first codec protocol and the second codec protocol, converting the data to be projected into a format matching the second codec protocol to obtain target projection data; and displaying the target projection data through the first device.

[0009] In the seventh aspect, an embodiment of the present application provides a computer program, which enables a processor to execute the acquisition of data to be projected; determine a first codec protocol for the data to be projected and a second codec protocol supported by a first device; based on the first codec protocol and the second codec protocol, convert the data to be projected into a format that matches the second codec protocol to obtain target projection data; and display the target projection data through the first device.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, serve to illustrate the technical solutions of the present application. Obviously, the drawings described below are merely some embodiments of the present application. Those skilled in the art can, without inventive effort, derive other drawings from these drawings.

[0012] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0013] Figure 1 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 1 ;

[0014] Figure 2 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 2 ;

[0015] Figure 3 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 3 ;

[0016] Figure 4 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 4 ;

[0017] Figure 5 A schematic diagram of a screen projection source and a receiving end provided in an embodiment of the present application;

[0018] Figure 6 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 5 ;

[0019] Figure 7 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 6 ;

[0020] Figure 8 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 7 ;

[0021] Figure 9 A schematic structural diagram of a screen projection device provided in an embodiment of the present application;

[0022] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions of this application are further elaborated in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the embodiments of the present application belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0026] The descriptions such as "first, second, third" appearing in the embodiments of this application are only for illustration and distinction of the described objects. There is no order, nor does it indicate any special limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application.

[0027] Before further describing the embodiments of the present application in detail, the nouns and terms that may be involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0028] (1) Digital Living Network Alliance (DLNA) Screen Casting Protocol

[0029] The DLNA screen mirroring protocol is a standards-based protocol designed to enable multimedia content sharing and screen mirroring between devices within a home network (such as TVs, mobile phones, computers, game consoles, etc.). It is built on a universal network protocol and allows compatible devices (different devices that support DLNA) to automatically discover and transfer audio, video, pictures, and other content.

[0030] The DLNA screen mirroring protocol is used for media sharing, allowing you to push videos, audio, and images from devices like phones and computers to larger screens or playback devices like TVs and speakers. It also supports interconnecting devices from different brands (subject to DLNA certification). The DLNA screen mirroring protocol allows data to be transmitted over a local network, either over a home wireless Wi-Fi network or a wired network.

[0031] (2) Miracast screen projection protocol

[0032] Miracast is a wireless screen mirroring protocol based on Wi-Fi Direct, developed by the Wi-Fi Alliance. It allows devices (such as phones, computers, and tablets) to mirror their screens directly to display devices like TVs and projectors in real time, without relying on a router or internet connection. Real-time screen mirroring can be understood as projecting a phone or computer screen 1:1 onto a larger screen, supporting videos, games, presentations (PowerPoint, PPT), and more. The DLNA screen mirroring protocol, on the other hand, only supports file push.

[0033] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and all of them fall within the scope of protection of the embodiments of the present application.

[0034] If the receiving end of the screen projection (TV, set-top box or projector, etc.) does not support the DLNA screen projection protocol and only supports the Miracast screen projection protocol. Then, the DLNA screen projection protocol of the online video application cannot be used. In a related technology, software that supports DLNA Media Render is installed on the receiving end (Sink end), and the DLNA screen projection protocol is supported through software extension. However, most Sink ends do not support this kind of software after installation. Alternatively, use screen mirroring to play the video on the Android device, and then project the screen content to the Sink end through the Miracast screen projection protocol. However, this will lead to increased power consumption and decreased video quality.

[0035] If the receiving end only supports the DLNA protocol and does not support Miracast, then screen mirroring cannot be achieved. In another related technology, such as demonstrating screen operations or PPT screen projection, DLNA's Media Player generally exists in video applications, and these applications are generally used for video streaming and do not support screen encoding. However, if the screen is recorded and then the video is projected via DLNA, real-time screen projection cannot be achieved.

[0036] In view of this, an embodiment of the present application provides a screen projection method. Figure 1 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes the following steps 101 to 104:

[0037] Step 101, obtaining the data to be projected;

[0038] Step 102: Determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol;

[0039] Step 103: Based on the first codec protocol and the second codec protocol, convert the to-be-projected screen data into a format that matches the second codec protocol to obtain target projection screen data.

[0040] Step 104: display the target projection data through the first device.

[0041] It can be understood that in the embodiment of the present application, when the projection protocols supported by the sending end (i.e., source end, also known as Source end) of the projection screen are inconsistent with those supported by the receiving end (i.e., Sink end) of the projection screen, that is, when the first codec protocol of the screen data to be projected is inconsistent with the second codec protocol supported by the first device, the screen data to be projected is converted into a format that matches the second codec protocol supported by the first device to obtain the target screen data; thereby enabling the first device that supports the second codec protocol to display the screen data to be projected obtained based on the first codec protocol. In this way, compatibility between the source device of the projection screen and the receiving device of the projection screen is achieved.

[0042] The following describes further optional implementations and related terms of each of the above steps.

[0043] In step 101, the data to be projected is obtained.

[0044] It should be understood that in the embodiment of the present application, there is no limitation on the method of obtaining the data to be projected, and the method of obtaining the data to be projected can be determined based on the source device of the data to be projected and the receiving device of the data to be projected.

[0045] In the embodiments of the present application, the data to be projected is not limited; the data to be projected may be different depending on the sending end of the data to be projected. In some embodiments, the data to be projected includes: video data in an online video application. In other embodiments, the data to be projected includes: non-video data displayed by the second device, such as: PPT, etc.

[0046] In some embodiments, the data to be projected includes: video data in an online video application. Accordingly, in some embodiments, obtaining the data to be projected includes: receiving the data to be projected sent by a video application. In other embodiments, obtaining the data to be projected includes: receiving the data to be projected sent by a second device; wherein the data to be projected is video data encoded based on a first encoding protocol, and the first device and the second device are located in the same local area network.

[0047] It should be understood that in the embodiments of this application, the online video application is not limited. In some embodiments, the online video application refers to software or platforms that provide video playback services over the Internet, and users can access its content library through mobile phones, tablets, computers, and other devices to watch movies, TV series, variety shows, short videos, etc.

[0048] In some embodiments, the screen data to be projected includes non-video data displayed by the second device, such as a PPT. Accordingly, in some embodiments, obtaining the screen data to be projected includes: obtaining the display content of at least one window currently displayed on the third device; and synthesizing the display content to obtain the screen data to be projected.

[0049] In some embodiments, the display content of the at least one window can be understood as a Surface. Surface is for all user interface (UI) applications in the Android system and fulfills the UI display requirements of the application process. Surface is a concept in the Windows operating system that is a window area that can be used to display application content.

[0050] In step 102, a first codec protocol of the data to be projected and a second codec protocol supported by the first device are determined; wherein the first codec protocol is different from the second codec protocol.

[0051] It should be understood that in the embodiments of the present application, the first codec protocol and the second codec protocol are not limited. In some embodiments, the first codec protocol is the DLNA screen projection protocol, and the second codec protocol is the Miracast screen projection protocol. In other embodiments, the first codec protocol is the Miracast screen projection protocol, and the second codec protocol is the DLNA screen projection protocol.

[0052] In the embodiments of the present application, the "first codec protocol of the data to be projected" is not limited. In some embodiments, the first codec protocol of the data to be projected may be the first codec protocol used by the data to be projected. In other embodiments, the first codec protocol of the data to be projected may be the first codec protocol to be used by the data to be projected.

[0053] In step 103, based on the first coding protocol and the second coding protocol, the data to be projected is converted into a format that matches the second coding protocol to obtain target projection data.

[0054] It should be understood that in the embodiments of the present application, the target projection data is not limited, and the target projection data supports the second codec protocol of the first device. In some embodiments, the data to be projected includes: video data in an online video application; accordingly, the target projection data includes: video data that supports the second codec protocol. In other embodiments, the data to be projected includes: non-video data displayed by the second device; accordingly, the target projection data includes: non-video data that supports the second codec protocol.

[0055] In some embodiments, based on the first codec protocol and the second codec protocol, the data to be projected is converted into a format that matches the second codec protocol to obtain target projection data, including: based on the first codec protocol, the data to be projected is demultiplexed to obtain the original code stream of the data to be projected; based on the second codec protocol, the original code stream of the data to be projected is converted into a format that matches the second codec protocol to obtain target projection data.

[0056] It should be understood that in the embodiments of the present application, the demultiplexing is not limited. In some embodiments, the demultiplexing (i.e., Demux) is to extract independent audio and video data streams from the container file in the encapsulation format. The container file is essentially a "container" for multimedia data, which integrates the video stream, audio stream, subtitle stream and metadata (such as resolution and frame rate) into a single file through encapsulation technology. Demultiplexing is to separate these compressed and encoded original data streams.

[0057] In some embodiments, based on the second codec protocol, the original code stream of the data to be projected is converted into a format that matches the second codec protocol to obtain target projection data, including: based on the encapsulation format of the second codec protocol, the original code stream is encapsulated to obtain the encapsulated data of the original code stream; based on the transmission format of the second codec protocol, the encapsulated data of the original code stream is packaged to obtain the target projection data; wherein, the target projection data meets the transmission format of the second codec protocol.

[0058] It should be understood that in the embodiments of the present application, the encapsulation is not limited. In some embodiments, the encapsulation refers to combining the encoded audio data, video data, subtitles, metadata, etc. in a standard format to form a file or stream that can be transmitted or stored (i.e., encapsulated data).

[0059] In the embodiments of the present application, the packaging is also not limited. In some embodiments, the packaging refers to dividing the data into units suitable for transmission (such as network packets, storage blocks), and adding a protocol header.

[0060] In some embodiments, when the first codec protocol is the DLNA screen projection protocol and the second codec protocol is the Miracast screen projection protocol, the data to be projected is demultiplexed based on the DLNA screen projection protocol to obtain the original code stream of the data to be projected; based on the encapsulation format of the Miracast screen projection protocol, the original code stream is encapsulated to obtain the encapsulated data of the original code stream; based on the transmission format of the Miracast screen projection protocol, the encapsulated data of the original code stream is packaged to obtain the target screen projection data; wherein, the target screen projection data meets the transmission format of the second codec protocol.

[0061] For example, in one possible implementation, the Sink supports the Miracast screen projection protocol, and the Source supports the DLNA screen projection protocol. For example, the Source uses the video data of the online video application as the data to be projected. By implementing DLNA MediaRender locally, the video uniform resource identifier (Video Uniform Resource Identifier, Video URI) passed in by the online video application (ie Video App) is used as the source of HttpSource, and after protocol conversion, the screen projection of the Sink end that supports the Miracast screen projection protocol is realized. In other words, the local protocol is used to convert the code stream that supports the DLNA screen projection protocol into the code stream that supports the Miracast screen projection protocol.

[0062] You can achieve screen projection by following steps 11 to 14:

[0063] Step 11: Use the Video App to cast the screen to the TV, search for DLNA MediaRender, and select the local MediaRender.

[0064] Step 12: Create a Miracast Session. Parse the video data's resolution, frame rate, encoding format, and other information from the HttpSource. Based on this information, negotiate Wi-Fi Display Session (WFD) session parameters with a sink that supports the Miracast protocol.

[0065] In step 13, the original stream demultiplexed (demuxed) by HttpSource is passed to TS Packetizer and repackaged into the transport (TS) format (i.e., the encapsulation format specified by the Miracast screen projection protocol). The transport stream (TS stream) is sent to the Sink end via Wi-Fi point-to-point communication as the RTP payload (i.e., the actual audio and video data carried in the RTP data packet, i.e., the transmission format specified by the Miracast screen projection protocol) of the RTPSource.

[0066] The stream received by HttpSource is encoded using the DLNA projection protocol, resulting in an HTTP stream. When implementing a TS stream as an RTP payload, the TS needs to be packaged into an RTP packet. This includes: reading the TS stream; creating an RTP header for each TS stream; adding the TS stream as an RTP payload; and sending the RTP packet.

[0067] Step 14: End the TV screen projection and destroy the Miracast Session.

[0068] In some embodiments, based on the first codec protocol and the second codec protocol, the data to be projected is converted into a format that matches the second codec protocol to obtain target projection data, including: encoding the data to be projected based on the codec format of the first codec protocol to obtain compressed data of the data to be projected; based on the transmission format of the second codec protocol, packaging the compressed data of the data to be projected to obtain the target projection data; wherein, the target projection data meets the transmission format of the second codec protocol.

[0069] In some embodiments, when the first codec protocol is the Miracast screen projection protocol and the second codec protocol is the DLNA screen projection protocol, the data to be projected is encoded based on the codec format of the Miracast screen projection protocol to obtain compressed data of the data to be projected; based on the transmission format of the DLNA screen projection protocol, the compressed data of the data to be projected is packaged to obtain the target screen projection data; wherein, the target screen projection data satisfies the transmission format of the second codec protocol.

[0070] For example, in one possible implementation, the Sink supports the DLNA projection protocol. However, when screen operations or PPT projection need to be displayed, the media is packaged into a Hypertext Transfer Protocol stream (HTTP stream) by reusing the Miracast projection protocol.

[0071] You can achieve screen projection by following steps 21 to 24:

[0072] Step 21. Go to system settings and select "Screen DLNA projection protocol playback".

[0073] Step 22: Search for DLNA MediaRender and select the DLNA device to which you want to cast the screen.

[0074] Step 23, reuse the protocol stack in the Miracast screen projection protocol to encode the screen source (PPT or the screen to be demonstrated). If sound is required, encode the pulse code modulation stream (Pulse-Code Modulation Stream, PCM stream) of the audio source (i.e. Audio Source). Package the encoded stream into an HTTP stream (i.e. the transport stream format of the DLNA screen projection protocol). Send it to the DLNA MediaRender (i.e. the receiving end) via Wi-Fi.

[0075] Step 24: End the screen DLNA playback and destroy the DLNA MediaPlayer.

[0076] In some embodiments, based on the first coding protocol and the second coding protocol, the data to be projected is converted into a format that matches the second coding protocol to obtain target projection data, including: based on the first coding protocol and the second coding protocol, determining the conversion method of the data to be projected; wherein, different first coding protocols and second coding protocols correspond to different conversion methods of the data to be projected; based on the conversion method of the data to be projected, the first coding protocol and the second coding protocol, the data to be projected is converted into a format that matches the second coding protocol to obtain target projection data.

[0077] For example, in one possible implementation, other devices within the domain network can perform protocol conversion through CastBridge, so that when the projection protocols supported by the projection sending end (i.e., the source end, also known as the Source end) and the projection receiving end (i.e., the Sink end) are inconsistent, the projection receiving end can display the projection data sent by the projection sending end. For example: the source end is an online video application on an Android phone (i.e., the source end supports the DLNA projection protocol), and the Sink end is a device that only supports the Miracast projection protocol. At this time, the Android phone can search for the DLNAMediaRender provided by CastBridge and start DLNA projection. CastBridge converts the received code stream into the RTP stream corresponding to the Miracast projection protocol and displays it on the Sink end that supports the Miracast projection protocol.

[0078] In step 104, the target projection data is displayed through the first device.

[0079] The embodiment of the present application provides a screen projection method. Figure 2 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 2 ,like Figure 2 As shown, the method includes the following steps 201 to 206:

[0080] Step 201: receiving the screen projection data sent by the video application;

[0081] Step 202: Determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol;

[0082] Step 203: Demultiplexing the data to be projected based on the first encoding and decoding protocol to obtain an original code stream of the data to be projected;

[0083] Step 204: encapsulate the original bitstream based on the encapsulation format of the second codec protocol to obtain encapsulated data of the original bitstream;

[0084] Step 205: Pack the encapsulated data of the original code stream based on the transmission format of the second codec protocol to obtain the target screen projection data; wherein the target screen projection data meets the transmission format of the second codec protocol;

[0085] Step 206: Display the target projection data through the first device.

[0086] It can be understood that in the screen projection method provided in the embodiment of the present application, when the first codec protocol is the DLNA screen projection protocol and the second codec protocol is the Miracast screen projection protocol, the video data of the online video application is converted into a format that matches the Miracast screen projection protocol supported by the first device to obtain the target screen projection data; thereby enabling the first device that supports the Miracast screen projection protocol to display the screen projection data obtained based on the DLNA screen projection protocol. In this way, compatibility between the source device of the screen projection and the receiving device of the screen projection is achieved.

[0087] The embodiment of the present application provides a screen projection method. Figure 3 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 3 ,like Figure 3 As shown, the method includes the following steps 301 to 306:

[0088] Step 301: Obtain display content of at least one window currently displayed on a third device;

[0089] Step 302: synthesize the display content to obtain the screen projection data;

[0090] Step 303: Determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol;

[0091] Step 304: Encode the data to be projected based on the codec format of the first codec protocol to obtain compressed data of the data to be projected;

[0092] Step 305: Packing the compressed data of the screen projection data based on the transmission format of the second codec protocol to obtain the target screen projection data; wherein the target screen projection data meets the transmission format of the second codec protocol;

[0093] Step 306: display the target projection data through the first device.

[0094] It can be understood that in the screen projection method provided in the embodiment of the present application, when the first codec protocol is the Miracast screen projection protocol and the second codec protocol is the DLNA screen projection protocol, the non-video data is converted into a format that matches the DLNA screen projection protocol supported by the first device to obtain the target screen projection data; thereby enabling the first device that supports the DLNA screen projection protocol to display the screen projection data obtained based on the Miracast screen projection protocol. In this way, compatibility between the source device of the screen projection and the receiving device of the screen projection is achieved.

[0095] The embodiment of the present application provides a screen projection method. Figure 4 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 4 ,like Figure 4 As shown, the method includes the following steps 401 to 405:

[0096] Step 401: Receive the screen data to be projected sent by the second device; wherein the screen data to be projected is video data encoded based on a first encoding protocol, and the first device and the second device are located in the same local area network;

[0097] Step 402: Determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol;

[0098] Step 403: Determine a conversion method for the data to be projected based on the first coding protocol and the second coding protocol; wherein different first coding protocols and second coding protocols correspond to different conversion methods for the data to be projected;

[0099] Step 404: Based on the conversion mode of the screen projection data, the first codec protocol, and the second codec protocol, convert the screen projection data into a format that matches the second codec protocol to obtain target projection data.

[0100] Step 405: Display the target projection data through the first device.

[0101] It can be understood that in the screen projection method provided in the embodiment of the present application, when the screen projection protocols supported by the sending end (i.e., the source end, also known as the Source end) of the screen projection and the receiving end (i.e., the Sink end) of the screen projection are inconsistent, that is, when the first codec protocol of the screen projection data is inconsistent with the second codec protocol supported by the first device, the conversion method of the screen projection data is determined based on the first codec protocol and the second codec protocol; and the conversion methods of the screen projection data corresponding to different first coding protocols and second codec protocols are different; based on the conversion method of the screen projection data, the first codec protocol and the second codec protocol, the screen projection data is converted into a format that matches the second codec protocol supported by the first device to obtain the target screen projection data; thereby, the first device that supports the second codec protocol can display the screen projection data obtained based on the first codec protocol. In this way, compatibility between the source device of the screen projection and the receiving device of the screen projection is achieved, and adaptation to different application scenarios is achieved.

[0102] The following describes an exemplary application of the screen projection method provided in an embodiment of the present application in a practical scenario.

[0103] Figure 5 A schematic diagram of a screen projection source and a receiving end provided in an embodiment of the present application is shown as follows: Figure 5 As shown, device 501 represents an Apple device, device 502 represents an Android device, device 503 represents a computer with a Windows operating system, device 504 represents an online video application, device 505 represents a high-definition TV, device 506 represents a projector, device 507 represents a car computer, and device 508 represents a High Definition Multimedia Interface (HDMI); device 501 supports the AirPlay screen projection protocol, devices 502 and 503 support the Miracast screen projection protocol, and device 504 supports the DLNA screen projection protocol.

[0104] Figure 6 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 5 ,like Figure 6 As shown, VideoApp sends video data to HttpSource in DLNAMediaRender; HttpSource demultiplexes the video data and passes the demultiplexed raw stream to the extractor (i.e., Extractor); Extractor extracts the video stream (i.e., Video) and audio stream (i.e., Audio) and sends them to TS Packetizer; TS Packetizer repackages them into the transport (TS) format (i.e., the encapsulation format specified by the Miracast screen projection protocol). The transport stream (TransportStream, TS stream) is sent to the Sink end as the RTP payload of RTPSource via Wi-Fi point-to-point (i.e., Wi-Fi P2P) communication.

[0105] In some embodiments, the Sink supports the Miracast screen projection protocol, and the Source supports the DLNA screen projection protocol. For example, the Source uses the video data of the online video application as the data to be projected. By implementing DLNAMediaRender locally, the video uniform resource identifier (Video Uniform Resource Identifier, Video URI) passed in by the online video application (i.e., Video App) is used as the source of HttpSource, and after protocol conversion, the screen projection of the Sink end that supports the Miracast screen projection protocol is realized. In other words, the local protocol is used to convert the code stream that supports the DLNA screen projection protocol into the code stream that supports the Miracast screen projection protocol.

[0106] You can achieve screen projection by following steps 11 to 14:

[0107] Step 11: Use the Video App to cast the screen to the TV, search for DLNAMediaRender, and select the local MediaRender.

[0108] Step 12: Create a Miracast Session. Parse the video data's resolution, frame rate, encoding format, and other information from the HttpSource. Based on this information, negotiate Wi-Fi Display Session (WFD) session parameters with a sink that supports the Miracast protocol.

[0109] In step 13, the original stream demultiplexed (demuxed) by HttpSource is passed to TS Packetizer and repackaged into the transport (TS) format (i.e., the encapsulation format specified by the Miracast screen projection protocol). The transport stream (TS stream) is sent to the Sink end via Wi-Fi point-to-point communication as the RTP payload (i.e., the actual audio and video data carried in the RTP data packet, i.e., the transmission format specified by the Miracast screen projection protocol) of the RTPSource.

[0110] The stream received by HttpSource is encoded using the DLNA projection protocol, resulting in an HTTP stream. When implementing a TS stream as an RTP payload, the TS needs to be packaged into an RTP packet. This includes: reading the TS stream; creating an RTP header for each TS stream; adding the TS stream as an RTP payload; and sending the RTP packet.

[0111] Step 14: End the TV screen projection and destroy the Miracast Session.

[0112] Figure 7 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 6 ,like Figure 7 As shown, the screen source (Source) is sent to SurfaceComposer for synthesis, and the synthesized data is sent to Video Encoder for video encoding. If sound is required, the audio source (Audio Source) is sent to Audio Encoder for encoding. The encoded stream is sent to Http Sink, packaged into an HTTP stream (the transport stream format of the DLNA screen projection protocol), and sent to DLNAMediaRender via Wi-Fi.

[0113] In some embodiments, the Sink supports the DLNA projection protocol. However, when screen operations or PPT projection need to be displayed, the media is packaged into a Hypertext Transfer Protocol stream (HTTP stream) by reusing the Miracast projection protocol.

[0114] You can achieve screen projection by following steps 21 to 24:

[0115] Step 21. Go to system settings and select "Screen DLNA projection protocol playback".

[0116] Step 22: Search for DLNAMediaRender and select the DLNA device to which you want to cast the screen.

[0117] Step 23, reuse the protocol stack in the Miracast screen projection protocol to encode the screen source (PPT or the screen to be demonstrated). If sound is required, encode the pulse code modulation stream (Pulse-Code Modulation Stream, PCM stream) of the audio source (i.e. Audio Source). Package the encoded stream into an HTTP stream (i.e. the transport stream format of the DLNA screen projection protocol). Send it to the DLNA MediaRender (i.e. the receiving end) via Wi-Fi.

[0118] Step 24: End the screen DLNA playback and destroy the DLNA MediaPlayer.

[0119] Figure 8 Schematic diagram of the implementation process of a screen projection method provided in an embodiment of the present application Figure 7 ,like Figure 8 As shown, device 501 represents an Apple device, device 502 represents an Android device, device 505 represents a high-definition TV, device 506 represents a projector, device 801 represents CastBridge, 802 represents an online video application, and 803 represents a protocol conversion operation. Device 801 can determine the conversion method of the data to be projected based on the first codec protocol and the second codec protocol, and different first coding protocols and second codec protocols correspond to different conversion methods of the data to be projected.

[0120] In some embodiments, other devices within the domain network can perform protocol conversion through CastBridge, so that when the projection protocols supported by the projection sending end (i.e., the source end, also known as the Source end) and the projection receiving end (i.e., the Sink end) are inconsistent, the projection receiving end can display the projection data sent by the projection sending end. For example: the source end is an online video application on an Android phone (i.e., the source end supports the DLNA projection protocol), and the Sink end is a device that only supports the Miracast projection protocol. At this time, the Android phone can search for the DLNA MediaRender provided by CastBridge and start DLNA projection. CastBridge converts the received code stream into the RTP stream corresponding to the Miracast projection protocol and displays it on the Sink end that supports the Miracast projection protocol.

[0121] It can be understood that in the embodiments of the present application, different screen projection receiving ends can be adapted through protocol conversion. That is to say. There is no need to modify the Sink end or purchase other Sink ends. By supporting different Sink ends through local protocol conversion, different application scenarios can be adapted. By reusing the Miracast screen projection protocol and the DLNA screen projection protocol, only a small amount of modification is required to adapt to different protocols. In addition, protocol conversion will not cause quality loss, and the power consumption overhead of reuse is small.

[0122] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps; or steps in different embodiments may be combined to form a new technical solution.

[0123] Based on the foregoing embodiments, an embodiment of the present application provides a screen projection device, which includes the modules included and the units included in each module, and can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an AI acceleration engine (such as NPU, etc.), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0124] Figure 9 A schematic diagram of the structure of a screen projection device provided in an embodiment of the present application is shown in FIG. Figure 9 As shown, the screen projection device 90 includes an acquisition module 901, a determination module 902, a conversion module 903 and a display module 904, wherein:

[0125] Acquisition module 901, configured to acquire data to be projected;

[0126] Determining module 902, configured to determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol;

[0127] The conversion module 903 is configured to convert the to-be-projected screen data into a format that matches the second codec protocol based on the first codec protocol and the second codec protocol, thereby obtaining target projection screen data;

[0128] The display module 904 is configured to display the target projection data through the first device.

[0129] In some embodiments, the conversion module 903 is configured to demultiplex the data to be projected based on the first codec protocol to obtain the original code stream of the data to be projected; based on the second codec protocol, convert the original code stream of the data to be projected into a format matching the second codec protocol to obtain the target projection data.

[0130] In some embodiments, the conversion module 903 is configured to encapsulate the original code stream based on the encapsulation format of the second codec protocol to obtain the encapsulation data of the original code stream; based on the transmission format of the second codec protocol, the encapsulation data of the original code stream is packaged to obtain the target screen projection data; wherein, the target screen projection data satisfies the transmission format of the second codec protocol.

[0131] In some embodiments, the acquisition module 901 is configured to receive the data to be projected sent by the video application; or, to receive the data to be projected sent by the second device; wherein, the data to be projected is video data encoded based on a first encoding protocol, and the first device and the second device are located in the same local area network.

[0132] In some embodiments, the conversion module 903 is configured to determine the conversion method of the data to be projected based on the first coding protocol and the second coding protocol; wherein different first coding protocols and second coding protocols correspond to different conversion methods of the data to be projected; based on the conversion method of the data to be projected, the first coding protocol and the second coding protocol, the data to be projected is converted into a format matching the second coding protocol to obtain target projection data.

[0133] In some embodiments, the acquisition module 901 is configured to acquire the display content of at least one window currently displayed by the third device; synthesize the display content to obtain the data to be projected.

[0134] In some embodiments, the conversion module 903 is configured to encode the data to be projected based on the codec format of the first codec protocol to obtain compressed data of the data to be projected; and to package the compressed data of the data to be projected based on the transmission format of the second codec protocol to obtain the target projection data; wherein the target projection data satisfies the transmission format of the second codec protocol.

[0135] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0136] It should be noted that the division of modules in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units. They may also be implemented in the form of a combination of software and hardware.

[0137] It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0138] The embodiment of the present application provides a first controller, Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 10 As shown, the electronic device 100 includes a memory 1001 and a processor 1002. The memory 1001 stores a computer program that can be run on the processor 1002. When the processor 1002 executes the program, the steps in the method provided in the above embodiment are implemented.

[0139] It should be noted that the memory 1001 is configured to store instructions and applications executable by the processor 1002, and can also cache data to be processed or processed by the processor 1002 and various modules in the electronic device 100, which can be implemented through flash memory (Flash) or random access memory (Random Access Memory, RAM).

[0140] In some embodiments, the electronic device is the source of the data to be projected. In other embodiments, the electronic device is a CastBridge.

[0141] It should be understood that when the electronic device is CastBridge, the source end of the screen projection, the receiving end of the screen projection and CastBridge are located in the same local area network.

[0142] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method provided in the above embodiment are implemented.

[0143] An embodiment of the present application provides a computer program product comprising instructions, which, when executed on a processor, enables the processor to execute the steps of the method provided in the above method embodiment.

[0144] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0145] It should be understood that the "one embodiment" or "an embodiment" or "some embodiments" or "in one possible implementation" or "example" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" or "some embodiments" or "in a possible implementation" or "example" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments. The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0146] The term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, object A and / or object B can mean: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0147] It should be noted that, in this article, the terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In the several embodiments provided in this application, it should be understood that the disclosed equipment and devices, etc., can be implemented in other ways. The embodiments described above are merely illustrative. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined to obtain new product embodiments without conflict.

[0148] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A screen projection method, comprising: Get the data to be projected; Determining a first codec protocol for the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol; Based on the first coding and decoding protocols and the second coding and decoding protocols, the screen projection data is converted into a format that matches the second coding and decoding protocol to obtain target screen projection data; The target screen projection data is displayed through the first device.

2. The method according to claim 1, wherein, based on the first codec protocol and the second codec protocol, converting the to-be-projected screen data into a format that matches the second codec protocol to obtain target projection data comprises: Demultiplexing the data to be projected based on the first encoding and decoding protocol to obtain an original code stream of the data to be projected; Based on the second encoding and decoding protocol, the original code stream of the data to be projected is converted into a format that matches the second encoding and decoding protocol to obtain the target projection data.

3. The method according to claim 2, wherein, based on the second codec protocol, converting the original code stream of the screen projection data into a format that matches the second codec protocol to obtain the target screen projection data comprises: Encapsulating the original bitstream based on the encapsulation format of the second codec protocol to obtain encapsulated data of the original bitstream; Based on the transmission format of the second codec protocol, the encapsulated data of the original code stream is packaged to obtain the target screen projection data; wherein, the target screen projection data meets the transmission format of the second codec protocol.

4. The method according to claim 1, wherein obtaining the data to be projected comprises: Receiving the screen projection data sent by the video application; or, Receiving the screen projection data sent by the second device; The data to be projected is video data that has been encoded based on a first encoding protocol, and the first device and the second device are located in the same local area network.

5. The method according to claim 4, wherein the converting the to-be-projected screen data into a format matching the second codec protocol based on the first codec protocol and the second codec protocol to obtain target projection screen data comprises: Determine, based on the first encoding and decoding protocol and the second encoding and decoding protocol, a conversion method of the data to be projected; wherein different first encoding protocols and second encoding and decoding protocols correspond to different conversion methods of the data to be projected; Based on the conversion method of the data to be projected, the first encoding and decoding protocol and the second encoding and decoding protocol, the data to be projected is converted into a format that matches the second encoding and decoding protocol to obtain the target projection data.

6. The method according to claim 1, wherein obtaining the data to be projected comprises: Obtaining display content of at least one window currently displayed on the third device; The display content is synthesized to obtain the data to be projected.

7. The method according to claim 6, wherein the step of converting the screen projection data into a format that matches the second codec protocol based on the first codec protocol and the second codec protocol to obtain target screen projection data comprises: Encoding the data to be projected based on the encoding and decoding format of the first encoding and decoding protocol to obtain compressed data of the data to be projected; Based on the transmission format of the second codec protocol, the compressed data of the data to be projected is packaged to obtain the target projection data; wherein, the target projection data meets the transmission format of the second codec protocol.

8. A screen projection device, comprising: An acquisition module configured to acquire data to be projected; a determination module configured to determine a first codec protocol of the screen projection data and a second codec protocol supported by the first device; wherein the first codec protocol is different from the second codec protocol; a conversion module configured to convert the to-be-projected screen data into a format matching the second codec protocol based on the first codec protocol and the second codec protocol, thereby obtaining target projected screen data; A display module is configured to display the target projection data through the first device.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor obtains data to be projected when executing the program; Determine a first codec protocol for the data to be projected and a second codec protocol supported by the first device; Based on the first coding and decoding protocols and the second coding and decoding protocols, the screen projection data is converted into a format that matches the second coding and decoding protocol to obtain target screen projection data; The target screen projection data is displayed through the first device.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables acquisition of data to be projected; Determine a first codec protocol for the data to be projected and a second codec protocol supported by the first device; Based on the first coding and decoding protocols and the second coding and decoding protocols, the screen projection data is converted into a format that matches the second coding and decoding protocol to obtain target screen projection data; The target screen projection data is displayed through the first device.

Images