Screen projection method, device, system, computer device and storage medium

Abstract

The application relates to a screen projection method, device, system, computer equipment and storage medium. The method comprises the following steps: a playing end receives a sending end signaling from a sending end; the sending end signaling is a WebRTC signaling transmitted by the sending end through a UPnP protocol; the playing end generates a playing end signaling based on the sending end signaling, and sends the playing end signaling to the sending end through the UPnP protocol, so that the sending end and the playing end establish a WebRTC communication connection; and the playing end receives and displays real-time audio stream or mirror data from the sending end based on the WebRTC communication connection. The method can use the UPnP protocol as a signaling exchange channel of the WebRTC, realizes support and expansion of the WebRTC screen projection on the basis of the current mainstream DLNA screen projection function, does not increase hardware cost, does not reduce the popularity, and realizes mirror screen projection and real-time audio and video stream screen projection functions.

Description

Technical Field

[0001] This application relates to the field of near-field communication technology, and in particular to a screen projection method, device, system, computer equipment, and storage medium. Background Technology

[0002] The Digital Living Network Alliance (DLAN) is dedicated to building home media sharing solutions through the integration of various technologies. It mainly relies on the UPnP (Universal Plug and Play) protocol, which aims to enable interconnection between devices through a network.

[0003] However, although DLNA has been widely supported in media sharing between devices, and DLNA screen mirroring technology covers almost all media sharing between smart TVs and set-top boxes, the actual application of DLNA is still limited to the casting of videos and pictures, without involving more types of device interoperability and functional expansion, and lacking capabilities such as mobile phone screen mirroring and real-time audio and video. Summary of the Invention

[0004] Therefore, it is necessary to provide a screen mirroring method, device, system, computer equipment, and storage medium that can realize screen mirroring and real-time audio and video functions to address the above-mentioned technical problems.

[0005] Firstly, this application provides a screen mirroring method. The method includes:

[0006] Receive delivery terminal signaling from the delivery terminal; the delivery terminal signaling is WebRTC signaling transmitted by the delivery terminal via the UPnP protocol;

[0007] Based on the sending end signaling, a playback end signaling is generated, and the playback end signaling is sent to the sending end via the UPnP protocol, so that the sending end and the playback end establish a WebRTC communication connection;

[0008] Based on the WebRTC communication connection, the system receives and displays real-time audio streams or mirror data from the sending end.

[0009] In one embodiment, sending the playback terminal signaling to the delivery terminal via the UPnP protocol includes:

[0010] The playback terminal signaling is sent to the delivery terminal via the UPnP protocol through the playback terminal's configured UPnP non-standard service;

[0011] The non-standard UPnP service on the playback end is configured through predefined service elements while meeting the UPnP protocol rules; the predefined service elements include service function elements, service type elements, and service identification elements.

[0012] In one embodiment, the playback-side UPnP non-standard service is configured through the following process:

[0013] Based on the UPnP protocol rules, define the service elements that support WebRTC signaling exchange;

[0014] Compile a service description document based on the aforementioned service elements;

[0015] Based on the standard UPnP service framework, the non-standard UPnP service for the playback terminal is implemented by combining the service elements and the service description file.

[0016] In one embodiment, before receiving the delivery end signaling from the delivery end, the process includes:

[0017] In response to the device search request sent by the casting terminal via the UPnP protocol, a description of the playback terminal device is sent to the casting terminal.

[0018] If the playback device description meets the WebRTC detection conditions, an initial connection is established with the delivery device.

[0019] In one embodiment, generating playback signaling based on the sending end signaling includes:

[0020] The session description information and interactive connection candidate information of the sending end are obtained from the sending end signaling.

[0021] The signaling of the playback terminal is determined based on the session description information of the sending terminal and the interactive connection candidate information of the sending terminal.

[0022] Secondly, this application also provides a screen projection device. The device includes:

[0023] The receiving module is used to receive signaling from the sending end; the signaling is WebRTC signaling transmitted by the sending end via the UPnP protocol.

[0024] The generation module is used to generate playback signaling based on the delivery end signaling, and send the playback end signaling to the delivery end via the UPnP protocol, so that the delivery end and the playback end can establish a WebRTC communication connection.

[0025] The display module is used to receive and display real-time audio streams or mirror data from the sending end based on the WebRTC communication connection.

[0026] Thirdly, this application also provides a screen mirroring system. The screen mirroring system includes:

[0027] A screen projection system, characterized in that the system includes a webRTC signaling exchange module and a UPnP non-standard service module configured on the projection end, and the system also includes a webRTC signaling exchange module and a UPnP non-standard service module configured on the playback end.

[0028] The WebRTC signaling exchange module at the delivery end is used to generate delivery end signaling;

[0029] The delivery end UPnP non-standard service module is connected to the delivery end WebRTC signaling exchange module and the playback end UPnP non-standard service module respectively, and is used to receive the delivery end signaling from the delivery end WebRTC signaling exchange module, and send the delivery end signaling to the playback end UPnP non-standard service module through the UPnP protocol.

[0030] The playback end UPnP non-standard service module is connected to the playback end WebRTC signaling exchange module, and is used to receive the delivery end signaling from the delivery end, and transmit the delivery end signaling to the playback end WebRTC signaling exchange module.

[0031] The playback-end WebRTC signaling exchange module is used to generate playback-end signaling based on the sending-end signaling, and transmit the playback-end signaling to the playback-end UPnP non-standard service module.

[0032] The playback terminal UPnP non-standard service module is also used to send the playback terminal signaling to the delivery terminal UPnP non-standard service module via the UPnP protocol.

[0033] The UPnP non-standard service module of the delivery end is also used to receive the playback end signaling from the playback end and transmit the playback end signaling to the WebRTC signaling exchange module of the delivery end.

[0034] The WebRTC signaling exchange module of the sending end is also used to establish a WebRTC communication connection between the sending end and the playback end based on the signaling of the playback end, so that the playback end can receive and display real-time audio streams or mirror data from the sending end based on the WebRTC communication connection.

[0035] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the screen projection method provided in the first aspect of this application.

[0036] Fifthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the screen projection method provided in the first aspect of this application.

[0037] Sixthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the steps of the screen projection method provided in the first aspect of this application.

[0038] The aforementioned screen mirroring method, apparatus, system, computer equipment, and storage medium involve a playback terminal receiving signaling from the sending terminal. This sending terminal signaling is WebRTC signaling transmitted by the sending terminal via the UPnP protocol. The playback terminal generates playback terminal signaling based on this sending terminal signaling and sends it to the sending terminal via the UPnP protocol, thereby establishing a WebRTC communication connection between the sending and playback terminals. Based on this WebRTC communication connection, the playback terminal receives and displays real-time audio streams or mirrored data from the sending terminal. This application configures WebRTC casting and playback functions on existing DLNA-supporting sending and playback terminals, respectively, using the UPnP protocol as the WebRTC signaling exchange channel. This achieves support and extension of WebRTC screen mirroring based on the current mainstream DLNA screen mirroring function, without increasing hardware costs or reducing its popularity, and realizes mirrored screen mirroring and real-time audio / video stream casting functions. Attached Figure Description

[0039] Figure 1 This is a structural block diagram of a screen projection system in one embodiment;

[0040] Figure 2 This is an interaction timing diagram of the screen projection system in one embodiment;

[0041] Figure 3 This is a flowchart illustrating a screen projection method in one embodiment;

[0042] Figure 4 This is a flowchart illustrating the process of configuring a non-standard UPnP service on the playback end in one embodiment.

[0043] Figure 5 This is a schematic diagram of the interaction process of a standard UPnP service in one embodiment;

[0044] Figure 6This is a diagram of a non-standard UPnP service architecture in one embodiment;

[0045] Figure 7 This is a structural block diagram of the projection device in one embodiment;

[0046] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] DLNA: DLNA (Digital Living Network Alliance) is not a technology in itself, but a solution. It is an integration of multiple technologies dedicated to building home media sharing solutions. Currently, the mainstream smart TV viewing technology is achieved through DLNA screen mirroring technology from mobile phones.

[0049] UPnP: The UPnP protocol stack is the main technology stack upon which DLNA relies. Based on the standard IP protocol, it operates independently of the underlying network medium. The UPnP protocol is primarily divided into five parts: discovery, description, control, events, and presentation. During the device description phase, the sending device provides the playback device with a list of supported services. The UPnP committee defines a series of standard service types, while also allowing device manufacturers to define custom non-standard services to suit their needs.

[0050] UPnP non-standard services: These are service types that are not defined as standard by the UPnP committee, but still follow the UPnP protocol rules. They are generally customized by equipment manufacturers based on the characteristics of their own equipment.

[0051] WebRTC: The name comes from the abbreviation of Web Real-Time Communication. It is a technology that enables real-time voice calls or video chats in web browsers. WebRTC is an open standard for real-time communication between web browsers and mobile applications. Its connection requires signaling exchange between devices, but WebRTC itself does not provide a signaling exchange channel; this needs to be implemented manually.

[0052] WebRTC signaling exchange: Signaling exchange is the process of coordinating communication. Before establishing a connection, WebRTC needs to exchange network information and media information so that both parties can confirm the optimal connection method.

[0053] While there are numerous screen mirroring technologies available, most struggle to simultaneously support the three key characteristics of widespread adoption, compatibility with all mainstream operating systems, and support for mirroring and real-time audio / video. Some screen mirroring technologies have technical barriers, requiring the installation of dedicated software; others are limited to their own ecosystems, resulting in ecosystem monopolies; and still others have low adoption rates and are not supported by some operating systems.

[0054] Compared to the screen mirroring technologies mentioned above, UPnP technology has a longer history and wider adoption since its inception in 1999. Almost all smartphones, smart TVs, and set-top boxes support the UPnP protocol. Moreover, the UPnP protocol has a low technical threshold; generally, as long as the device supports the TCP / IP protocol stack, it can access the UPnP network. Furthermore, the UPnP protocol is completely open source, avoiding ecosystem closure and compatibility issues. Therefore, it has become a widely applicable screen mirroring technology.

[0055] Currently, DLNA is widely supported in media sharing between devices. DLNA screen mirroring technology covers almost all media sharing between smart TVs and set-top boxes. For example, users can use DLNA screen mirroring to cast videos from their mobile phones to their TVs. However, the practical application of DLNA is still limited to casting videos and images, and does not involve more types of device interoperability and functional expansion, lacking capabilities such as mobile phone mirroring and real-time audio and video.

[0056] To achieve screen mirroring and real-time audio and video functions, this application combines UPnP and WebRTC technologies, using WebRTC as a non-standard protocol of UPnP technology, making it a sub-function of DLNA screen mirroring technology. On existing smart TVs and set-top boxes that support DLNA, screen mirroring and real-time audio and video are achieved through software.

[0057] WebRTC is an open-source real-time audio and video technology launched in 2010, designed to provide low-latency, high-quality audio and video communication and support adaptive audio and video transmission under different network conditions. Its core functions include audio and video capture, encoding / decoding, network transmission, and display, and it can run cross-platform on Windows, Linux, macOS, iOS, and Android systems. WebRTC connection establishment relies on signaling exchange, but it does not provide a signaling channel itself; this needs to be implemented manually.

[0058] This application combines DLNA with WebRTC, using DLNA to solve the signaling exchange problem of WebRTC and using WebRTC to enable devices equipped with DLNA technology to perform real-time audio and video streaming. After combining DLNA and WebRTC, this application searches for playback devices that support WebRTC functionality using the traditional method of searching for DLNA-enabled screen mirroring devices, and then performs real-time screen mirroring on these WebRTC-enabled playback devices. This method uses non-standard services defined in the UPnP protocol for WebRTC signaling exchange, supporting and extending WebRTC screen mirroring based on the current mainstream DLNA screen mirroring functions, thus enabling real-time audio and video screen mirroring using WebRTC.

[0059] This application's embodiments establish a non-standard UPnP service, enabling the UPnP protocol to serve as a signaling exchange channel for WebRTC. By exchanging WebRTC signaling through this non-standard UPnP service, support and extensions for WebRTC screen mirroring are achieved on top of the currently mainstream DLNA screen mirroring functionality. This expansion utilizes widely available and open-source technologies without increasing hardware costs or reducing market penetration. Based on DLNA, screen mirroring and real-time audio / video functions are implemented, significantly improving device functionality and user experience.

[0060] The following first introduces the screen projection system and the method for constructing the screen projection system provided in the embodiments of this application.

[0061] In one embodiment, please refer to Figure 1 and Figure 2 This application provides a screen projection system, which includes a screen projection end WebRTC signaling exchange module and a screen projection end UPnP non-standard service module configured on the projection end. The screen projection system also includes a playback end WebRTC signaling exchange module and a playback end UPnP non-standard service module configured on the playback end.

[0062] In this embodiment of the screen mirroring system, the sending end and the playback end can be located on the same local area network when performing screen mirroring operations, such as typically connecting to the same network via Wi-Fi. Both the sending end and the playback end support the UPnP protocol and have DLNA screen mirroring functionality. The UPnP non-standard service modules of the sending end and the playback end can establish an initial connection relationship through the UPnP protocol.

[0063] The system includes a delivery-end WebRTC signaling exchange module for generating delivery-end signaling; a delivery-end UPnP non-standard service module connected to both the delivery-end WebRTC signaling exchange module and the playback-end UPnP non-standard service module, receiving the delivery-end signaling from the delivery-end WebRTC signaling exchange module and transmitting it to the playback-end UPnP non-standard service module via the UPnP protocol; and a playback-end UPnP non-standard service module connected to the playback-end WebRTC signaling exchange module, receiving the delivery-end signaling from the delivery end and transmitting it to the playback-end WebRTC signaling exchange module. The RTC signaling exchange module is used to generate playback signaling based on the sending end signaling and transmit the playback end signaling to the playback end UPnP non-standard service module; the playback end UPnP non-standard service module is also used to send the playback end signaling to the sending end UPnP non-standard service module via the UPnP protocol; the sending end UPnP non-standard service module is also used to receive the playback end signaling from the playback end and transmit the playback end signaling to the sending end WebRTC signaling exchange module; the sending end WebRTC signaling exchange module is also used to establish a WebRTC communication connection between the sending end and the playback end based on the playback end signaling, so that the playback end can receive and display real-time audio streams or mirror data from the sending end based on the WebRTC communication connection.

[0064] In the construction process of the screen projection system provided in this application embodiment, a UPnP non-standard service that can perform WebRTC signaling exchange can be first formulated. This UPnP non-standard service enables WebRTC to perform WebRTC signaling exchange through the UPnP non-standard service.

[0065] After developing the UPnP non-standard services, the delivery equipment implements both UPnP delivery and WebRTC delivery functions. Specifically, the delivery equipment implements basic UPnP delivery functions, including device search, obtaining device descriptions, device control, and monitoring playback events. Building upon these basic UPnP delivery functions, it integrates UPnP non-standard services for WebRTC signaling exchange. The delivery equipment also implements WebRTC signaling exchange functionality based on these UPnP non-standard services.

[0066] After developing the UPnP non-standard services, the playback device implements both UPnP and WebRTC playback client functionalities. Specifically, the playback device implements basic UPnP playback client functions, including device search, sending device descriptions, receiving device control, and sending playback client events. Building upon these basic UPnP playback client functions, it integrates UPnP non-standard services capable of WebRTC signaling exchange. The playback device also implements WebRTC signaling exchange functionality based on these UPnP non-standard services.

[0067] In the screen projection system of this application embodiment, the UPnP non-standard service module of the projection end is used to implement the basic functions related to the UPnP projection end, including device search, obtaining device description, performing device control, and listening to playback end events. The UPnP non-standard service module of the projection end also integrates UPnP non-standard services that can perform WebRTC signaling exchange.

[0068] In the screen projection system of this application embodiment, the WebRTC signaling exchange module of the projection end is used to implement the WebRTC signaling exchange function based on UPnP non-standard services.

[0069] In the screen projection system of this application embodiment, the playback end UPnP non-standard service module is used to implement the basic functions related to the UPnP playback end, including device search, sending device description, receiving device control, sending playback end events, etc. The playback end UPnP non-standard service module also integrates UPnP non-standard services that can perform WebRTC signaling exchange.

[0070] In the screen projection system of this application embodiment, the WebRTC signaling exchange module of the playback end is used to implement the WebRTC signaling exchange function based on UPnP non-standard services.

[0071] The present application will be further described below from the perspective of the application method of the screen projection system provided in the embodiments of the present application.

[0072] In one embodiment, such as Figure 3 As shown, a screen mirroring method is provided, which can be applied to... Figure 1 Taking the playback terminal in the example, the following steps are included:

[0073] Step 202: Receive delivery end signaling from the delivery end.

[0074] Among them, the sending end signaling is the WebRTC signaling transmitted by the sending end through the UPnP protocol.

[0075] For example, in response to a connection request sent by the sender via the UPnP protocol, the playback end establishes an initial communication connection with the sender. The sender generates sender signaling, which is the sender WebRTC signaling that needs to be exchanged before WebRTC communication can be implemented. It may include the sender's session control information, metadata, network data, etc. The sender sends the sender signaling to the playback end via the UPnP protocol.

[0076] Step 204: Generate playback signaling based on the sending end signaling, and send the playback signaling to the sending end via the UPnP protocol, so that the sending end and the playback end establish a WebRTC communication connection.

[0077] For example, after receiving the signaling from the sender, the playback end parses and configures the sender signaling, generates playback signaling based on the sender signaling, and then sends the playback signaling to the sender via the UPnP protocol. After the sender receives, parses, and configures the playback signaling, that is, after exchanging configuration and network parameters between the two, it can try to directly use the other party's IP address and port to establish a direct WebRTC communication connection. During this process, different methods will be used to establish the connection depending on the network conditions of both parties.

[0078] Step 206: Based on the WebRTC communication connection, receive and display real-time audio streams or mirror data from the sending end.

[0079] In the above screen mirroring method, the playback terminal receives signaling from the sending terminal; the sending terminal signaling is WebRTC signaling transmitted by the sending terminal via the UPnP protocol. The playback terminal generates playback terminal signaling based on the sending terminal signaling and sends the playback terminal signaling to the sending terminal via the UPnP protocol, thereby establishing a WebRTC communication connection between the sending terminal and the playback terminal. Based on the WebRTC communication connection, the playback terminal receives and displays real-time audio streams or mirrored data from the sending terminal. This application embodiment configures WebRTC casting and playback functions on existing sending and playback terminals supporting DLNA technology, respectively, using the UPnP protocol as the signaling exchange channel for WebRTC. This achieves support and extension of WebRTC screen mirroring based on the current mainstream DLNA screen mirroring function, without increasing hardware costs or reducing its popularity, realizing mirrored screen mirroring and real-time audio / video stream casting functions.

[0080] In one embodiment, the step of sending the playback terminal signaling to the delivery terminal via the UPnP protocol includes: sending the playback terminal signaling to the delivery terminal via the UPnP protocol through a playback terminal UPnP non-standard service configured on the playback terminal; wherein the playback terminal UPnP non-standard service is configured through predefined service elements while satisfying the UPnP protocol rules.

[0081] The predefined service elements include service function elements, service type elements, and service identifier elements. Service function elements may include action elements and state variable elements, service type elements may include service type, and service identifier elements may include service ID.

[0082] In an optional implementation, the playback-side UPnP non-standard service can be configured through the following procedure:

[0083] Step 402: Based on the UPnP protocol rules, define the service elements that support WebRTC signaling exchange.

[0084] Step 404: Compile a service description document based on the service elements.

[0085] For example, this application embodiment first implements a UPnP non-standard service capable of WebRTC signaling exchange. This UPnP non-standard service requires the playback device to have WebRTC playback functionality and the delivery device to have WebRTC delivery functionality. The delivery device can obtain the IP address and port of the playback device's WebRTC service. Under existing UPnP rules, a UPnP non-standard service capable of WebRTC signaling exchange is defined.

[0086] For example, the action list can be, but is not limited to, the following format.

[0087]

[0088] For example, the state variable table can be, but is not limited to, the following format.

[0089]

[0090] For example, the device type, service type, and service ID of its playback device can be, but are not limited to: "deviceType: urn:schemas-upnp-org:device:MediaRenderer:1;

[0091] serviceType: urn:schemas-dlife-cn:service:WebRTCTransport:1;

[0092] serviceId: urn:dlife-cn:serviceId:WebRTCTransport;".

[0093] For example, the XML used to compile device descriptions can be, but is not limited to:

[0094] ".

[0095] For example, its compilation of service description XML can be, but is not limited to:

[0096] ".

[0097] Step 406: Based on the standard UPnP service framework, implement the non-standard UPnP service for the playback terminal by combining the service elements and the service description file.

[0098] The playback-side UPnP non-standard service module in this embodiment is used to implement playback-side UPnP non-standard services. In the implementation of the playback-side UPnP non-standard service module, firstly, based on the UPnP protocol specification, a standard UPnP service is implemented to enable it to operate normally. The interaction process between the delivery end and the playback end for operating the standard UPnP service is as follows: Figure 5 As shown. The normal implementation of standard UPnP services can be achieved, but is not limited to, using the cling library or implementing it according to the UPnP protocol. Then, based on steps 402 and 404, a non-standard UPnP service specification capable of WebRTC signaling exchange is implemented to achieve the corresponding playback terminal functionality. This can also be achieved, but is not limited to, using the cling library or implementing it according to the UPnP protocol. Based on the standard UPnP service framework, the code implements the non-standard services based on steps 402 and 404, and its service architecture diagram is shown below. Figure 6 As shown, its code implementation example is as follows:

[0099] ".

[0100] Furthermore, this application embodiment pre-configures a WebRTC signaling exchange module for the playback end. First, it implements the WebRTC service based on the WebRTC protocol specification, enabling it to function correctly. Then, based on the pre-built UPnP non-standard service module, it uses actions from the action list defined by the non-standard service to exchange WebRTC signaling with the playback end. It interfaces with UPnP non-standard service specifications capable of WebRTC signaling exchange, allowing WebRTC signaling exchange to be performed with the delivery end via the UPnP non-standard service module. Its service architecture diagram is as follows: Figure 6 As shown, a code implementation example can be found below:

[0101] ".

[0102] Understandably, taking the implementation of a non-standard UPnP service module for the delivery end as an example, this application embodiment can, based on the UPnP protocol specification, first implement the standard UPnP service to enable it to run normally. The implementation method can be, but is not limited to, using the cling library or implementing it according to the UPnP protocol. Based on steps 402 and 404, a non-standard UPnP service specification capable of WebRTC signaling exchange is implemented, and then the corresponding delivery end function is implemented. The implementation method can be, but is not limited to, using the cling library or implementing it according to the UPnP protocol. Based on the standard UPnP service framework, the code implements the non-standard service based on steps 402 and 404, and its service architecture diagram is as follows. Figure 6 As shown, a code implementation example can be found below:

[0103] ".

[0104] Understandably, taking the implementation of the WebRTC signaling exchange module on the delivery end as an example, this application embodiment can also be based on the WebRTC protocol specification. First, the WebRTC service on the delivery end is implemented, enabling it to run normally. This is achieved by using actions from the action list defined in the non-standard service definition, based on the established UPnP non-standard service module, to exchange WebRTC signaling with the playback end. The interface is then established with the UPnP non-standard service specification capable of WebRTC signaling exchange, allowing WebRTC signaling exchange to be performed with the playback end through the UPnP non-standard service module. Its service architecture diagram is as follows: Figure 6 As shown, a code implementation example can be found below:

[0105] ".

[0106] In one embodiment, before receiving signaling from the delivery end, the process includes: in response to a device search request sent by the delivery end via the UPnP protocol, sending a playback device description to the delivery end; and if the playback device description meets the WebRTC detection conditions, establishing an initial connection with the delivery end.

[0107] For example, the casting end and the playback end are on the same local area network (LAN), and the network router allows multicast communication and SSDP protocol broadcasting. The casting end actively searches for playback devices in the network, which can broadcast M-SEARCH messages via SSDP. The playback end listens for multicast messages and directly replies to the casting end in the form of an HTTP response message, containing basic information about the playback end device (such as device type, UUID, and URL of the description file). The casting end uses the basic information returned by the playback end to confirm whether it supports the casting function, and can also confirm whether it supports WebRTC playback function through the list of supported services provided by the playback end. If the playback end device description meets the WebRTC detection conditions, the casting end confirms that the playback end supports WebRTC casting function and begins to establish a control connection. For example, the casting end can subscribe to the playback end's status change notifications (GENA mechanism), send instructions to the playback end through the UPnP control service (SOAP request) to prepare, and establish an initial service control connection (SOAP communication) to prepare for subsequent casting operations. The above process relies on the automatic discovery and connection mechanism of UPnP configured in the UPnP non-standard service modules of the sending and playback ends, ensuring a fast and convenient device connection experience.

[0108] In one embodiment, generating playback signaling based on the delivery end signaling includes: obtaining session description information of the delivery end and interactive connection candidate information of the delivery end from the delivery end signaling; and determining the playback end signaling based on the session description information of the delivery end and the interactive connection candidate information of the delivery end.

[0109] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, this application also provides a screen projection device for implementing the screen projection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more screen projection device embodiments provided below can be found in the limitations of the screen projection method described above, and will not be repeated here.

[0111] In one embodiment, such as Figure 7 As shown, a screen projection device is provided, including: a receiving module 702, a generating module 704, and a display module 706, wherein:

[0112] The receiving module 702 is used to receive sending signaling from the sending end; the sending end signaling is WebRTC signaling transmitted by the sending end through the UPnP protocol.

[0113] The generation module 704 is used to generate playback signaling based on the sending end signaling, and send the playback signaling to the sending end via the UPnP protocol, so that the sending end and the playback end establish a WebRTC communication connection.

[0114] Display module 706 is used to receive and display real-time audio streams or mirror data from the sending end based on the WebRTC communication connection.

[0115] Each module in the aforementioned projection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0116] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a screen projection method.

[0117] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0119] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0120] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0122] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A screen projection method, characterized by, Applied to a playing terminal, the method comprises: receiving a sender signaling from a sender terminal; the sender signaling is a WebRTC signaling delivered by the sender terminal through a UPnP protocol; generating a playing terminal signaling based on the sender signaling, and sending the playing terminal signaling to the sender terminal through the UPnP protocol, so as to make the sender terminal and the playing terminal establish a WebRTC communication connection; based on the WebRTC communication connection, receiving and displaying real-time audio stream or mirror data from the sender terminal; the sending of the playing terminal signaling to the sender terminal through the UPnP protocol comprises: sending the playing terminal signaling to the sender terminal through the UPnP protocol by means of a playing terminal UPnP non-standard service configured by the playing terminal; wherein the playing terminal UPnP non-standard service is configured by means of a pre-defined service element under the condition of meeting the rules of the UPnP protocol; the pre-defined service element comprises a service function element, a service type element and a service identification element.

2. The method of claim 1, wherein, The service function element comprises an action element and a state variable element.

3. The method of claim 1, wherein, The playing terminal UPnP non-standard service is configured by means of the following process: defining a service element supporting WebRTC signaling exchange based on the rules of the UPnP protocol; compiling a service description file based on the service element; realizing the playing terminal UPnP non-standard service based on the standard UPnP service framework, in combination with the service element and the service description file.

4. The method of claim 1, wherein, Before the receiving of the sender signaling from the sender terminal, it comprises: sending a playing terminal device description to the sender terminal in response to a device search request sent by the sender terminal through the UPnP protocol; establishing a preliminary connection with the sender terminal under the condition that the playing terminal device description meets WebRTC detection conditions.

5. The method of claim 1, wherein, The generating of the playing terminal signaling based on the sender signaling comprises: obtaining session description information of the sender terminal and interactive connection candidate information of the sender terminal from the sender signaling; determining the playing terminal signaling based on the session description information of the sender terminal and the interactive connection candidate information of the sender terminal.

6. A screen projection device, characterized by, The device comprises: a receiving module for receiving a sender signaling from a sender terminal; the sender signaling is a WebRTC signaling delivered by the sender terminal through a UPnP protocol; a generating module for generating a playing terminal signaling based on the sender signaling, and sending the playing terminal signaling to the sender terminal through the UPnP protocol, so as to make the sender terminal and the playing terminal establish a WebRTC communication connection; a display module for receiving and displaying real-time audio stream or mirror data from the sender terminal based on the WebRTC communication connection; the sending of the playing terminal signaling to the sender terminal through the UPnP protocol comprises: sending the playing terminal signaling to the sender terminal through the UPnP protocol by means of a playing terminal UPnP non-standard service configured by the playing terminal; The UPnP non-standard service of the playing terminal is configured by pre-defined service elements under the condition of meeting the UPnP protocol rules; the pre-defined service elements include service function elements, service type elements and service identification elements.

7. A screen projection system, characterized by, The system comprises a sending terminal WebRTC signaling exchange module and a sending terminal UPnP non-standard service module configured in the sending terminal, and the system further comprises a playing terminal WebRTC signaling exchange module and a playing terminal UPnP non-standard service module configured in the playing terminal. The sending terminal WebRTC signaling exchange module is configured to generate a sending terminal signaling. The sending terminal UPnP non-standard service module is connected with the sending terminal WebRTC signaling exchange module and the playing terminal UPnP non-standard service module respectively, and is configured to receive the sending terminal signaling from the sending terminal WebRTC signaling exchange module, and send the sending terminal signaling to the playing terminal UPnP non-standard service module in the form of the UPnP protocol. The playing terminal UPnP non-standard service module is connected with the playing terminal WebRTC signaling exchange module, and is configured to receive the sending terminal signaling from the sending terminal, and transfer the sending terminal signaling to the playing terminal WebRTC signaling exchange module. The playing terminal WebRTC signaling exchange module is configured to generate a playing terminal signaling based on the sending terminal signaling, and transfer the playing terminal signaling to the playing terminal UPnP non-standard service module. The playing terminal UPnP non-standard service module is further configured to send the playing terminal signaling to the sending terminal UPnP non-standard service module in the form of the UPnP protocol. The sending terminal UPnP non-standard service module is further configured to receive the playing terminal signaling from the playing terminal, and transfer the playing terminal signaling to the sending terminal WebRTC signaling exchange module. The sending terminal WebRTC signaling exchange module is further configured to establish a WebRTC communication connection between the sending terminal and the playing terminal based on the playing terminal signaling, so that the playing terminal receives and displays real-time audio stream or mirror data from the sending terminal based on the WebRTC communication connection.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images