Screen projection method, device and system, computer equipment and storage medium

Abstract

The invention relates to a screen projection method, device and system, computer equipment and a storage medium. The method comprises the following steps: a playing end receives a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through a UPnP protocol, the playing end generates a playing end signaling based on the delivery end signaling and sends the playing end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playing end establish a WebRTC communication connection, and the playing end sends the playing end signaling to the delivery end based on the WebRTC communication connection. And receiving and displaying the real-time audio stream or mirror image data from the delivery end. By adopting the method, the UPnP protocol can be used as a signaling exchange channel of the WebRTC, the support and expansion of the WebRTC projection screen on the basis of the current mainstream DLNA projection screen function are realized, the hardware cost is not increased, the popularization degree is not reduced, and the mirror image projection screen and real-time audio and video stream projection screen functions are realized.

Description

Technical Field

[0001] The present application relates to the field of near field communication technology, and in particular to a screen projection method, device, system, computer equipment and storage medium. Background Art

[0002] The Digital Living Network Alliance (DLAN) is committed to building a home media sharing solution through the integration of multiple technologies. It mainly relies on the UPnP (Universal Plug and Play) protocol and aims to achieve interconnection between devices through the network.

[0003] However, although DLNA has been widely supported in media sharing between devices, and DLNA screen projection technology covers almost all media sharing between smart TVs and set-top boxes, the actual application of DLNA is still limited to the transmission of videos and pictures, and does not involve more types of device interoperability and function expansion, and lacks capabilities such as mobile phone mirroring screen projection and real-time audio and video. Summary of the invention

[0004] Based on this, it is necessary to provide a screen projection method, device, system, computer equipment and storage medium that can realize mirror screen projection and real-time audio and video functions in response to the above technical problems.

[0005] In a first aspect, the present application provides a screen projection method. The method comprises:

[0006] Receiving a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through the UPnP protocol;

[0007] Generate a playback end signaling based on the delivery end signaling, and send the playback end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playback end establish a WebRTC communication connection;

[0008] Based on the WebRTC communication connection, the real-time audio stream or mirror data from the sending end is received and displayed.

[0009] In one embodiment, sending the playback end signaling to the delivery end through the UPnP protocol includes:

[0010] The player end signaling is sent to the delivery end through the UPnP protocol by using the player end UPnP non-standard service configured by the player end;

[0011] The playback end UPnP non-standard service is configured through pre-defined service elements under the condition of satisfying UPnP protocol rules; the pre-defined service elements include service function elements, service type elements and service identification elements.

[0012] In one embodiment, the playback end UPnP non-standard service is configured by the following process:

[0013] Based on UPnP protocol rules, define service elements that support WebRTC signaling exchange;

[0014] Compile a service description file based on the service elements;

[0015] On the basis of the standard UPnP service framework, the service elements and the service description file are combined to realize the playback end UPnP non-standard service.

[0016] In one embodiment, before receiving the delivery end signaling from the delivery end, the method includes:

[0017] In response to a device search request sent by the delivery end through the UPnP protocol, sending a playback end device description to the delivery end;

[0018] When the playback end device description satisfies the WebRTC detection conditions, a preliminary connection is established with the delivery end.

[0019] In one embodiment, the generating the playback end signaling based on the delivery end signaling includes:

[0020] Acquire the session description information of the delivery end and the interactive connection candidate information of the delivery end from the delivery end signaling;

[0021] The playback end signaling is determined based on the session description information of the delivery end and the interactive connection candidate information of the delivery end.

[0022] In a second aspect, the present application also provides a screen projection device. The device includes:

[0023] A receiving module, used for receiving a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through the UPnP protocol;

[0024] A generating module, used for generating a playback end signaling based on the delivery end signaling, and sending the playback end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playback end establish a WebRTC communication connection;

[0025] The display module is used to receive and display the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

[0026] In a third aspect, the present application also provides a screen projection system. The screen projection system includes:

[0027] The screen projection system is characterized in that the system includes a projection-end WebRTC signaling exchange module and a projection-end UPnP non-standard service module configured at the projection end, and the system also includes a playback-end WebRTC signaling exchange module and a playback-end UPnP non-standard service module configured at the playback end;

[0028] The delivery-end WebRTC signaling exchange module is used to generate delivery-end signaling;

[0029] The delivery-end UPnP non-standard service module is connected to the delivery-end WebRTC signaling exchange module and the playback-end UPnP non-standard service module, respectively, and is used to receive the delivery-end signaling from the delivery-end WebRTC signaling exchange module, and send the delivery-end signaling to the playback-end UPnP non-standard service module through the UPnP protocol;

[0030] The playback end UPnP non-standard service module is connected to the playback end WebRTC signaling exchange module, and is used to receive the delivery end signaling from the delivery end, and transmit the delivery end signaling to the playback end WebRTC signaling exchange module;

[0031] The playback-end WebRTC signaling exchange module is used to generate playback-end signaling based on the delivery-end signaling, and transmit the playback-end signaling to the playback-end UPnP non-standard service module;

[0032] The playback end UPnP non-standard service module is further used to send the playback end signaling to the delivery end UPnP non-standard service module through the UPnP protocol;

[0033] The delivery-end UPnP non-standard service module is further used to receive the playback-end signaling from the playback end, and transmit the playback-end signaling to the delivery-end WebRTC signaling exchange module;

[0034] The sending-end WebRTC signaling exchange module is also used to establish a WebRTC communication connection between the sending end and the playing end based on the playing-end signaling, so that the playing end receives and displays the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

[0035] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the screen projection method provided in the first aspect of the present application when executing the computer program.

[0036] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the screen projection method provided in the first aspect of the present application are implemented.

[0037] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the screen projection method provided in the first aspect of the present application are implemented.

[0038] In the above-mentioned screen projection method, device, system, computer equipment and storage medium, the playback end receives the sending end signaling from the sending end; the sending end signaling is the WebRTC signaling transmitted by the sending end through the UPnP protocol, the playback end generates the playback end signaling based on the sending end signaling, and sends the playback end signaling to the sending end through the UPnP protocol, so that the sending end and the playback end establish a WebRTC communication connection, and the playback end receives and displays the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection. This application configures the WebRTC delivery function and the WebRTC playback function on the existing sending end and the playback end that support DLNA technology respectively, and uses the UPnP protocol as the signaling exchange channel of WebRTC, realizing the support and expansion of WebRTC screen projection based on the current mainstream DLNA screen projection function, without increasing the hardware cost and reducing the popularity, and realizing the mirror screen projection and real-time audio and video stream screen projection functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural block diagram of a screen projection system in one embodiment;

[0040] Figure 2 An interaction sequence diagram of a screen projection system in one embodiment;

[0041] Figure 3 Schematic diagram of a screen projection method in one embodiment;

[0042] Figure 4 A schematic diagram of a process for configuring a UPnP non-standard service on a playback end in one embodiment;

[0043] Figure 5 A schematic diagram of the interaction process of a standard UPnP service in one embodiment;

[0044] Figure 6A diagram of a UPnP non-standard service architecture in one embodiment;

[0045] Figure 7 is a structural block diagram of a screen projection device in one embodiment;

[0046] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] DLNA: DLNA (Digital Living Network Alliance) is not a technology itself, but a solution. It is an integration of multiple technologies and is committed to building a home media sharing solution. The current mainstream smart TV viewing technology is achieved through the DLNA screen projection technology of mobile phones.

[0049] UPnP: The UPnP protocol stack is the main technology stack that DLNA relies on. The UPnP protocol stack is based on the standard IP protocol and is independent of the underlying network medium. The UPnP protocol is mainly divided into five parts: discovery, description, control, event, and display. In the device description stage, the delivery device will provide the supported services to the playback device in the form of a service list. The UPnP committee defines a series of standard service types, and also allows device manufacturers to customize non-standard services that suit them.

[0050] UPnP non-standard service: It is not a standard service type defined by the UPnP Committee, but it also follows the UPnP protocol rules and is generally customized by device manufacturers based on the characteristics of their own devices.

[0051] WebRTC: The name comes from the abbreviation of Web Real-Time Communication. It is a technology that supports web browsers for real-time voice calls or video chats. WebRTC is an open standard for real-time communication between web browsers and mobile applications. It is a real-time communication technology. The premise of its connection is the signaling exchange between devices, but WebRTC itself does not provide a signaling exchange channel and needs to be implemented by itself.

[0052] WebRTC signaling exchange: Signaling exchange is the process of coordinating communication. Before a WebRTC connection is established, it is necessary to exchange network information and media information so that both parties can confirm the optimal connection method.

[0053] In the related technologies, although there are many ways to project screens, most of them are difficult to support the three major features of wide popularity, compatibility with all mainstream systems, and support for mirrored screen projection and real-time audio and video. Some screen projection technologies have technical barriers and must install special software; some screen projection technologies are limited to their own ecosystems and have ecological monopoly problems; some screen projection technologies are not widely popularized and are not supported by some operating systems.

[0054] Compared with the above screen projection technologies, UPnP technology has a longer history and wider popularity since its advent in 1999. Almost all smartphones, smart TVs, and set-top boxes support the UPnP protocol, and the technical threshold of the UPnP protocol is relatively low. Generally, as long as the device supports the TCP / IP protocol stack, it can access the UPnP network. The UPnP protocol is completely open source, avoiding ecological closure and compatibility issues. Therefore, it has become a widely applicable screen projection technology.

[0055] Currently, DLNA has been widely supported in media sharing between devices, and DLNA screen projection technology covers almost all media sharing between smart TVs and set-top boxes. For example, users can use DLNA screen projection to cast videos on their mobile phones to the TV for viewing. However, the actual application of DLNA is still limited to the transmission of videos and pictures, and does not involve more types of device interoperability and function expansion, and lacks the capabilities of mobile phone mirroring, real-time audio and video, etc.

[0056] In order to realize the functions of mirrored screen projection and real-time audio and video, the embodiment of the present application combines UPnP technology with WebRTC technology, uses WebRTC as a non-standard protocol of UPnP technology, and makes it a sub-function of DLNA screen projection technology. On existing smart TVs and set-top boxes that support DLNA, mirrored screen projection and real-time audio and video are realized at the software level.

[0057] WebRTC is an open source real-time audio and video technology launched in 2010. It aims to provide low-latency, high-quality audio and video communications and support adaptive audio and video transmission under different network conditions. Its core functions include audio and video acquisition, encoding and decoding, network transmission and display, and it can run cross-platform on Windows, Linux, macOS, iOS and Android systems. The establishment of WebRTC connections depends on signaling exchanges, but it does not provide signaling channels itself and needs to be implemented by itself.

[0058] The embodiment of the present application combines DLNA with WebRTC, solves the signaling exchange problem of WebRTC through DLNA, and enables devices equipped with DLNA technology to have the ability of real-time audio and video streaming media through WebRTC. After combining DLNA and WebRTC, the embodiment of the present application searches for playback devices that support the WebRTC function through the traditional way of searching for DLNA screen projection devices, and performs real-time mirroring on playback devices that support the WebRTC function. This method exchanges WebRTC signaling by formulating non-standard services in the UPnP protocol, supports and expands WebRTC screen projection on the basis of the current mainstream DLNA screen projection function, and supports WebRTC real-time audio and video screen projection.

[0059] The embodiment of the present application develops a UPnP non-standard service so that the UPnP protocol can be used as a signaling exchange channel for WebRTC. By exchanging WebRTC signaling through the UPnP non-standard service, support and expansion of WebRTC screen projection based on the current mainstream DLNA screen projection function are achieved. Existing widely used and open source technical means are used for combination and expansion without increasing hardware costs or reducing popularity. Mirror screen projection and real-time audio and video functions are realized on the basis of DLNA, which significantly improves the functionality of the device and user experience.

[0060] The following first introduces the screen projection system and the construction method of the screen projection system provided by the embodiments of the present application.

[0061] In one embodiment, please refer to Figure 1 and Figure 2 The present application provides a screen projection system, which includes a sending-end WebRTC signaling exchange module and a sending-end UPnP non-standard service module configured at the projection end, and the screen projection system also includes a playback-end WebRTC signaling exchange module and a playback-end UPnP non-standard service module configured at the playback end.

[0062] When the screen projection system of the embodiment of the present application performs the screen projection operation, the projection end and the playback end can be in the same local area network, for example, they can usually be connected to the same network via Wi-Fi. Both the projection end and the playback end support the UPnP protocol and have the DLNA screen projection function. The projection end UPnP non-standard service module and the playback end UPnP non-standard service module can establish a preliminary connection relationship through the UPnP protocol.

[0063] The WebRTC signaling exchange module at the delivery end is used to generate delivery end signaling; the UPnP non-standard service module at the delivery end is connected to the WebRTC signaling exchange module at the delivery end and the UPnP non-standard service module at the playback end respectively, and the UPnP non-standard service module at the delivery end is used to receive the delivery end signaling from the WebRTC signaling exchange module at the delivery end, and send the delivery end signaling to the UPnP non-standard service module at the playback end through the UPnP protocol; the UPnP non-standard service module at the playback end is connected to the WebRTC signaling exchange module at the playback end, and the UPnP non-standard service module at the playback end is used to receive the delivery end signaling from the delivery end, and transfer the delivery end signaling to the WebRTC signaling exchange module at the playback end; The RTC signaling exchange module is used to generate playback end signaling based on the sending end signaling, and pass the playback end signaling to the playback end UPnP non-standard service module; the playback end UPnP non-standard service module is also used to send the playback end signaling to the sending end UPnP non-standard service module through the UPnP protocol; the sending end UPnP non-standard service module is also used to receive the playback end signaling from the playback end, and pass the playback end signaling to the sending end WebRTC signaling exchange module; the sending end WebRTC signaling exchange module is also used to establish a WebRTC communication connection between the sending end and the playback end based on the playback end signaling, so that the playback end receives and displays the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

[0064] During the construction process of the above-mentioned screen projection system provided in the embodiment of the present application, a UPnP non-standard service that can perform WebRTC signaling exchange can be first formulated. This UPnP non-standard service enables WebRTC to perform WebRTC signaling exchange through the UPnP non-standard service.

[0065] After developing UPnP non-standard services, the delivery device implements the UPnP delivery terminal and WebRTC delivery terminal functions, where the delivery device is used to implement the basic functions related to the UPnP delivery terminal, including device search, obtaining device descriptions, device control, and monitoring playback terminal events. On the basis of the basic functions related to the UPnP delivery terminal, the UPnP non-standard service that can perform WebRTC signaling exchange is integrated. The delivery device is also used to implement the WebRTC signaling exchange function based on the UPnP non-standard service.

[0066] After developing UPnP non-standard services, the playback device implements the UPnP playback terminal and WebRTC playback terminal functions, where the playback device is used to implement the basic functions related to the UPnP playback terminal, including device search, sending device descriptions, receiving device control, sending playback terminal events, etc. And on the basis of this UPnP playback terminal related basic functions, the UPnP non-standard service that can perform WebRTC signaling exchange is integrated. The playback device is also used to implement the WebRTC signaling exchange function based on the UPnP non-standard service.

[0067] In the screen projection system of the embodiment of the present application, the UPnP non-standard service module on the sending end is used to implement basic functions related to the UPnP sending end, including device search, obtaining device description, device control, monitoring playback end events, etc. The UPnP non-standard service module on the sending end also integrates UPnP non-standard services that can perform WebRTC signaling exchange.

[0068] In the screen projection system of the embodiment of the present application, the WebRTC signaling exchange module at the sending end is used to implement the WebRTC signaling exchange function based on UPnP non-standard service.

[0069] In the screen projection system of the embodiment of the present application, the UPnP non-standard service module on the playback end is used to implement basic functions related to the UPnP playback end, including device search, sending device description, receiving device control, sending playback end events and other functions. The UPnP non-standard service module on the playback end also integrates UPnP non-standard services that can perform WebRTC signaling exchange.

[0070] In the screen projection system of the embodiment of the present application, the playback end WebRTC signaling exchange module is used to implement the WebRTC signaling exchange function based on UPnP non-standard service.

[0071] The following further introduces the present application from the perspective of the application method of the screen projection system provided in the embodiments of the present application.

[0072] In one embodiment, Figure 3 As shown, a screen projection method is provided, which is applied to Figure 1 The following steps are taken as an example to illustrate the playback end in FIG.

[0073] Step 202, receiving a delivery end signaling from a delivery end.

[0074] The sending end signaling is the WebRTC signaling transmitted by the sending end through the UPnP protocol.

[0075] Exemplarily, the player establishes a preliminary communication connection with the delivery end in response to a connection request sent by the delivery end through the UPnP protocol. The delivery end generates delivery end signaling, which is the delivery end WebRTC signaling that needs to be exchanged before WebRTC communication is realized, and may include the delivery end's session control information, metadata, network data, etc. The delivery end sends the delivery end signaling to the player through the UPnP protocol.

[0076] Step 204: Generate playback-end signaling based on the delivery-end signaling, and send the playback-end signaling to the delivery-end through the UPnP protocol, so that the delivery-end and the playback-end establish a WebRTC communication connection.

[0077] Exemplarily, after receiving the signaling from the sending end, the playback end parses and configures the signaling from the sending end, generates the playback end signaling based on the signaling from the sending end, and then sends the playback end signaling to the sending end through the UPnP protocol. After the sending end receives, parses and configures the playback end signaling, that is, after exchanging the configuration and network parameters of the two, it can try to directly use the other party's IP address and port for direct WebRTC communication connection. In this process, different methods will be used to establish the connection according to the network conditions of both parties.

[0078] Step 206: Based on the WebRTC communication connection, receive and display the real-time audio stream or mirror data from the sending end.

[0079] In the above-mentioned screen projection method, the playback end receives the sending end signaling from the sending end; the sending end signaling is the WebRTC signaling transmitted by the sending end through the UPnP protocol, the playback end generates the playback end signaling based on the sending end signaling, and sends the playback end signaling to the sending end through the UPnP protocol, so that the sending end and the playback end establish a WebRTC communication connection, and the playback end receives and displays the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection. The embodiment of the present application configures the WebRTC delivery function and the WebRTC playback function on the existing sending end and the playback end that support DLNA technology, respectively, and uses the UPnP protocol as the signaling exchange channel of WebRTC, thereby realizing the support and expansion of WebRTC screen projection based on the current mainstream DLNA screen projection function, without increasing the hardware cost and reducing the popularity, and realizing the mirror screen projection and real-time audio and video stream screen projection functions.

[0080] In one embodiment, the step of sending the playback end signaling to the delivery end through the UPnP protocol includes: sending the playback end signaling to the delivery end through the UPnP protocol through the playback end UPnP non-standard service configured by the playback end; wherein the playback end UPnP non-standard service is configured through pre-defined service elements while satisfying the UPnP protocol rules.

[0081] The predefined service elements include service function elements, service type elements and service identification elements. The service function elements may include action elements and state variable elements, the service type elements may include service types, and the service identification elements may include service IDs.

[0082] In an optional implementation, the playback end UPnP non-standard service can be configured through the following process:

[0083] Step 402: Based on UPnP protocol rules, define service elements that support WebRTC signaling exchange.

[0084] Step 404: compile a service description file based on the service elements.

[0085] For example, the embodiment of the present application first implements a UPnP non-standard service that can perform WebRTC signaling exchange. The UPnP non-standard service that can perform WebRTC signaling exchange requires that the playback device has a WebRTC playback function, the delivery device has a WebRTC delivery function, and the delivery device can obtain the IP and port of the WebRTC service of the playback device. Under the existing UPnP rules, a UPnP non-standard service that can perform WebRTC signaling exchange is defined.

[0086] For example, the action list may be in the following format but is not limited to it.

[0087]

[0088] For example, the state variable table may be in, but is not limited to, the following format.

[0089]

[0090] For example, the device type, service type and service ID of the playback device may be, but are not limited to: “deviceType:urn:schemas-upnp-org:device:MediaRenderer:1;

[0091] serviceType: urn:schemas-dlife-cn:service:WebRTCTransport:1;

[0092] serviceId: urn:dlife-cn:serviceId:WebRTCTransport;".

[0093] For example, the compiled device description XML may be, but is not limited to:

[0094] ”.

[0095] For example, the compiled service description XML may be but is not limited to:

[0096] ”.

[0097] Step 406: Based on the standard UPnP service framework, the service elements and the service description file are combined to implement the playback end UPnP non-standard service.

[0098] The playback end UPnP non-standard service module of the embodiment of the present application is used to implement the playback end UPnP non-standard service. In the process of implementing the playback end UPnP non-standard service module, firstly, based on the UPnP protocol specification, the standard UPnP service is first implemented so that the standard UPnP service can be normally operated. The interaction process of the normal operation of the standard UPnP service between the delivery end and the playback end is as follows: Figure 5 As shown. The implementation method of normal operation of standard UPnP service can be, but not limited to, using the cling library or implementing it according to the UPnP protocol. Then, based on steps 402 and 404, the UPnP non-standard service specification that can perform WebRTC signaling exchange is implemented to realize the corresponding playback terminal function. The implementation method can be, but not limited to, using the cling library or implementing it according to the UPnP protocol. Based on the standard UPnP service framework, the code implements the non-standard service based on steps 402 and 404, and its service architecture diagram is shown as follows Figure 6 As shown, the code implementation example is as follows: "

[0099] ”.

[0100] In addition, the embodiment of the present application also pre-configures the WebRTC signaling exchange module on the playback end, firstly implements the WebRTC service based on the WebRTC protocol specification so that it can run the WebRTC service normally. Then, based on the established UPnP non-standard service module, the actions in the action list defined by the non-standard service are used to perform WebRTC signaling exchanges with the playback end. Docking with the UPnP non-standard service specification that can perform WebRTC signaling exchanges enables the WebRTC signaling exchanges to be exchanged with the delivery end through the UPnP non-standard service module. The service architecture diagram is shown below. Figure 6 As shown, the code implementation example can be as follows: "

[0101] ”.

[0102] It can be understood that, taking the implementation of the UPnP non-standard service module on the delivery side as an example, the embodiment of the present application can be based on the UPnP protocol specification, first implement the standard UPnP service so that it can normally operate the standard UPnP service, and the implementation method may be but not limited to using the cling library or implementing it according to the UPnP protocol. Based on steps 402 and 404, the UPnP non-standard service specification that can perform WebRTC signaling exchange is implemented, and then the corresponding delivery side function is implemented. The implementation method may be but not limited to using the cling library or implementing it according to the UPnP protocol. On the basis of the standard UPnP service framework, the code implements the non-standard service implemented based on steps 402 and 404, and its service architecture diagram is shown in the figure. Figure 6 As shown, a code implementation example can be as follows: "

[0103] ”.

[0104] It can be understood that, taking the implementation of the WebRTC signaling exchange module on the delivery side as an example, the embodiment of the present application can also be based on the WebRTC protocol specification, firstly implement the WebRTC service on the delivery side so that it can operate the WebRTC service normally, and the implementation method is to use the actions in the action list defined by the non-standard service on the basis of the built UPnP non-standard service module to perform WebRTC signaling exchange with the playback side. The UPnP non-standard service specification that can perform WebRTC signaling exchange is connected, so that the WebRTC signaling exchange can be exchanged with the playback side through the UPnP non-standard service module. Its service architecture diagram is shown as follows: Figure 6 As shown, the code implementation example can be as follows: "

[0105] ”.

[0106] In one embodiment, before receiving the sending end signaling from the sending end, it includes: sending the playback end device description to the sending end in response to the device search request sent by the sending end through the UPnP protocol; and establishing a preliminary connection with the sending end when the playback end device description meets the WebRTC detection conditions.

[0107] Exemplarily, the delivery end and the playback end are in the same local area network, and the network router allows multicast communication (Multicast) and SSDP protocol broadcasting. The delivery end actively searches for playback devices in the network, and it can broadcast M-SEARCH messages through SSDP. The playback end listens to multicast messages and directly replies to the delivery end in the form of an HTTP response message, which contains basic information of the playback end device (such as device type, UUID, and URL of the description file). The delivery end uses the basic information returned by the playback end to confirm whether it supports the screen projection function, and can also confirm whether it supports the WebRTC playback function through the list of supported services fed back by the playback end. When the playback end device description meets the WebRTC detection conditions, the delivery end confirms that the playback end supports the WebRTC screen projection function and starts to establish a control connection. For example, the delivery end can subscribe to the status change notification of the playback end (GENA mechanism), send instructions to the playback end through the UPnP control service (SOAP request) to prepare, and establish a preliminary service control connection (SOAP communication) to prepare for subsequent screen projection operations. The above process can rely on the UPnP automatic discovery and connection mechanism configured in the UPnP non-standard service module on the delivery side and the UPnP non-standard service module on the playback side, ensuring a fast and convenient device connection experience.

[0108] In one embodiment, generating playback-end signaling based on the delivery-end signaling includes: obtaining the session description information of the delivery end and the interactive connection candidate information of the delivery end from the delivery-end signaling; and determining the playback-end signaling based on the session description information of the delivery end and the interactive connection candidate information of the delivery end.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a screen projection device for implementing the screen projection method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more screen projection device embodiments provided below can refer to the limitations on the screen projection method above, and will not be repeated here.

[0111] In one embodiment, Figure 7 As shown, a screen projection device is provided, including: a receiving module 702, a generating module 704 and a display module 706, wherein:

[0112] The receiving module 702 is used to receive a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through the UPnP protocol.

[0113] The generating module 704 is used to generate a playback end signaling based on the delivery end signaling, and send the playback end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playback end establish a WebRTC communication connection.

[0114] The display module 706 is used to receive and display the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

[0115] Each module in the above-mentioned projection device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0116] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a screen projection method is implemented.

[0117] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0120] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0123] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A screen projection method, characterized in that: Applied to the playback end, the method includes: Receiving a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through the UPnP protocol; Generate a playback end signaling based on the delivery end signaling, and send the playback end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playback end establish a WebRTC communication connection; Based on the WebRTC communication connection, the real-time audio stream or mirror data from the sending end is received and displayed.

2. The method according to claim 1, characterized in that The step of sending the playback end signaling to the delivery end through the UPnP protocol includes: The player end signaling is sent to the delivery end through the UPnP protocol by using the player end UPnP non-standard service configured by the player end; The playback end UPnP non-standard service is configured through pre-defined service elements under the condition of satisfying UPnP protocol rules; the pre-defined service elements include service function elements, service type elements and service identification elements.

3. The method according to claim 2, characterized in that The playback end UPnP non-standard service is configured through the following process: Based on UPnP protocol rules, define service elements that support WebRTC signaling exchange; Compile a service description file based on the service elements; On the basis of the standard UPnP service framework, the service elements and the service description file are combined to realize the playback end UPnP non-standard service.

4. The method according to claim 1, characterized in that: Before receiving the delivery end signaling from the delivery end, the method includes: In response to a device search request sent by the delivery end through the UPnP protocol, sending a playback end device description to the delivery end; When the playback end device description satisfies the WebRTC detection conditions, a preliminary connection is established with the delivery end.

5. The method according to claim 1, characterized in that The generating the playback end signaling based on the delivery end signaling comprises: Acquire the session description information of the delivery end and the interactive connection candidate information of the delivery end from the delivery end signaling; The playback end signaling is determined based on the session description information of the delivery end and the interactive connection candidate information of the delivery end.

6. A screen projection device, characterized in that: The device comprises: A receiving module, used for receiving a delivery end signaling from a delivery end; the delivery end signaling is a WebRTC signaling transmitted by the delivery end through the UPnP protocol; A generating module, used for generating a playback end signaling based on the delivery end signaling, and sending the playback end signaling to the delivery end through the UPnP protocol, so that the delivery end and the playback end establish a WebRTC communication connection; The display module is used to receive and display the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

7. A screen projection system, characterized in that: The system includes a delivery-end WebRTC signaling exchange module and a delivery-end UPnP non-standard service module configured at the delivery end, and the system also includes a playback-end WebRTC signaling exchange module and a playback-end UPnP non-standard service module configured at the playback end; The delivery-end WebRTC signaling exchange module is used to generate delivery-end signaling; The delivery-end UPnP non-standard service module is connected to the delivery-end WebRTC signaling exchange module and the playback-end UPnP non-standard service module, respectively, and is used to receive the delivery-end signaling from the delivery-end WebRTC signaling exchange module, and send the delivery-end signaling to the playback-end UPnP non-standard service module through the UPnP protocol; The playback end UPnP non-standard service module is connected to the playback end WebRTC signaling exchange module, and is used to receive the delivery end signaling from the delivery end, and transmit the delivery end signaling to the playback end WebRTC signaling exchange module; The playback-end WebRTC signaling exchange module is used to generate playback-end signaling based on the delivery-end signaling, and transmit the playback-end signaling to the playback-end UPnP non-standard service module; The playback end UPnP non-standard service module is further used to send the playback end signaling to the delivery end UPnP non-standard service module through the UPnP protocol; The delivery-end UPnP non-standard service module is further used to receive the playback-end signaling from the playback end, and transmit the playback-end signaling to the delivery-end WebRTC signaling exchange module; The sending-end WebRTC signaling exchange module is also used to establish a WebRTC communication connection between the sending end and the playing end based on the playing-end signaling, so that the playing end receives and displays the real-time audio stream or mirror data from the sending end based on the WebRTC communication connection.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images