Media stream playing method, device and system

Abstract

The invention provides a media stream playing method, device and system, and the method applied to a first video recording server comprises the steps: receiving a playing request for a target media stream sent by a webpage player based on a signaling transmission channel between the first video recording server and the webpage player; acquiring a target media stream from a second video recording server based on a first audio and video media transmission channel with the second video recording server; the target media stream is sent to the webpage player based on a second audio and video media transmission channel between the first audio and video media transmission channel and the webpage player, transmission protocols adopted by the first audio and video media transmission channel and the second audio and video media transmission channel are different, and the target media stream is used for being played in the webpage player. Therefore, even if the webpage player does not support the target media stream which is stored in the second video recording server and is recorded by adopting an RTSP protocol, the target media stream can be played on the webpage player, and additional plug-ins do not need to be installed or a specific server framework does not need to be depended on.

Description

Technical Field

[0001] The present disclosure relates to the technical field of media stream transmission and playback, and in particular to a media stream playback method, device and system. Background Art

[0002] With the rapid development of Internet technology, audio and video services have become an indispensable part of modern network applications. There are many challenges in building a comprehensive and efficient audio and video service platform. For example, with the development of Web (network) technology, audio and video resources on demand in web browsers can meet users' seamless browsing experience, while the audio and video resources played in web browsers may be stored on third-party platforms. Since most of the audio and video resources on third-party platforms are recorded using RTSP (Real Time Streaming Protocol), and web browsers do not natively support this protocol for security and compatibility considerations, it is difficult to play RTSP streams directly on the web browser side. In addition, considering the cost-effectiveness and the need for smooth system transition, when building a new audio and video service platform, it is often hoped that existing infrastructure and services can continue to be used, including those old systems that have not yet been upgraded to support more modern Web-friendly protocols. This means that the newly built audio and video service platform must not only be able to process the content generated by itself, but also be able to effectively connect with third-party platforms that use old technical standards such as the RTSP protocol to achieve cross-system audio and video playback. In this context, how to play media streams recorded using protocols that are not supported by web browsers, such as the RTSP protocol, has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the present disclosure is to solve one of the above technical problems at least to some extent.

[0004] To this end, the first purpose of the present disclosure is to propose a method for playing a media stream, which can be used to play a media stream recorded using a protocol that the web player of a web browser does not support, such as the RTSP protocol, without installing additional plug-ins or relying on a specific server architecture.

[0005] A second objective of the present disclosure is to provide a method for playing a media stream.

[0006] A third objective of the present disclosure is to provide a media stream playback device.

[0007] A fourth objective of the present disclosure is to provide a media stream playback device.

[0008] A fifth objective of the present disclosure is to provide an electronic device.

[0009] A sixth objective of the present disclosure is to provide a media stream playback system.

[0010] A seventh objective of the present disclosure is to provide a computer-readable storage medium.

[0011] An eighth objective of the present disclosure is to provide a computer program product.

[0012] To achieve the above-mentioned purpose, an embodiment of the first aspect of the present disclosure proposes a method for playing a media stream, which is applied to a first recording server, and the method includes: based on a signaling transmission channel between the web page player, receiving a playback request for a target media stream sent by the web page player; in response to the playback request, based on a first audio and video media transmission channel between the web page player and the second recording server, obtaining the target media stream from the second recording server; based on a second audio and video media transmission channel between the web page player and the web page player, sending the target media stream to the web page player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used to be played on the web page player.

[0013] To achieve the above-mentioned purpose, the second aspect embodiment of the present disclosure proposes a method for playing a media stream, which is applied to a web page player, and the method includes: based on a signaling transmission channel between the first recording server and the first recording server, sending a playback request for a target media stream to the first recording server; based on a second audio and video media transmission channel between the first recording server and the first recording server, obtaining the target media stream from the first recording server, wherein the target media stream is obtained by the first recording server from the second recording server in response to the playback request based on the first audio and video media transmission channel between the first recording server and the second recording server, and the transmission protocols used by the first audio and video media transmission channel and the second audio and video media transmission channel are different; playing the target media stream.

[0014] To achieve the above-mentioned purpose, an embodiment of the third aspect of the present disclosure proposes a media stream playback device, which is applied to a first recording server, and the device includes: a receiving module, which is used to receive a playback request for a target media stream sent by the web page player based on a signaling transmission channel between the web page player; an acquisition module, which is used to respond to the playback request and obtain the target media stream from the second recording server based on a first audio and video media transmission channel between the web page player and the second recording server; a sending module, which is used to send the target media stream to the web page player based on a second audio and video media transmission channel between the web page player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used to be played on the web page player.

[0015] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present disclosure proposes a media stream playback device, which is applied to a web page player, and the device includes: a sending module, which is used to send a playback request for a target media stream to the first recording server based on a signaling transmission channel between the first recording server and the second recording server; an acquisition module, which is used to obtain the target media stream from the first recording server based on a second audio and video media transmission channel between the first recording server and the second recording server, wherein the target media stream is obtained by the first recording server from the second recording server in response to the playback request based on the first audio and video media transmission channel between the first recording server and the second recording server, and the transmission protocols used by the first audio and video media transmission channel and the second audio and video media transmission channel are different; and a playback module, which is used to play the target media stream.

[0016] To achieve the above-mentioned purpose, the fifth aspect embodiment of the present disclosure proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the media stream playback method described in the first aspect embodiment or the media stream playback method described in the second aspect embodiment is implemented.

[0017] To achieve the above-mentioned purpose, the sixth aspect embodiment of the present disclosure proposes a media stream playback system, including a web page player, a first recording server and a second recording server; wherein the web page player is used to implement the media stream playback method described in the second aspect embodiment; the first recording server is used to implement the media stream playback method described in the first aspect embodiment; the second recording server is used to store the target media stream to be played.

[0018] To achieve the above-mentioned purpose, the seventh aspect embodiment of the present disclosure proposes a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the media stream playback method described in the first aspect embodiment above, or the media stream playback method described in the second aspect embodiment.

[0019] According to an eighth aspect of the embodiments of the present disclosure, a computer program product is provided, including: a computer program, which, when executed by a processor, implements the media stream playback method as described in the first aspect of the embodiments of the present disclosure, or the media stream playback method as described in the second aspect of the embodiments.

[0020] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A structural diagram of a media stream playback system provided by an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of signaling and media interaction of a media stream playback system provided in an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a method for playing a media stream provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of another method for playing a media stream provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the structure of a media stream playback device provided in an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of another media stream playback device provided in an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below by the candidate drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0030] With the development of Web technology, on-demand audio and video resources in web browsers can meet users' seamless browsing experience. The audio and video resources played in web browsers may be stored on third-party platforms. Since most of the audio and video resources on third-party platforms are recorded using RTSP, and web browsers do not natively support this protocol for security and compatibility reasons, it is difficult to play RTSP streams directly on the web browser side. In addition, considering the cost-effectiveness and the need for smooth system transition, when building a new audio and video service platform, it is often hoped that the existing infrastructure and services can be continued to be used, including those old systems that have not yet been upgraded to support more modern Web-friendly protocols. This means that the newly built audio and video service platform must not only be able to process the content generated by itself, but also be able to effectively connect with third-party platforms that use old technical standards such as the RTSP protocol to achieve cross-system audio and video playback. In this context, how to play media streams recorded by protocols that are not supported by web browsers, such as the RTSP protocol, has become a technical problem that needs to be solved urgently.

[0031] As an emerging standard, the WebRTC (Web Real-Time Communication) protocol is widely used in the field of real-time communication. It allows web browsers to directly establish point-to-point connections to transmit audio and video data without installing additional plug-ins. In view of the good compatibility of WebRTC and its powerful multimedia processing capabilities, the present disclosure uses it as a unified transport layer protocol to bridge the web player as a web client and the back-end recording service, thereby proposing a media stream playback method, device and system to achieve the playback of media streams recorded by protocols that are not supported by the web player of the web browser, such as the RTSP protocol, without installing additional plug-ins or relying on a specific server architecture.

[0032] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0033] First combine Figure 1 , a media stream playback system provided in an embodiment of the present application is described. Figure 1 A structural diagram of a media stream playback system provided in an embodiment of the present application.

[0034] like Figure 1 As shown, the media stream playback system includes: a web player 110 , a first video recording server 120 and a second video recording server 130 .

[0035] The web page player 110 is a player in a web page browser.

[0036] The first video recording server 120 is a video recording server at the back end of the web player 110. Considering that the web player 110 supports limited media encoding types, the first video recording server 120 may have transcoding capability, and may convert media streams of various encoding types into encoding types supported by the web player 110. The encoding type of video data supported by the web player 110 may be H264 type, and the encoding type of audio data supported by the web player 110 may be OPUS type.

[0037] The second video recording server 130 is a third-party video recording server that stores the media stream to be played.

[0038] In the disclosed embodiment, a signaling transmission channel may be established between the web player 110 and the first video recording server 120, wherein the signaling transmission channel may transmit control instructions such as pause playback, resume playback, and seek (fast forward or rewind) accurate to seconds.

[0039] In some embodiments, the transmission protocol used by the signaling transmission channel may be WebSocket, and accordingly, the signaling transmission channel may be a websocket connection. As the signaling transmission protocol between the web player 110 and the first video server 120, websocket may establish a persistent connection between the web player 110 as a client and the first video server 120 as a server to transmit control instructions such as pause, resume, and seek (fast forward or rewind) with precision to seconds.

[0040] After the web player 110 establishes a signaling transmission channel with the first video recording server 120, when the web player 110 needs to play the target media stream, it can send a play request for the target media stream to the first video recording server 120 based on the signaling transmission channel, wherein the play request is used to request the target media stream from the first video recording server. The target media stream is the media stream to be played, and can be at least one of an audio media stream, a video media stream, and an audio and video media stream. In response to the play request, the first video recording server 120 can establish a second audio and video media transmission channel with the web player 110. In addition, the first video recording server 120 can establish a first audio and video media transmission channel with the second video recording server 130.

[0041] The first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols. The first audio and video media transmission channel is used to transmit media streams between the first video recording server 120 and the second video recording server 130. The second audio and video media transmission channel is used to transmit media streams between the web page player 110 and the first video recording server 120. The transmission protocol used by the second audio and video media transmission channel is a transmission protocol supported by the web page player 110.

[0042] In some embodiments, the transmission protocol used by the first audio and video media transmission channel may be the RTSP protocol, and accordingly, the first audio and video media transmission channel may be an RTSP connection. The transmission protocol used by the second audio and video media transmission channel may be the WebRTC protocol, and accordingly, the second audio and video media transmission channel may be a WebRTC connection.

[0043] After establishing the first audio and video media transmission channel and the second audio and video media transmission channel, the first recording server 120 can obtain the target media stream to be played from the second recording server 130 based on the first audio and video media transmission channel, and send the target media stream to the web player 110 based on the second audio and video media transmission channel.

[0044] After receiving the target media stream, the web player 110 can play the target media stream.

[0045] In the case where the transmission protocol used by the first audio and video media transmission channel is the RTSP protocol, the first video recording server 120 can obtain the target media stream from the second video recording server 130 based on the RTP transmission in the RTSP protocol. In the case where the transmission protocol used by the second audio and video media transmission channel is the WebRTC protocol, the first video recording server 120 can send the target media stream to the web player 110 based on the SRTP (Secure Real-time Transport Protocol) transmission in the WebRTC. In the case where the SRTP transmission is used, the target media stream can be encrypted and protected to ensure the security and privacy of the communication process.

[0046] Therefore, the first recording server 120 obtains the target media stream to be played from the second recording server 130 based on the first audio and video media transmission channel between the first recording server 120 and the second recording server 130, and sends the target media stream to the web player 110 based on the second audio and video media transmission channel between the first recording server 120 and the web player 110, so that even if the web player 110 does not support the protocol used by the target media stream stored in the second recording server 130, such as the RTSP protocol, the target media stream can be played on the web player 110, thereby achieving the playback of media streams recorded by protocols that are not supported by the web player of the web browser, such as the RTSP protocol, without the need to install additional plug-ins or rely on a specific server architecture.

[0047] In some embodiments, the playback request sent by the web player 110 to the first video recording server 120 may carry the IP (Internet Protocol) address and port number of the second video recording server 130 where the target media stream is located, as well as an identifier of the target media stream; accordingly, the process of the first video recording server 120 obtaining the target media stream from the second video recording server 130 based on the first audio and video media transmission channel between the first video recording server 120 and the second video recording server 130 may include: the first video recording server 120 obtains the target media stream matching the above identifier from the target port of the second video recording server corresponding to the IP address based on the first audio and video media transmission channel, wherein the target port is the port corresponding to the port number carried in the playback request.

[0048] In some embodiments, the process in which the first video recording server 120 obtains the target media stream matching the above-mentioned identifier from the target port of the second video recording server corresponding to the IP address based on the first audio and video media transmission channel may include: the first video recording server 120 sends a data acquisition request to the second video recording server 130 corresponding to the IP address, and the data acquisition request carries the identifier of the target media stream; in response to receiving the data acquisition request, the second video recording server 130 may return the target media stream matching the above-mentioned identifier to the first video recording server 120 through the target port based on the established first audio and video media transmission channel. Accordingly, the first video recording server 120 may receive the target media stream matching the above-mentioned identifier returned by the second video recording server 130 based on the target port in response to the data acquisition request based on the established first audio and video media transmission channel.

[0049] The data acquisition request is used to request the second video recording server 130 for a target media stream.

[0050] The data acquisition request may be sent from the first video recording server 120 to the second video recording server 130 during the process of establishing the first audio and video media transmission channel.

[0051] In some embodiments, after the first video recording server 120 sends a data acquisition request to the second video recording server 130, a timer can be started and the connection status of the first video media transmission channel can be monitored. If the first video media transmission channel is not successfully established within a preset time period or the target media stream is not obtained, the web page player 110 can be notified to terminate the current operation and display an error message to the user.

[0052] In some embodiments, the play request sent by the web player 110 to the first video recording server 120 may carry the network transmission protocol type specified by the web player 110. In the case where the transmission protocol used by the second audio and video media transmission channel is the WebRTC protocol and the transmission protocol used by the first audio and video media transmission channel is the RTSP protocol, since both the SRTP transmission in WebRTC and the RTP transmission in the RTSP transmission support both the UDP (User Datagram Protocol) type and the TCP (Transmission Control Protocol) type, the network transmission protocol type specified by the web player 110 may be the UDP and / or TCP type. The first video recording server 120 may select a suitable RTP transmission method according to the network transmission protocol type specified by the web player 110, obtain the target media stream from the second video recording server 130, and adjust the forwarding strategy of the media stream data.

[0053] In some embodiments, the first video recording server 120 may determine the encoding type of the target media stream returned by the second video recording server 130. For ease of distinction, the encoding type may be referred to as a first encoding type, and the first encoding type may not belong to the media encoding type supported by the web player 110. In this case, after the first video recording server 120 obtains the target media stream from the second video recording server 130, it may convert the target media stream from being encoded with the first encoding type to being encoded with the second encoding type, wherein the media encoding types supported by the web player include the second encoding type, and then based on the second audio and video media transmission channel, the target media stream encoded with the second encoding type is sent to the web player 110, so that the web player 110 can decode the target media stream encoded with the second encoding type, and then play the decoded target media stream.

[0054] The media encoding types supported by the web player 110 may be pre-stored in the first video recording server 120 .

[0055] Among them, the process of the first video recording server 120 converting the target media stream from being encoded in the first encoding type to being encoded in the second encoding type may include: decoding the target media stream encoded in the first encoding type, and then encoding the decoded target media stream in the second encoding type to obtain the target media stream encoded in the second encoding type.

[0056] In some embodiments, the first recording server 120 determines the first encoding type of the target media stream returned by the second recording server 130 by: receiving the audio and video media capability information of the second recording server 130 returned by the second recording server 130 during the process of establishing the first audio and video media transmission channel; and determining the first encoding type of the target media stream based on the audio and video media capability information.

[0057] The audio and video media capability information of the second video recording server 130 may include the network transmission protocol supported by the second video recording server 130, such as TCP and / or UDP, the IP address and port number of the second video recording server 130, and the audio and video media information supported by the second video recording server 130. The audio and video media information may include the encoding type and encoding details of the media stream supported by the second video recording server 130.

[0058] Thus, the first video recording server 120 can determine the encoding type of the media stream supported by the second video recording server 130 based on the audio and video media capability information of the second video recording server 130 returned by the second video recording server 130 , and use the encoding type as the first encoding type.

[0059] In some embodiments, during the playback of the target media stream, the web player 110 may also send a control instruction to the first video recording server 120 based on the signaling transmission channel, wherein the control instruction is used to perform playback control such as pausing, resuming, and seeking on the playback status of the target media stream. The control instruction may include any instruction such as a pause instruction for pausing the playback of the media stream, a resume instruction for resuming the playback of the media stream, and a seek instruction for jumping to a specified timestamp for playback.

[0060] In response to receiving the control instruction sent by the web player 110 based on the signaling transmission channel, the first recording server 120 can forward the control instruction to the second recording server 130 based on the first audio and video media transmission channel. Thus, the second recording server 130 can perform corresponding operations based on the control instruction, obtain the control result corresponding to the control instruction, and return the control result corresponding to the control instruction to the first recording server 120 based on the first audio and video media transmission channel. In response to receiving the control result corresponding to the control instruction returned by the second recording server 130 based on the first audio and video media transmission channel, the first recording server 120 can send the control result to the web player 110 based on the signaling transmission channel. Thus, the web player 110 can obtain the control result of the control instruction.

[0061] Among them, for the pause playback instruction in the control instruction, the corresponding operation performed by the second video recording server after obtaining the instruction may include: pausing to return the media stream to the first video recording server 120, and maintaining the first audio and video media transmission channel between the first video recording server 120. The control result of the instruction may include, for example, a response message for notifying that the playback has been paused.

[0062] The resume playback instruction in the control instruction may carry a target timestamp, wherein the target timestamp may be a timestamp corresponding to a frame of data being played in the target media stream when the web player 110 sends a pause playback instruction. The corresponding operation performed by the second video recording server after obtaining the instruction may include: resuming the return of the media stream to the first video recording server 120, wherein the returned media stream may include a media stream corresponding to the target timestamp in the target media stream. The control result of the instruction may include, for example, a response message for notifying that playback has been resumed. The media stream corresponding to the target timestamp may be a media stream including a frame of data corresponding to the target timestamp and each frame of data after the target timestamp.

[0063] For the jump instruction in the control instruction, which may carry the timestamp to be jumped to, the corresponding operation performed by the second video recording server after obtaining the instruction may include: returning the media stream corresponding to the timestamp in the target media stream to the first video recording server 120. The control result of the instruction may include, for example, a response message for notifying that the playback has been jumped. The media stream corresponding to the timestamp may be a media stream including a frame of data corresponding to the timestamp and each frame of data after the timestamp.

[0064] Therefore, during the playback of the target media stream, the web player 110 can send a control instruction to the back-end first video recording server 120 to achieve smooth control of the target media stream and ensure a good interactive experience.

[0065] In some embodiments, when the user actively stops watching the target media stream or the target media stream is played to the end, the web page player 110 may be triggered to send a stop playback instruction to the first video recording server 120 based on the signaling transmission channel. In response to receiving the control instruction sent by the web page player 110, the first video recording server 120 may forward the stop playback instruction to the second video recording server 130 based on the first audio and video media transmission channel. Thus, the second video recording server 130 may stop returning the media stream to the first video recording server 120, and disconnect the first audio and video media transmission channel between the second video recording server 130 and the first video recording server 120, thereby releasing occupied resources.

[0066] In addition, the first video recording server 120 can also disconnect the second audio and video media transmission channel and release the occupied resources when receiving the stop playback instruction sent by the web player 110. For any unexpected interruption, such as disconnection caused by network failure, the first video recording server 120 can quickly detect and take appropriate measures to avoid resource leakage or other potential problems.

[0067] Combine the following Figure 2 , the media stream playback method of the embodiment of the present disclosure is exemplified. Here, the signaling transmission channel between the web player 110 and the first video server 120 is a WebSocket connection, the second audio and video media transmission channel between the web player 110 and the first video server 120 is a WebRTC connection, and the first audio and video media transmission channel between the first video server 120 and the second video server 130 is an RTSP connection.

[0068] refer to Figure 2 , the web page player 110 can send a play request for the target media stream to the first video server 120 based on the WebSocket connection between the web page player 110 and the first video server 120. The play request can carry necessary parameters, such as the IP address and port number of the second video server 130 where the target media stream is located, the identifier of the target media stream, the network transmission protocol type such as UDP and / or TCP type, the instruction type (such as a start play instruction, a pause play instruction, a jump instruction, etc.), etc.

[0069] After receiving the playback request, the first video recording server 120 can parse all the parameters contained therein, then create a response message, and return it to the web player 110 through the WebSocket connection, wherein the response message carries an SDP (Session Description Protocol) string, which contains key information about the media session, such as the encoding type of the media stream supported by the first video recording server 120, the network transmission protocol type such as TCP and / or UDP, the IP address and port number of the first video recording server 120 used to transmit the media stream, etc.

[0070] After receiving the response message returned by the first video recording server 120, the web player 110 can parse and obtain the IP address of the first video recording server 120 for transmitting the media stream, and based on the IP address, send a STUN (Session Traversal Utilities for NAT) request to the first video recording server 120 ( Figure 2 The STUN request is used to discover the public IP address of the web player 110 and its NAT (Network Address Translation) type, and to help penetrate the NAT device. Once the web player 110 receives the response to the STUN request from the first video server 120 ( Figure 2 If the BINDING RESP in the protocol is found, it indicates that a valid communication path has been found. At this time, the web player 110 can start the DTLS (Datagram Transport Layer Security) handshake process. DTLS is based on the TLS (Transport Layer Security) protocol and is designed specifically for UDP, providing a secure data transmission mechanism. The DTLS handshake process may include Figure 2 The DTLS handshake process can refer to the relevant technology and will not be described here. By completing the DTLS handshake, it can be ensured that the web player 110 and the first video server 120 can exchange data on an encrypted and verified channel, thereby protecting the security of the transmitted media stream. Finally, with the successful completion of the DTLS handshake, the WebRTC connection is officially established.

[0071] Further, the first video recording server 120 may send an RTSP request to the corresponding second video recording server 130 based on the IP address and port number of the second video recording server 130 where the target media stream carried in the play request is located, wherein the RTSP request corresponds to Figure 2 OPTIONS (options) request, DESCRIPTION (description) request, VIDEO SETUP (video settings) request, AUDIO SETUP (audio settings) request, PLAY (playback) request, and receive the response message corresponding to the RTSP request returned by the second video server (corresponding to Figure 2 200OK, SDP in the RTSP request to establish an RTSP connection. Among them, any RTSP request can be a data acquisition request in the above embodiment, which can carry the identifier of the target media stream. Among them, for any RTSP request and the corresponding response message, please refer to the relevant technology, which will not be repeated here. It should be noted that the response message corresponding to the DESCRIPTION request is Figure 2 The SDP response returned by the second video recording server 130 in the video recording server 130 may include the audio and video media capability information of the second video recording server 130, which is used for the first video recording server 120 to determine the first encoding type of the target media stream based on the audio and video media capability information. After the first video recording server 120 sends an RTSP request, such as the first request OPTION, a timer may be started and the connection status of the RTSP connection may be monitored. When the first video recording server 120 determines that the first video media transmission channel has not been successfully established within a preset time period (for example, an SDP response has not been obtained within a preset time period or a 200OK corresponding to PLAY has not been obtained) or the target media stream has not been obtained, the web page player 110 may be notified to terminate the current operation and display an error message to the user.

[0072] Furthermore, the second video recording server 130 may return the target media stream to the first video recording server 120 through RTP transmission in RTSP based on the identifier of the target media stream in the RTSP request.

[0073] During the RTSP interaction process, the first recording server 120 can parse the audio and video media capability information carried in the SDP response returned by the second recording server 130, and determine the first encoding type of the target media stream returned by the second recording server 130. If the first encoding type does not belong to the media encoding type supported by the web player 110 known to the first recording server 120, after the first recording server obtains the target media stream from the second recording server 130, it can convert the target media stream from being encoded in the first encoding type to being encoded in the second encoding type, wherein the media encoding types supported by the web player 110 include the second encoding type. Then, the first recording server 120 can send the target media stream encoded in the second encoding type to the web player 110 based on the SRTP transmission in WebRTC, so that the web player 110 can decode the target media stream encoded in the second encoding type, and then play the decoded target media stream.

[0074] In addition, the first video recording server 120 can select a suitable RTP transmission mode according to the network transmission protocol type specified by the web player 110, obtain the target media stream from the second video recording server 130, and adjust the data forwarding strategy.

[0075] refer to Figure 2 In the process of playing the target media stream, the web player 110 can also send a control instruction for controlling the playing state of the target media stream to the first video server 120 based on the WebSocket connection, wherein the control instruction can include any instruction of a pause play instruction, a resume play instruction, a seek instruction, etc. In response to receiving the control instruction sent by the web player, the first video server 120 can forward the control instruction to the second video server 130 based on the RTSP connection. Thus, the second video server 130 can perform corresponding operations based on the control instruction, obtain the control result corresponding to the control instruction, and return the control result corresponding to the control instruction to the first video server 120 based on the first audio and video media transmission channel.

[0076] When the user actively stops watching the target media stream or the target media stream is played to the end, the web page player 110 can be triggered to send a stop playback instruction to the first video recording server 120 based on the WebSocket connection. In response to receiving the control instruction sent by the web page player 110, the first video recording server 120 can forward the stop playback instruction to the second video recording server 130 based on the RTSP connection. Thus, the second video recording server 130 can stop returning the media stream to the first video recording server 120, and disconnect the first audio and video media transmission channel between the first video recording server 120 and the first video recording server 120, thereby releasing occupied resources.

[0077] Therefore, by using the above-mentioned flexible, efficient and well-scalable media stream transmission method, WebRTC is applied to the transmission of RTSP streams, which can bypass the restrictions of web browsers on the RTSP protocol, so that the Web-based web player 110 can seamlessly access a variety of different target media streams such as audio and video on-demand streams, so that high-quality media stream content can be smoothly viewed on various audio and video platforms without installing additional plug-ins or relying on a specific server architecture. This not only solves the problem that web browsers do not support RTSP, but also enhances cross-platform compatibility and user experience.

[0078] It is understandable that the common packets supported by the front-end decoder usually require the media stream to be encapsulated in a segmented format such as FMP4 (Fragmented MP4), and the common packet formats of the back-end are RTP, mp4 or flv format, which means that the back-end recording server needs to convert the original RTP data into FMP4 format, which not only increases the complexity of the front-end and back-end processing, but also brings additional computing overhead to the entire playback process. By adopting the above-mentioned media stream transmission method in this application, since WebRTC naturally supports the RTP protocol, after the first recording server 120 receives the RTP data packet from the second recording server 130 through the RTSP connection, it can transmit the RTP packet to the web page player 110 on the Web side without the need for additional packet conversion steps, thereby simplifying the processing flow of the first recording server 120 at the back end, reducing the burden of the first recording server 120, and greatly improving the compatibility with various RTP encapsulation formats, so that the system can support a wider range of application scenarios, while shortening the time to obtain the target media stream and improving the overall efficiency of the system.

[0079] In addition, browsers generally only support http or WebSocket protocols. Such protocols are based on TCP at the bottom layer and rely on stable network connections to continuously receive media segments. Once the network is not good and the network channel is blocked, it may cause increased delays or even interruptions in playback. The WebRTC protocol used in this application has a powerful adaptive flow control and error recovery mechanism, which can dynamically adjust the bit rate according to the current network conditions, and request the retransmission of lost data packets through technologies such as NACK (Negative Acknowledgment, negative acknowledgment), so as to ensure a smooth playback experience in an unstable network environment. In addition, if frame loss occurs in a weak network environment, especially the loss of key frames, the traditional decoding scheme may not be able to decode subsequent video segments, thereby causing playback interruption. Even if you try to discard the data of the entire slice, you may not be able to restore the normal playback process. The WebRTC protocol used in this application has strong robustness. The design takes into account the situation of frame loss. It can compensate for the lost data packets through forward error correction (FEC) and retransmission. Even if part of the data is lost, WebRTC can restore the picture quality as much as possible through other mechanisms to ensure continuous playback, thereby greatly improving the user experience. Therefore, the WebRTC protocol is used for media stream transmission in this application. Due to WebRTC's excellent ability to resist weak networks, it can maintain a relatively stable media stream quality even when the network conditions are poor, thereby significantly improving the robustness of the entire playback system and user experience.

[0080] In some embodiments, for audio and video on demand scenarios, in order to display the playback progress and support related playback operations such as pause playback, resume playback, seek, etc., the present application also proposes a web player. Correspondingly, the web player 110 in the media stream playback system can be the web player proposed in the present application. Among them, the web player includes a video tag and a progress bar. Among them, the video tag is used to play the media stream and monitor the processing timing of the event. The progress bar is used for the playback operation of the media stream.

[0081] Accordingly, the first video recording server 120 can obtain the total duration information of the target media stream from the second video recording server 130, and send the total duration information to the web player 110 based on the signaling transmission channel. Then the web player 110 can initialize the total duration information corresponding to the progress bar based on the received total duration information.

[0082] The total duration information of the target media stream may be obtained by the first recording server 120 from the second recording server 130 and sent to the web player 110 before the web player 110 displays the total duration information corresponding to the progress bar. For example, the first recording server 120 may obtain the total duration information of the target media stream from the second recording server 130 during the process of establishing the first audio and video media transmission channel, such as the RTSP connection, and send the total duration information of the target media stream to the web player 110. Accordingly, if the total duration information of the target media stream is not obtained within the preset time period after the first recording server 120 starts the timer, it can be considered that the first audio and video media transmission channel is not successfully established, so that the web player 110 can be notified to terminate the current operation and display an error message to the user.

[0083] As an example, continuing with Figure 2 For example, during the RTSP connection establishment process, the first video server 120 can send a DESCRIPTION request to the second video server 130 to obtain an SDP response returned by the second video server 130, wherein the SDP response may include the total duration information of the target media stream, and then the first video server 120 can send a message carrying the total duration information of the target media stream to the web player 110 ( Figure 2 (not shown) to asynchronously send the total duration information of the target media stream to the web player 110 via the WebSocket connection.

[0084] As another example, continuing with Figure 2 For example, you can Figure 2 The OPTIONS request and the corresponding 200OK, DESCRIPTION and the corresponding SDP response exchanged between the first video recording server 120 and the second video recording server 130 are moved to before the first video recording server 120 returns the SDP response to the web page player 110. The SDP response corresponding to the DESCRIPTION may include the total duration information of the target media stream, so that the first video recording server 120 can carry the total duration information of the target media stream in the SDP response returned to the web page player 110, so as to send the total duration information of the target media stream to the web page player 110 through the WebSocket connection.

[0085] It should be noted that the above two examples are only illustrative. In actual applications, the first recording server 120 can also obtain the total duration information of the target media stream from the second recording server 130 in other ways, and send the total duration information of the target media stream to the web player 110. The first recording server 120 only needs to obtain and send the total duration information corresponding to the progress bar from the second recording server 130 to the web player 110 before the web player 110 displays the total duration information.

[0086] In some embodiments, the current playback position of the progress bar may be updated during the process of the web player 110 playing the target media stream. The current playback position of the progress bar indicates the position of a frame of data currently being played in the total duration of the target media stream, which may be expressed in the form of time, indicating how much time has passed since the start of the target media stream playback to the current moment.

[0087] As an example, the current playback position of the progress bar can be updated in the following way:

[0088] During the playback of the target media stream, the first video recording server 120 sends the first current playback time of the target media stream to the web player 110 in real time or at a preset time interval based on the signaling transmission channel. The first current playback time may be a timestamp corresponding to a frame of data currently being played determined by the first video recording server 120 according to the actual playback progress of the target media stream. The frame of data is a frame of data in the target media stream.

[0089] The web player 110 starts to load the target media stream, which can trigger the loadeddata (data loading completed) event of the video tag. At this time, the web player 110 has parsed part of the media data internally, and the current playback position can be determined. When the loadeddata event is triggered, the web player 110 starts a timer, and regularly verifies the second current playback time recorded by the web player 110 itself based on the first current playback time sent by the first video recording server 120, that is, compares whether the two current playback times are consistent, and updates the current playback position of the progress bar based on the verification result. Among them, when the two current playback times are consistent, the web player 110 can maintain the original current playback position, and when the two current playback times are inconsistent, the web player 110 can update the current playback position of the progress bar based on the first current playback time. Among them, the second current playback time can be a timestamp recorded by the web player 110, which indicates the timestamp corresponding to a frame of data currently being played.

[0090] In some embodiments, during the process of the web player 110 playing the target media stream, the user is allowed to jump to the corresponding playing time by clicking any position on the progress bar. Then, the web player 110 can also handle the click event of the progress bar in the following manner:

[0091] When the user clicks on the progress bar, the web player 110 determines the timestamp corresponding to the click position according to the ratio of the click position of the progress bar (also called the jump position of the progress bar) to the total width of the progress bar, and sends the timestamp to the first recording server 120 through the signaling transmission channel. After receiving the timestamp, the first recording server 120 sends the timestamp to the second recording server 130. The second recording server 130 returns the media stream corresponding to the timestamp to the first recording server 120 based on the first audio and video media transmission channel. Then, the first recording server 120 sends the media stream corresponding to the timestamp to the web player 110 based on the second audio and video media transmission channel. The web player 110 plays the target media stream from the position corresponding to the timestamp based on the media stream corresponding to the timestamp. The total width of the progress bar represents the total duration of the target media stream.

[0092] The media stream corresponding to the timestamp may be a media stream including a frame of data corresponding to the timestamp and each frame of data after the timestamp. The timestamp refers to the timestamp corresponding to the click position.

[0093] The timestamp corresponding to the click position determined by the web player 110 may be carried in the jump instruction sent by the web player 110 to the first video recording server 120 .

[0094] In some embodiments, determining the timestamp corresponding to the click position according to the ratio of the click position of the progress bar to the total width of the progress bar can be achieved in the following manner:

[0095] timestamp=T*(D / W)*K

[0096] Among them, timestamp represents the timestamp corresponding to the click position; T represents the total duration of the target media stream, and the unit can be seconds; D represents the distance from the click position to the starting point of the progress bar, and the distance can be expressed in pixels; W represents the total width of the progress bar, and can be expressed in pixels; K is a value used for unit conversion, for example, when the unit of T is seconds and the unit of timestamp is milliseconds, the value of K is 1000.

[0097] Therefore, by integrating the progress bar function of the web player 110 and working closely with the first video recording server 120 at the back end, and utilizing the forwarding capability of the first video recording server 120, users can achieve precise control over the audio and video on-demand streams, and users can easily perform playback operations such as pause, resume, and seek, thereby enriching the user's viewing method and interactive experience.

[0098] The following is an explanation of the media stream playback method of the embodiment of the present application. For the implementation process and principle of the media stream playback method, reference can be made to the description of the media stream playback system of the embodiment of the present application, which will not be repeated here.

[0099] Combine the following Figure 3 , a method for playing a media stream applied to a first video recording server provided in an embodiment of the present application is described. Figure 3 A flowchart of a method for playing a media stream provided in an embodiment of the present application.

[0100] It should be noted that the media stream playback method provided in the embodiment of the present application can be executed by a media stream playback device, wherein the media stream playback device can be the first recording server, or can be configured in the first recording server, such as a recording service in the first recording server, and the present application does not impose any restrictions on this.

[0101] like Figure 3 As shown, the media stream playing method may include the following steps:

[0102] Step 301: Based on the signaling transmission channel between the web page player and the web page player, a request for playing a target media stream sent by the web page player is received.

[0103] Step 302: In response to the play request, a target media stream is obtained from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the second video recording server.

[0104] In one embodiment, the play request carries the IP address and port number of the second video recording server where the target media stream is located, and the identifier of the target media stream;

[0105] In step 302, based on the first audio and video media transmission channel between the second video recording server and the second video recording server, the target media stream is obtained from the second video recording server, which can be achieved by:

[0106] Based on the first audio and video media transmission channel, a target media stream matching the identifier is obtained from a target port of the second video server corresponding to the IP address, wherein the target port is a port corresponding to the port number of the second video server carried in the play request.

[0107] In one embodiment, based on the first audio and video media transmission channel, obtaining a target media stream matching the identifier from a target port of a second video recording server corresponding to an IP address includes:

[0108] In the process of establishing the first audio and video media transmission channel, a data acquisition request is sent to the second video recording server corresponding to the IP address, where the data acquisition request carries an identifier of the target media stream;

[0109] Based on the established first audio and video media transmission channel, a target media stream matching the identifier and returned by the second video recording server in response to the data acquisition request based on the target port is received.

[0110] Step 303: Send a target media stream to the web player based on the second audio and video media transmission channel between the web player and the web player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used for playing on the web player.

[0111] In one embodiment, the transmission protocol used by the signaling transmission channel is WebSocket, the transmission protocol used by the first audio and video media transmission channel is the real-time streaming protocol RTSP, and the transmission protocol used by the second audio and video media transmission channel is the web instant communication protocol WebRTC.

[0112] In one embodiment, before step 303, the following steps may also be included:

[0113] Determine a first encoding type of a target media stream returned by the second video recording server;

[0114] In the case where the first encoding type does not belong to the media encoding type supported by the web page player, converting the target media stream from being encoded with the first encoding type to being encoded with the second encoding type, wherein the media encoding type supported by the web page player includes the second encoding type; accordingly, step 303 may include:

[0115] Based on the second audio and video media transmission channel, the target media stream encoded by the second encoding type is sent to the web player.

[0116] In one embodiment, determining the first encoding type of the target media stream returned by the second video recording server includes:

[0117] In the process of establishing the first audio and video media transmission channel, receiving the audio and video media capability information of the second video recording server returned by the second video recording server;

[0118] Based on the audio and video media capability information, a first encoding type of the target media stream is determined.

[0119] In one embodiment, the method for playing a media stream may further include:

[0120] During the playback of the target media stream, in response to receiving a control instruction sent by the web player based on the signaling transmission channel, forwarding the control instruction to the second recording server based on the first audio and video media transmission channel, wherein the control instruction is used to control the playback state of the target media stream;

[0121] In response to receiving a control result corresponding to the control instruction returned by the second video recording server based on the first audio and video media transmission channel, the control result is sent to the web player based on the signaling transmission channel.

[0122] In one embodiment, the web page player includes a progress bar, and the media stream playing method further includes:

[0123] In the process of establishing the first audio and video media transmission channel, obtaining total duration information of the target media stream from the second video recording server;

[0124] Based on the signaling transmission channel, the total duration information is sent to the web page player, wherein the total duration information is used for the total duration information corresponding to the progress bar of the web page player initialization.

[0125] In one embodiment, the method for playing a media stream may further include:

[0126] During the playback of the target media stream, the first current playback time of the target media stream is sent to the web player based on the signaling transmission channel; wherein the first current playback time is used by the web player to verify the recorded second current playback time, and update the current playback position of the progress bar based on the verification result.

[0127] In one embodiment, the control instruction includes a jump instruction, and the jump instruction carries a timestamp corresponding to the jump position of the progress bar; the media stream playback method may further include:

[0128] Based on the first audio and video media transmission channel, receiving a media stream corresponding to the timestamp returned by the second video recording server;

[0129] Based on the second audio and video media transmission channel, a media stream corresponding to the timestamp is sent to the web player, wherein the media stream corresponding to the timestamp is used by the web player to play the target media stream from the position corresponding to the timestamp.

[0130] In summary, the first recording server obtains the target media stream to be played from the second recording server based on the first audio and video media transmission channel between the first recording server and the second recording server, and sends the target media stream to the web player based on the second audio and video media transmission channel between the first recording server and the web player. Even if the web player does not support the protocol used by the target media stream stored in the second recording server, such as the RTSP protocol, the target media stream can be played on the web player, thereby realizing the playback of media streams recorded by protocols that the web player of the web browser does not support, such as the RTSP protocol, without installing additional plug-ins or relying on a specific server architecture.

[0131] Combine the following Figure 4 , a method for playing a media stream applied to a web player provided in an embodiment of the present application is described. Figure 4 A flowchart of another method for playing a media stream provided in an embodiment of the present application.

[0132] It should be noted that the media stream playback method provided in the embodiment of the present application can be executed by a media stream playback device, wherein the media stream playback device can be a web player, or can be configured in a web player, and the present application does not limit this. This embodiment is described by taking the media stream playback device as a web player as an example.

[0133] like Figure 4 As shown, the media stream playing method may include the following steps:

[0134] Step 401: Send a request for playing a target media stream to the first video recording server based on a signaling transmission channel between the first video recording server and the first video recording server.

[0135] In some embodiments, the transmission protocol used by the signaling transmission channel is WebSocket.

[0136] Step 402: obtaining a target media stream from the first recording server based on a second audio and video media transmission channel between the first recording server and the second recording server, wherein the target media stream is obtained from the second recording server based on the first audio and video media transmission channel between the first recording server and the second recording server in response to a playback request, and the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols.

[0137] In some embodiments, the playback request carries the Internet Protocol IP address and port number of the second recording server where the target media stream is located, as well as an identifier of the target media stream, so that the first recording server can obtain the target media stream matching the identifier from the target port of the second recording server corresponding to the IP address based on the first audio and video media transmission channel, wherein the target port is the port corresponding to the port number.

[0138] In some embodiments, the transmission protocol used by the first audio and video media transmission channel is the real-time streaming protocol RTSP, and the transmission protocol used by the second audio and video media transmission channel is the web instant messaging WebRTC protocol.

[0139] Step 403: play the target media stream.

[0140] In some embodiments, the target media stream to be played is a media stream encoded with a second encoding type obtained by the first recording server transcoding the target media stream encoded with a first encoding type. The first encoding type is the encoding type of the target media stream returned by the second recording server, and the first encoding type does not belong to the media encoding type supported by the web player. The media encoding types supported by the web player include the second encoding type.

[0141] In some embodiments, during the playback of the target media stream, the web page player may also send a control instruction for controlling the playback status of the target media stream to the first recording server based on the signaling transmission channel, and receive a control result corresponding to the control instruction returned by the first recording server based on the signaling transmission channel.

[0142] In some embodiments, the web player includes a progress bar. The web player can obtain the total duration information of the target media stream from the first recording server based on the signaling transmission channel, and initialize the total duration information of the progress bar based on the total duration information of the target media stream.

[0143] In some embodiments, during the playback of the target media stream, the web player can obtain a first current playback time of the target media stream from the first recording server based on a signaling transmission channel, and based on the first current playback time, verify the second current playback time recorded by the web player, and update the current playback position of the progress bar based on the verification result.

[0144] In some embodiments, the control instruction sent by the web player may include a jump instruction, the jump instruction carries a timestamp corresponding to the jump position of the progress bar, and the web player may also obtain the media stream corresponding to the timestamp from the first recording server based on the second audio and video media transmission channel, and play the target media stream from the position corresponding to the timestamp based on the media stream corresponding to the timestamp. The media stream corresponding to the timestamp obtained from the first recording server is obtained by the first recording server from the second recording server based on the first audio and video media transmission channel.

[0145] In summary, the web page player sends a request for playing the target media stream to the first recording server based on the signaling transmission channel between the web page player and the first recording server; based on the second audio and video media transmission channel between the web page player and the first recording server, the target media stream is obtained from the first recording server in response to the play request, based on the first audio and video media transmission channel between the web page player and the second recording server, and the transmission protocols used by the first audio and video media transmission channel and the second audio and video media transmission channel are different; the target media stream is played. As a result, even if the web page player does not support the protocol used by the target media stream stored in the second recording server, such as the RTSP protocol, the target media stream can be played on the web page player, thereby realizing the playback of media streams recorded by protocols that it does not support, such as the RTSP protocol, in the web page player of the web browser, without the need to install additional plug-ins or rely on a specific server architecture.

[0146] In order to implement the above embodiment, the present disclosure proposes a media stream playback device, wherein the media stream playback device may be a first video recording server, or may be configured in the first video recording server, and the present application does not limit this.

[0147] Figure 5 A schematic diagram of the structure of a media stream playback device provided in an embodiment of the present disclosure.

[0148] like Figure 5 As shown, the media stream playback device 500 includes: a receiving module 510 , an acquiring module 520 and a sending module 530 .

[0149] The receiving module 510 is used to receive a request for playing a target media stream sent by a web player based on a signaling transmission channel with the web player;

[0150] The acquisition module 520 is used to obtain the target media stream from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the second video recording server in response to the play request;

[0151] The sending module 530 is used to send a target media stream to the web player based on the second audio and video media transmission channel between the web player and the web player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used to be played on the web player.

[0152] In some embodiments, the play request carries the Internet Protocol IP address and port number of the second video recording server where the target media stream is located, and the identifier of the target media stream; the acquisition module 520 is used to:

[0153] Based on the first audio and video media transmission channel, a target media stream matching the identifier is obtained from a target port of the second video recording server corresponding to the IP address, wherein the target port is a port corresponding to the port number.

[0154] In some embodiments, the media stream playback device further includes:

[0155] A determination module, used to determine a first encoding type of a target media stream returned by the second video recording server;

[0156] a processing module, configured to convert the target media stream from being encoded with the first encoding type to being encoded with the second encoding type when the first encoding type does not belong to the media encoding types supported by the web page player, wherein the media encoding types supported by the web page player include the second encoding type;

[0157] The sending module 530 is used for:

[0158] Based on the second audio and video media transmission channel, the target media stream encoded by the second encoding type is sent to the web player.

[0159] In some embodiments, the determining module is configured to:

[0160] In the process of establishing the first audio and video media transmission channel, receiving the audio and video media capability information of the second video recording server returned by the second video recording server;

[0161] Based on the audio and video media capability information, a first encoding type of the target media stream is determined.

[0162] In some embodiments, the sending module 530 is further configured to:

[0163] During the playback of the target media stream, in response to receiving a control instruction sent by the web player based on the signaling transmission channel, forwarding the control instruction to the second recording server based on the first audio and video media transmission channel, wherein the control instruction is used to control the playback state of the target media stream;

[0164] In response to receiving a control result corresponding to the control instruction returned by the second video recording server based on the first audio and video media transmission channel, the control result is sent to the web player based on the signaling transmission channel.

[0165] In some embodiments, the web page player includes a progress bar, and the acquisition module 520 is further used to:

[0166] In the process of establishing the first audio and video media transmission channel, obtaining total duration information of the target media stream from the second video recording server;

[0167] The sending module 530 is further used to send the total duration information to the web player based on the signaling transmission channel, wherein the total duration information is used for the total duration information corresponding to the progress bar of the web player initialization.

[0168] In some embodiments, the sending module 530 is further configured to:

[0169] During the playback of the target media stream, the first current playback time of the target media stream is sent to the web player based on the signaling transmission channel; wherein the first current playback time is used by the web player to verify the recorded second current playback time, and update the current playback position of the progress bar based on the verification result.

[0170] In some embodiments, the control instruction includes a jump instruction, and the jump instruction carries a timestamp corresponding to the jump position of the progress bar; the receiving module 510 is further used to:

[0171] Based on the first audio and video media transmission channel, receiving a media stream corresponding to the timestamp returned by the second video recording server;

[0172] The sending module 530 is further used to: send the media stream corresponding to the timestamp to the web player based on the second audio and video media transmission channel, wherein the media stream corresponding to the timestamp is used by the web player to play the target media stream from the position corresponding to the timestamp.

[0173] In some embodiments, the transmission protocol used by the signaling transmission channel is WebSocket, the transmission protocol used by the first audio and video media transmission channel is the real-time streaming protocol RTSP, and the transmission protocol used by the second audio and video media transmission channel is the web instant communication protocol WebRTC.

[0174] It should be noted that, for the implementation process and principle of the media stream playback device, reference may be made to the description of the media stream playback system in the embodiment of the present application, which will not be described in detail here.

[0175] The media stream playback device of the disclosed embodiment obtains the target media stream to be played from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the web page player, and sends the target media stream to the web page player based on the second audio and video media transmission channel between the web page player, so that even if the web page player does not support the protocol adopted by the target media stream stored in the second video recording server, such as the RTSP protocol, the target media stream can be played on the web page player, thereby achieving the playback of media streams recorded by protocols that the web page player of the web browser does not support, such as the RTSP protocol, without installing additional plug-ins or relying on a specific server architecture.

[0176] In order to implement the above embodiment, the present disclosure proposes another media stream playback device, wherein the media stream playback device can be a web page player, or can be configured in a web page player, and the present application does not limit this.

[0177] Figure 6 A schematic diagram of the structure of another media stream playback device provided in an embodiment of the present disclosure.

[0178] like Figure 6 As shown, the media stream playing device 600 includes: a sending module 610 , an acquiring module 620 and a playing module 630 .

[0179] The sending module 610 is used to send a request for playing the target media stream to the first video recording server based on the signaling transmission channel between the first video recording server and the first video recording server;

[0180] The acquisition module 620 is configured to acquire a target media stream from the first recording server based on a second audio and video media transmission channel between the first recording server and the first recording server, wherein the target media stream is acquired from the second recording server by the first recording server in response to a playback request based on the first audio and video media transmission channel between the first recording server and the second recording server, and the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols;

[0181] The playing module 630 is used to play the target media stream.

[0182] It should be noted that, for the implementation process and principle of the media stream playback device, reference may be made to the description of the media stream playback system in the embodiment of the present application, which will not be described in detail here.

[0183] The media stream playback device of the disclosed embodiment sends a playback request for the target media stream to the first video recording server based on the signaling transmission channel between the first video recording server and the first video recording server; obtains the target media stream from the first video recording server based on the second audio and video media transmission channel between the first video recording server and the first video recording server, wherein the target media stream is obtained from the second video recording server by the first video recording server in response to the playback request based on the first audio and video media transmission channel between the first audio and video media transmission channel and the second audio and video media transmission channel, and the transmission protocols used by the first audio and video media transmission channel and the second audio and video media transmission channel are different; and plays the target media stream. As a result, even if the web page player does not support the protocol used by the target media stream stored in the second video recording server, such as the RTSP protocol, the target media stream can be played on the web page player, thereby achieving the playback of media streams recorded by protocols that it does not support, such as the RTSP protocol, on the web page player of the web browser, without the need to install additional plug-ins or rely on a specific server architecture.

[0184] In order to implement the above embodiment, the present disclosure also provides an electronic device, Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device includes:

[0185] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0186] When the processor 702 executes the program, the method for playing audio and video data provided in the above embodiment is implemented.

[0187] Furthermore, the electronic device further comprises:

[0188] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0189] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0190] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0191] The processor 702 is used to implement the method for playing audio and video data described in the above embodiment when executing the program.

[0192] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0193] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0194] The processor 702 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0195] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for playing audio and video data as described in the above embodiments is implemented.

[0196] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including: a computer program, which implements the method for playing audio and video data as described in the above embodiments when executed by a processor.

[0197] In the description of this specification, the description of the candidate terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0198] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0199] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0200] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0201] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0202] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0203] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0204] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for playing a media stream, characterized in that: The method is applied to a first video recording server, and the method includes: Based on the signaling transmission channel between the web page player and the web page player, receiving a playback request for the target media stream sent by the web page player; In response to the play request, acquiring the target media stream from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the second video recording server; Based on a second audio and video media transmission channel between the web page player, the target media stream is sent to the web page player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used to be played on the web page player.

2. The method according to claim 1, characterized in that The play request carries the Internet Protocol IP address and port number of the second video recording server where the target media stream is located, and an identifier of the target media stream; The step of acquiring the target media stream from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the second video recording server includes: Based on the first audio and video media transmission channel, the target media stream matching the identifier is obtained from the target port of the second video recording server corresponding to the IP address, wherein the target port is a port corresponding to the port number.

3. The method according to claim 1, characterized in that Before sending the target media stream to the web page player based on the second audio and video media transmission channel between the web page player and the web page player, the method further includes: Determine a first encoding type of the target media stream returned by the second video recording server; In a case where the first encoding type does not belong to the media encoding types supported by the web page player, converting the target media stream from being encoded with the first encoding type to being encoded with a second encoding type, wherein the media encoding types supported by the web page player include the second encoding type; The sending the target media stream to the web page player based on the second audio and video media transmission channel between the web page player and the web page player includes: Based on the second audio and video media transmission channel, the target media stream encoded by the second encoding type is sent to the web player.

4. The method according to claim 3, characterized in that The determining the first encoding type of the target media stream returned by the second video recording server includes: In the process of establishing the first audio and video media transmission channel, receiving the audio and video media capability information of the second video recording server returned by the second video recording server; Based on the audio and video media capability information, the first encoding type of the target media stream is determined.

5. The method according to claim 1, characterized in that: The method further comprises: During the playback of the target media stream, in response to receiving a control instruction sent by the web player based on the signaling transmission channel, forwarding the control instruction to the second recording server based on the first audio and video media transmission channel, wherein the control instruction is used to control the playback state of the target media stream; In response to receiving a control result corresponding to the control instruction returned by the second video recording server based on the first audio and video media transmission channel, the control result is sent to the web player based on the signaling transmission channel.

6. The method according to claim 5, characterized in that The web page player includes a progress bar, and the method further includes: In the process of establishing the first audio and video media transmission channel, obtaining the total duration information of the target media stream from the second video recording server; Based on the signaling transmission channel, the total duration information is sent to the web page player, wherein the total duration information is used by the web page player to initialize the total duration information corresponding to the progress bar.

7. The method according to claim 6, characterized in that The method further comprises: During the playback of the target media stream, the first current playback time of the target media stream is sent to the web player based on the signaling transmission channel; wherein the first current playback time is used by the web player to verify the recorded second current playback time, and update the current playback position of the progress bar based on the verification result.

8. The method according to claim 6, characterized in that The control instruction includes a jump instruction, and the jump instruction carries a timestamp corresponding to the jump position of the progress bar; The method further comprises: Based on the first audio and video media transmission channel, receiving the media stream corresponding to the timestamp returned by the second video recording server; Based on the second audio and video media transmission channel, the media stream corresponding to the timestamp is sent to the web player, wherein the media stream corresponding to the timestamp is used by the web player to play the target media stream from the position corresponding to the timestamp.

9. The method according to claims 1-8, characterized in that The transmission protocol used by the signaling transmission channel is WebSocket, the transmission protocol used by the first audio and video media transmission channel is the real-time streaming protocol RTSP, and the transmission protocol used by the second audio and video media transmission channel is the web instant communication protocol WebRTC.

10. A method for playing a media stream, characterized in that: The method is applied to a web page player, and the method comprises: Based on the signaling transmission channel between the first video recording server and the first video recording server, sending a request for playing the target media stream to the first video recording server; The target media stream is obtained from the first recording server based on a second audio and video media transmission channel between the first recording server and the first recording server, wherein the target media stream is obtained from the second recording server by the first recording server in response to the play request based on a first audio and video media transmission channel between the first recording server and the second recording server, and the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols; Play the target media stream.

11. A media stream playback device, characterized in that: The device is applied to a first video recording server, and the device includes: A receiving module, configured to receive a playback request for a target media stream sent by the web player based on a signaling transmission channel between the web player and the web player; an acquisition module, configured to acquire the target media stream from the second video recording server based on the first audio and video media transmission channel between the second video recording server and the second video recording server in response to the playback request; A sending module is used to send the target media stream to the web page player based on a second audio and video media transmission channel between the web page player and the web page player, wherein the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols, and the target media stream is used to be played on the web page player.

12. A media stream playback device, characterized in that: The device is applied to a web player, and comprises: A sending module, configured to send a play request for a target media stream to the first video recording server based on a signaling transmission channel between the first video recording server and the first video recording server; an acquisition module, configured to acquire the target media stream from the first recording server based on a second audio and video media transmission channel between the first recording server and the first recording server, wherein the target media stream is acquired by the first recording server from the second recording server in response to the play request based on the first audio and video media transmission channel between the first recording server and the second recording server, and the first audio and video media transmission channel and the second audio and video media transmission channel use different transmission protocols; The playing module is used to play the target media stream.

13. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 9 or 10 when executing the program.

14. A media stream playback system, characterized in that: It includes a web player, a first video server and a second video server; wherein, The web player is used to implement the method according to claim 10; The first video recording server is used to implement the method according to any one of claims 1 to 9; The second video recording server is used to store the target media stream to be played.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 or 10 is implemented.

Images