Entrance guard two-way visual voice talkback method and system based on RTSP (Real Time Streaming Protocol)

Abstract

The invention discloses an access control two-way visual voice talkback method and system based on an RTSP protocol, and relates to the real-time communication technology and the streaming media technology, and the method comprises the steps that an access control end generates a dialing event; the application platform receives the dialing event and issues the dialing event to the media server; the media server returns a forward audio and video stream playing address and a reverse audio stream pushing address; the access control end establishes an audio long link channel for the media server according to the reverse audio stream pushing address; the application platform collects audio data and reversely pushes an audio stream to the media server; the media server pushes an audio stream to the access control end in the audio long link channel through an RTSP protocol; the access control end establishes an audio and video long link channel according to the forward playing address; and the media server actively pulls the audio and video streams of the access control end in the audio and video long link channel through the RTSP protocol. The method has the advantages of efficient penetrating power, flexible expansibility and low time delay, and the real-time performance and stability of audio transmission are improved.

Description

Technical Field

[0001] The present invention relates to real-time communication technology and streaming media technology, and particularly to an access control two-way visual voice intercom method and system implemented based on the RTSP protocol. Background Art

[0002] RTSP (Real Time Streaming Protocol): Real-time Streaming Protocol, which is an application layer protocol jointly proposed by Real network and Netscape on how to effectively transmit streaming media data over an IP network.

[0003] Long connection: Also known as persistent connection. After the TCP layer handshake is successful, the connection is not immediately disconnected, and multiple messages (including heartbeats) are exchanged based on this connection until either party (client or server) of the connection actively disconnects the connection.

[0004] RTMP (Real Time Messaging Protocol): Real Time Messaging Protocol, which is a network protocol designed for real-time data communication, mainly used for audio and video and data communication between the Flash / AIR platform and streaming media / interactive servers that support the RTMP protocol.

[0005] In existing access control intercom solutions, voice intercom is mainly implemented based on the GB28181 protocol. The following problems exist in this technical solution:

[0006] (1) As a national standard protocol based on sip transmission, the GB28181 protocol does not support a rich enough variety of common media stream audio and video codec formats and has poor generality. (2) The GB28181 protocol not only covers the transmission and control of video streams but also involves multiple aspects of video surveillance systems, including device registration, authentication, management, control, etc., which may require more technical investment and time costs. (3) Due to the complexity of the GB28181 protocol and the differences in protocol implementation among different manufacturers, it is difficult to implement the voice intercom process that is compatible with different devices from different manufacturers. (4) The latency is relatively high. (5) The voice intercom process of the GB28181 protocol does not support penetration of the internal and external networks. When the device is located in the internal network, two-way intercom cannot be achieved in a public network environment. Summary of the Invention

[0007] In view of the deficiencies in the prior art, the present invention provides an access control two-way visual voice intercom method and system implemented based on the RTSP protocol, which solves the problem of penetration of the internal and external networks between the platform and the device in a public network environment, overcomes network barriers, realizes visual two-way voice intercom between the access control and the platform, and utilizes the low latency feature of rtsp to reduce the dialogue latency between the two parties and improve the user's call experience.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] An access control two-way visual voice intercom method implemented based on the RTSP protocol, comprising the following steps:

[0010] The access control end generates a dialing event;

[0011] The application platform receives the dialing event and forwards it to the media server;

[0012] The media server returns the forward audio-video stream playback address and the reverse audio stream push address;

[0013] The access control end establishes an audio long link channel with the media server according to the reverse audio stream push address;

[0014] The application platform collects audio data and reversely pushes the audio stream to the media server;

[0015] The media server pushes the audio stream to the access control end through the RTSP protocol in the audio long link channel;

[0016] The access control end establishes an audio-video long link channel according to the forward playback address;

[0017] The media server actively pulls the audio-video stream of the access control end through the RTSP protocol in the audio-video long link channel.

[0018] To optimize the above technical solutions, the specific measures taken also include:

[0019] Further, the specific process of the application platform collecting audio data and reversely pushing the audio stream to the media server is as follows:

[0020] The application platform collects audio data on the client side, and the client side pushes the collected audio data to the media server based on the RTMP protocol;

[0021] The specific process of the client side pushing the collected audio data to the media server based on the RTMP protocol includes:

[0022] The client side and the media server perform three-way handshaking based on the RTMP protocol. After the handshaking is completed, a connection is established between the client side and the media server, a network stream is established between the client side and the media server, and the client side pushes the audio stream to the media server.

[0023] Further, the media server pushing the audio stream to the access control end through the RTSP protocol in the audio long link channel is specifically as follows:

[0024] The media server converts the RTMP protocol to the RTSP protocol, and the media server sends a DESCRIBE message to the access control terminal to request media stream description information;

[0025] The access control terminal responds to the media server initialization information;

[0026] The media server sends a SETUP message to the access control terminal once to specify the audio stream transmission method, and adds an extended custom field backchannel to indicate that this stream is a reverse audio stream; the media server generates a session ID;

[0027] The access control terminal responds to the session ID;

[0028] The media server sends a PLAY message to the access control terminal to request playback;

[0029] The media server pushes the reverse audio stream to the access control terminal;

[0030] The media server or the access control terminal sends a TEARDOWN message to the other party to abort the playback.

[0031] Further, the specific process for the media server to actively pull the audio and video streams of the access control terminal through the RTSP protocol in the audio and video long connection channel is as follows:

[0032] The media server sends a DESCRIBE message to the access control terminal to request media stream description information;

[0033] The access control terminal responds to the media server initialization information;

[0034] The media server sends a SETUP message to the access control terminal twice to specify the audio stream and video stream transmission methods, and adds extended custom fields video and audio to indicate that this stream is a forward audio and video stream; the media server generates a session ID;

[0035] The access control terminal responds to the session ID;

[0036] The media server sends a PLAY message to the access control terminal to request playback;

[0037] The access control terminal pushes the forward audio and video stream to the media server;

[0038] The media server or the access control terminal sends a TEARDOWN message to the other party to abort the playback.

[0039] Further, the specific process for the three-way handshake between the client and the media server based on the RTMP protocol is as follows:

[0040] The client sends the version number C0 and the random string C1 to the media server;

[0041] After the media server receives version number C0 or random string C1, it sends response version number S0 and random string S1 to the client;

[0042] After the client receives the response version number S0 and random string S1 from the media server, it sends the client's final response data C2 to the media server. After the media server receives the client's final response data C2, it sends the media server's final response data S2 to the client;

[0043] After the client and the media server respectively receive the other party's final response data S2 and C2, the handshake is completed.

[0044] Furthermore, establishing a connection between the client and the media server specifically is as follows:

[0045] The client sends a Connect instruction to the media server to request establishing a connection with the media server;

[0046] After the media server receives the Connect instruction, it sends a window protocol to the client and simultaneously connects to the specified application;

[0047] The media server sends a protocol for setting bandwidth to the client;

[0048] After the client processes the bandwidth protocol, it sends a protocol for confirming the window size to the media server;

[0049] The media server sends a user control information instruction Stream Begin to the client, indicating the start of stream transmission;

[0050] The media server sends a user message instruction _result to notify the client of the connection status.

[0051] Furthermore, establishing a network stream between the client and the media server specifically is as follows:

[0052] The client sends a createStream instruction to the media server;

[0053] After the media server receives the createStream instruction, it successfully creates a network stream and sends a _result instruction to notify the client.

[0054] Furthermore, the client pushing an audio stream to the media server specifically is as follows:

[0055] The client sends a publish instruction to the media server, telling the media server that it is about to push a stream;

[0056] The media server returns an onStatus instruction, reporting the status of the stream and error information;

[0057] The client sends a metaData instruction, which contains the metadata of the audio stream;

[0058] The client sends the audio stream.

[0059] The present invention also proposes an access control two-way visual voice intercom system implemented based on the RTSP protocol, including: an access control end, an application platform, and a media server;

[0060] The access control end generates a dialing event;

[0061] The application platform receives the dialing event and forwards it to the media server;

[0062] The media server returns the forward audio-video stream playback address and the reverse audio stream push address;

[0063] The access control end establishes an audio long connection channel with the media server according to the reverse audio stream push address;

[0064] The application platform collects audio data and reversely pushes the audio stream to the media server;

[0065] The media server pushes the audio stream to the access control end through the RTSP protocol in the audio long connection channel;

[0066] The access control end establishes an audio-video long connection channel according to the forward playback address;

[0067] The media server actively pulls the audio-video stream of the access control end through the RTSP protocol in the audio-video long connection channel.

[0068] The beneficial effects of the present invention are:

[0069] (1) High penetration ability: The access control actively establishes a TCP long connection with the media service to break through the network barrier between the internal and external networks, providing a transmission channel for the forward audio-video stream and the reverse audio stream between the platform and the access control;

[0070] (2) Flexible scalability: Based on the custom fields extended in the rtsp protocol, the addresses of the forward audio-video stream and the reverse audio stream are identified, and the rtsp signaling and two-way media streams are transmitted in the long connection channel;

[0071] (3) Low-latency transmission: Based on the RTSP protocol, a voice and video call experience with higher real-time performance and lower latency is provided.

[0072] (4) Reverse push mechanism: The platform notifies the access control device to actively establish a long connection through the reverse push address, rather than waiting for the platform's request, improving the real-time performance and stability of audio transmission. Description of the Drawings

[0073] Figure 1Flowchart of the access control two-way visual voice intercom method implemented based on the RTSP protocol proposed by the present invention.

[0074] Figure 2 It is a signaling media interaction diagram of the forward audio-video stream.

[0075] Figure 3 It is a signaling media interaction diagram of the reverse audio stream. Detailed implementation manners

[0076] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0077] Embodiment 1

[0078] The present invention proposes an access control two-way visual voice intercom method implemented based on the RTSP protocol. The flowchart of this method is as Figure 1 shown, and it includes the following steps:

[0079] S1. The access control end generates a dialing event (dialing the room number);

[0080] S2. The application platform receives the dialing event and forwards it to the media server;

[0081] S3. The media server returns the forward audio-video stream playback address and the reverse audio stream push address;

[0082] S4. The access control end establishes an audio long connection channel with the media server according to the reverse audio stream push address;

[0083] S5. The application platform collects audio data and reversely pushes the audio stream to the media server; specifically:

[0084] The application platform collects audio data on the client (web client or CS client), and the client pushes the collected audio data to the media server based on the RTMP protocol;

[0085] The specific process of the client pushing the collected audio data to the media server based on the RTMP protocol includes:

[0086] The client and the media server perform three-way handshaking based on the RTMP protocol:

[0087] The client sends the version number C0 and the random string C1 to the media server;

[0088] After the media server receives version number C0 or random string C1, it sends response version number S0 and random string S1 to the client;

[0089] After the client receives the response version number S0 and random string S1 from the media server, it sends the client's final response data C2 to the media server. After the media server receives the client's final response data C2, it sends the media server's final response data S2 to the client;

[0090] After the client and the media server respectively receive each other's final response data S2 and C2, the handshake is completed.

[0091] After the handshake is completed, a connection is established between the client and the media server:

[0092] The client sends a Connect instruction to the media server to request a connection with the media server;

[0093] After the media server receives the Connect instruction, it sends the window protocol to the client and simultaneously connects to the specified application;

[0094] The media server sends a protocol for setting the bandwidth to the client;

[0095] After the client processes the bandwidth protocol, it sends a protocol for confirming the window size to the media server;

[0096] The media server sends the user control information instruction Stream Begin to the client, indicating the start of the stream transmission;

[0097] The media server sends the user message instruction _result to notify the client of the connection status.

[0098] A network stream is established between the client and the media server:

[0099] The client sends a createStream instruction to the media server;

[0100] After the media server receives the createStream instruction, it successfully creates a network stream and sends a _result instruction to notify the client.

[0101] The client pushes an audio stream to the media server:

[0102] The client sends a publish instruction to the media server, telling the media server that it is about to push a stream;

[0103] The media server returns an onStatus instruction, reporting the status and error information of the stream;

[0104] The client sends a metaData instruction, containing the metadata of the audio stream;

[0105] The client sends an audio stream.

[0106] S6. The media server pushes the audio stream to the access control terminal through the RTSP protocol in the audio long connection channel. Specifically:

[0107] The media server converts the RTMP protocol to the RTSP protocol, and the media server sends a DESCRIBE message to the access control terminal to request media stream description information.

[0108] The access control terminal responds to the media server initialization information.

[0109] The media server sends a SETUP message to the access control terminal once to specify the audio stream transmission method, and adds an extended custom field backchannel to indicate that this stream is a reverse audio stream. The media server generates a session ID.

[0110] The access control terminal responds to the session ID.

[0111] The media server sends a PLAY message to the access control terminal to request playback.

[0112] The media server pushes the reverse audio stream to the access control terminal.

[0113] The media server or the access control terminal sends a TEARDOWN message to the other party to abort playback.

[0114] The signaling media interaction diagram of the reverse audio stream is as Figure 3 shown.

[0115] S7. The access control terminal establishes an audio and video long connection channel according to the forward playback address.

[0116] S8. The media server actively pulls the audio and video stream of the access control terminal through the RTSP protocol in the audio and video long connection channel. The specific process is as follows:

[0117] The media server sends a DESCRIBE message to the access control terminal to request media stream description information.

[0118] The access control terminal responds to the media server initialization information.

[0119] The media server sends two SETUP messages to the access control terminal to specify the audio stream and video stream transmission methods, and adds extended custom fields video and audio to indicate that this stream is a forward audio and video stream. The media server generates a session ID.

[0120] The access control terminal responds to the session ID.

[0121] The media server sends a PLAY message to the access control terminal to request playback.

[0122] The access control terminal pushes the forward audio-video stream to the media server;

[0123] The media server or the access control terminal sends a TEARDOWN message to the other party to terminate the playback.

[0124] The signaling media interaction diagram of the forward audio-video stream is as Figure 2 shown.

[0125] Embodiment 2

[0126] The present invention proposes an access control two-way visual voice intercom system implemented based on the RTSP protocol corresponding to the method of Embodiment 1, including: an access control terminal, an application platform, and a media server;

[0127] The access control terminal generates a dialing event;

[0128] The application platform receives the dialing event and sends it to the media server;

[0129] The media server returns the forward audio-video stream playback address and the reverse audio stream push address;

[0130] The access control terminal establishes an audio long connection channel with the media server according to the reverse audio stream push address;

[0131] The application platform collects audio data and reversely pushes the audio stream to the media server;

[0132] The media server pushes the audio stream to the access control terminal through the RTSP protocol in the audio long connection channel;

[0133] The access control terminal establishes an audio-video long connection channel according to the forward playback address;

[0134] The media server actively pulls the audio-video stream of the access control terminal through the RTSP protocol in the audio-video long connection channel.

[0135] The implementation methods of each module and module function in the system are exactly the same as the steps of the method in Embodiment 1, so they will not be elaborated here.

[0136] Those of ordinary skill in the art can realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in the present application, they can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0137] The above are only the preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. Any technical solutions falling within the concept of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A two-way visual voice intercom method for access control implemented based on the RTSP protocol, characterized in that, It includes the following steps: The access control terminal generates a dialing event; The application platform receives the dialing event and forwards it to the media server; The media server returns the forward audio - video stream playback address and the reverse audio stream pushing address; The access control terminal establishes an audio long - link channel with the media server according to the reverse audio stream pushing address; The application platform collects audio data and pushes the audio stream to the media server in the reverse direction; The media server pushes the audio stream to the access control terminal through the RTSP protocol in the audio long - link channel; The access control terminal establishes an audio - video long - link channel according to the forward playback address; The media server actively pulls the audio - video stream of the access control terminal through the RTSP protocol in the audio - video long - link channel.

2. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 1, characterized in that, The specific process of the application platform collecting audio data and pushing the audio stream to the media server in the reverse direction is as follows: The application platform collects audio data on the client side, and the client pushes the collected audio data to the media server based on the RTMP protocol; The specific process of the client pushing the collected audio data to the media server based on the RTMP protocol includes: The client and the media server perform three - way handshakes based on the RTMP protocol. After the handshakes are completed, a connection is established between the client and the media server, a network stream is established between the client and the media server, and the client pushes the audio stream to the media server.

3. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 1, characterized in that, The specific process of the media server pushing the audio stream to the access control terminal through the RTSP protocol in the audio long - link channel is as follows: The media server converts the RTMP protocol to the RTSP protocol, and the media server sends a DESCRIBE message to the access control terminal to request media stream description information; The access control terminal responds to the media server initialization information; The media server sends a SETUP message to the access control terminal once to specify the audio stream transmission method, and adds an extended custom field "backchannel" to indicate that this stream is a reverse audio stream; The media server generates a sessionID; The access control terminal responds to the sessionID; The media server sends a PLAY message to the access control terminal to request playback; The media server pushes the reverse audio stream to the access control terminal; The media server or the access control terminal sends a TEARDOWN message to the other party to abort the playback.

4. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 1, wherein, The specific process of the media server actively pulling the audio - video stream of the access control terminal through the RTSP protocol in the audio - video long - link channel is as follows: The media server sends a DESCRIBE message to the access control terminal to request media stream description information; The access control terminal responds to the media server initialization information; The media server sends a SETUP message to the access control terminal twice to specify the audio stream and video stream transmission methods, and adds extended custom fields "video" and "audio" to indicate that this stream is a forward audio - video stream; the media server generates a sessionID; The access control terminal responds to the sessionID; The media server sends a PLAY message to the access control terminal to request playback; The access control terminal pushes the forward audio - video stream to the media server; The media server or the access control terminal sends a TEARDOWN message to the other party to abort the playback.

5. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 2, wherein, The specific process of the client and the media server performing three - way handshakes based on the RTMP protocol is as follows: The client sends the version number C0 and a random string C1 to the media server; After the media server receives version number C0 or random string C1, it sends response version number S0 and random string S1 to the client; After the client receives the response version number S0 and random string S1 from the media server, it sends the client's final response data C2 to the media server. After the media server receives the client's final response data C2, it sends the media server's final response data S2 to the client; After the client and the media server respectively receive the other party's final response data S2 and C2, the handshake is completed.

6. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 2, characterized in that The establishment of a connection between the client and the media server is specifically as follows: The client sends a Connect instruction to the media server to request to establish a connection with the media server; After receiving the Connect instruction, the media server sends a window protocol to the client and simultaneously connects to the specified application; The media server sends a protocol for setting the bandwidth to the client; After processing the bandwidth protocol, the client sends a protocol for confirming the window size to the media server; The media server sends a user control information instruction Stream Begin to the client to indicate the start of the stream transmission; The media server sends a user message instruction _result to notify the client of the connection status.

7. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 2, characterized in that, The establishment of a network stream between the client and the media server is specifically as follows: The client sends a createStream instruction to the media server; After receiving the createStream instruction, the media server successfully creates a network stream and sends a _result instruction to notify the client.

8. The access control two-way visual voice intercom method implemented based on the RTSP protocol according to claim 2, characterized in that, The client's pushing of the audio stream to the media server is specifically as follows: The client sends a publish instruction to the media server to tell the media server that it is about to push the stream; The media server returns an onStatus instruction to report the status and error information of the stream; The client sends a metaData instruction containing the metadata of the audio stream; The client sends the audio stream.

9. An access control two-way visual voice intercom system implemented based on the RTSP protocol, characterized in that, Including: The access control terminal, the application platform, and the media server; The access control terminal generates a dialing event; The application platform receives the dialing event and forwards it to the media server; The media server returns the forward audio and video stream playback address and the reverse audio stream push address; The access control terminal establishes an audio long connection channel with the media server according to the reverse audio stream push address; The application platform collects audio data and reversely pushes the audio stream to the media server; The media server pushes the audio stream to the access control terminal through the RTSP protocol in the audio long connection channel; The access control terminal establishes an audio and video long connection channel according to the forward playback address; The media server actively pulls the audio and video stream of the access control terminal through the RTSP protocol in the audio and video long connection channel.

Images