Method and device for managing and scheduling multi-channel video streams based on SSRC

Abstract

The present invention discloses a method and device for managing and scheduling multiple video streams based on SSRC. When a device management server (DMS) receives a video stream acquisition request initiated by a client, it requests an SSRC that has not been occupied by other clients from a data management module (DBS), requests a video stream from a device (such as a camera), and then returns the SSRC of the stream to the client. A media server (MMS) only needs to open a stream sending port and a stream receiving port to send and receive all video streams, and uses SSRC to distinguish different streams. The client can use URLs (video stream acquisition network addresses) carrying different SSRCs to watch real-time videos or recorded playback videos generated by different devices, thereby avoiding the network information security risks of obtaining multiple video streams by traditional methods due to too many ports opened by MMS. By effectively distinguishing the source of the video stream, the cross-stream phenomenon can be prevented at the same time, and the call initiator can be prevented from receiving the video stream from the wrong source.

Description

Technical Field

[0001] The present invention relates to the technical field of streaming media scheduling, and in particular to a method and device for managing and scheduling multi-channel video streams based on SSRC. Background Art

[0002] The video stream management method commonly used by current video platforms is: on the host where the media server is located, a range of ports is allocated, and for each video stream, at least one receiving port and one sending port are opened to transmit the video stream, and different video streams are distinguished by port type. This approach consumes a large amount of port resources, and the platform needs to frequently allocate and recycle ports. When the number of video streams is relatively large, the port resources in the range will be exhausted, making it impossible to start new video stream transmission tasks. In addition, the more ports a server opens, the more vulnerable it is to network attacks, which does not meet the current requirements for network information security. Summary of the invention

[0003] The present invention aims to reduce port opening and improve data transmission security in multi-channel video stream transmission, and provides a method and device for managing and scheduling multi-channel video streams based on SSRC.

[0004] To achieve this object, the present invention adopts the following technical solutions:

[0005] A method for managing and scheduling multiple video streams based on SSRC is provided, comprising:

[0006] After receiving the video stream acquisition request initiated by the client, the device management server DMS requests the data management module DBS to allocate the SSRC and the media server MMS for sending and receiving streams;

[0007] After the DMS carries the allocated SSRC, MMS information and the video stream acquisition request to establish a call with the device, the device starts to send the stream to the MMS. The RTP packet of the video stream sent by the device carries the SSRC;

[0008] The central management module CMS generates a URL for obtaining the video stream according to the streaming IP and streaming port of the MMS and the SSRC, and returns it to the client. The client initiates a request to the URL, and the MMS sends the stream to the client.

[0009] Preferably, the video stream acquisition request initiated by the client carries a video channel ID. After the DBS completes each SSRC allocation, the DMS generates a video stream acquisition task corresponding to the video channel ID and adds it to a video stream acquisition task list;

[0010] Before forwarding the video stream acquisition request initiated by the client to the DMS, the CMS first queries the video stream acquisition task list to see whether the video stream acquisition task corresponding to the video channel ID already exists.

[0011] If yes, directly return the URL corresponding to the video channel ID generated in advance to the client;

[0012] If not, the video stream acquisition request is forwarded to the DMS registered with the video channel.

[0013] As a preferred method, the DBS allocates SSRC as follows:

[0014] Randomly assign an unenabled SSRC from the SSRC session table stored in the database, and then set the occupancy state of the SSRC to enabled;

[0015] The SSRC session table stores a plurality of SSRC session data in columns, each of which includes an SSRC value, a video channel ID, an ID of a DMS registered with the video channel, an SSRC occupancy status, and an SSRC occupancy status change timestamp.

[0016] Preferably, when the video stream acquisition request initiated by the client is a real-time video stream acquisition request, the data content in the video stream acquisition request includes the client ID of the client initiating the request, the video channel ID of the video transmission channel to be established between the client and the device, the request timestamp and the video stream transmission mode of the device;

[0017] When the video stream acquisition request initiated by the client is a front-end video playback request, the data content in the video stream acquisition request includes the client ID of the client initiating the request, the video channel ID of the video transmission channel to be established between the client and the device, the request timestamp, the video stream transmission mode of the device, the video playback start time and the video playback end time.

[0018] Preferably, if the video stream obtained by the client from the MMS is a real-time video stream generated by the device, the process of the client obtaining the video stream from the MMS specifically includes the following steps:

[0019] After receiving the video stream acquisition request forwarded by the CMS to the client, the DMS requests the DBS to allocate an SSRC and an MMS for sending and receiving streams;

[0020] The DMS carries the SSRC assigned by the DBS, the MMS's receiving IP, receiving port, and video stream transmission mode to initiate a formal request for obtaining the video stream to the device where the video transmission channel is located;

[0021] After receiving the formal request for obtaining the video stream, the device returns a status response code indicating that the request is successful to the DMS;

[0022] After receiving the status response code, the DMS sends an ACK character to the device to confirm receipt, and generates response data carrying SSRC, MMS ID, MMS streaming IP, and streaming port and returns it to the CMS; at the same time, the video channel starts to stream to the MMS after receiving the ACK response;

[0023] After receiving the response data, the CMS generates a corresponding URL according to the live broadcast protocol and the received SSRC and the streaming IP and streaming port of the MMS and returns it to the client;

[0024] Finally, the client obtains the video stream corresponding to the SSRC from the corresponding MMS according to the path guidance of the MMS streaming IP and streaming port carried in the returned URL.

[0025] Preferably, the method for managing and scheduling multiple video streams based on SSRC also includes an SSRC recovery process, specifically:

[0026] MMS parses the SSRC from the URL sent by the client, then transcodes the video stream corresponding to the SSRC into a live broadcast protocol specified by the URL (common network streaming media transmission protocols, such as RTSP protocol, RTMP protocol, HTTP-FLV protocol, HLS protocol), and then sends the video stream to the client from the streaming port specified by the URL;

[0027] The MMS monitors the streaming status, and when detecting a streaming abnormality, adds the SSRC corresponding to the video stream with the streaming abnormality to a recycling list;

[0028] The MMS sends each SSRC in the recycling list to the DBS, and initiates a stop flow request to the DMS corresponding to each SSRC in the recycling list;

[0029] The DMS sends a stop streaming instruction to the corresponding device. After receiving the stop streaming instruction, the device stops sending the video stream corresponding to the SSRC and returns a stop streaming success signal;

[0030] After receiving the signal of successfully stopping the streaming, the DMS notifies the DBS to recycle the SSRC corresponding to the stopped video stream.

[0031] Preferably, the types of stream transmission exceptions include any one or more of an error in the format of the transmitted video stream, an abnormal transmission bit rate, an abnormal transcoding, and a timeout in the client receiving the video stream.

[0032] Preferably, if the video stream obtained by the client from the MMS is a video stream generated by video playback, the process of the client obtaining the video stream from the MMS specifically includes:

[0033] After receiving the video stream acquisition request forwarded by the CMS to the client, the DMS generates a SessionID and records it in the corresponding session data in the SSRC session table in the DBS, and requests the DBS to allocate an SSRC and an MMS for sending and receiving video streams;

[0034] The DMS carries the SSRC assigned by the DBS, the receiving IP, receiving port and SessionID of the MMS, the national standard transmission mode, the video playback start time and the video playback end time contained in the video stream acquisition request initiated by the client to initiate a formal video playback request to the device to establish the video transmission channel;

[0035] After receiving the formal request for video playback, the device returns a status response code indicating that the request is successful to the DMS;

[0036] After receiving the status response code, the DMS returns an ACK character to the device to confirm receipt, and generates response data carrying the video channel ID, SSRC, SessionID, MMS ID, MMS streaming IP, and streaming port, and returns it to the CMS; at the same time, after receiving the ACK response, the device sends the video playback video stream carrying the SSRC to the MMS receiving IP and receiving port;

[0037] After receiving the response data, the CMS generates a corresponding URL according to the SSRC, the MMS ID, the streaming IP, and the streaming port, and returns it to the client together with the SessionID and the video channel ID;

[0038] The client saves the SessionID, and obtains the video playback stream corresponding to the SSRC from the corresponding MMS according to the path guidance of the MMS streaming IP and streaming port carried in the returned URL.

[0039] Preferably, the process of stopping the video playback by the client includes:

[0040] The client initiates a request to the CMS to stop the video playback, which carries the client ID, SessionID, and the video channel ID;

[0041] The CMS searches the DBS for the DMS registered with the video channel corresponding to the video channel ID carried in the request, and forwards the request to stop video playback to the found DMS;

[0042] The DMS searches the DBS for the session data containing the SessionID according to the SessionID carried in the request.

[0043] If not found, an error message is returned;

[0044] If found, an INFO command is sent to the corresponding device along with the request to stop video playback, and a BYE command is sent to the device. The INFO command is a command to pause, resume, fast forward, slow down, or play at double speed, and the BYE command is used to stop video playback;

[0045] The device sends a status response code indicating successful receipt of the INFO command and the BYE command to the DMS, and stops sending the stream;

[0046] After receiving the status response code, the DMS returns an HTTP response to the CMS, and carries the video channel ID and the SessionID to request the DBS to delete the session data corresponding to the SessionID;

[0047] The CMS returns a response carrying the video channel ID and the SessionID to the client to notify that the playback of the video stream has stopped.

[0048] The present invention also provides an SSRC management and recovery device, including a data management module DBS, a device management server DMS and a media server MMS, wherein the DMS is used to allocate an SSRC to each video stream request DBS, and the MMS is used to push the video stream sent by the device to the corresponding client, and is responsible for managing the status of the video stream, and when the video stream is monitored to be abnormal, recovering the SSRC corresponding to the abnormal video stream.

[0049] The present invention has the following beneficial effects:

[0050] 1. When the device management server (DMS) receives a video stream acquisition request initiated by a client, it requests an SSRC (Synchronization source, a 32-bit identifier in the RTP protocol header, used to indicate the source of the media stream) that has not been occupied by other clients from the data management module (DBS), and requests the video stream from the device (such as a camera), and then returns the SSRC of this stream to the client. The media server (MMS) only needs to open one stream sending port and one stream receiving port to send and receive all video streams, and use SSRC to distinguish different streams. The client can use URLs (video stream acquisition network addresses) with different SSRCs to watch real-time videos and recorded playback videos generated by different devices, avoiding the network information security risks of traditional methods of obtaining multiple video streams due to too many ports opened by MMS. By effectively distinguishing the source of the video stream, it can also prevent the phenomenon of streaming and prevent the call initiator from receiving the video stream from the wrong source.

[0051] 2. SSRC is used to record and manage the session status of all streams in the platform. If there is an abnormality, the device will stop sending streams in time and recycle related resources such as SSRC, eliminating the dead stream phenomenon (the device keeps sending RTP packets of the video stream but there is no media server responsible for receiving and distributing it, wasting network resources). BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 This is a schematic diagram of the process of a client requesting a real-time video stream;

[0054] Figure 2 This is a flow chart of the client requesting the front-end video playback;

[0055] Figure 3 This is a flowchart of how the client stops front-end video playback. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0057] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0058] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as a limitation on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0059] In the description of the present invention, unless otherwise clearly specified and limited, if the term "connection" or the like appears to indicate the connection relationship between components, the term should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] The present invention provides a method for managing and scheduling multiple video streams based on SSRC, such as Figure 1 As shown, the following steps are included:

[0061] The client initiates a call to the central management module CMS to request the establishment of a video channel (video stream acquisition request);

[0062] The CMS forwards the call initiated by the client to the device management server DMS registered with the video channel;

[0063] The DMS requests the data management module DBS to allocate the SSRC corresponding to the video stream and the media server MMS for sending and receiving the stream. Then the DMS carries the SSRC, the MMS's receiving IP, receiving port, and video stream transmission mode to establish a call with the device to establish the video transmission channel.

[0064] After DMS establishes a call with the device, the device starts to send the stream to MMS. The RTP packet of the video stream sent by the device carries SSRC. MMS distinguishes different video streams by the SSRC in the RTP packet and distributes them to the corresponding client.

[0065] CMS generates a URL for obtaining the video stream based on the MMS streaming port, MMS streaming IP, and SSRC, and returns it to the client. An example of the URL format is: rtmp: / / MMS streaming IP:MMS streaming port / rtp / SSRC value.

[0066] During the video stream transmission and reception process, MMS monitors the video stream transmission and reception status at the same time, and determines whether the video stream transmission and reception is abnormal through the built-in model. If an abnormality occurs, the transmission and reception of the video stream associated with the SSRC is interrupted, and the abnormality is reported to DMS. Then DMS sends a BYE message to the device and notifies DBS to recycle the SSRC. DBS recycles the SSRC with abnormality or discontinued use for the next allocation.

[0067] The method of DBS allocating SSRC is as follows: an SSRC session table storing all SSRCs is established in the database. Each SSRC in the table is different. Each column in the table represents a piece of SSRC session data. Each piece of SSRC session data includes SSRC value, video channel ID, ID of the DMS registered with the video channel, SSRC occupancy status, and SSRC occupancy status change timestamp. The SSRC occupancy status is off (not enabled) by default. If the SSRC is occupied, the status of the SSRC is set to on (enabled). When allocating SSRC, a SSRC with a status of off is randomly selected from the table, and then the occupancy status of the SSRC is set to on.

[0068] When DBS allocates SSRC, considering the two usage scenarios of real-time video stream acquisition and front-end video playback, it is preferred to use two tables to store SSRC session data, one for allocating SSRC when requesting real-time video stream acquisition, and the other for allocating SSRC when requesting front-end video playback. The two tables are independent of each other and do not interfere with each other, so there will be no allocation errors.

[0069] MMS needs to store all the task details of video stream reception and distribution in the database. The task details include the IP and port of MMS sending and receiving streams, request timestamp, start timestamp of sending and receiving stream tasks (MMS is responsible for storing them in the database), IP and port list of MMS sending streams, SSRC, the current specific status of the video stream distribution task (a list of clients distributed to, including the IP and port of these clients, and whether the distributed code stream transmission is normal, which is stored in the database by MMS), video stream encoding format (such as MPEG-4 video stream / H.265 video stream / H.265 video stream / SVAC video stream, etc.), device-side streaming mode (three video and audio stream transmission modes specified in the national standard 28181 protocol, UDP or TCPACTIVE or TCPPASSIVE), live broadcast protocol distributed to the client (common network streaming media transmission protocols, such as HLS or HTTP-FLV or RTMP, which are stored in the database by MMS), task type is real-time video stream acquisition or video playback, and SIP header of INVITE (request to obtain video stream from the device).

[0070] When a client initiates a request to obtain a real-time video stream for a video channel, the CMS checks whether an SSRC has been assigned to the video channel. If so, it directly obtains the URL corresponding to the video channel and returns the URL to the client. The client obtains the real-time video stream based on the returned URL. If an SSRC has not been assigned to the video channel, the CMS forwards the video stream acquisition request initiated by the client to the DMS registered with the video channel.

[0071] When a client initiates a real-time video stream acquisition request, the data content in the real-time video stream acquisition request includes the client ID (the device uses the client ID to identify which client initiated the request), the video channel ID (the device uses the video channel ID to identify from which video channel the video stream is returned), the request timestamp, and the device's data stream transmission mode (UDP or TCP ACTIVE or TCP PASSIVE, and the subsequent device transmits the video stream according to the data stream transmission mode parsed from the request).

[0072] When the client initiates a front-end video playback request for a video channel, the CMS directly sends the front-end video playback request to the DMS registered with the video channel.

[0073] When the client initiates a front-end video playback request, the data content in the request includes the client ID, video channel ID, current timestamp, device playback stream transmission mode (UDP or TCP ACTIVE or TCP PASSIVE, the subsequent device performs video stream transmission according to the data stream transmission mode parsed from the request), video playback start time, and video playback end time. The device plays back the recorded video according to the video playback start time and end time carried in the request.

[0074] like Figure 1 As shown in the figure, after receiving the video stream acquisition request forwarded by the CMS, the DMS requests an SSRC and an MMS for sending and receiving streams from the DBS; after completing the allocation of SSRC and MMS, the DBS returns the allocation information to the DMS, and the allocation information includes the allocated SSRC and MMS's receiving IP, sending IP, receiving port, and sending port. Then, DMS carries the video stream acquisition request initiated by the client and the allocated SSRC and the MMS streaming IP and port, and sends sipINVITE (formal video stream acquisition request) to the device as the call object; the corresponding device returns 200ok (success status response code, used to indicate that the request has been successful) to DMS, and DMS sends ACK (acknowledgement character, indicating that 200ok is received successfully), and returns it to CMS with SSRC and MMS streaming IP and port. After receiving the response from DMS, CMS generates a URL based on the live broadcast protocol (common network streaming media transmission protocol, such as HLS or HTTP-FLV or RTMP), SSRC and MMS streaming IP and port, and returns it to the client. The URL format is as follows: rtmp: / / MMS streaming IP:MMS streaming port / rtp / SSRC value. Finally, the client obtains the video stream corresponding to the SSRC from the corresponding MMS according to the path guidance of the streaming IP and port carried in the returned URL.

[0075] In this application, multiple MMSs are deployed on one platform, and when the DBS allocates an MMS for sending and receiving flows to the DMS, a load balancing algorithm is adopted.

[0076] After receiving the RTP packet of the video stream sent by the video channel, MMS distinguishes different video streams according to the SSRC carried in the RTP message header (SSRC is unique, and each SSRC corresponds to a different video stream).

[0077] After receiving the URL returned by CMS, the client requests the URL from MMS. MMS parses the URL and obtains the SSRC carried in the URL, then transcodes the RTP stream corresponding to the SSRC into the live broadcast protocol specified by the URL and sends it to the client.

[0078] After the client no longer receives the video stream for a preset interval, the MMS no longer distributes the video stream to the client.

[0079] When the RTP stream associated with SSRC is not received by the client for a long time, MMS marks the stream as unwatched and adds it to the recycling list.

[0080] When an exception occurs in the RTP stream associated with the SSRC, the MMS marks the stream as abnormal and also adds it to the recycling list.

[0081] MMS sends the SSRC in the recycling list to DBS, obtains the ID of the corresponding DMS, and initiates a stop stream request to the DMS corresponding to the ID. After receiving the request, DMS sends BYE to the device that establishes a video channel with it. The device stops sending the RTP stream corresponding to the SSRC and notifies DBS to recycle the SSRC.

[0082] Among them, the types of RTP stream abnormalities include RTP stream format error, client RPT packet reception timeout, RTP packet bit rate abnormality, and RTP stream transcoding error.

[0083] The following describes in detail the specific implementation of the method for managing and scheduling multiple video streams based on SSRC provided by the present invention through two specific embodiments:

[0084] Embodiment 1: Multi-channel real-time video stream management and scheduling process

[0085] The data structure of SSRC is shown in Table 1 below:

[0086]

[0087] Table 1

[0088] Please combine Figure 1 The method for managing and scheduling multiple video streams in the first embodiment includes the following steps:

[0089] S101, the client carries the video channel ID, client ID, and the national standard transmission mode of the device-side video stream in the body of the HTTP POST to form a video stream acquisition request and sends it to the CMS. The JSON format of the video stream acquisition request is as follows:

[0090]

[0091] Among them, DeviceID represents the national standard device ID, for example, DeviceID here is "33010100001180000001";

[0092] ClientID represents the client ID, for example, here ClientID is "client01";

[0093] ChannelID indicates the video channel ID. A national standard device may have multiple video channels, indicating different video sources. For example, the ChannelID here is "33010100001320000001"

[0094] GBVideoTransMode indicates the national standard transmission mode for transmitting video streams on the device side. For example, GBVideoTransMode is "UDP" here.

[0095] S102, CMS searches for the ID of the DMS corresponding to the ChannelID in the DBS, and whether the ChannelID has an assigned SSRC for real-time video streaming. If an SSRC has been assigned, the previously generated URL is directly returned to the client. The specific return example is as follows:

[0096]

[0097] Wherein, code indicates the error code. Here, code "0" indicates that there is no error, otherwise it indicates that there is an error.

[0098] Message indicates the specific error information. Here, if message is "ok", it means there is no error. Otherwise, it means there is an error.

[0099] RTMP indicates the URL for obtaining video stream from MMS using RTMP protocol. 127.0.0.1 is the streaming IP of MMS, 554 is the RTMP protocol port of MMS, and 0010000002 is the SSRC value.

[0100] If there is no assigned SSRC, an HTTP request is sent to the DMS with the video channel ID, client ID, and national standard transmission mode of the device. The specific JSON format is as follows:

[0101]

[0102] S103, after receiving the HTTP request sent by CMS, DMS requests DBS to allocate an available MMS and SSRC, and then carries the SSRC, MMS receiving IP, receiving port, and national standard transmission mode, and initiates INVITE to the video channel;

[0103] S104, after receiving the INVITE, the video channel returns 200ok;

[0104] S105, after receiving 200ok, DMS sends ACK to the video channel;

[0105] S106, after receiving the ACK, the video channel transmits the RTP stream carrying the SSRC to the receiving IP and receiving port of the MMS;

[0106] S107, DMS returns an HTTP response to CMS. The HTTP response carries the video channel ID, SSRC, MMS ID, MMS streaming IP, and streaming port. The specific JSON format is as follows:

[0107]

[0108] Wherein, MMSID indicates the ID of MMS, and "mms01" here indicates that the ID of MMS is 01;

[0109] SendIP indicates the IP address of the MMS stream. Here, the IP address is "127.0.0.1";

[0110] SendPort indicates the MMS streaming port, and "554" here indicates the port number of the MMS streaming port.

[0111] S108, CMS returns an HTTP response carrying the video channel ID and URL to the client. The specific format is as follows:

[0112]

[0113]

[0114] Among them, rtmp represents the live broadcast protocol;

[0115] 0010000002 indicates SSRC;

[0116] 127.0.0.1 indicates the IP address of MMS traffic;

[0117] 554 indicates the MMS streaming port.

[0118] S109, the client pulls the real-time video stream corresponding to the SSRC carried in the URL from the corresponding MMS streaming port according to the path guidance of the streaming IP and streaming port carried in the URL.

[0119] Embodiment 2: Front-end video playback and stopping front-end video playback process

[0120] 1. Front-end video playback process is as follows Figure 2 As shown, the following steps are included:

[0121] S201, the client carries the video channel ID, client ID, device-side national standard transmission mode, video playback start time and video playback end time in the body of HTTP POST, and then sends an RTSP SETUP request to the CMS. The specific json format is as follows:

[0122]

[0123] Among them, BeginTime indicates the start time of the video playback. Here, "2022-07-15 00:00:00" means that the start time of the video playback is 00:00:00 on July 15, 2022;

[0124] EndTime indicates the end time of the video playback. Here, "2022-07-15 04:00:00" means that the end time of the video playback is 4:0:00 am on July 15, 2022.

[0125] RecordAction indicates whether it is video playback or video download. Here, "0" indicates playback and "1" indicates download.

[0126] S202, the CMS searches for the DMS registered with the ChannelID in the DBS, and then forwards an HTTP request carrying the video channel ID, the client ID, and the national standard transmission mode of the device to the found DMS. An example of the forwarded HTTP request content is as follows:

[0127]

[0128]

[0129] S203, after receiving the HTTP request sent by the CMS, the DMS generates a SessionID (session ID is a unique identifier of a session), and forms a session record including the SessionID and the video channel ID, client ID and current timestamp of establishing the SessionID carried in the HTTP request into the DBS.

[0130] DSM requests DBS to allocate SSRC and MMS for sending and receiving video streams;

[0131] Then, DMS initiates INVITE (here is the formal request for video playback) to the device that is the object of the client request, carrying the SSRC assigned by DBS, the receiving IP and port of MMS, and SessionID, the national standard transmission mode, the start time of video playback, and the end time of video playback contained in the video stream acquisition request initiated by the client. The video channel ID is an ID used to identify the media source (such as a network camera). Generally, one device can have one or more video channels.

[0132] S204, after receiving the formal request for video playback, the corresponding device returns a status response code indicating that the request is successful to the DMS, for example, returns a status response code of "200ok" to the DMS;

[0133] S205, after receiving "200 ok", the DMS returns ACK to the corresponding device;

[0134] S206, after receiving the ACK, the corresponding device sends the video playback stream carrying the SSRC to the receiving IP and receiving port of the MMS;

[0135] S207, DMS returns an HTTP response to CMS, which carries the video channel ID, SSRC, SessionID, MMS ID, MMS streaming IP, and streaming port. The specific JSON format is as follows:

[0136]

[0137]

[0138] S208, CMS returns an HTTP response carrying the video channel ID, URL, and SessionID to the client. The specific format is as follows:

[0139]

[0140] S209, the client saves the SessionID as a unique session identifier, and obtains the video stream corresponding to the SSRC from the corresponding MMS according to the MMS ID, MMS streaming IP and streaming port path guidance carried in the returned URL.

[0141] It should be noted here that SessionID is a unique session identifier in the RTSP protocol. After a session is established, the pause / fast forward / stop operations sent to the device require SessionID. The video stream is obtained from MMS, but the pause / fast forward / stop operations are sent to DMS. MMS can use SSRC to mark the stream, so only SSRC is needed, and SessionID is not needed for marking.

[0142] 2. Stop the front-end video playback process as follows Figure 3 As shown, including:

[0143] S301, the client initiates a request to stop video playback. The request to stop video playback carries the client ID, video channel ID, and SessionID. The specific JSON format is as follows:

[0144]

[0145]

[0146] S302, the CMS searches the DBS for the DMS registered with the video channel ID, and forwards the request to stop video playback to the DMS;

[0147] S303, DMS searches for session information in DBS according to the SessionID carried in the request.

[0148] If not found, an error message is returned;

[0149] If found, go to step S304;

[0150] S304, the DMS sends an INFO message carrying a request to stop video playback to the corresponding device, and at the same time sends a BYE instruction to the device;

[0151] S305, the device sends a 200 ok (status response code) indicating that the request to stop video playback has been successfully received and a 200 ok indicating that the BYE instruction has been successfully received to the DMS, and stops sending the stream;

[0152] S306, after receiving the 200 OK of INFO, DMS returns an HTTP response to CMS and carries the video channel ID and SessionID to request DBS to delete the session corresponding to SessionID;

[0153] S307, CMS returns a response carrying the video ID and SessionID to the client to notify that the video stream has stopped. The specific JSON format of the returned response is as follows:

[0154]

[0155] The present invention also provides an SSRC management and recovery device, including a data management module DBS, a device management server DMS and a media server MMS, wherein the DMS is used to allocate an SSRC to each video stream request DBS, and the MMS is used to push the video stream sent by the device to the corresponding client, and is responsible for managing the status of the video stream, and when an abnormal video stream is monitored, recovering the SSRC corresponding to the abnormal video stream.

[0156] The table structure responsible for storing SSRC sessions in the data management module DBS is shown in Table 2 and Table 3 below:

[0157]

[0158] Table 2 Real-time video SSRC session table structure

[0159]

[0160] Table 3 Front-end video playback SSRC session table structure

[0161] DMS carries the video channel ID and its own ID to request DBS to allocate SSRC. DBS randomly returns an SSRC with Status off, then sets the Status of the SSRC to on, and sets LastUpdateTime to the current timestamp.

[0162] MMS pushes the video stream sent by the device to the corresponding client and is responsible for managing the status of the video stream. When it monitors that the video stream has not been watched for a long time or an abnormality occurs (such as video stream format error, video stream reception timeout, video stream bit rate abnormality, transcoding error, etc.), it notifies DBS to recycle the SSRC corresponding to the video stream.

[0163] The steps to recover SSRC are:

[0164] S401, the MMS carries the SSRC and queries the DBS for the DMS ID and the DMS API interface corresponding to the SSRC;

[0165] S402, MMS carries SSRC and timestamp and pushes Notify to the API interface of DMS;

[0166] S403, DMS receives Notify, queries the video channel ID corresponding to SSRC, and initiates BYE to the corresponding device (if it is front-end video playback, INFO needs to be sent at the same time);

[0167] S404: The device returns 200 OK.

[0168] S405, DMS receives 200ok and notifies DBS to recycle the SSRC;

[0169] S406, DBS sets the ChannelID and DMSID corresponding to the SSRC to NULL, Status to off, and LastUpdateTime to the current timestamp. If it is front-end video playback, SessionID also needs to be set to NULL to complete the SSRC recovery process.

[0170] It should be noted that the above specific implementations are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art should understand that various modifications, equivalent substitutions, changes, etc. can be made to the present invention. However, as long as these changes do not deviate from the spirit of the present invention, they should be within the scope of protection of the present invention. In addition, some terms used in the specification and claims of this application are not restrictive, but are only for the convenience of description.

Claims

1. A method for managing and scheduling multi-channel video streams based on SSRC, characterized in that: include: After receiving the video stream acquisition request initiated by the client, the device management server DMS requests the data management module DBS to allocate the SSRC and the media server MMS for sending and receiving streams; After the DMS carries the allocated SSRC, MMS information and the video stream acquisition request to establish a call with the device, the device starts to send the stream to the MMS. The RTP packet of the video stream sent by the device carries the SSRC; The central management module CMS generates a URL for acquiring the video stream according to the streaming port of the MMS and the SSRC, and returns the URL to the client. The client initiates a request to the URL, and the MMS sends the stream to the client.

2. The method for managing and scheduling multiple video streams based on SSRC according to claim 1, characterized in that: The video stream acquisition request initiated by the client carries the ID of the video channel. After the DBS completes each SSRC allocation, the DMS generates the video stream acquisition task details corresponding to the video channel ID and adds them to the video stream acquisition task list; Before forwarding the video stream acquisition request initiated by the client to the DMS, the CMS first queries the video stream acquisition task list to see whether the video stream acquisition task corresponding to the video channel ID already exists. If yes, directly return the URL corresponding to the video channel ID generated in advance to the client; If not, the video stream acquisition request is forwarded to the DMS registered with the video channel.

3. The method for managing and scheduling multiple video streams based on SSRC according to claim 1, characterized in that: The method of assigning SSRC by DBS is: Randomly assign an unenabled SSRC from the SSRC session table stored in the database, and then set the occupancy state of the SSRC to enabled; The SSRC session table stores a plurality of SSRC session data in columns, each of which includes an SSRC value, a video channel ID, an ID of a DMS registered with the video channel, an SSRC occupancy status, and an SSRC occupancy status change timestamp.

4. The method for managing and scheduling multiple video streams based on SSRC according to claim 1, characterized in that: When the video stream acquisition request initiated by the client is a real-time video stream acquisition request, the data content in the video stream acquisition request includes the client ID of the client initiating the request, the video channel ID of the video transmission channel to be established between the client and the device, the request timestamp, and the video stream transmission mode of the device; When the video stream acquisition request initiated by the client is a front-end video playback request, the data content in the video stream acquisition request includes the client ID of the client initiating the request, the video channel ID of the video transmission channel to be established between the client and the device, the request timestamp, the video stream transmission mode of the device, the video playback start time and the video playback end time.

5. The method for managing and scheduling multiple video streams based on SSRC according to claim 1, characterized in that: If the video stream obtained by the client from the MMS is a real-time video stream generated by the device, the process of the client obtaining the video stream from the MMS specifically includes the following steps: After receiving the video stream acquisition request forwarded by the CMS to the client, the DMS requests the DBS to allocate an SSRC and an MMS for sending and receiving streams; The DMS carries the SSRC assigned by the DBS, the receiving IP and port of the MMS, and the video stream transmission mode to initiate a formal request for obtaining the video stream to the device for which the video transmission channel is to be established; After receiving the formal request for obtaining the video stream, the device returns a status response code indicating that the request is successful to the DMS; After receiving the status response code, the DMS sends an ACK character to the device to confirm receipt, and generates response data carrying SSRC, MMS ID, MMS streaming IP, and streaming port and returns it to the CMS; at the same time, the device starts to send streams to the MMS after receiving the ACK response; After receiving the response data, CMS generates a corresponding URL based on the live broadcast protocol and the received SSRC, MMS streaming IP, and streaming port and returns it to the client. An example of the URL format is: rtmp: / / MMS streaming IP:MMS streaming port / rtp / SSRC value; Finally, the client requests the video stream corresponding to the SSRC from the corresponding MMS according to the path guidance of the MMS streaming IP and streaming port carried in the returned URL.

6. The method for managing and scheduling multiple video streams based on SSRC according to any one of claims 1 to 5, characterized in that: It also includes an SSRC recycling process, specifically: The MMS parses the SSRC from the URL sent by the client, then transcodes the video stream corresponding to the SSRC into the live broadcast protocol specified by the URL, and then sends the video stream to the client from the streaming port specified by the URL; The MMS monitors the streaming status, and when detecting a streaming abnormality, adds the SSRC corresponding to the video stream with the streaming abnormality to a recycling list; The MMS sends each SSRC in the recycling list to the DBS, and initiates a stop flow request to the DMS corresponding to each SSRC in the recycling list; The DMS sends a stop streaming instruction to the corresponding device. After receiving the stop streaming instruction, the device stops sending the video stream corresponding to the SSRC and returns a stop streaming success signal; After receiving the successful signal for stopping the streaming, the DMS notifies the DBS to recycle the SSRC corresponding to the stopped video stream.

7. The method for managing and scheduling multiple video streams based on SSRC according to claim 6, characterized in that: The types of stream transmission anomalies include any one or more of an error in the format of the transmitted video stream, an abnormal transmission bit rate, an abnormal transcoding, and a timeout in the client receiving the video stream.

8. The method for managing and scheduling multiple video streams based on SSRC according to claim 1, characterized in that: If the video stream obtained by the client from the MMS is a video stream generated by video playback, the process of the client obtaining the video stream from the MMS specifically includes: After receiving the video stream acquisition request forwarded by the CMS to the client, the DMS generates a SessionID and records it in the corresponding session data in the SSRC session table in the DBS, and requests the DBS to allocate an SSRC and an MMS for sending and receiving video streams; The DMS carries the SSRC assigned by the DBS, the receiving IP, receiving port and SessionID of the MMS, the national standard transmission mode, the video playback start time and the video playback end time contained in the video stream acquisition request initiated by the client to initiate a formal video playback request to the device to establish the video transmission channel; After receiving the formal request for video playback, the device returns a status response code indicating that the request is successful to the DMS; After receiving the status response code, the DMS returns an ACK character to the device to confirm receipt, and generates response data carrying the video channel ID, SSRC, SessionID, MMS ID, MMS streaming IP, and streaming port, and returns it to the CMS; at the same time, after receiving the ACK response, the device sends the video playback video stream carrying the SSRC to the MMS receiving IP and receiving port; After receiving the response data, the CMS generates a corresponding URL according to the SSRC, the MMS ID, the streaming IP, and the streaming port, and returns it to the client together with the SessionID and the video channel ID; The client saves the SessionID, and obtains the video playback stream corresponding to the SSRC from the corresponding MMS according to the path guidance of the MMS streaming IP and streaming port carried in the returned URL.

9. The method for managing and scheduling multiple video streams based on SSRC according to claim 8, characterized in that: The process of stopping video playback on the client includes: The client initiates a request to the CMS to stop the video playback, which carries the client ID, SessionID, and the video channel ID; The CMS searches the DBS for the DMS registered with the video channel corresponding to the video channel ID carried in the request, and forwards the request to stop video playback to the found DMS; The DMS searches the DBS for the session data containing the SessionID according to the SessionID carried in the request. If not found, an error message is returned; If found, an INFO command is sent to the corresponding device along with the request to stop video playback, and a BYE command is sent to the device. The INFO command is a command to pause, resume, fast forward, slow down or play at double speed, and the BYE command is used to stop video playback; The device sends a status response code indicating successful receipt of the INFO command and the BYE command to the DMS, and stops sending the stream; After receiving the status response code, the DMS returns an HTTP response to the CMS and carries the video channel ID and the SessionID to request the DBS to delete the session data corresponding to the SessionID; The CMS returns a response carrying the video channel ID and the SessionID to the client to notify that the playback of the video stream has stopped.

10. An SSRC management and recovery device, characterized in that: It includes a data management module DBS, a device management server DMS and a media server MMS. The MMS is used to push the video stream sent by the device to the corresponding client, and is responsible for managing the status of the video stream. When the video stream is monitored to be abnormal, it recovers the SSRC corresponding to the abnormal video stream; After receiving the video stream acquisition request initiated by the client, the device management server DMS requests the data management module DBS to allocate the SSRC and the media server MMS for sending and receiving streams; After the DMS carries the allocated SSRC, MMS information and the video stream acquisition request to establish a call with the device, the device starts to send the stream to the MMS. The RTP packet of the video stream sent by the device carries the SSRC; The central management module CMS generates a URL for acquiring the video stream according to the streaming port of the MMS and the SSRC, and returns the URL to the client. The client initiates a request to the URL, and the MMS sends the stream to the client.

Images