A real-time media stream caching system and method applicable to satellite networks

Abstract

The present invention discloses a real-time media stream caching system and method applicable to satellite networks, and relates to the field of computer network communications. The present invention includes a data caching module, which is responsible for caching and querying business data within the system; a media processing module, which is responsible for storing, processing and acquiring real-time media streams; a management module, which is responsible for managing the media processing module and processing business logic; a message processing module, which is responsible for processing message interactions with the media core; an access module, which is responsible for connection management and maintenance with the media core; and an authentication module, which is responsible for connection and interface authentication. The media core implements storage and acquisition of media streams based on a real-time media stream caching system. The present invention can reduce the number of connections between media cores of a distributed multimedia conference system, reduce the complexity of the system, improve the maintainability of the system, and can also reduce the impact of media stream packet loss, disorder, and delayed arrival problems caused by network reasons, thereby ensuring smooth viewing of audio and video streams.

Description

Technical Field

[0001] The invention relates to the field of computer network communication, and in particular to a real-time media stream caching system and method applicable to a satellite network. Background Art

[0002] As an efficient means of communication, satellite networks have significant advantages such as wide coverage, long transmission distance, and no geographical restrictions. They play an important role in emergency rescue and other scenarios. In recent years, with the development of high-throughput satellites, it has become a reality to transmit audio and video via satellite links. Using satellite networks through multimedia conferencing systems, multi-party audio and video services can be interconnected. However, due to the characteristics of satellite links such as high latency, high jitter, and susceptibility to weather and cloud factors, the video quality of multimedia conferencing systems has also brought certain challenges.

[0003] A multimedia conference system generally consists of two parts: a client and a server. The client is responsible for the production and consumption of media streams, while the server is responsible for client access, routing and forwarding of media streams, and other functions.

[0004] like Figure 1 As shown in the figure, client A collects audio and video data, encodes and decodes them, and then forwards them through the routing of a series of devices such as satellite portable stations, satellites, modems, and network switches, and sends the media stream to the multimedia conference system. The multimedia conference system forwards the media stream to the participating client B according to the business. After client B decodes the data, it displays the audio and video data in the upper-layer application. In this process, client A is the producer of the media stream, and client B is the consumer of the media stream.

[0005] In the process of media stream processing within the multimedia conferencing system, the media core module is responsible for the sending, receiving and routing of media streams. It is a key module of the system and determines the performance indicators of the entire system. In a multimedia conferencing system with a single media core, due to the influence of factors such as hardware performance and network bandwidth, the system has certain performance bottlenecks and cannot meet the application scenarios of large concurrent users. In a distributed multimedia conferencing system, by introducing multiple media cores, the system's hardware and bandwidth resources are expanded, solving the problem of concurrent access on the end side. However, in order to achieve interoperability of media streams between different media cores, it is necessary to establish connection channels between multiple media cores participating in the conference. For example Figure 2 As shown in the figure, when there are N media cores participating in a conference, if all media stream producers in the conference are concentrated on one media core, and other media cores only consume media streams, then N-1 connection channels need to be established between the media cores. However, for a multimedia conference system, each end side may become a media stream producer, and may be distributed on different media cores. Figure 3As shown in the figure, in this case, at most N*(N-1) / 2 connection channels need to be established between N media cores to meet the interconnection of media streams between all media cores. This undoubtedly greatly increases the complexity of the system, and once a problem occurs, it is very difficult to locate. Summary of the invention

[0006] In view of this, the present invention proposes a real-time media stream caching system and method applicable to satellite networks. The method can reduce the number of connections between media cores of a distributed multimedia conference system, reduce the complexity of the system, and improve the maintainability of the system. In addition, it can also reduce the impact of media stream packet loss, disorder, and delayed arrival caused by network reasons, and ensure the smooth viewing of audio and video streams.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A real-time media stream caching system applicable to a satellite network is applied to the storage, processing and acquisition of real-time media stream data between a multimedia conference system and each media core. The real-time media stream caching system includes a data caching module, a management module, a message processing module, an access module, an authentication module and a plurality of media processing modules;

[0009] Among them, the data cache module is responsible for caching and querying the internal business data of the real-time media stream cache system;

[0010] The media processing module is responsible for the storage, processing and acquisition of real-time media streams, and supports hot expansion;

[0011] The management module is responsible for the management of the media processing module of the real-time media stream caching system and the processing of business logic;

[0012] The message processing module is responsible for handling message interactions with the media core;

[0013] The access module is responsible for the management and maintenance of the connection with the media core;

[0014] The authentication module is responsible for connection and interface authentication.

[0015] Furthermore, a media processing module management list is provided in the management module. After the media processing module passes the authentication of the authentication module, it registers with the management module, establishes a connection, maintains a heartbeat and regularly reports load information. When the connection between the media processing module and the management module is interrupted or the heartbeat times out three times in a row, the management module determines that the media processing module is currently unavailable and removes it from the media processing module management list. After that, the real-time media stream cache system cleans up the cache data associated with the media processing module, and notifies the media core that has a business relationship with the media processing module, triggering the media core's business reconnection. In addition, the newly connected media processing module does not affect the currently ongoing business in the real-time media stream cache system.

[0016] A real-time media stream caching method applicable to a satellite network, implemented based on any one of the above-mentioned real-time media stream caching systems applicable to a satellite network, including storage and acquisition of real-time media streams;

[0017] Specifically, the storage of real-time media streams includes the following steps:

[0018] Step a1, each media core sends an authentication request to the authentication module through the access module. After the authentication module passes the authentication, the connection between the media core and the real-time media stream cache system is completed. After that, the media core periodically sends a heartbeat message to the message processing module through the access module. The message processing module forwards the heartbeat message to the authentication module to maintain normal connection. If the authentication module detects that a certain media core has three consecutive heartbeat timeouts, the real-time media stream cache system determines that the media core is unavailable and clears all data caches related to the media core. The media core executes step a1 to trigger the service reconnection of the media core;

[0019] Step a2: When a producer on a certain media core releases a media stream, the media core sends a media stream storage message to the message processing module through the access module, and the message processing module forwards the media stream storage message to the management module;

[0020] Step a3: The management module traverses the media processing module management list in a loop, and selects the media processing module closest to the media core and with the lowest load as the working module of the media stream according to the media stream storage message combined with the load balancing method;

[0021] Step a4, the management module generates a media stream identifier and a media stream random verification code corresponding to the current media stream storage message, and parses the producer information in the media stream storage message, and records the media stream identifier, the media stream random verification code, the correspondence between the producer ID and the media stream identifier, the correspondence between the media stream identifier and the media processing module, and the correspondence between the media stream and the media core into the data cache module, and then encodes the IP and port of the media processing module corresponding to the current media stream storage message, the media stream identifier and the media stream random verification code as a response message, and returns it to the media core corresponding to the current media stream storage message through the message processing module and the access module;

[0022] Step a5, after receiving the response message, the media core corresponding to the current media stream storage message parses the content in the response message and pushes the media stream, media stream identifier and media stream random verification code to the corresponding media processing module;

[0023] Step a6, after the media processing module receives the media stream, the media stream identifier and the media stream random verification code, it triggers a new media stream joining event according to the media stream identifier, and reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream, and the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core; if the management module verifies that the media stream random verification code is legal, the media processing module continuously monitors the status of the media stream, performs disorderly reordering and packet loss retransmission processing on the data packets of the media stream, and automatically adjusts the length of the real-time buffer of the media stream according to the specific network environment; when the media processing module detects that the media stream is abnormally disconnected, it reports to the management module, and then the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core;

[0024] The acquisition of real-time media stream includes the following steps:

[0025] Step b1, when a certain media core has a consumer subscribing to a media stream, the media core sends a media stream acquisition message to the message processing module through the access module, and the message processing module forwards the media stream acquisition message to the management module;

[0026] Step b2: the management module queries the data cache module for information about the media stream according to the media stream producer ID information in the media stream acquisition message. If the information is not found, an error alarm is returned to the current media core. Otherwise, the management module encodes the media stream identifier, the media stream random verification code, the correspondence between the media stream identifier and the media processing module, and the IP and port of the media processing module corresponding to the media stream according to the queried media stream information as a response message, and returns it to the media core corresponding to the current media stream acquisition message through the message processing module and the access module.

[0027] Step b3, after receiving the response message, the media core corresponding to the current media stream acquisition message parses the content in the response message, takes the media stream identifier and the media stream random verification code as parameters, and requests the media stream information from the media processing module corresponding to the media stream;

[0028] Step b4, after the media processing module receives the media stream identifier and the media stream random verification code, it reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream; if the management module verifies that the media stream random verification code is legal, the media processing module searches for the corresponding media stream information according to the media stream identifier. If the media stream does not exist, an error alarm is returned to the current media core. If the media stream exists, the media stream is sent to the current media core, and the current media core sends the media stream to the corresponding consumer.

[0029] Due to the adoption of the above technical solution, the beneficial effects of the present invention compared with the prior art are:

[0030] 1. The present invention can reduce the number of connections between media cores of a distributed multimedia conference system based on the storage and acquisition of a media processing module, reduce the complexity of the system, and improve the maintainability of the system.

[0031] 2. The present invention can reduce the impact of media stream packet loss, disorder, and delayed arrival problems caused by network reasons, and ensure the viewing smoothness of audio and video streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The figure is a schematic diagram of the deployment of a multimedia conference system based on a satellite network in an embodiment of the present invention.

[0033] Figure 2 , Figure 3 This is a connection diagram of the existing multimedia core system.

[0034] Figure 4 The figure is a schematic diagram of the connection between the real-time media stream caching system and the media core in an embodiment of the present invention.

[0035] Figure 5Schematic diagram of various modules of the real-time media stream caching system in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0037] A real-time media stream caching system applicable to satellite networks, such as Figure 4 , Figure 5 As shown, it is applied to the storage, processing and acquisition of real-time media stream data between a multimedia conference system and each media core. The multimedia conference system is applied to the application scenario of transmitting real-time media streams through an IP network, and is particularly suitable for the application scenario of transmitting real-time media streams using a satellite network as a link. The real-time media stream cache system includes a data cache module, a management module, a message processing module, an access module, an authentication module and multiple media processing modules;

[0038] Among them, the data cache module is responsible for caching and querying the internal business data of the real-time media stream cache system;

[0039] The media processing module is responsible for the storage, processing and acquisition of real-time media streams, and supports hot expansion;

[0040] Specifically, the media processing module is responsible for the storage, processing and acquisition of media streams. In the system, multiple instances of this module can be deployed to expand the system's concurrent capabilities. The newly deployed media processing module registers with the management module, establishes a connection, and maintains a heartbeat. After passing the authentication of the authentication module, the management module adds it to the media processing module management list, and regularly receives load information reported by the media processing module. When a new service is connected, the management module can select a module with the closest distance and lower load as the working module for the current service based on the current load and geographical conditions of each media processing module. The newly connected media processing module does not affect the current services in the system;

[0041] In this embodiment, the media processing module stores and processes the real-time media stream pushed by the first media core, specifically: the first media core pushes the real-time media stream to the media processing module, the media processing module reports the new media stream event to the management module, and the management module verifies the legality of the new media stream. If it is illegal, the connection is directly disconnected. If it is legal, the state of the media stream is continuously monitored, and the data packets of the media stream are reordered and the packet loss and retransmission processing are performed.

[0042] The media processing module also responds to the second media core's request for obtaining real-time media streams, specifically: the second media core sends a media stream acquisition message to the real-time media stream cache system according to the producer ID that needs to be subscribed. The management module queries the media processing module where the media stream is located based on the producer ID. If it does not exist, an error is directly returned. Otherwise, the corresponding media processing module information is returned to the second media core. The second media core sends a media stream acquisition request to the corresponding media processing module. After receiving the request, the media processing module sends the corresponding media stream to the second media core.

[0043] The management module is responsible for the management of the media processing module of the real-time media stream caching system and the processing of business logic;

[0044] The message processing module is responsible for handling message interactions with the media core;

[0045] The access module is responsible for the management and maintenance of the connection with the media core;

[0046] Specifically, the access module provides TCP services for media core connections and is responsible for connection management and maintenance.

[0047] Specifically, the message processing module is responsible for processing the message interaction with the media core. The main message types include: get (get media stream), set (store media stream), ping (heartbeat). The message processing module performs different processing logic according to the message type. If the message type is ping, the timeout clock of the connection is reset and a response is returned. When the message type is get or set, the message is handed over to the management module for processing;

[0048] The authentication module is responsible for connection and interface authentication.

[0049] Furthermore, a media processing module management list is provided in the management module. After the media processing module passes the authentication of the authentication module, it registers with the management module, establishes a connection, maintains a heartbeat and regularly reports load information. When the connection between the media processing module and the management module is interrupted or the heartbeat times out three times in a row, the management module determines that the media processing module is currently unavailable and removes it from the media processing module management list. After that, the real-time media stream cache system cleans up the cache data associated with the media processing module, and notifies the media core that has a business relationship with the media processing module, triggering the media core's business reconnection. In addition, the newly connected media processing module does not affect the currently ongoing business in the real-time media stream cache system.

[0050] Specifically, the management module's control over the media processing module can minimize the impact of a single media processing module failure on upper-layer services. If there is no available media processing module, an error is returned.

[0051] The management module receives the media stream storage request message from the first media core, loops through the media processing module management list, and selects the media processing module with the closest distance and the lowest load as the working module for media stream storage.

[0052] The management module receives the media stream acquisition request message from the second media core, and queries the data cache module for media stream information according to the producer ID. If no media stream information is found, an error is returned. If the media stream information is found, the found media stream information is returned to the second media core.

[0053] A real-time media stream caching method applicable to a satellite network, implemented based on any one of the above-mentioned real-time media stream caching systems applicable to a satellite network, including storage and acquisition of real-time media streams;

[0054] Specifically, the storage of real-time media streams includes the following steps:

[0055] Step a1, each media core sends an authentication request to the authentication module through the access module. After the authentication module passes the authentication, the connection between the media core and the real-time media stream cache system is completed. After that, the media core periodically sends a heartbeat message to the message processing module through the access module. The message processing module forwards the heartbeat message to the authentication module to maintain normal connection. If the authentication module detects that a certain media core has three consecutive heartbeat timeouts, the real-time media stream cache system determines that the media core is unavailable and clears all data caches related to the media core. The media core executes step a1 to trigger the service reconnection of the media core;

[0056] Step a2: When a producer on a certain media core releases a media stream, the media core sends a media stream storage message to the message processing module through the access module, and the message processing module forwards the media stream storage message and the message routing to the management module;

[0057] Step a3: The management module traverses the media processing module management list in a loop, and selects the media processing module closest to the media core and with the lowest load as the working module of the media stream according to the media stream storage message combined with the load balancing method;

[0058] Step a4, the management module generates a media stream identifier and a media stream random verification code corresponding to the current media stream storage message, and parses the producer information in the media stream storage message, and records the media stream identifier, the media stream random verification code, the correspondence between the producer ID and the media stream identifier, the correspondence between the media stream identifier and the media processing module, and the correspondence between the media stream and the media core into the data cache module, and then encodes the IP and port of the media processing module corresponding to the current media stream storage message, the media stream identifier and the media stream random verification code as a response message, and returns it to the media core corresponding to the current media stream storage message through the message processing module and the access module;

[0059] Step a5, after receiving the response message, the media core corresponding to the current media stream storage message parses the content in the response message and pushes the media stream, media stream identifier and media stream random verification code to the corresponding media processing module;

[0060] Step a6, after the media processing module receives the media stream, the media stream identifier and the media stream random verification code, it triggers a new media stream joining event according to the media stream identifier, and reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream, and the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core; if the management module verifies that the media stream random verification code is legal, the media processing module continuously monitors the status of the media stream, performs disorderly reordering and packet loss retransmission processing on the data packets of the media stream, and automatically adjusts the length of the real-time buffer of the media stream according to the specific network environment; when the media processing module detects that the media stream is abnormally disconnected, it reports to the management module, and then the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core;

[0061] The acquisition of real-time media stream includes the following steps:

[0062] Step b1, when a certain media core has a consumer subscribing to a media stream, the media core sends a media stream acquisition message to the message processing module through the access module, and the message processing module forwards the media stream acquisition message to the management module;

[0063] Step b2: the management module queries the data cache module for information about the media stream according to the media stream producer ID information in the media stream acquisition message. If the information is not found, an error alarm is returned to the current media core. Otherwise, the management module encodes the media stream identifier, the media stream random verification code, the correspondence between the media stream identifier and the media processing module, and the IP and port of the media processing module corresponding to the media stream according to the queried media stream information as a response message, and returns it to the media core corresponding to the current media stream acquisition message through the message processing module and the access module.

[0064] Step b3, after receiving the response message, the media core corresponding to the current media stream acquisition message parses the content in the response message, takes the media stream identifier and the media stream random verification code as parameters, and requests the media stream information from the media processing module corresponding to the media stream;

[0065] Step b4, after the media processing module receives the media stream identifier and the media stream random verification code, it reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream; if the management module verifies that the media stream random verification code is legal, the media processing module searches for the corresponding media stream information according to the media stream identifier. If the media stream does not exist, an error alarm is returned to the current media core. If the media stream exists, the media stream is sent to the current media core, and the current media core sends the media stream to the corresponding consumer.

[0066] In summary, the present invention can reduce the number of connections between media cores of a distributed multimedia conference system, reduce the complexity of the system, and improve the maintainability of the system. In addition, it can also reduce the impact of media stream packet loss, disorder, and delayed arrival problems caused by network reasons, and ensure the smooth viewing of audio and video streams.

[0067] Those skilled in the art will appreciate that the embodiments described herein take a satellite network as an example. In the case of a non-satellite network, any modifications, equivalent substitutions, improvements, etc. made outside the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims

1. A real-time media stream caching system applicable to satellite networks, used for the storage, processing and acquisition of real-time media stream data between multimedia conference systems and various media cores, characterized in that: The real-time media stream caching system includes a data caching module, a management module, a message processing module, an access module, an authentication module and a plurality of media processing modules; Among them, the data cache module is responsible for caching and querying the internal business data of the real-time media stream cache system; The media processing module is responsible for the storage, processing and acquisition of real-time media streams, and supports hot expansion; The management module is responsible for the management of the media processing module of the real-time media stream caching system and the processing of business logic; The message processing module is responsible for handling message interactions with the media core; The access module is responsible for the management and maintenance of the connection with the media core; The authentication module is responsible for connection and interface authentication.

2. A real-time media stream caching system applicable to a satellite network according to claim 1, characterized in that: The management module is provided with a media processing module management list. After the media processing module passes the authentication of the authentication module, it registers with the management module, establishes a connection, maintains a heartbeat, and regularly reports load information. When the connection between the media processing module and the management module is interrupted or the heartbeat times out three times in a row, the management module determines that the media processing module is currently unavailable and removes it from the media processing module management list. After that, the real-time media stream cache system cleans up the cache data associated with the media processing module, and notifies the media core that has a business relationship with the media processing module, triggering the media core's business reconnection. In addition, the newly connected media processing module does not affect the currently ongoing business in the real-time media stream cache system.

3. A real-time media stream caching method applicable to a satellite network, characterized in that: A real-time media stream caching system applicable to a satellite network is implemented based on claim 1 or 2, including storage and acquisition of real-time media streams; Specifically, the storage of real-time media streams includes the following steps: Step a1, each media core sends an authentication request to the authentication module through the access module. After the authentication module passes the authentication, the connection between the media core and the real-time media stream cache system is completed. After that, the media core periodically sends a heartbeat message to the message processing module through the access module. The message processing module forwards the heartbeat message to the authentication module to maintain normal connection. If the authentication module detects that a certain media core has three consecutive heartbeat timeouts, the real-time media stream cache system determines that the media core is unavailable and clears all data caches related to the media core. The media core executes step a1 to trigger the service reconnection of the media core; Step a2: When a producer on a certain media core releases a media stream, the media core sends a media stream storage message to the message processing module through the access module, and the message processing module forwards the media stream storage message to the management module; Step a3: The management module traverses the media processing module management list in a loop, and selects the media processing module closest to the media core and with the lowest load as the working module of the media stream according to the media stream storage message combined with the load balancing method; Step a4, the management module generates a media stream identifier and a media stream random verification code corresponding to the current media stream storage message, and parses the producer information in the media stream storage message, and records the media stream identifier, the media stream random verification code, the correspondence between the producer ID and the media stream identifier, the correspondence between the media stream identifier and the media processing module, and the correspondence between the media stream and the media core into the data cache module, and then encodes the IP and port of the media processing module corresponding to the current media stream storage message, the media stream identifier and the media stream random verification code as a response message, and returns it to the media core corresponding to the current media stream storage message through the message processing module and the access module; Step a5, after receiving the response message, the media core corresponding to the current media stream storage message parses the content in the response message and pushes the media stream, media stream identifier and media stream random verification code to the corresponding media processing module; Step a6, after the media processing module receives the media stream, the media stream identifier and the media stream random verification code, it triggers a new media stream joining event according to the media stream identifier, and reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream, and the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core; if the management module verifies that the media stream random verification code is legal, the media processing module continuously monitors the status of the media stream, performs disorderly reordering and packet loss retransmission processing on the data packets of the media stream, and automatically adjusts the length of the real-time buffer of the media stream according to the specific network environment; when the media processing module detects that the media stream is abnormally disconnected, it reports to the management module, and then the real-time media stream cache system clears all data caches related to the media core, and the media core executes step a1 to trigger the service reconnection of the media core; The acquisition of real-time media stream includes the following steps: Step b1, when a certain media core has a consumer subscribing to a media stream, the media core sends a media stream acquisition message to the message processing module through the access module, and the message processing module forwards the media stream acquisition message to the management module; Step b2: the management module queries the data cache module for information about the media stream according to the media stream producer ID information in the media stream acquisition message. If the information is not found, an error alarm is returned to the current media core. Otherwise, the management module encodes the media stream identifier, the media stream random verification code, the correspondence between the media stream identifier and the media processing module, and the IP and port of the media processing module corresponding to the media stream according to the queried media stream information as a response message, and returns it to the media core corresponding to the current media stream acquisition message through the message processing module and the access module. Step b3, after receiving the response message, the media core corresponding to the current media stream acquisition message parses the content in the response message, takes the media stream identifier and the media stream random verification code as parameters, and requests the media stream information from the media processing module corresponding to the media stream; Step b4, after the media processing module receives the media stream identifier and the media stream random verification code, it reports the media stream random verification code to the management module. If the management module verifies that the media stream random verification code is illegal, the media processing module disconnects the connection with the media core corresponding to the media stream; if the management module verifies that the media stream random verification code is legal, the media processing module searches for the corresponding media stream information according to the media stream identifier. If the media stream does not exist, an error alarm is returned to the current media core. If the media stream exists, the media stream is sent to the current media core, and the current media core sends the media stream to the corresponding consumer.

Images