Media stream forwarding processing method and apparatus, electronic device, and storage medium

Abstract

The embodiment of the application discloses a media stream forwarding processing method and device, electronic equipment and a storage medium. The method comprises the following steps: determining at least two target forwarding links to be sequentially used by a target media stream during transmission from a target encoding end to a target decoding end; sequentially adopting each target forwarding link to sequentially forward the target media stream from the target encoding end to the target decoding end through at least one target forwarding server located between the target encoding end and the target decoding end; one-time issuing the planned forwarding link; and continuously recursing and completing the negotiation forwarding by communicating between each link node of the forwarding link, so as to realize the control of the media stream through the specified forwarding servers in sequence during the process from the encoding end to the decoding end, ensure that the media stream can be forwarded according to the optimal route in the complex network environment, and improve the environmental adaptability of the media stream in the forwarding process.

Description

Technical Field

[0001] The present invention relates to the field of communication processing technology, and in particular to a method, apparatus, electronic device and storage medium for forwarding media streams. Background Technology

[0002] As video surveillance systems continue to expand, the network environment between the encoding end (hereinafter referred to as the encoding end) and the decoding end (hereinafter referred to as the decoding end) of the media stream becomes increasingly complex. The encoding end can directly send the media stream to the decoding end via the network, or it can be sent by forwarding through a media stream forwarding server (hereinafter referred to as the forwarding server). However, the media stream sending methods cannot cope with the complex network environment, nor can they flexibly forward the media stream across multiple forwarding servers. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, and storage medium for forwarding media streams, enabling control of media streams to be forwarded along an optimal route in complex network environments.

[0004] In a first aspect, embodiments of the present invention provide a method for forwarding media streams, the method comprising:

[0005] Determine the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoder to the target decoder;

[0006] The target media stream is forwarded step by step from the target encoding end to the target decoding end by using the at least two target forwarding links in sequence;

[0007] The link nodes in the at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

[0008] Secondly, this invention also provides a media stream forwarding processing apparatus, the apparatus comprising:

[0009] The forwarding link determination module is used to determine the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end.

[0010] The media stream forwarding module is used to sequentially forward the target media stream from the target encoding end to the target decoding end using the at least two target forwarding links.

[0011] The link nodes in the at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

[0012] Thirdly, this invention also provides an electronic device, comprising:

[0013] One or more processors;

[0014] Storage device for storing one or more programs;

[0015] The one or more programs are executed by the one or more processors, causing the one or more processors to implement the media stream forwarding processing method as provided in any of the embodiments of the present invention.

[0016] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the media stream forwarding processing method as described in any of the embodiments of the present invention.

[0017] According to the media stream forwarding method provided in the embodiments of the present invention, at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end are determined. The target media stream is then forwarded step by step from the target encoding end to the target decoding end through at least one target forwarding server located between the target encoding end and the target decoding end. The planned forwarding links are issued at once, and the links between the links communicate with each other and recursively complete the negotiated forwarding. This enables the media stream to pass through a number of designated forwarding servers in sequence from the encoding end to the decoding end, ensuring that the media stream can be forwarded along a better route in a complex network environment and improving the environmental adaptability of the media stream during the forwarding process.

[0018] The above description of the invention is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0020] Figure 1 This is a flowchart of a media stream forwarding processing method provided in an embodiment of the present invention;

[0021] Figure 2This is a schematic diagram of a media stream forwarding processing architecture provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a forwarding link for media stream forwarding processing provided in an embodiment of the present invention;

[0023] Figure 4 This is a flowchart of another media stream forwarding processing method provided in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the forwarding channel construction in a media stream forwarding process provided in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the forwarding channel construction in another media stream forwarding process provided in this embodiment of the invention;

[0026] Figure 7 This is a schematic diagram illustrating the negotiation and construction interaction of the media stream forwarding channel during the media stream forwarding process provided in an embodiment of the present invention;

[0027] Figure 8 This is a flowchart of media stream forwarding and multiplexing during the media stream forwarding process provided in an embodiment of the present invention;

[0028] Figure 9 This is a schematic diagram of a media stream forwarding tree structure during the media stream forwarding process provided in an embodiment of the present invention;

[0029] Figure 10 This is a structural block diagram of a media stream forwarding and processing device provided in an embodiment of the present invention;

[0030] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0032] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0033] Figure 1 This is a flowchart of a media stream forwarding processing method provided in an embodiment of the present invention. This embodiment of the present invention is applicable to situations where a media stream is transmitted from the encoding end to the decoding end. This method can be executed by a media stream forwarding processing device, which can be implemented in software and / or hardware and integrated into any electronic device with network communication capabilities. Figure 1 As shown, the media stream forwarding processing method in this application embodiment may include the following steps:

[0034] S110. Determine the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end.

[0035] See Figure 2 To better transmit media streams from the encoding end to the decoding end, a large number of encoding ends, decoding ends, and forwarding servers can be connected to a single control platform. This control platform centrally controls the forwarding and transmission of media streams between the encoding ends, decoding ends, and forwarding servers, ensuring the platform's involvement throughout the media stream creation process. The media stream can be an audio stream, video stream, or multimedia file stream transmitted over a network.

[0036] See Figure 2 A media stream is sent directly from a transmitting device to a receiving device. A single media stream transmission between these two devices can be defined as a forwarding link (Dialog). Each forwarding link (Dialog) can be represented using a unique forwarding link identifier (DialogID). When media stream transmission is required, at least two target forwarding links can be selected to sequentially transmit the target media stream from the target encoding end to the target decoding end. For example, when the encoding end is used as the transmitting device and the forwarding server as the receiving device, the following can be selected: Figure 2 The media stream transmission from the encoding end to the forwarding server is forwarding link 1; when the forwarding server is used as the streaming device and the decoding end is used as the receiving device, the following is selected: Figure 2 The media stream transmission from the forwarding server to the decoding end is forwarding link 2. Following the above process, and so on, at least two target forwarding links can be obtained.

[0037] In one alternative embodiment, determining the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end may include the following steps A1-A2:

[0038] Step A1: Determine the pre-built media stream forwarding relationship information between the target encoding end and the target decoding end.

[0039] Step A2: From the pre-built media stream forwarding relationship information, select at least two target forwarding links that will be used sequentially to transmit the target media stream from the target encoding end to the target decoding end.

[0040] See Figure 2 When establishing a forwarding link between a streaming device and a receiving device, the control platform needs to negotiate with both the streaming and receiving devices. This negotiation can be defined as a negotiation session, and each session can be represented by a unique session ID. For example, for forwarding link 1, the negotiation between the control platform and the encoding end is streaming negotiation session 1 (denoted as SendSession1), and the negotiation between the control platform and the forwarding server is receiving negotiation session 1 (denoted as RecvSession1). For forwarding link 2, the negotiation between the control platform and the forwarding server is streaming negotiation session 2 (denoted as SendSession2), and the negotiation between the control platform and the decoding end is receiving negotiation session 2 (denoted as RecvSession2). Following this process, the negotiation establishment of different forwarding links can be obtained, thus constructing the media stream forwarding relationship information between the target encoding end and the target decoding end.

[0041] Optionally, in media streaming scenarios, the media stream forwarding relationship established between the target encoding end (Enc) and the target decoding end (Dec) may include a forwarding link that sequentially passes the media stream from the target encoding end (Enc) through different forwarding servers to the target decoding end (Dec). See [link / reference] Figure 3 The media stream forwarding relationship established between the target encoding end Enc and the target decoding end Dec can include not only the forwarding links that the media stream travels from the target encoding end through forwarding servers M1, M2...Mn to the target decoding end, but also the forwarding links that the media stream travels from the target encoding end through other forwarding servers to the target decoding end, etc.

[0042] See Figure 2 and Figure 3After connecting the encoding end (Enc), forwarding server (M1), forwarding server (M2)... forwarding server (Mn), and decoding end (Dec) to the same control platform, the control platform can uniformly manage each media stream processing node. In this case, the control platform can obtain the addresses and signaling ports of the encoding end (Enc), forwarding server (M1), forwarding server (M2)... forwarding server (Mn), and decoding end (Dec).

[0043] See Figure 3 The control platform can select the forwarding servers that the target encoding end (Enc) must traverse to reach the target decoding end (Dec) based on the constructed media stream forwarding relationship information, and obtain the forwarding links that the target encoding end (Enc) must traverse through the selected forwarding servers to reach the target decoding end (Dec). This consists of n+2 media stream processing nodes and n+1 forwarding links. In this way, the control platform can obtain at least two target forwarding links at once and distribute them.

[0044] Using the above optional solution, it is not necessary to build a multi-level network platform in accordance with the GB / T 28181 protocol. Instead, all media stream processing nodes only need to be connected to a unified control platform, and the media streams are forwarded between the corresponding media stream processing nodes in the order of forwarding links issued by the control platform at one time. This allows the media streams to be transmitted from the encoding end to the decoding end.

[0045] S120. At least two target forwarding links are used sequentially to forward the target media stream from the target encoding end to the target decoding end step by step.

[0046] Among them, the link nodes in at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

[0047] The control platform sequentially links at least two target forwarding links. Adjacent target forwarding links share a common link node, which connects two different target forwarding links. Given these sequentially linked target forwarding links, the target media stream can be received from the previous target forwarding link through the current target forwarding link and forwarded to the next target forwarding link, until the target media stream is transmitted sequentially from the encoding end through at least one target forwarding server located between the target encoding end and the target decoding end, according to the sequentially linked target transmission links, to the decoding end.

[0048] According to the media stream forwarding processing method provided in the embodiments of the present invention, a concept for multimedia stream processing nodes to forward media streams is proposed. A one-time plan is issued to forward the target media stream from the target encoding end through at least one target forwarding server to the target decoding end in a step-by-step manner. The links between the links in the forwarding link communicate with each other and recursively complete the negotiated forwarding. This realizes the control of the media stream from the encoding end to the decoding end to pass through a number of designated forwarding servers in sequence, ensuring that the media stream is forwarded along a better route in complex network environments and improving the environmental adaptability of the media stream during the forwarding process.

[0049] Figure 4 This is a flowchart of another media stream forwarding processing method provided in this embodiment of the invention. The technical solution of this embodiment is further optimized based on the above embodiments, and the technical solution of this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 4 As shown, the media stream forwarding processing method provided in this application embodiment may include the following steps:

[0050] S410. Determine the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoder to the target decoder.

[0051] Among them, the link nodes in at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

[0052] S420: Control the construction of a media stream forwarding channel between the first and second link nodes of the target forwarding link; each link node has its corresponding media stream processing node.

[0053] See Figure 3 For sending a target media stream from the target encoding end to the target decoding end, at least two target forwarding links are required. Each target forwarding link is configured with a first-end link node and a second-end link node at its two ends. Two adjacent target forwarding links have a common link node, which can be used to connect two different target forwarding links together.

[0054] Each target forwarding link has a media stream processing node at its first and second ends. The media stream processing nodes can be devices such as a target encoder, a target decoder, or at least one target forwarding server located between the target encoder and the target decoder. To forward the target media stream using the target forwarding link, the GB / T 28181 protocol and the RTSP protocol can be used to construct a media stream forwarding channel between the media stream processing nodes at the first and second ends of the target forwarding link.

[0055] S430. Through the constructed media stream transmission channel, the target media stream is forwarded from the first link node of the target transmission link to the second link node, so as to sequentially transmit the target media stream from the target encoding end to the target decoding end by using the media stream forwarding channels corresponding to each target forwarding link.

[0056] After establishing a corresponding media stream forwarding channel for each target forwarding link, the media stream can be controlled to be forwarded through the media stream forwarding channel corresponding to a target forwarding link. This allows the media stream to be forwarded from the first link node of a target transmission link to the second link node, and then enters the media stream forwarding channel corresponding to the next target forwarding link. Furthermore, the media stream is controlled to be forwarded through the media stream forwarding channel corresponding to the next target forwarding link, allowing the media stream to be forwarded from the first link node of the next target transmission link to the second link node, and so on. The target media stream is then transmitted step-by-step from the target encoding end to the target decoding end using the media stream forwarding channels corresponding to each target forwarding link. In adjacent target forwarding links, the second link node of one target forwarding link corresponds to the same media stream processing node as the first link node of the adjacent target forwarding link.

[0057] In one optional embodiment, controlling the construction of a media stream forwarding channel between the first and second end links of the target forwarding link may include the following steps B1-B2:

[0058] Step B1: Determine the target forwarding method to be used when forwarding the target media stream between the two end links of the target forwarding link.

[0059] Optionally, the target forwarding method may include a point-to-point streaming method in which the first-end link node and the second-end link node corresponding to the target forwarding link know each other's transmit and receive stream addresses and ports. In this way, as long as the media stream receiver and the media stream sender know each other's transmit and receive stream addresses and ports, the transmission relationship between the media stream receiver and the media stream sender can be established.

[0060] Optionally, the target forwarding method may also include an address retrieval method in which the first end link node corresponding to the target forwarding link obtains the push address of the second end link node. In this way, the media stream receiver obtains the RTSP URL of the media stream sender, and the media stream transmission relationship between the media stream receiver and the media stream sender can be established.

[0061] Step B2: Based on the target forwarding method to be used, control the construction of a media stream forwarding channel that matches the target forwarding method between the first and second link nodes of the target forwarding link.

[0062] See Figure 5 Taking a point-to-point streaming approach as an example, for the first and second end links in the target forwarding link, after receiving the media stream forwarding channel establishment request, the media stream processing node 2 corresponding to the second end link of the target forwarding link requests to initiate a session. The media stream processing node 2 corresponding to the second end link of the target forwarding link returns its receiving address and port information to the media stream processing node 1 corresponding to the first end link of the target forwarding link. The media stream processing node 1 corresponding to the first end link of the target forwarding link returns its sending address and port information to the media stream processing node 2 corresponding to the second end link of the target forwarding link.

[0063] See also Figure 5 Under the same target forwarding link, after exchanging the aforementioned send and receive stream addresses and port information, using the GB / T28181 protocol, the media stream processing node 1 corresponding to the first link node can obtain the receive stream address and port information of the media stream processing node 2 corresponding to the second link node, and the media stream processing node 2 corresponding to the second link node can also obtain the send stream address and port of the media stream processing node 1 corresponding to the first link node. In this way, the media stream processing node corresponding to the first link node can construct a media stream forwarding channel capable of forwarding media streams based on the forwarding link between the first and second link nodes.

[0064] See Figure 6Taking the address-based streaming method as an example, for the first and second end links in the target forwarding link, the media stream processing node 1 corresponding to the first end link can return the URL for obtaining the media stream to the media stream processing node 2 corresponding to the second end link. The media stream processing node 2 corresponding to the second end link can then establish an RTSP connection with the media stream processing node 1 corresponding to the first end link using the received URL address, thus obtaining a media stream forwarding channel capable of forwarding the media stream and retrieving it. Through this information exchange, as long as the media stream processing node 2 corresponding to the second end link obtains the URL address information of the media stream processing node 1 corresponding to the first end link, the media stream can be sent.

[0065] Using the above optional scheme, the control platform can issue a plan at once to forward the target media stream from the target encoding end through at least one target forwarding server to the target decoding end in a step-by-step manner. After issuing the target forwarding link at once, the media stream processing nodes communicate with each other and recursively complete the process of negotiating and establishing the media stream forwarding channel.

[0066] In one optional embodiment, at least one target forwarding server included in the link nodes of at least two target forwarding links is configured as follows:

[0067] Configuration 1: When both the target encoding end and the target decoding end use the same stream retrieval method, at least one target forwarding server ends both of the same stream retrieval method as the target encoding end and the target decoding end.

[0068] Optionally, if the target encoding end Enc uses a point-to-point streaming method and the target decoding end also uses a point-to-point streaming method, then all target forwarding servers located between the target encoding end and the target decoding end in the at least two target forwarding links use a point-to-point streaming method to build a media stream forwarding channel on the basis of the target forwarding links.

[0069] Optionally, if the target encoding end Enc uses the URL address retrieval method and the target decoding end Dec uses the URL address retrieval method, then all target forwarding servers located between the target encoding end and the target decoding end in the at least two target forwarding links use the URL address retrieval method.

[0070] Configuration 2: When different stream retrieval methods are used at the target encoding end and the target decoding end, at least one of the target forwarding servers must have both ends matched with the stream retrieval methods used by the target encoding end and the target decoding end, respectively.

[0071] Optionally, if the target encoding end Enc uses the URL address retrieval method and the target decoding end Dec uses the point-to-point retrieval method, then there must be a target forwarding server that ensures that the receiving side of the target forwarding server retrieves the stream from the target encoding end Enc using the URL address retrieval method, and the sending side of the target forwarding server retrieves the stream from the target decoding end using the point-to-point retrieval method.

[0072] Optionally, if the target encoding end Enc uses a point-to-point streaming method and the target decoding end Dec uses a URL-based streaming method, then there must be a target forwarding server that ensures that the receiving side of the target forwarding server uses a point-to-point streaming method to the target encoding end Enc, and the sending side of the target forwarding server uses a URL-based streaming method to the target decoding end.

[0073] In one optional embodiment of this application, to better describe the media stream forwarding process, the following example illustrates that the target encoding end Enc uses a point-to-point streaming method, the target decoding end Dec uses a URL streaming method, and the stream passes through three forwarding servers M1, M2, and M3, with the streaming method being converted at M2. See also... Figure 7 The negotiation process for establishing a media stream forwarding channel and forwarding media streams is as follows:

[0074] Step C1: The control platform plans at least two target forwarding links (including Enc->M1->M2->M3->Dec) to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end, and sends this information to the decoding end Dec all at once. Furthermore, it determines that the point-to-point streaming method to URL address streaming method will be converted at the target forwarding server M2, which includes the following four target forwarding links, as well as the address and port information of each media stream processing point:

[0075] Target forwarding link dialog1: Enc->M1, point-to-point flow retrieval method

[0076] Target forwarding link dialog2: M1->M2, point-to-point flow retrieval method

[0077] Target forwarding link dialog3: M2->M3, URL address retrieval method

[0078] Target forwarding link dialog4: M3->Dec, URL address retrieval method

[0079] Step C2: After receiving the request information from the control platform, the target decoding terminal Dec retrieves the last dialog4 (M3->Dec) and sends a GetURL request to the target forwarding server M3 to obtain the URL address. The request carries information about dialog1, dialog2, and dialog3.

[0080] Step C3: After receiving the request from the upstream, the target forwarding server M3 retrieves dialog3 (M2->M3) and sends a GetURL request to the target forwarding server M2, which carries the information of dialog1 and dialog2.

[0081] Step C4: After receiving the request from the upstream, the target forwarding server M2 retrieves dialog2 (M1->M2) and sends a SIP invite request message to the target forwarding server M1. The request carries the incoming address and port information of M2 in dialog2, as well as the information of dialog1.

[0082] Step C5: After receiving the request from the upstream, the target forwarding service M1 retrieves dialog1 (Enc->M1) and sends a SIP invite request message to the target encoding end Enc. The request carries the receiving address and port information of the target forwarding service M1 in dialog1.

[0083] Step C6: After receiving the request from the upstream, the target encoding end Enc retrieves the receiving address and port information of dialog1 and returns a SIP 200 OK response, which carries the sending address and port of the target encoding end Enc.

[0084] Step C7: After receiving the response message from the downstream, the target forwarding server M1 retrieves the downstream streaming address and port information carried in the message, and then returns a SIP ACK to the downstream Enc. The media stream forwarding channel corresponding to dialog1 is established, and then the target encoding end Enc sends the media stream to the target forwarding server M1.

[0085] Step C8: After receiving the media stream from dialog1, M1 replies to upstream M2 with a SIP 200 OK response, which carries the source address and port of M1.

[0086] Step C9: After M2 receives the response message from the downstream, it retrieves the downstream's streaming address and port information, and then returns a SIP ACK to the downstream M1. The media stream forwarding channel corresponding to dialog2 is established, and then M1 will forward the media stream received from dialog1 to M2.

[0087] Step C10: After M2 receives the media stream from dialog2, it replies to the upstream M3 with a response containing the URL for retrieving the stream from M2. Once M3 receives this URL, the media stream forwarding channel corresponding to dialog3 is established, and the media stream can be retrieved via the RTSP protocol.

[0088] Step C11: After M3 receives the media stream from dialog3, it replies to the upstream Dec response, which carries the URL for retrieving the stream from M3. After Dec receives this URL, the media stream forwarding channel corresponding to dialog4 is established, and the media stream can be retrieved through the RTSP protocol.

[0089] Step C12: After the entire media stream forwarding channel is established, the target decoding end Dec replies to the control platform and successfully responds.

[0090] According to the media stream forwarding method provided in the embodiments of the present invention, a concept for multimedia stream processing nodes to forward media streams is proposed. A one-time plan is issued to forward the target media stream sequentially from the target encoding end through at least one target forwarding server to the target decoding end via a step-by-step forwarding link. The nodes of each link communicate with each other, recursively negotiating and forwarding, thereby controlling the media stream to pass through a specified number of forwarding servers in sequence from the encoding end to the decoding end. This ensures that the media stream is forwarded along a more optimal route in complex network environments. Compared to the multi-level networking platform method in GB / T28181, this method is simpler to operate and supports encoding and decoding ends using the RTSP protocol, improving the environmental adaptability of the media stream during the forwarding process.

[0091] Based on the above embodiments, optionally, the media stream forwarding processing method provided in this application embodiment further includes the following steps D1-D3:

[0092] Step D1: Determine at least two other forwarding links to be used sequentially from the target encoding end to other decoding ends for the transmission of the target media stream; wherein, the at least two other forwarding links share a forwarding link with the at least two target forwarding links starting from the encoding end.

[0093] Step D2: Reuse the target media stream forwarded by the last link in the shared forwarding link.

[0094] Step D3: Sequentially use at least two other forwarding links, excluding the shared forwarding link, to forward the multiplexed target media stream to other decoding ends level by level.

[0095] See Figure 8After establishing a media stream forwarding channel (from the target encoding end Enc to the target decoding end Dec) according to the embodiment of this application, it is assumed that media stream multiplexing needs to be performed at one of the target forwarding servers Mx (1≤x≤n) in at least two target forwarding links. The copied media stream passes through Mx...My in sequence and finally reaches other decoding ends Dec.

[0096] See Figure 8 Assuming the forwarding relationship (Mx-1)->(Mx) is dialog x-1, the above negotiation process can be reused. The target forwarding server Mx, which needs to be reused, is treated as the target encoding server Enc to negotiate and establish the media stream forwarding channel that needs to be reused. Only the dialog x-1 information needs to be carried to Mx, so that Mx instructs that the media stream corresponding to dialog x-1 needs to be reused. Once the media stream forwarding can pass through multiple nodes and the bitstream can be reused at any node, the media stream forwarding channel relationship will be formed. Figure 9 The tree structure shown is shown.

[0097] By adopting the above optional schemes, in complex network environments, media streams can be controlled to be forwarded along a better route. Alternatively, media streams can be reused on a specific forwarding server. Based on the establishment of the above multi-node media stream forwarding channel relationship, media stream reuse is performed on a specific forwarding server, forming a tree-shaped media stream forwarding channel relationship. The media stream processing points still communicate with each other and continuously recursively complete the negotiation to establish media stream forwarding channels for reuse and forwarding media streams.

[0098] Figure 10 This is a structural block diagram of a media stream forwarding processing device provided in an embodiment of the present invention. The embodiments of the present invention are applicable to situations where media streams are transmitted from the encoding end to the decoding end. This device can be implemented in software and / or hardware and integrated into any electronic device with network communication capabilities. Figure 1 As shown, the media stream forwarding processing method in this embodiment may include: a forwarding link determination module 1010 and a media stream forwarding module 1020.

[0099] Forwarding link determination module 1010 is used to determine at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end;

[0100] The media stream forwarding module 1020 is used to sequentially forward the target media stream from the target encoding end to the target decoding end using the at least two target forwarding links.

[0101] The link nodes in the at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

[0102] Based on the above embodiments, optionally, the forwarding link determination module 1010 includes:

[0103] Determine the pre-built media stream forwarding relationship information between the target encoding end and the target decoding end;

[0104] From the pre-built media stream forwarding relationship information, select at least two target forwarding links that will be used sequentially to transmit the target media stream from the target encoding end to the target decoding end.

[0105] Based on the above embodiments, optionally, the media stream forwarding module 1020 includes:

[0106] Control the construction of a media stream forwarding channel between the first and second link nodes of the target forwarding link; the link nodes correspond to media stream processing nodes;

[0107] By constructing a media stream transmission channel, the target media stream is forwarded from the first link node of the target transmission link to the second link node, so as to sequentially transmit the target media stream from the target encoding end to the target decoding end through the media stream forwarding channels corresponding to each target forwarding link.

[0108] Based on the above embodiments, optionally, controlling the construction of a media stream forwarding channel between the first end link node and the second end link node of the target forwarding link includes:

[0109] Determine the target forwarding method to be used when forwarding the target media stream between the two end links of the target forwarding link;

[0110] Based on the target forwarding method to be used, a media stream forwarding channel matching the target forwarding method is constructed between the first and second link nodes of the target forwarding link.

[0111] Based on the above embodiments, optionally, the target forwarding method includes a point-to-point flow retrieval method in which the first end link node and the second end link node corresponding to the target forwarding link know each other's transmit and receive stream address ports, and an address flow retrieval method in which the first end link node knows the push stream address of the second end link node.

[0112] Based on the above embodiments, optionally, the at least one target forwarding server included in the link nodes of the at least two target forwarding links is configured as follows:

[0113] When both the target encoding end and the target decoding end use the same stream retrieval method, both ends of the at least one target forwarding server use the same stream retrieval method as the target encoding end and the target decoding end.

[0114] When the target encoding end and the target decoding end adopt different stream retrieval methods, one of the at least target forwarding servers has two ends that match the stream retrieval methods adopted by the target encoding end and the target decoding end, respectively.

[0115] Optionally, based on the above embodiments, the device further includes:

[0116] Identify at least two other forwarding links to be used sequentially from the target encoding end to other decoding ends for the transmission of the target media stream; wherein, the at least two other forwarding links share some forwarding links with the at least two target forwarding links starting from the encoding end;

[0117] The target media stream forwarded by the last link in the shared forwarding link is reused;

[0118] The remaining forwarding links, excluding the shared forwarding link, are used sequentially from the at least two other forwarding links to forward the multiplexed target media stream to other decoding ends level by level.

[0119] The media stream forwarding processing device provided in the embodiments of the present invention can execute the media stream forwarding processing method provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the media stream forwarding processing method. For details, please refer to the relevant operations of the media stream forwarding processing method in the foregoing embodiments.

[0120] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 11 The structure shown in this embodiment of the invention includes an electronic device comprising one or more processors 1110 and a storage device 1120; the processors 1110 in this electronic device may be one or more. Figure 11 Taking a processor 1110 as an example; storage device 1120 is used to store one or more programs; the one or more programs are executed by the one or more processors 1110, so that the one or more processors 1110 implement the video interaction method as described in any one of the embodiments of the present invention.

[0121] The electronic device may also include an input device 1130 and an output device 1140.

[0122] The processor 1110, storage device 1120, input device 1130, and output device 1140 in this electronic device can be connected via a bus or other means. Figure 11Taking the example of a connection between China and Israel via a bus.

[0123] The storage device 1120 in this electronic device serves as a computer-readable storage medium, which can be used to store one or more programs. These programs can be software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the video interaction method provided in this embodiment of the invention. The processor 1110 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the storage device 1120, thereby implementing the video interaction method in the above-described method embodiment.

[0124] Storage device 1120 may include a stored program area and a stored data area, wherein the stored program area may store the operating system and applications required for at least one function; the stored data area may store data created based on the use of the electronic device, etc. Furthermore, storage device 1120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, storage device 1120 may further include memory remotely located relative to processor 1110, and this remote memory may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0125] Input device 1130 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 1140 may include display devices such as a display screen.

[0126] Furthermore, when one or more programs included in the aforementioned electronic device are executed by one or more processors 1110, and are executed by the intranet monitoring device deployed in the video private network, the program performs the following operations:

[0127] Determine the current stimulus value required for video interaction between the management device and the target node device;

[0128] The target video data exchanged between the management device and the target node device is encrypted and decrypted based on the current incentive value.

[0129] Each node device can generate a unique feedback value under the stimulation of an excitation value, and node devices with different physical characteristics can generate different feedback values ​​under the same excitation value; the feedback value is used to encrypt and decrypt video data exchanged between devices.

[0130] Of course, those skilled in the art will understand that when one or more programs included in the above-mentioned electronic device are executed by one or more processors 1110, the programs can also perform related operations in the video interaction method provided in any embodiment of the present invention.

[0131] This invention provides a computer-readable medium storing a computer program that, when executed by a processor, performs a video interaction method. This method is executed by an intranet monitoring device deployed in a video private network, and includes:

[0132] Determine the current stimulus value required for video interaction between the management device and the target node device;

[0133] The target video data exchanged between the management device and the target node device is encrypted and decrypted based on the current incentive value.

[0134] Each node device can generate a unique feedback value under the stimulation of an excitation value, and node devices with different physical characteristics can generate different feedback values ​​under the same excitation value; the feedback value is used to encrypt and decrypt video data exchanged between devices.

[0135] Optionally, when executed by a processor, the program can also be used to execute the video interaction method provided in any embodiment of the present invention.

[0136] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination thereof. A computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0137] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device.

[0138] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, radio frequency (RF), etc., or any suitable combination thereof.

[0139] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for forwarding and processing media streams, characterized in that, The method includes: The at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end are determined; the target forwarding links in the at least two target forwarding links are sequentially linked, and two adjacent target forwarding links have a common link node, which can connect two different target forwarding links together; The target media stream is forwarded from the target encoding end to the target decoding end through the at least two target forwarding links in sequence. The links of the at least two target forwarding links communicate with each other and recursively complete the negotiation forwarding. The media stream is controlled to pass through a number of designated forwarding servers in sequence from the encoding end to the decoding end. The link nodes in the at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

2. The method according to claim 1, characterized in that, Determine the at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoder to the target decoder, including: Determine the pre-built media stream forwarding relationship information between the target encoding end and the target decoding end; From the pre-built media stream forwarding relationship information, select at least two target forwarding links that will be used sequentially to transmit the target media stream from the target encoding end to the target decoding end.

3. The method according to claim 1, characterized in that, The target media stream is forwarded step-by-step from the target encoding end to the target decoding end using at least two target forwarding links in sequence, including: Control the construction of a media stream forwarding channel between the first and second link nodes of the target forwarding link; the link nodes correspond to media stream processing nodes; By constructing a media stream transmission channel, the target media stream is forwarded from the first link node of the target transmission link to the second link node, so as to sequentially transmit the target media stream from the target encoding end to the target decoding end through the media stream forwarding channels corresponding to each target forwarding link.

4. The method according to claim 3, characterized in that, Controlling the construction of a media stream forwarding channel between the first and second link nodes of the target forwarding link includes: Determine the target forwarding method to be used when forwarding the target media stream between the two end links of the target forwarding link; Based on the target forwarding method to be used, a media stream forwarding channel matching the target forwarding method is constructed between the first and second link nodes of the target forwarding link.

5. The method according to claim 4, characterized in that, The target forwarding method includes a point-to-point stream retrieval method in which the first end link node and the second end link node of the target forwarding link know each other's transmit and receive stream address ports, and an address-based stream retrieval method in which the first end link node knows the push stream address of the second end link node.

6. The method according to claim 4, characterized in that, The configuration of at least one target forwarding server included in the link nodes of the at least two target forwarding links is as follows: When both the target encoding end and the target decoding end use the same stream retrieval method, both ends of the at least one target forwarding server use the same stream retrieval method as the target encoding end and the target decoding end. When the target encoding end and the target decoding end adopt different stream retrieval methods, one of the at least target forwarding servers has two ends that match the stream retrieval methods adopted by the target encoding end and the target decoding end, respectively.

7. The method according to claim 1, characterized in that, The method further includes: Identify at least two other forwarding links to be used sequentially from the target encoding end to other decoding ends for the transmission of the target media stream; wherein, the at least two other forwarding links share some forwarding links with the at least two target forwarding links starting from the encoding end; The target media stream forwarded by the last link in the shared forwarding link is reused; The remaining forwarding links, excluding the shared forwarding link, are used sequentially from the at least two other forwarding links to forward the multiplexed target media stream to other decoding ends level by level.

8. A media stream forwarding and processing apparatus, characterized in that, The device includes: The forwarding link determination module is used to determine at least two target forwarding links to be used sequentially for the transmission of the target media stream from the target encoding end to the target decoding end; each of the at least two target forwarding links is sequentially linked, and two adjacent target forwarding links have a common link node, which can connect two different target forwarding links together; The media stream forwarding module is used to sequentially use the at least two target forwarding links to forward the target media stream from the target encoding end to the target decoding end step by step. The link nodes of the at least two target forwarding links communicate with each other and continuously recursively complete the negotiated forwarding. The module controls the media stream to pass through a number of specified forwarding servers in sequence from the encoding end to the decoding end. The link nodes in the at least two target forwarding links include a target encoding end, a target decoding end, and at least one target forwarding server located between the target encoding end and the target decoding end.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the media stream forwarding processing method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the media stream forwarding processing method as described in any one of claims 1-7.

Images