Peer-to-peer streaming

Abstract

A relaying peer, producing an upstream media stream from a downstream media stream during an ongoing peer-to-peer session between a plurality of peers in a peer- to-peer network. Computing a media profile of the downstream media stream and peer profiles having limiting constraints according to capabilities subscribing peers. Receiving the downstream media stream of the ongoing session. Decoding the downstream media stream into a decoded media stream. Computing the decoded media stream into an upstream media stream in accordance with the upstream media profile. Transmitting the upstream media stream from the relaying peer to the one or more subscribing peers. Monitoring, healing and improving reliability of the peer-to-peer network.

Description

PEER-TO-PEER STREAMINGPRIORITY

[0001] This non-provisional patent application claims priority based upon the prior U.S. provisional patent application entitled “PEER-TO-PEER STREAMING,” application number 63 / 621 ,042, filed on January 15, 2024, in the name of TECHNOLOGIES CREWDLE INC, incorporated herein by reference in its entirety.TECHNICAL FIELD

[0002] The present invention relates to systems, devices and methods for computer network communication, more particularly for peer-to-peer networking.BACKGROUND

[0003] Many online systems rely on dedicated servers to offload workflows from users. With dedicated servers, the hardware requirements to interact with the system may be lowered, enabling a larger number of users to interact with the system. The use of dedicated servers may however be problematic in many scenarios.

[0004] For example, users located in remote areas and rural regions may not have any nearby dedicated servers. Having to interact with remote dedicated servers may result in high communication latency and degraded experience. Deploying dedicated servers proximate to all users may prove impractical considering the costs and logistics associated with configuring, managing, and monitoring hundreds or thousands of locations across multiple infrastructure service providers.

[0005] In addition, as locations are added to better serve the users, each location serves a reduced number of users to a point where most of the dedicated servers may be idle most of the time. In practice, service providers may balance costs and locations according to the geographical distribution of their users and optimize the location of the dedicated servers such that most but not all of their users are well served. This strategy may result in a reputational risk with a proportion of users experiencing poor service and missed opportunities.

[0006] Dedicated servers may also present challenges related to maintenance and system updates while sessions are active. It may be necessary to first drain thededicated servers, a process which can take several hours or more depending on the nature of the workloads.

[0007] Confidentiality of data may be yet another concern when using dedicated servers. In order to execute workflows, the data being processed by the dedicated servers is typically interpreted, which implies that the data may no longer be confidential to the users.

[0008] Based on principles of peer-to-peer connectivity, the present disclosure presents examples of solutions to at least some of the exemplified problems.SUMMARY

[0009] This summary is provided to introduce a selection of concepts in a simplified form that are further described below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0010] In a first aspect, the technique described herein relates to a system for relaying a media stream in a peer-to-peer network, where the system may comprise a plurality of peers participating in an ongoing peer-to-peer session and a relaying peer comprising one or more processors. The one or more processors may be configured to produce an upstream media stream from a downstream media stream during the ongoing peer-to-peer session between the plurality of peers in the peer-to-peer network. The one or more processors may compute a media profile of the downstream media stream and peer profiles into an upstream media profile of the upstream media stream. The peer profiles may comprise limit constraints according to capabilities of one or more subscribing peers participating in the ongoing peer-to-peer session through the relaying peer and limit constraints according to the capabilities of the relaying peer, thereby receiving the downstream media stream of the ongoing peer-to- peer session. The one or more processors may decode the downstream media stream, at least partially, into a decoded media stream. The one or more processors may compute the decoded media stream into an upstream media stream in accordance with the upstream media profile. The one or more processors may transmit the upstream media stream from the relaying peer to the one or more subscribing peers.

[0011] In embodiments, the upstream media profile may be configured to reduce a bitrate of the upstream media stream when compared to the downstream media stream.

[0012] In embodiments, the ongoing peer-to-peer session may be any one of a conferencing session, a live stream, and an on-demand source.

[0013] In embodiments, the downstream media stream may include at least one of a text stream, a video stream and an audio stream.

[0014] In embodiments, computing into the upstream media stream may include at least one lossy transformation. Optionally, the lossy transformation may include at least one of a spatial re-sampling, a temporal re-sampling, and a re-quantization operation.

[0015] In embodiments, the one or more processors may transform the decoded media stream into a transformed media stream in accordance with the upstream media profile and encode the transformed media stream into the upstream media stream in accordance with the upstream media profile.

[0016] In embodiments, the one or more processors may process a web browser configured to produce the upstream media stream, compute the media profile, limit constraints according to capabilities of the one or more subscribing peers and the capabilities of the relaying peer, receive the downstream media stream, decode the downstream media stream, compute the decoded media stream, and transmit the upstream media stream. The downstream media stream may include a video stream, and the one or more processors may render the video stream into a content of a source canvas, encode the content from the source canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile, and transmit the upstream media stream using a stream publisher over a RTC connection. Optionally, when the downstream media stream includes a video stream, the one or more processors may render the video stream into a source content of a source canvas, transform the source content from the source canvas into a destination content in a destination canvas in accordance with the media profile, encode the destination content from the destination canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile, and transmit the upstream media stream using a stream publisher over a RTC connection. Optionally, the stream encoder may use at least one of a Mediastream API and a WebCodecs API provided by the web browser. Optionally, the stream publisher may use at least one of a WebRTC framework and a QUIC framework provided by the web browser. Optionally, a framerate of the video stream may be decreased by executing the transforming at a lower frequency than the rendering. Optionally, a resolution of the video stream may bedecreased by using a destination canvas with a lower resolution than the source canvas. Optionally, the source canvas and the destination canvas may be HTML canvases.

[0017] In a second aspect, the technique described herein relates to a relaying peer for relaying a media stream in a peer-to-peer network, where the network comprises a plurality of peers participating in an ongoing peer-to-peer session. The relaying peer may include one or more processors. The one or more processors may produce an upstream media stream from a downstream media stream during the ongoing peer-to- peer session between the plurality of peers in the peer-to-peer network. The one or more processors may compute a media profile of the downstream media stream and peer profiles into an upstream media profile of the upstream media stream. The peer profiles may include limiting constraints according to capabilities of one or more subscribing peers participating in the ongoing peer-to-peer session through the relaying peer and limiting constraints according to the capabilities of the relaying peer, whereby the downstream media stream of the ongoing peer-to-peer session is received. The one or more processors may decode the downstream media stream, at least partially, into a decoded media stream. The one or more processors may compute the decoded media stream into an upstream media stream in accordance with the upstream media profile. The one or more processors may transmit the upstream media stream from the relaying peer to the one or more subscribing peers.

[0018] In embodiments, a relaying peer may be configured such that the upstream media profile is configured to reduce a bitrate of the upstream media stream when compared to the downstream media stream.

[0019] In embodiments, the ongoing peer-to-peer session may be any one of a conferencing session, a live stream, and an on-demand source.

[0020] In embodiments, the downstream media stream may include at least one of a text stream, a video stream and an audio stream.

[0021] In embodiments, computing into the upstream media stream may include at least one lossy transformation. Optionally, the lossy transformation may include at least one of a spatial re-sampling, a temporal re-sampling, and a re-quantization operation.

[0022] In embodiments, the one or more processors of a relaying peer may transform the decoded media stream into a transformed media stream in accordance with the upstream media profile. The one or more processors may also encode thetransformed media stream into the upstream media stream in accordance with the upstream media profile.

[0023] In embodiments, the one or more processors of a relaying peer may process a web browser configured to produce the upstream media stream, compute the media profile, limit constraints according to capabilities of the one or more subscribing peers and the capabilities of the relaying peer, receive the downstream media stream, decode the downstream media stream, compute the decoded media stream, and transmit the upstream media stream. The downstream media stream may include a video stream, and the one or more processors may render the video stream into a content of a source canvas, encode the content from the source canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile, and transmit the upstream media stream using a stream publisher over a RTC connection. Optionally, when the downstream media stream includes a video stream, the one or more processors may render the video stream into a source content of a source canvas, transform the source content from the source canvas into a destination content in a destination canvas in accordance with the media profile, encode the destination content from the destination canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile, and transmit the upstream media stream using a stream publisher over a RTC connection. Optionally, the stream encoder may use at least one of a Mediastream API and a WebCodecs API provided by the web browser. Optionally, the stream publisher uses at least one of a WebRTC framework and a QUIC framework provided by the web browser. Optionally, a framerate of the video stream is decreased by executing the transforming at a lower frequency than the rendering. Optionally, a resolution of the video stream is decreased by using a destination canvas with a lower resolution than the source canvas. Optionally, the source canvas and the destination canvas may be HTML canvases.

[0024] In a third aspect, the technique described herein relates to a method for relaying a media stream in a peer-to-peer network. At a relaying peer, an upstream media stream may be produced from a downstream media stream during an ongoing peer-to-peer session between a plurality of peers in the peer-to-peer network. A media profile of the downstream media stream and peer profiles may be computed into an upstream media profile of the upstream media stream. The peer profiles may comprise limiting constraints according to capabilities of one or more subscribing peers participating in the ongoing session through the relaying peer and limiting constraintsaccording to the capabilities of the relaying peer. The downstream media stream of the ongoing session may be received. The downstream media stream may be at least partially decoded into a decoded media stream. The decoded media stream may be computed into an upstream media stream in accordance with the upstream media profile. The upstream media stream from the relaying peer may be transmitted to the one or more subscribing peers.

[0025] In embodiments, the upstream media profile may be configured to reduce a bitrate of the upstream media stream when compared to the downstream media stream.

[0026] In embodiments, the ongoing peer-to-peer session may be any one of a conferencing session, a live stream, and an on-demand source.

[0027] In embodiments, the downstream media stream may comprise at least one of a text stream, a video stream and an audio stream.

[0028] In embodiments, computing into an upstream media stream may include at least one lossy transformation. Optionally, the lossy transformation may comprise at least one of a spatial re-sampling, a temporal re-sampling, and a re-quantization operation.

[0029] In embodiments, computing into an upstream media stream may further comprise transforming the decoded media stream into a transformed media stream in accordance with the upstream media profile, and encoding the transformed media stream into the upstream media stream in accordance with the upstream media profile.

[0030] In embodiments, the method may be executed within a web browser, wherein the downstream media stream may comprise a video stream. The decoding may render the video stream into a content of a source canvas, and computing the decoded media stream into the upstream media stream may comprise encoding the content from the source canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile. The transmitting may transmit the upstream media stream using a stream publisher over a RTC connection. Optionally, the downstream media stream may comprise a video stream, where the decoding may render the video stream into a source canvas. The transforming may transform a source content from the source canvas into a destination content in a destination canvas in accordance with the media profile. The encoding may encode the content from the destination canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile, and the transmitting maytransmit the upstream media stream using a stream publisher over a RTC connection. Optionally, the stream encoder may use at least one of a Mediastream API and a WebCodecs API provided by the web browser. Optionally, the stream publisher may use at least one of a WebRTC framework and a QUIC framework provided by the web browser. Optionally, a framerate of the video stream may be decreased by executing the transforming at a lower frequency than the rendering. Optionally, a resolution of the video stream may be decreased by using a destination canvas with a lower resolution than the source canvas. Optionally, the source canvas and the destination canvas may be HTML canvases.

[0031] In a fourth aspect, the technique described herein relates to a system for real-time monitoring and detection of problems in a peer-to-peer network. The system comprises a monitoring module configured, considering a plurality of connections between two or more participants of the peer-to-peer network in real-time, to collect connection statistics from at least two connections from the plurality of connections using a weighted round-robin collection mechanism, wherein the connection statistics comprise one or more of latency, packet loss, jitter, and bandwidth. The monitoring module may generate an alert when one or more of the collected connection statistics is outside of one or more of a plurality of statistics thresholds.

[0032] In embodiments, the connection statistics may be collected during a transmission of a media stream between a content peer and a subscribing peer over one of the plurality of connections.

[0033] In embodiments, the content peer and the subscribing peer may be participating in a conferencing session over the peer-to-peer network.

[0034] In embodiments, the monitoring module may be configured to sequentially arrange the at least two connections for the weighted round-robin collection mechanism by prioritizing weaker connections.

[0035] In embodiments, the monitoring module may dynamically adjust one or more of the statistics thresholds from the plurality of statistics thresholds for detecting problems based on the connection statistics, wherein the adjusting is based on the number of the connections between participants. Optionally, the monitoring module may continuously collect and compare connection statistics against one or more dynamically adjusted thresholds for each connection.

[0036] In a fifth aspect, the technique described herein relates to a monitoring peer for real-time monitoring and detection of problems in a peer-to-peer network. The monitoring peer may include a monitoring module configured, considering a plurality of connections between two or more participants of the peer-to-peer network in real-time, to collect connection statistics from at least two of the plurality of connections using a weighted round-robin collection mechanism. The connection statistics may comprise one or more of latency, packet loss, jitter, and bandwidth. The monitoring module may generate an alert when one or more of the collected connection statistics is outside of one or more of a plurality of statistics thresholds.

[0037] In embodiments, the connection statistics may be collected during a transmission of a media stream between a content peer and a subscribing peer over one of the plurality of connections.

[0038] In embodiments, the content peer and the subscribing peer may be participating in a conferencing session over the peer-to-peer network.

[0039] In embodiments, the monitoring module may be configured to sequentially arrange the at least two connections for the weighted round-robin collection mechanism by prioritizing weaker connections.

[0040] In embodiments, the monitoring module may dynamically adjust one or more statistics thresholds from the plurality of statistics thresholds for detecting problems based on the connection statistics, wherein the adjusting is based on a number of the connections between participants. Optionally, the monitoring module may continuously collect connection statistics and continuously compare the connection statistics against one or more dynamically adjusted thresholds for each connection.

[0041] In a sixth aspect, the technique described herein relates to a method for realtime monitoring and detection of problems in a peer-to-peer network. The method comprises collecting, in real-time, a plurality of connection statistics from at least two of a plurality of connections using a weighted round-robin collection mechanism, wherein the plurality of connections is defined between two or more participants of the peer-to- peer network. The connection statistics may comprise one or more of latency, packet loss, jitter, and bandwidth. The method may involve generating an alert when one or more of the collected connection statistics is outside of one or more of a plurality of statistics thresholds.

[0042] In embodiments, the connection statistics may be collected during a transmission of a media stream between a content peer and a subscribing peer over one of the plurality of connections.

[0043] In embodiments, the content peer and the subscribing peer may be participating in a conferencing session over the peer-to-peer network.

[0044] In embodiments, the method may further comprise arranging the at least two connections for the weighted round-robin collection mechanism by prioritizing weaker connections.

[0045] In embodiments, the method may further adjust one or more thresholds for detecting problems based on the statistics, wherein the thresholds are adjusted based on the number of the connections between participants. Optionally, collecting may be performed continuously, and the method may further comprise comparing the connection statistics to the one or more dynamically adjusted thresholds for each of the plurality of connections.

[0046] In a seventh aspect, the technique described herein relates to a system for autonomous real-time problem resolution in a peer-to-peer network. The system comprises a first peer transmitting a media stream to a second peer over a connection between the first peer and the second peer within the peer-to-peer network, wherein the media stream is configured according to a peer profile of the second peer providing limiting constraints over the media stream. The system further includes a monitoring module configured, considering the connection between the first peer and the second peer in real-time, to, upon receiving an alert indicating that a collected connection statistic is outside of one or more of a plurality of statistics thresholds for the connection between the first and the second peer, adjust the peer profile of the second peer for the first peer. Optionally, the second peer may also transmit a second media stream to the first peer over a connection between the second peer and the first peer within the peer- to-peer network, wherein the second media stream is configured according to a peer profile of the first peer, providing limiting constraints over the media stream. The monitoring module may be further configured, upon receiving the alert, to adjust the peer profile of the first peer for the second peer. Optionally, the media stream may be a video stream, and the limiting constraints of the peer profile may comprise at least one of a maximum resolution, a maximum frame rate, and a maximum bitrate.

[0047] In an eighth aspect, the technique described herein relates to a self-healing peer for autonomous real-time problem resolution in a peer-to-peer network. The self- healing peer may be a first peer transmitting a media stream to a second peer over a connection between the self-healing peer and the second peer within the peer-to-peer network. The media stream may be configured according to a peer profile of the second peer, providing limiting constraints over the media stream. The self-healing peer may include a monitoring module configured, considering the connection between the self- healing peer and the second peer in real-time, to adjust the peer profile of the second peer for the self-healing peer, upon receiving an alert indicating that a collected connection statistic is outside of one or more of a plurality of statistics thresholds for the connection between the self-healing peer and the second peer. Optionally, the second peer may also transmit a second media stream to the self-healing peer over a connection between the second peer and the self-healing peer within the peer-to-peer network. The second media stream may, according to a peer profile of the self-healing peer, provide limiting constraints over the media stream, and the monitoring module may, upon receiving the alert, adjust the peer profile of the self-healing peer for the second peer. Optionally, the media stream may be a video stream, and the limiting constraints of the peer profile may comprise at least one of a maximum resolution, a maximum frame rate, and a maximum bitrate.

[0048] In a ninth aspect, the technique described herein relates to a method for real-time problem resolution in a peer-to-peer network. The method comprises transmitting of a media stream from a first peer to a second peer over a connection between the first peer and the second peer within the peer-to-peer network, wherein the media stream is configured according to a peer profile of the second peer, which provides limiting constraints over the media stream. The method includes receiving an alert indicating that a collected connection statistic is outside of one or more of a plurality of statistics thresholds for the connection between the first peer and the second peer, and adjusting the peer profile of the second peer for the first peer. In embodiments, the method may further comprise transmitting a second media stream from the second peer to the first peer over a connection within the peer-to-peer network, wherein the second media stream is configured according to a peer profile of the first peer, which provides limiting constraints over the media stream, and, upon considering the alert, adjusting the peer profile of the second peer for the first peer. Optionally, the media stream may be a video stream, and the limiting constraints of the peer profilemay comprise at least one of a maximum resolution, a maximum frame rate, and a maximum bitrate.

[0049] In a tenth aspect, the technique described herein relates to a system for establishing a stable connection in a peer-to-peer network. The system comprises a first peer having a first one-way connection for sending media streams to a second peer and a second one-way connection for receiving media streams from the second peer.

[0050] In embodiments, the first and second one-way connections may be established using a PeerConnection API or a WebSocket.

[0051] In embodiments, the first peer and the second peer may implement a heartbeat mechanism to monitor the stability of the first one-way connection.

[0052] In embodiments, the first peer and the second peer may implement a collision prevention protocol for establishing the first one-way connection and the second one-way connection without collision.

[0053] In an eleventh aspect, the technique described herein relates to a dualconnection peer for establishing a stable connection in a peer-to-peer network. The dual-connection peer may be a first peer having a first one-way connection for sending media streams to a second peer and may have a second one-way connection for receiving media streams from the second peer.

[0054] In embodiments, the first and second one-way connections may be established using a PeerConnection API or a WebSocket.

[0055] In embodiments, the dual-connection peer and the second peer may implement a heartbeat mechanism to monitor the stability of the connection.

[0056] In embodiments, the dual-connection peer and the second peer may implement a collision prevention protocol for establishing the first one-way connection and the second one-way connection without collision.

[0057] In a twelfth aspect, the technique described herein relates to a method for establishing a stable connection in a peer-to-peer network. The method comprises establishing, from a first peer, a first one-way connection for sending media streams from the first peer to a second peer, and establishing, from a second peer, a second one-way connection for sending media streams from the second peer to the first peer.

[0058] In embodiments, establishing the first one-way connection and establishing the second one-way connection may be performed using a PeerConnection API or a WebSocket.

[0059] In embodiments, the method may further implement a heartbeat mechanism between the first and second peers to monitor the stability of the connection.

[0060] In embodiments, the method may further implement a collision prevention protocol for establishing the first one-way connection and the second one-way connection without collision.BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Further features and exemplary advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the appended drawings, in which:

[0062] Figure 1 is a network topology and flow chart diagram of an exemplary fully connected peer-to-peer network in accordance with the teachings of the present invention;

[0063] Figure 2 is a flow chart of an exemplary relaying peer in a peer-to-peer network in accordance with the teachings of the present invention;

[0064] Figure 3 is a flow chart and nodal operation diagram of an exemplary relaying peer in a peer-to-peer network in accordance with the teachings of the present invention;

[0065] Figure 4 is a network topology and flow chart diagram of an exemplary simplified connected peer-to-peer network in accordance with the teachings of the present invention;

[0066] Figure 5 is a flow chart of an exemplary relaying peer producing multiple upstream media streams in accordance with the teachings of the present invention;

[0067] Figure 6 is a flow chart of a relaying peer implemented within a web browser in accordance with the teachings of the present invention;

[0068] Figure 7 is a flow chart of a method to monitor connections within a peer-to- peer network in accordance with the teachings of the present invention;

[0069] Figure 8 is a modular representation of a system for monitoring connections within a peer-to-peer network in accordance with the teachings of the present invention;

[0070] Figure 9 is a flow chart and nodal operation diagram for problem resolution in a peer-to-peer network in accordance with the teachings of the present invention;

[0071] Figure 10 is a flow chart and nodal operation diagram for establishing a connection in a peer-to-peer network in accordance with the teachings of the present invention; and

[0072] Figure 11 is a logical modular representation of an exemplary peer deployed in a system in accordance with the teachings of the present invention.DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0073] Peer-to-peer networks may comprise computer systems where each computer, or “node,” function as both a client and a server, enabling direct data sharing without requiring a central authority or dedicated server. In the context of the present disclosure, a node may refer to any device or computer connected to a network with the capability to send, receive, or forward data. A node may consist, for instance, of a personal computer, a personal device such as a mobile phone, a gaming console, a server, a router, a switch, or any other networked device participating in communication and data transmission within the peer-to-peer network. A node may be physical or exist in a virtualized environment.

[0074] Disadvantages associated with dedicated servers for streaming media may be avoided through the use of a peer-to-peer network; however, other issues associated with peer-to-peer networks, typically stemming from heterogeneity of peers, may render this alternative impractical for media streams. Media streams may typically be presented according to an actual time clock at a constant rate to a user while both transmission and processing of the media stream may comprise delays and packet losses, thereby impacting the stream rate. A delayed audio stream may result in a buffer underflow and unacceptable audio degradation, perceived as audio stuttering, for the receiving user. Similarly, a delayed video stream may result in unacceptable video degradation, such as a frozen image. Methods to improve the robustness of peer-to- peer media may include buffering the stream to compensate for rate variations, which may result in increased memory requirements for the peers and delays in mediapresentation. In a live context where media is captured and presented with as short a delay as possible, large buffers may not be an option.

[0075] Another challenge of peer-to-peer networks may relate to scalability. Reference is made to the drawings in which Figure 1 exemplifies a fully connected peer-to-peer network 100. For a small number of peers, such as peers 101 , 102, 103, and 104, it may be feasible for each peer to connect to one another in a fully connected peer network 100 topology as exemplified in Figure 1. The number of connections between peers in a fully connected topology may grow with a complexity of O(2), with each peer 101 , 102, 103, and 104 having to connect to n-1 other peers. In systems where each peer captures media and presents media from other peers, the fully connected topology may result in an exponential cost in total bandwidth, where each peer may be responsible for transmitting the captured data to all other peers of the system. In the depicted example, peers 101 , 102, 103, and 104 may have to transmit and receive streams 112, 113, 114, 121 , 123, 124, 131 , 132, 134, 141 , 142, and 143.

[0076] In a first set of embodiments, a method for relaying a media stream in a peer- to-peer network may be described. Figure 2 depicts an exemplary peer acting as a relaying peer 220 in a peer-to-peer network 200. In the example of Figure 2, the relaying peer 220 may be connected to more than one subscriber peer 230-1 ...n. The relaying peer 220 may receive a downstream media stream 215 from a media source 210. The media source 210 may capture a wide range of signals, broadly described as data over time. For instance, the captured signals may include sound waves, video frames, coordinates, poses, or words. The media source 210 may also transmit a previously captured media stream 215. The media stream 215 may have been stored to a storage device or originate from another node in the peer-to-peer network 200, or even from outside the peer-to-peer network 200. Persons skilled in the art will readily understand that other types of media sources 210 may be used with necessary adaptations, such as generating the streams procedurally or by combining existing streams, for instance, without affecting the teachings provided herein.

[0077] In the context of the present disclosure, for the purpose of providing illustrative examples, the downstream media streams 215 may include audio streams, video streams or any other stream of data sharing the common characteristic of being presented according to actual time. Actual time may refer to time based on a standard reference clock or a calendar with the passage of time flowing at the usual rate. Data captured over a period of one second may typically be presented over a period of onesecond in actual time. Actual time may be different from real-time, which may refer to the instantaneous or immediate response to an event. Real-time may emphasize the idea of reacting to events or data in a timely manner, without any noticeable delay and with minimal latency. Other examples of types of streams for which the teachings found herein may be applicable may include text, multimedia presentations, animations, object models and meshes, point clouds, coordinates, measurements such as temperature, velocity, pressure, etc. The downstream media streams 215 may originate from a live media capture (audio, video or others), a teleconference (with or without audio, video and text), or an on-demand stream, including, for example, video streams, audio streams, binary assets, etc. stored on storage, or generated on demand. Persons skilled in the art will readily understand that in the context of live video or audio capture, conferencing and on-demand streams, the main audio or video streams may be accompanied by other media streams, such as text and metadata, for example. T ext streams may include monitoring data such as logs, conversations, configurations data, task progression and may be provided raw or formatted in a JSON, XML, comma- separated values (CSV), YAML, Avro. In addition to specialized formats adapted to each stream type, data may use a binary representation such as Parquet, Protobuf, Gzip, or other similar formats to transport text and other types of streams. On demand media streams, when generated may include text-to-speech streams, music generation, speech synthesis, rendered graphics and 2D / 3D models, generated subtitles, simulated environments, transcriptions, prompts and responses from chatbots and virtual assistants.

[0078] The term “relaying” may possess slightly different meanings across technological fields. Within the field of network administration, relaying may be understood as the forwarding of data as received. In the context of the present disclosure, “relaying” may be understood as forwarding media such that the presented media resembles the captured media. Presented media may resemble captured media when, despite potential differences in format, encoding, or medium of support, the content or message conveyed is similar. When one media form resembles another, the underlying message or information may be perceived by a recipient as the same or equivalent, even if the data representation or sensory modality through which the message is delivered differs. Resemblance may hinge on the ability of the recipient to recognize and consistently interpret the embedded message across various forms of media. Characteristics of relayed audio may be modified, but meaning should remainintelligible from the perspective of a user, whether a human participant or a technological participant. For video streams, images may be altered, but the content of the images should still be recognizable such that two recipients would agree to having received equivalent content. Different contents may be recognizable differently depending on their nature. An audio stream may be modified to remove certain frequencies to achieve aggressive compression ratios, but the audio stream would remain recognizable if the speech remains intelligible and unaltered. Other types of media, such as meshes and textures, may be altered as long as the objects represented thereby may still be recognized once presented. Presentation of media streams should be understood with proper adaptation to each type of media stream. Presentation of a video stream may consist of displaying the frames to a display device such as a monitor or a projector, while presentation of an audio stream may consist of playing back the waveform through an audio device such as a speaker or headphones. Presentation of a text stream may consist of rendering the text to a display device or may consist of playing back a synthetic voice. Broadly speaking, presentation of a media stream may consist of making the information embedded in the media stream accessible to the mind of a participant. In some cases, the term “relaying” may further encompass additional actions such as converting the media type into another form, such that the underlying meaning is still conveyed. The underlying meaning, in the context of this disclosure, may refer to the fundamental content or information conveyed through the media stream, independent of the form, method, or technology used for its delivery. The underlying meaning of a media stream is what may be intended to be received, understood, or acted upon by the participant, regardless of the medium. For example, a video sequence may be relayed as a stream of text describing the content of the video sequence.

[0079] Subscribing peers, from the perspective of a relaying peer, may be defined as one or more peers connected to the relaying peer that have expressed interest in the downstream media stream. The communication of interest may be achieved by first transferring a list of available streams over a data channel, each represented by an identifier and description, from the relaying peer to the subscribing peer. The subscribing peer may then communicate back, over the data channel, a list of stream identifiers that the subscribing peer wishes to receive and subscribe to. In one embodiment where peers participate in a teleconference, the subscribing peer may request any media streams associated with other participants of the teleconference. Insuch an embodiment, a subscribing peer may wish to receive the media stream from any participant it is not already receiving. In other embodiments, a subscribing peer may be interested in a particular media stream for the purpose of relaying it and making it available to other peers. Subscribing peers receiving the downstream media stream may be referred to as downstream peers. The downstream peers may themselves be relaying peers for other peers and may alternatively or additionally be peers presenting the media stream. Any given peer may subscribe to a media stream from more than one source. For instance, a subscribing peer may receive the media stream from two different relaying peers, or from both a capturing peer and a relaying peer, such that when one of the connections is lost, the stream may continue to be relayed or presented without interruption.

[0080] The relaying peers 220 may not capture or present any part of the media stream and may act purely as facilitators within the peer-to-peer network 200. A facilitator within the peer-to-peer network 200 may be a node that subscribes to a media stream for the purpose of making it available to other peers rather than for presenting the stream to a participant at the node. A peer may be considered reliable due to characteristics contributing to the resilience of the peer-to-peer network, such as high bandwidth capacity, low jitter, high CPU and memory availability. Reliable peers may be added to the peer-to-peer network as dedicated relaying peers. Dedicated peers may help to improve resilience and performance of the peer-to-peer network in certain scenarios, such as by providing high trans-oceanic bandwidth and managing quality of service for a system spanning America and Europe. In such an example, two dedicated peers may relay most, or all, of the media streams across the ocean. When justified, dedicated peers may be allocated within the infrastructure of major network services providers, such as Google GCP, Amazon AWS, or Microsoft Azure, to utilize the related backbone network reliability. Although a scenario where dedicated relaying peers are used may resemble the use of dedicated servers, these embodiments may still be advantageous since the dedicated relaying peers may not require over-provisioning, and maintenance and updates thereof may be performed without interrupting ongoing sessions, as subscription from the subscribed peers may be adjusted as the connection of the dedicated relaying peer changes.

[0081] Reference is made concurrently to Figure 2 and Figure s, depicting an example of a relaying peer connection 1000 and relaying 2000 sequence. The relaying peer 220 may forward the downstream media stream 215 into an upstream mediastream 225 to the subscribing peers 230-1 ...n in the peer-to-peer network 200. Upon connecting 1010 and receiving the subscribing peer profile 1020, the relaying peer 220 may compute 1030 an upstream media profile for the upstream stream 225. The downstream media stream 215 may be partially or fully decoded 2010 while being received by the relaying peer 220. The decoded media stream may be transformed 2020 into an upstream media stream in accordance with the computed upstream media profile. The upstream media stream 225 may then be transmitted to the subscribing peers 230.

[0082] Decoding 2010, in the context of the media streams, may refer to the process of converting encoded data back into an original or a usable form. This decoding 2010 process may vary in complexity and scope, encompassing simple protocol-level decoding to extensive decompression and reorganization of data. Partial decoding might involve interpreting the header information of a media packet to understand its metadata, such as format and size, allowing the relaying peer to manage the stream without altering the core data payload. Full decoding, on the other hand, may involve decompressing the encoded media to retrieve the original audio, video, or text content. For example, in video streaming, full decoding may entail converting compressed video files into viewable frames, a process that might involve decompressing the data using codecs like H.264 or VP9, and reorganizing the data into a sequence of images that may be displayed or transformed. Similarly, in audio streaming, decoding may involve decompressing audio files encoded in formats like MP3 or AAC into waveforms that can be output through speakers. Decoding 2010 may range from low-level operations, such as parsing protocols, to complex processes that restore data to its full fidelity, further processing or presentation to the end-user.

[0083] Transforming 2020 the downstream media stream 215 in accordance with the profile of subscribing peers 230-1 ...n may improve the robustness of the system. A robust peer-to-peer system may exhibit characteristics such as low latency, low data loss, few disconnections, high availability and redundancy of the data, and high availability of processing resources. By accounting for the processing requirements or bandwidth required for the subscribing peers 230-1... n to present the upstream media stream 225, data loss and latency may be reduced while increasing availability of processing resources when compared to the downstream media stream 215.

[0084] Reference is made concurrently to Figure 2 and Figure 4, depicting an example of a simplified peer-to-peer network 200 topology. Reliability of the networkmay be improved by reducing the number of connections between peers. For instance, in one embodiment, each peer 110, 120, 130, and 140 may connect to only two other peers, thereby reducing complexity from O(2) to O( ). In such an embodiment, media streams 112, 123, and 143 may be relayed to non-connected peers as upstream media streams 3014, 3013, 3024, 3031 , 3041 , and 3042. More precisely, in the depicted example of Figure 4, peer 110 may transmit a downstream media stream 112 to peer 120. Peer 120 may present, such as by displaying or playing back, the stream 112, but may also relay downstream media stream 112 as upstream media stream 3013 to peer 130. Peer 130 may present stream 3013, but may also relay downstream media stream 3013 as upstream media stream 3014 to peer 140. Similarly, peer 140 may transmit stream 143 to peer 130. Peer 130 may present stream 143, but may also relay downstream media stream 143 as upstream media stream 3042 to peer 120. Peer 120 may present media stream 3042, but may also relay downstream media stream 3042 as upstream media stream 3041 to peer 110. By relaying media streams in the manner depicted in Figure 4, all media streams may be shared among 4 peers using (4 - 1) x 2 = 6 connections instead of 4 x 3 = 12 when compared to a fully connected topology. In a peer-to-peer network comprising 100 peers, scalability advantages may be more apparent: this method may result in (100 - 1) x 2 = 198 connections instead of 100 x 99 = 9900 connections for a fully connected topology.

[0085] Although reducing the total number of connections may improve network stability, it may still be desirable in certain circumstances to use separate channels or network connections to transmit each stream individually, even when the originating peer and the destination peer are the same. For example, streams 121 , 3031 , and 3041 transmitted from peer 120 to peer 110 may be transmitted over the same data channel using a single network connection, multiple data channels over a single network connection, or multiple data channels over multiple network connections. In some embodiments, media streams may be merged as a single media stream and transmitted over a single data channel over a single connection. For instance, multiple video streams may be combined as a mosaic and transmitted as a single video stream. Similarly, multiple audio streams may be combined as a single multi-channel audio stream and transmitted over a single data channel over a single network connection.

[0086] In other embodiments, peers may be connected with more than two peers, according to the circumstances. For example, in a scenario where several peers are on the same local network and another peer is remote, all peers on the local network maybe fully connected to one another, and only one of these peers may be connected to the remote peer, thereby relaying the media streams of all the other local peers.

[0087] Reference is now made back to Figure 2 and Figure 3 concurrently. Computing 1030 the upstream media profile by the relaying peer 220 may be performed from the downstream media profile 214 associated with the downstream media stream 215, and the peer profile 1020. The downstream media profile 214 may be provided by the downstream peer or the media source 210 and may include similar properties as the computed 1030 upstream media profile but applicable to the downstream media stream 215. The peer profiles 1020 include the limiting constraints of the subscribing peers 230-1... n and the limiting constraints of the relaying peer 220. Such limiting constraints may include network and processing capabilities as further defined hereinbelow.

[0088] In the context of the present disclosure, “limiting constraints” may refer to a set of predefined conditions or parameters that guide and restrict the processing and transmission of media streams within the peer-to-peer network. The limiting constraints may be established to ensure that the media streaming process remains within the operational capabilities of the participating peers and the network infrastructure. Limiting constraints may help maintain a balance between performance, quality, and resource utilization. Limiting constraints may include, for example, network and processing capabilities, such as available bandwidth, network jitter, packet loss, processing power, and memory capacity of the peers, codec and format compatibility, such as compatible codec families or profiles, ensuring that media streams are encoded and decoded in a manner that is supported by all participating peers, security and compliance constraints, such as acceptable encryption methods and strengths to ensure data confidentiality and compliance with security standards, or quality and performance standards, such as thresholds for minimum accepted quality, including resolution and frame rate.

[0089] The limiting constraints may originate from various sources, including but not limited to configurations, estimates, benchmarks, historical data, and any combination thereof. The capabilities may include performance characteristics such as availability of processor and memory storage, bandwidth, network jitter and packet loss, as well as qualitative characteristics such as availability of codecs, hardware accelerations, network protocols, supported media formats, and supported encryption methods and strengths. The limiting constraints may include thresholds, ranges, labels,flags, complete or partial media format, or any other specifier expressing media characteristics. Examples of limiting constraints may include maximum bitrate, minimum and maximum frame size, sampling or frame rate, depth, codec family or codec profiles, range of quantization parameters or other codec-specific settings, encryption strategy and strength, etc. The limiting constraints may ensure aspects such as minimum accepted quality, maximum processing requirements, security compliance, and compatibility. Persons skilled in the art will readily recognize that the limiting constraints may be adapted to the specific media type of the stream and the configuration of the system.

[0090] The computed 1030 upstream media profile may consist of a common denominator, accommodating several subscribing peers 230-1 ... n limitations while also being achievable, given the processing resources and bandwidth available to the relaying peer 220 and compatible with the downstream media profile 214. When the downstream media profile 214 is compatible with the peer profiles 1020, the media may be forwarded “as-is,” without transformation.

[0091] Reference is made concurrently to Figure 2, Figure s, and Figure s, depicting relaying using two different media profiles 1032 and 1034. In some embodiments, the relaying peer 220 may choose to split the subscribing peers 230- 1...n into multiple subscriber groups 242 and 244 and compute multiple upstream media profiles 1032 and 1034. Generating multiple subscriber groups 242 and 244 may prevent scenarios where a single strongly limiting subscribing peer, such as 230-1 , forces the relaying peer 220 to generate a common denominator profile that is overly limited when compared to the other subscribing peers 230-2... n. The number of subscriber groups may depend on the processing capabilities of the relaying peer 220, such as if the relaying peer 220 has computing resources to handle a maximum of two different media profiles. The clustering of groups may be achieved using various strategies such as partition clustering, including K-Means clustering, hierarchical clustering, or density-based clustering. In embodiments where multiple subscriber groups are configured, the relaying peer 220 may further decode 2010 the downstream media stream 215 only once into a decoded stream and transform 2022 and 2024, encode 2032 and 2034, and transmit upstream media streams 222 and 224 according to the media profiles 1032 and 1034 of each subscriber group 242 and 244. Reusing the decoded downstream media stream for multiple upstream media streams may allow a single relaying peer 220 to be more efficient at providing multiple optimized upstreammedia streams compared to using multiple relaying peers, each providing a single upstream media stream.

[0092] In one embodiment, the computed 1030 upstream media profile 1034 may be configured to dynamically apply different parameters to the processing of the downstream media stream 215 based on changing requirements of the subscribing peer 230 connection. As an example, when the available bandwidth of a subscribing peer 230-1 connection varies over time, the relaying peer 220 may determine the current available bandwidth and may compute 1030 the upstream media stream to transform the downstream media stream 215 into the upstream media stream 225 according to the current available bandwidth.

[0093] Similarly, when the capabilities of a subscribing peer 230-1 change, the subscribing peer 230-1 may communicate a new peer profile 1020 to the relaying peer 220. The relaying peer 220 may then compute 1030 the upstream media profile 1034 to accommodate the new peer profile. When the relaying peer 220 is processing multiple media profiles concurrently, the relaying peer 220 may assign the subscribing peer 230-1 to a better fitting subscribing group. Alternatively, a new subscribing group may be created to accommodate the new subscribing peer profile.

[0094] Reference is made back to Figure 2 and Figure 3 concurrently. When no common denominator for the upstream media profile 1034 may be found to satisfy all the subscribing peers 230-1... n, and the relaying peer 220 is unable to split the subscribing peers 230-1... n into subscriber groups, the relaying peer 220 may choose to ignore some limiting requirements and compute 1030 an upstream media profile 1034 accommodating a subset of the subscribing peers 230-1... n. The subset may encompass the largest possible number of subscribing peers 230-1... n, which may, for instance, exclude peer 230-3. Upon receiving 1020 an incompatible upstream media profile, the subscribing peer 230-3 may choose to accept the incompatible upstream media profile or may alternatively choose to subscribe to another relaying peer (not shown) in the peer-to-peer network 200. In some embodiments, the incompatible subscribers, such as 230-3, may temporarily accept the incompatible media stream while searching for an alternative, thereby presenting a possibly degraded media stream rather than not being subscribed. In other embodiments, weights or cost functions may be associated with limiting constraints to guide the search for an optimal media profile. For example, a video stream with heavier weight assigned to the video resolution than to the video framerate may result in a media profile that isslightly outside the framerate limits but within the limits of the video resolution. Similarly, cost functions may increase in a nonlinear fashion as the media profile diverges from the limiting constraints, allowing for compromises. When weights are provided, computing 1030 of an upstream media profile may be performed by resolving conflicting requirements considering the weights. More specifically, the relaying peer 220 may compute several media profiles (not shown) and assign a score (not shown) to the several media profiles considering the weights. Optimization (not shown) of the media profiles may be performed in accordance with various strategies, as provided by different weights. Examples of strategies to select the upstream media profile for a group of subscribing peers may include choosing an upstream media profile that maximizes the score for any one peer while being compatible with other peers, or one that maximizes the average score of all peers, or one that maximizes the minimum score across all subscribing peers.

[0095] In certain circumstances, when transforming 2020 the downstream media stream 215 according to the computed 1030 upstream media profile into the upstream media stream 225, it may not be necessary to fully decode the downstream media stream 215. For example, in some embodiments, the downstream media stream 215 may already include alternative sub-streams multiplexed together to support different media profiles within a single media stream. In these embodiments, the relaying peer 220 may partially decode 2010 the downstream media stream 215 up to the demultiplexing of the sub-streams and may thereafter relay only one of the sub-streams to the subscribing peer, such as 230-4. In other embodiments, a structure of the media stream may support transcoding (not shown) instead of decoding 2010. Likewise, the structure of the media stream may support encoding 2030 without being fully decoded 2010. For example, with MPEG and AVC, it may be possible to decode 2010 the stream into macroblocks, transform 2020 the stream by re-quantizing the stream, and reuse the original stream’s information including motion vectors and motion compensation when re-encoding 2030 to reduce the bitrate. Other embodiments may choose to decode 2010 a video stream, transform 2020 the stream using spatial resampling, for example, and skip B-type frames, thus reducing the frame rate without fully decoding 2010 and re-encoding 2030 the video stream.

[0096] Transformations 2020 of the media stream 215 may include resampling spatially, resampling temporally, re-quantizing, mesh decimation, changing the complexity of the stream by adding or removing predicted data such as the frequencyof P-type and B-type frames in video streams, joining or combining audio channels in audio streams, recoding to a different codec family or profile, discarding, merging, or aggregating data points, etc. Persons skilled in the art will readily understand that these transformations may need to be adapted to the specific media type. For example, spatial resampling fora video stream may be adapted to adjusting the width and height of the video canvas, while temporal resampling may be adapted to changing the frame rate. Re-quantization, for a video stream, may be adapted by changing the pixel component depth, such as reducing from 10 bit to 8 bit, increasing chroma subsampling, or increasing the quantization factor applied to DCT or wavelet coefficients. When the media stream 215 is an audio stream, spatial resampling may be adapted by adjusting the number of audio samples captured for each audio channel per second. Re-quantification may be adapted for audio by reducing the depth of the sample, such as reducing from 24 bit to 16 bit, or the quantization coefficient applied to the MDCT factors, for instance. Similarly, spatial and temporal resampling may be adapted to streams of data points through the aggregation of samples, for example, and re-quantification may be performed by reducing the precision of their measurements. The transformation of the downstream media stream 215 may decrease or maintain the quality of the upstream media stream 225 when compared to the downstream media stream 215. In one embodiment, the upstream media profile may be configured to reduce the bitrate of the upstream media stream 225 when compared to the downstream media stream 215. The transformation may apply lossy or lossless operations. When lossy transformation is performed, some of the original data may be lost, and the upstream media stream 225 may be considered degraded when compared to the downstream media stream 215. Lossless operations may include transforming a structure of the media stream 215 such as to reduce the entropy thereof, resulting in the upstream media stream 225 that can be represented with fewer bytes without being degraded when compared to the media stream 215.

[0097] When the computed 1030 upstream media profile requires reducing the bitrate of a downstream media stream 215 for which lossy encodings are possible, such as for audio and video streams, applying more aggressive compression settings may typically help reduce the bitrate of the upstream media stream 225 by 30 to 40% while retaining a good quality within a PSNR range of 30 dB to 40 dB. With a video stream, further savings may be achieved by reducing the frame rate and video frame dimensions. When combining multiple strategies aimed at reducing the size of theupstream media stream 225, reductions of up to 70 to 80% of the initial size of the downstream media stream 215 may be envisioned.

[0098] The transformation 2020 of the downstream media stream 215 may also increase the quality thereof. For example, in another embodiment, a relaying peer 220 may be connected to a capturing peer with limited processing capabilities and apply upscaling algorithms, such as including neural filters, to improve the quality of the upstream media stream 225. When the downstream media stream 215 contains a video stream, a relaying peer 220 may interpolate frames therein to increase the frame rate of a related video stream in the upstream video stream 225. In another embodiment, audio may be improved by applying noise filtering, thereby improving both perceived quality and bandwidth.

[0099] The transformation 2020 of the downstream media stream 215 may also involve converting from one media type into another. For example, in one embodiment, a relaying peer 220 may transcribe an audio stream into a text stream using a speech- to-text algorithm. In another embodiment, a video stream may be converted into an animation stream where the gestures of a participant may be streamed as gestures of an avatar. In other embodiments, the transformation may involve converting an audio stream carrying speech in one language into synthetic speech in another language. In yet another embodiment, a mesh stream may be relayed as rendered images.

[0100] The transformation 2020 of the downstream media stream 215 may also involve combining several media streams into one. In one embodiment, a text stream may be rendered onto a video stream, for example, as subtitles rendered over the frames of the video. Multiple audio or video streams may be combined into one. For instance, in one embodiment where the subscriber peers 230-1 ... n are participating in a teleconference, the relaying peer 220 may combine the downstream audio stream from a plurality of the subscriber peers 230-1... n into a single upstream audio stream. Similarly, multiple downstream video streams from more than one of the subscriber peers 230-1... n may be rendered together as a mosaic where multiple video stream frames are rendered side by side.

[0101] When multiple media streams are combined, such as merging several audio streams into a single output or rendering multiple video streams as a mosaic, the merged stream may be managed through a central processing module that aggregates input streams and maintains a synchronized timeline across all participating peers. Thismay involve aligning timestamps of incoming media packets to ensure synchronization of all streams in real time, irrespective of variations in latency or network conditions. The system may dynamically adjust buffer sizes and employ jitter buffers to accommodate variations in network delay, ensuring synchronization of audio streams with each other and with corresponding video streams. Additionally, audio leveling techniques may be applied by the system to balance the volume levels of merged audio streams, providing a consistent listening experience. For video mosaics, management of layout and scaling may be executed by the system, dynamically adjusting the size and position of individual video frames to fit the composite display. A feedback loop may continuously monitor the synchronization quality, allowing real-time adjustments by the system to maintain alignment across all streams, thereby enhancing the overall user experience.

[0102] In some embodiments, the transformation 2020 of the downstream media stream 215 into the upstream media stream 225 may not require a specific transformation step other than representing the downstream media stream 215 as the upstream media stream 225 using a different encoding. In the process of decoding 2010 the downstream media stream 215 and encoding the upstream media stream 225 using another format as a transformation 2020, it may be possible to perform some of the necessary transformations combined with the decoding or encoding process. For example, adjusting the frame rate may be performed without any transformation filter by simply skipping frames. In other embodiments, the media stream 215 may be first decoded 2010 into a memory buffer, then transformed 2020 once or several times before being encoded 2030 into the upstream media stream 225.

[0103] The encoded upstream media stream 225 may be transmitted to subscribing peers 230-1... n using various network protocols. In one embodiment, the subscribing peers 230-1 ...n may receive the upstream media stream 225 through a data channel, such as a WebRTC data channel or media channel, comprising audio or video tracks, for example. The data channel may optionally be configured to retransmit lost packets, depending on the media type of the downstream media stream 215 and the computed 1030 upstream media profile. Activation or deactivation of the retransmission of lost packet feature may be performed considering that some media types are more resilient than others to packet loss. The upstream media stream 225 may be transmitted to each subscribing peer 230-1 ...n configured with the upstream media profile, meaning the same media stream using the same upstream media profile.

[0104] In some embodiments, one of the subscribing peers 230-1...n may be a peer-to-peer client participating in a conferencing system in a peer-to-peer network 200 embodied as a web browser. In such embodiments, the conferencing system may typically use a WebRTC framework for processing content streams. In other embodiments, the peer-to-peer client may be a native application, such as a desktop application, mobile application, or any other type of application running on a specific computerized device.

[0105] Reference is made concurrently to Figure 2 and Figure s, depicting an example of an in-browser embodiment for a teleconference system. In one embodiment, the relaying peer 220 may be implemented within a web browser 3000. A web browser 3000 may considerably limit access to operating system resources, including access to specialized hardware, threads, and networking devices. In one embodiment, the relaying peer 220 participating in a teleconference system may execute within a web browser 3000 by receiving into a downstream media buffer 3010 a media stream 215 received from a media source 210 and comprising a video stream. The relaying peer 220 may decode 2010 the video stream of the downstream media stream 215 into a source canvas 3020, encode 2030 the decoded media stream from the source canvas as a way to transform 2020 into an upstream media stream 225 in accordance with an upstream media profile, and transmit the upstream media stream 225 using a stream publisher 3040 over an RTC connection. When the upstream media profile requires transformation of the downstream media profile and the transformation cannot be performed during the decoding 2010 and encoding 2030 steps, an additional step may transform 2020 the decoded video stream from the source canvas 3020 into a transformed video stream in a destination canvas 3030. When a transforming 2020 step is performed, the encoding 2030 step may use the destination canvas 3030 instead of the source canvas 3020. The encoding 2030 may use the Mediastream API or the WebCodecs API provided by the web browser 3000. The stream publisher 3040 may use the WebRTC framework provided by the web browser 3000 to manage the RTC connection. In one embodiment, decreasing the framerate of a video stream may be achieved by performing the encoding 2030 step at a lower frequency than the decoding 2010. In one embodiment, the resampling of a video stream from a downstream media stream 215 may be achieved by copying the content of the source canvas 3020 into a destination canvas 3030 with different dimensions. In someembodiments, the source canvas 3020 and the destination canvas 3030 may be HTML canvases.

[0106] In some circumstances, such as when implementing execution in a web browser 3000, changing characteristics of an upstream media stream 225 may cause the stream publisher 3040 to renegotiate a media channel with the subscribing peers 230-1 ...n. The transform 2020 step may be used to adjust from the downstream media stream 215 to the upstream media stream 225 without altering the characteristics of the media channel. For example, when adjusting for constrained bandwidth, the source canvas 3020 may be transformed 2020 before being encoded 2030 such that the media channel used by the stream publisher 3040 remains untouched. Transformations 2020 that reduce the bandwidth without disrupting the media channel may include adjusting the sampling rate, compression settings, or applying low-pass filters on video or bandpass filters on audio, for example.

[0107] In a second set of embodiments, the disclosure may focus on providing examples of solutions in the form of various systems, devices, and methods for realtime monitoring and detection of problems in peer-to-peer connectivity.

[0108] Monitoring peers as a peer-to-peer network scales up may be difficult as the effort to monitor each peer of the network grows linearly with the number of peers. In some implementations, such as within a web browser or a similar single-threaded JavaScript engine, peers may not have access to all the resources of the operating system and may not be allowed to dispatch new threads for monitoring purposes. Saturating the JavaScript thread with monitoring operations may result in overall degradation of the peer as the number of monitored connections grows. When degraded, processing resources of the peer may be saturated, and the peer may be unable to perform critical operations such as receiving, capturing, transmitting, and presenting the media streams. Interactivity of the system may also be degraded as the system may be unable to dequeue and process user events, resulting in a frustrating experience for the participant.

[0109] Experimentation has demonstrated that most modern web browsers may execute a single WebRTC “GetStatsQ” operation at any given time, blocking further operations until the operation completes. The “GetStatsQ” operation, in the context of the described examples, may aim at gathering monitoring statistics related to a WebRTC connection and may therefore be executed individually for each of the peersreachable via one of the WebRTC connections. The gathered statistics may then be used to troubleshoot issues and potentially make improvements. The execution time of the “GetStatsQ” operation may depend on I / O operations, including latency and round trip time, and may execute over long periods independently of the running hardware processing power. As the number of monitored peers increases, the delay to obtaining monitoring statistics may also increase, making it impossible to continuously monitor a large number of peers. Paradoxically, as the number of peers grows, the likelihood of detecting a peer with degraded conditions may also increase. Historically, the paradox of a high number of connections leading to a high likelihood of condition degradation has been a limiting factor in the number of peers considered stable in a peer-to-peer topology.

[0110] Figure 7 depicts a flow chart of a weighted round-robin method 5000. The method 5000 may be provided to get statistics for each connection one at a time, thereby mitigating the potential blocking issue associated with existing statistics gathering methods. The method 5000 may collect real-time connection statistics from the connections using a weighted round-robin collection mechanism.

[0111] A monitoring queue may be used to collect statistics for all connections that should be monitored. The monitoring queue may be altered as new connections are made by the peer or as connections are closed by the peer. Each connection in the monitoring queue may be associated with a weight computed based on several data points. Weights for each connection may be recomputed periodically or upon reaching a trigger such as a timeout or an event such as a failure to transmit or to receive acknowledgment from the connected peer. Examples of data points considered at the time of computing or recomputing the weights may include delay since the last statistics gathering, where connections that have not been monitored for a long period may receive a higher weight, and quality of service measurement, where connections that have been found to be weaker may also be associated with a higher weight. Skilled persons will readily understand that identification of a peer connection may be performed in different ways, such as by a connection identifier or by a peer identifier.

[0112] A weighted round-robin scheduling process may be used by the monitoring module to facilitate efficient monitoring of multiple peer-to-peer connections by dynamically prioritizing them based on computed weights. The weighted round-robin scheduling may ensure that connections showing signs of potential degradation receivetimely attention, thereby maintaining the overall stability and performance of the network.

[0113] For example, a weight VK for each connection may be calculated using a combination of key network performance indicators, represented byW =a / L+ p x P + Y Xj +5 / B+ s x T

[0114] where L may represent latency, measured in milliseconds; P may represent the packet loss rate, as a percentage; J may represent jitter, measured in milliseconds; B may represent the available bandwidth, measured in kilobits per second; and T may represent the time elapsed since the last statistics gathering, measured in seconds. The coefficients (a, / ?, y, 6, s) may determine the relative influence of each parameter on the overall weight. These coefficients may be adjusted based on the specific requirements of the network or application. Weights may be recalculated periodically or in response to significant changes in network conditions. The system may prioritize connections with higher weights, thereby indicating a greater need for monitoring due to potential performance issues. The dynamic adjustment may allow adaptation to changing conditions in real-time. For example, consider a connection with the following characteristics: latency of 200 ms, packet loss of 0.005 (equivalent to 0.5% or one every 200 packets), jitter of 100 ms, bandwidth of 1000 kbps, and last monitored 30 seconds prior. Using the formula, the weight may be calculated as follows:0.2 0.4W = - + 30 x 0.005 + 0.1 x 100 + - + 0.5 x 30 = 25.1514200 1000

[0115] Connections may be sorted based on their computed weights, with those having higher weights being monitored more frequently. This may ensure that connections at risk of degradation receive timely attention, thereby maintaining overall network stability and performance.

[0116] The monitoring queue may indicate the next connection to monitor as the connection with the highest weight. Identifying the next connection to monitor may be achieved, for example, by sorting 5010 the monitoring queue in descending order and selecting 5020 the first connection in the queue.

[0117] Once a connection has been selected 5020, statistics for the connection may be gathered 5030. When implemented using the WebRTC framework, monitoring may be achieved using a modified GetStatQ operation specifically executed on the selectedconnection, meaning the modified GetStat() operation is executed for the purpose of gathering 5030 the statistics for the selected connection only and does not trigger additional GetStat() operations for other connections.

[0118] Issues may be detected 5040 by comparing statistics against thresholds. Thresholds for detecting issues based on the statistics may be dynamically adjusted according to the number of connections, allowing higher effectiveness, which may be particularly advantageous when the number of connections increases. For instance, peer-to-peer connectivity used for teleconferencing may require multiple streams of sounds and images to be shared at a minimum frame rate, such as 15 frames per second or more, with a synchronicity between the sound and images not exceeding a predetermined threshold of jitter of 125 ms, to be considered acceptable. The thresholds may represent a range of acceptable values for the connection to be considered healthy.

[0119] Dynamic adjustment of thresholds may account for various parameters influencing network performance. Key parameters may include the number of connections, average connection quality, historical performance data, and specific network conditions such as latency, packet loss, and jitter. Initially, base thresholds may be established for each statistic under minimal load, serving as a baseline. As connections increase, thresholds may be adjusted incrementally; for instance, latency thresholds may increase linearly by 10 milliseconds for every additional five connections, up to a predefined maximum. Conversely, when connections decrease, thresholds may revert to the baseline to maintain optimal conditions. The algorithm may incorporate adaptive mechanisms, dynamically adjusting thresholds based on connection quality. If a significant percentage of connections exhibit poor quality, thresholds may be temporarily relaxed more aggressively to stabilize the network. Machine learning algorithms may enhance this process by analyzing historical data and predicting optimal thresholds to preemptively mitigate network degradation. A feedback loop mechanism may continuously evaluate the effectiveness of these adjustments, refining the algorithm based on data collected about the success rate in maintaining connection quality. Illustrative examples, such as a scenario where increased connections lead to incremental threshold adjustments, may be depicted using charts or graphs, showing how the mechanism operates in real-time as connection numbers fluctuate.

[0120] An alert representing an issue may be generated when the connection statistics collected from a connection fall outside of their corresponding statistics thresholds. Problems may then be detected 5050 by identifying patterns in the detected issues. For example, a pattern involving CPU saturation and low frame rate may be associated with a problem related to the dimensions of a video stream being too large.

[0121] From the detected problems, a solution may be found 5060 to mitigate each problem. For example, a solution associated with a problem of video stream dimensions being too large may be resolved by reducing the dimensions of the video stream being forwarded to the peer.

[0122] The connection profile may then be updated 5070 to implement the solution. In the context of the present disclosure, weaker connections may be scored according to alerts suggesting degradation in the transmission of data. Characteristics of degradation may include CPU cycle saturation, higher latency, higher rates of packet loss, lower bandwidth, and higher jitter, for example. When a remote peer has subscribed to a media stream, additional statistics may be considered in the monitoring and the scoring of a weaker connection. For instance, the availability of CPU cycles and memory may be considered, with lower CPU and memory availability being associated with signs of degradation. When transmitting a video stream, additional characteristics may be considered, such as the negotiated frame rate and rate of frames being dropped during the video stream presentation.

[0123] Once the connection profile is updated 5070, the process may be repeated for as long as there are connections to monitor 5090.

[0124] The monitoring peer may allocate a monitoring budget, such as a budget of 500 ms renewed every second, and prioritize connections that warrant more attention within the monitoring budget. Upon exhausting the monitoring budget, the process may await or yield 5080 for a duration of time before resuming allowing other processes to execute within the same thread. Yielding 5080 may be useful in some embodiments to ensure that the monitoring thread is not starving other processes executing on the same thread.

[0125] Figure 8 depicts a modular representation of a monitoring peer 5100. In the examples depicted herein, a monitoring module 6010 may be described, such as part of a device and / or a system. The monitoring module 6010 may be configured to monitor multiple connections in real-time and collect connection statistics using a weightedround-robin mechanism. The connection statistics may include information such as latency, packet loss, jitter, and bandwidth. The monitoring peer 5100 may further comprise a network interface module 8170 for communicating with other peers, and a processor module 8120.

[0126] To avoid the problem of degraded performance and instability associated with the current method of getting connection statistics, the monitoring module 6010 may use a weighted round-robin mechanism to get the statistics of each connection one at a time. The round-robin mechanism may ensure that the CPU is not blocked and that a large number of connections can be monitored without degrading performance.

[0127] In operation, the monitoring module 6010 may continuously collect connection statistics or may be limited by a processing budget to ensure that other components of the peer are not starved.

[0128] The collected statistics may then be compared to one or more thresholds for each connection. The thresholds may be adjusted dynamically. When a connection exceeds a threshold set for a particular statistic, the monitoring module 6010 may generate an alert indicating that a problem has been detected. For instance, thresholds on latency may be set at 500 ms. A dynamic threshold on latency may increase from 200 ms with 3-5 peers and increase to 800 ms when connected to more than 50 peers. Thresholds on packet loss may be in the range of 0.5%. A dynamic threshold on latency may start at 0.05% with 3-5 peers and increase to 1 % when connected to more than 50 peers. Thresholds on jitter may be set to 125 ms. A dynamic threshold on jitter may be 50 ms with 3 to 5 peers and increase to 250 ms when connected to more than 50 peers. Thresholds on bandwidth may be around 500 kbps. A dynamic threshold on bandwidth may be set at 1 mbps for each peer with 3 to 5 peers and decrease to 250 kbps for each peer when connected to over 50 peers.

[0129] In the context of the present disclosure, “continuous” monitoring may refer to a repeated sequence of data-gathering operations carried out at intervals that are short enough to detect transient or evolving issues in real time, rather than requiring manual reactivation or being confined to a single discrete inspection. Each repetition of the sequence may collect the latest relevant connection statistics, compares them to a set of dynamically adjusted thresholds, and, if necessary, triggers an alert. In practical scenarios, continuous monitoring could involve performing these data-gathering operations every two seconds, every five seconds, or in another relatively short cyclebased on the constraints of processing budgets and the available system resources. For example, a peer-to-peer teleconferencing system may check jitter, latency, packet loss, and bandwidth usage every five seconds while allowing for more frequent checks (such as every two seconds) when spikes in latency appear. Another example could involve a system that monitors connections in batches at one-second intervals to identify drops in audio or video quality for a set of over fifty connected peers. While these intervals are typically finite, the monitoring process remains ongoing over the lifespan of the connection, ensuring that any detected degradation can be addressed as soon as it becomes apparent.

[0130] The present disclosure exemplifies several advantages over existing mechanisms for monitoring peer-to-peer connectivity. By using a weighted round-robin mechanism for connection statistics collection, the CPU blocking issue associated with existing solutions may be at least partially averted, resulting in improved performance and stability. Prioritization of weaker connections may also increase the real-time characteristics thereof. When thresholds are dynamically adjusted, detecting problems may be based on the number of connections, thereby further improving effectiveness and real-time monitoring capabilities.

[0131] In a third set of embodiments, the disclosure may focus on providing examples of solutions in the form of various systems, devices, and methods for self- healing peer-to-peer media streaming in real-time. In particular, the disclosure may provide examples for detecting problems in peer-to-peer media streaming and for automatically adjusting the resolution, frame rate, and / or bitrate to ensure stable streaming while also considering latency.

[0132] Figure 9 depicts an exemplary flow and nodal operation chart of a method 6000 for autonomous real-time problem resolution in a peer-to-peer network. In the examples depicted herein, a streaming peer 6005 may be depicted as having a monitoring module 6010, a profiling module 6020, and a streaming module 6030. In other non-depicted embodiments, the monitoring module 6010 and / or the profiling module 6020 may be implemented as distinct nodes. In the context of the example of Figure 9, the streaming module 6030 of the streaming peer 6005 may currently maintain a monitored connection with an altering peer 6040.

[0133] The monitoring module 6010 may be configured to monitor each connection of the peer in real-time for the streaming peer 6005, including the connection with thealerting peer 6040, and collect connection stats, which may advantageously, but not necessarily, be performed in accordance with the teachings presented in relation to the second set of embodiments described hereinabove.In this disclosure, “real-time” may refer to the ability of a peer-to-peer network system or module to detect, process, or respond to changing conditions without introducing perceptible delays to participants. It implies that the collection of statistics, triggering of alerts, and application of remedial actions happen on a schedule allowing any corrective measure to be executed before media quality becomes noticeably degraded. For instance, real-time may involve sampling connection metrics, such as latency and jitter, at intervals of a few seconds or less to maintain an active stream with minimal disruption. Real-time responses may include end-to-end delays spanning from well under one hundred milliseconds up to a few hundred milliseconds, depending on network size and conditions. In smaller-scale scenarios (for example, fewer than five peers), the application references jitter thresholds as low as 50 ms, indicating that latencies at or below this level are considered acceptable for maintaining synchronized media streams. At the upper range, the application allows latency thresholds to increase toward 800 ms when more than 50 peers are connected. Some statistics may require longer assessment windows, and therefore real-time may span in circumstances to several seconds, after which it can be safely determined that a threshold has been reached and that fluctuations are not merely noise.

[0134] The profiling module may be configured to maintain at least one upstream media profile 6105 comprising a peer profile along with the streaming peer profile and the media stream profile. The upstream media profile may be used at the streaming module 6030 to configure the characteristics of the upstream media stream for one or more peer connections, such as the connection from the streaming module 6030 to the alerting peer 6040. The upstream media profile may prescribe any transformations necessary over the original media stream. In embodiments, computation of the upstream media stream may be performed in accordance with the teachings presented in relation to the first set of embodiments described hereinabove. The profiles 6105 may be available to the profiling module 6020, the monitoring module 6010, and the streaming module 6030.

[0135] During the monitoring 6110 of the connections, the monitoring module 6010 may detect 6120 problems therein by gathering and analyzing connection statistics.The detected problems may include issues related to the CPU, network conditions, or other factors that may affect the stability of the media streaming.

[0136] Once a problem is detected 6120 through monitoring 6110, the monitoring module 6010 may adjust 6220 the peer profile associated with the problematic peer. Adjustments may include changes to the resolution, frame rate, and / or bitrate of the media streaming to ensure stable streaming with minimal latency. After adjusting 6220 the peer profile, the profiling module 6020 may compute 6230 a new upstream media stream profile and, in response to the parameters of the new upstream media stream profile, the streaming module 6030 may adjust 6240 the encoding pipeline accordingly. The decision on which parameter(s) to adjust 6220 and by how much may be determined based on, among other factors, the nature of the issue detected 6120 during monitoring 6110. For example, when the issue is related to network conditions, the bitrate may be reduced to decrease the amount of data sent over the network. When the issue is associated with memory or CPU of the presenting peer, the peer profile may be adjusted to request a simplified version of the stream. For video streams, simplifying the stream may be achieved by reducing the frame size or the frame rate, for example.

[0137] In some cases, the streaming module 6030 connected to the alerting peer 6040 may also be a subscribed peer from the perspective of the alerting peer 6040. This may occur when connections exist in both directions, such as during a teleconference system where each peer subscribes to the other concurrently. In these cases, problems detected on one peer may also allow for the detection or anticipation of a problem from the perspective of the other peer. It may be advantageous to generate self-healing actions for the other peer. For instance, upon adjusting 6240 the profile of the alerting peer 6040, the streaming module 6030 may also communicate 6310 a new peer profile to the alerting peer 6040 concerning the inbound connection from the alerting peer 6040 in order to further reduce pressure on the alerting peer 6040, such as regarding bandwidth, CPU, or memory.

[0138] In operation, the monitoring module 6010 may continuously collect connection stats and adjust the media streaming parameters as necessary to ensure stable streaming while considering latency.

[0139] By providing self-healing capabilities, it may be possible to improve the stability and performance of media streaming, keep latency within prescribed limits, andprovide uninterrupted streaming without intervention from a central authority. The two- way nature of the solution may allow problems detected on one peer to generate self- healing actions on the other peer, further improving the effectiveness of the system.

[0140] Self-healing logic within a system may be designed to dynamically adjust parameters such as resolution, frame rate, and bitrate to maintain optimal media streaming performance. A prioritization strategy may be used to initiate the decisionmaking process for such adjustments, wherein the nature of the media stream and user preferences may be considered. For video streams, maintaining a higher frame rate may be prioritized to ensure smooth playback, while for audio streams, adjustments to bitrate may take precedence to preserve sound quality.

[0141] Specific decision criteria may be employed to determine which parameter to adjust first. Such criteria may include assessment of current network conditions, the capabilities of the receiving device, and the type of content being streamed. In scenarios where network bandwidth is constrained, the system may initially reduce the bitrate to decrease data transmission demands. Incremental adjustments may be executed using predefined steps or adaptive methods based on real-time monitoring of network performance. For example, the system may initially reduce bitrate by 10% and assess the impact before proceeding to adjust frame rate or resolution.

[0142] An algorithmic approach may guide the adjustment process, utilizing rulebased systems or more complex algorithms, such as decision trees or machine learning models. Such algorithms may evaluate various factors and determine the most effective adjustment strategy. A feedback mechanism may continuously monitor the quality of the media stream after adjustments, providing data that refines future decision-making. A feedback loop may ensure responsiveness to changing network conditions, resulting in necessary adjustments to maintain the desired streaming quality.

[0143] A scenario in which network conditions suddenly degrade may illustrate the decision-making process. The system may first reduce bitrate, followed by a slight decrease in frame rate, to stabilize streaming. Should conditions further deteriorate, resolution adjustments may be applied. User preferences and profiles may also influence the adjustment strategy, allowing the system to prioritize clarity over frame rate or vice versa, depending on individual user requirements.

[0144] In accordance with a fourth set of embodiments, the disclosure may focus on providing examples of solutions in the form of various systems, devices, andmethods for establishing a stable connection in peer-to-peer communication. Figure 10 depicts an exemplary method 7000 for establishing a stable connection in a peer-to- peer network. Instead of relying on a centralized server, such as a signaling server, to perform peer discovery 7100 and establish a bidirectional connection between a first peer 7010 and a new peer 7030, discovery may be achieved by communicating 7110 with an existing peer 7020 and obtaining a peer list 7120 therefrom.

[0145] During connection establishment 7200 with the new peer 7030, two one-way connections may be established 7210 and 7220 between the participants for sending and receiving media streams. Establishing a stable peer-to-peer connection between two participants may be performed by establishing two one-way connections for transmitting the media streams between the peers, instead of establishing a single two- way connection, thereby creating a dual-connection peer. Each of the one-way connections may actually be a bidirectional connection, allowing traffic in either direction, but each may only be used to transfer data in one direction. As discussed hereinbelow, the other direction may still be used for other purposes, such as transferring feedback or heartbeats back to a peer. The two connections may be established 7210 and 7220 independently, without the need for a signaling server during the negotiation of each connection. The participants may be autonomous in establishing these connections, which enables a stable and efficient communication channel. In one embodiment, the two connections may use the PeerConnection API or WebSockets. The PeerConnection API, or Web Real-Time Communication PeerConnection, may be an application programming interface that facilitates direct peer-to-peer communication between browsers or devices and allows for the transfer of audio, video, and data directly without the need for an intermediary server, enabling real-time communication in web applications. WebSockets are a communication protocol providing a full-duplex communication channel over a single, long-lived connection established between a client and a server. Unlike the traditional requestresponse model, WebSockets may allow for persistent connections where both the client and the server can send data at any time, facilitating real-time data transfer and interactive communication in web applications.

[0146] In the field of network communications, User Datagram Protocol (UDP) may be a protocol that facilitates the exchange of messages between devices in a network with a focus on speed and efficiency, without guaranteeing delivery, order, or data integrity. Transmission Control Protocol (TCP), in contrast, may be a protocol designedto provide reliable, ordered, and error-checked data exchange between devices, establishing a firm connection before any data transfer. A TCP socket may be an interface for the point-to-point communication pathway established for TCP, allowing for bidirectional data transfer. Network Address Translation (NAT) may be a technique used to manage and map the connectivity and communications between devices within a private network and external networks, often altering the network address information for the purpose of conserving addresses and securing the network.

[0147] The data channel of each connection may use a heartbeat mechanism 7300 to further improve stability of the peer-to-peer network. The first peer 7010 and the new peer 7030 may transmit a heartbeat 7310 and 7340 and await a heartbeat response 7320 and 7350 to verify that the peers are still connected to each other. In practice, it is not unusual for TCP connections to become disconnected without the peers noticing. Such TCP disconnections may occur, for example, when an intermediate network component stealthily closes a port used for establishing the connection, such as when detecting that the connection has been idle for some time. To the peers, the TCP socket may appear to still be connected until data is transmitted. When the socket is used to receive data, the peer may not be aware that the connection has been lost until it reaches a time threshold. A heartbeat 7300 may be initiated on either side of each connection, and the connection may be considered disconnected 7330 and 7360 if no reply 7320 and 7350 is received. In the case of a TCP connection, a single missing reply may be sufficient to conclude that the connection has been lost. In the case of a UDP connection, the peers may await a certain number of missing replies to conclude that a connection has been lost. For example, a heartbeat may be initiated every 5 seconds, and a UDP connection may be considered terminated after three missing replies. While UDP is considered connectionless, in the context of peer- to-peer, UDP communication between two peers may require NAT traversal or forwarding servers, which results in network device ports remaining open to provide a route. In the context of the present disclosure, when these ports are closed, the UDP connection between the peers may be considered closed.

[0148] In traditional peer-to-peer connection establishment, a signaling server may be used to coordinate the negotiation process between nodes, requiring a centralized server and adding load thereon, which may in turn cause scalability and reliability issues. Furthermore, when the signaling server is unavailable, the peer-to-peer connection may not be able to connect.

[0149] When a first peer opens a first connection to a second peer, the second peer may attempt to connect to the first peer in order to establish the second connection. If the second peer is already attempting to connect to the first peer, there is a risk that the two peers may connect to each othertwice. In one embodiment, at least one remote peer may be discovered using a centralized server. When a second peer joins the peer- to-peer network, the centralized server may provide a list of peers in the peer-to-peer network to connect to. A collision prevention protocol whereby the centralized server never instructs a second peer to connect to a first peer may prevent scenarios where the two connections are opened twice.

[0150] Upon negotiating the initial connection and establishing a data channel, all subsequent connections may be negotiated using the data channel without interacting with the centralized server. Negotiation of a subsequent connection between two peers may rely on the data channel of intermediate peers acting as relay peers.

[0151] It is suggested herein in accordance with the fourth set of embodiments that the two participants establish two one-way connections for transmitting the media streams, which may diminish reliance on a centralized server. Each peer may be responsible for the egress connection, such as the connection used to send media streams, which makes the connection more autonomous and robust.

[0152] The first, second, third and fourth set of embodiments may be implemented as methods, systems or devices. When implemented as a device, the device may be a peer within the peer-to-peer network.

[0153] More specifically, when the first set of embodiments is implemented as a peer within the peer-to-peer network, the peer may be the relaying peer for relaying a media stream in a peer-to-peer network, as described hereinabove, where the network comprises a plurality of peers participating in an ongoing peer-to-peer session. The relaying peer may include one or more processors. The one or more processors may produce an upstream media stream from a downstream media stream during the ongoing peer-to-peer session between the plurality of peers in the peer-to-peer network. The one or more processors may compute a media profile of the downstream media stream and peer profiles into an upstream media profile of the upstream media stream. The peer profiles may include limiting constraints according to capabilities of one or more subscribing peers participating in the ongoing peer-to-peer session through the relaying peer and limiting constraints according to the capabilities of therelaying peer, whereby the downstream media stream of the ongoing peer-to-peer session is received. The one or more processors may decode the downstream media stream, at least partially, into a decoded media stream. The one or more processors may compute the decoded media stream into an upstream media stream in accordance with the upstream media profile. The one or more processors may transmit the upstream media stream from the relaying peer to the one or more subscribing peers.

[0154] When the second set of embodiments is implemented as a peer within the peer-to-peer network, the peer may be the monitoring peer 5100 for real-time monitoring and detection of problems in a peer-to-peer network, as described hereinabove. The monitoring peer 5100 may include a monitoring module 6010 configured, considering a plurality of connections between two or more participants of the peer-to-peer network in real-time, to collect connection statistics from at least two of the plurality of connections using a weighted round-robin collection mechanism. The connection statistics may comprise one or more of latency, packet loss, jitter, and bandwidth. The monitoring module 6010 may generate an alert when one or more of the collected connection statistics is outside of one or more of a plurality of statistics thresholds.

[0155] When the third set of embodiments is implemented as a peer within the peer- to-peer network, the peer may be the self-healing peer for autonomous real-time problem resolution in a peer-to-peer network, as described hereinabove. The self- healing peer may be a first peer transmitting a media stream to a second peer over a connection between the self-healing peer and the second peer within the peer-to-peer network. The media stream may be configured according to a peer profile of the second peer, providing limiting constraints over the media stream. The self-healing peer may include a monitoring module 6010 configured, considering the connection between the self-healing peer and the second peer in real-time, to adjust the peer profile of the second peer for the self-healing peer, upon receiving an alert indicating that a collected connection statistic is outside of one or more of a plurality of statistics thresholds for the connection between the self-healing peer and the second peer.

[0156] When the third set of embodiments is implemented as a peer within the peer- to-peer network, the dual-connection peer for establishing a stable connection in a peer-to-peer network as described hereinabove. The dual-connection peer may be a first peer having a first one-way connection for sending media streams to a second peerand may have a second one-way connection for receiving media streams from the second peer.

[0157] Reference is now made to the drawings in which Figure 11 shows a logical modular representation of an exemplary peer-to-peer network 200 comprising a peer 8100 interacting with other peers 8100’ directly 8178 or across a network 8200. The peer 8100 comprises a memory module 8160, a processor module 8120, a functionality module 8130 and a network interface module 8170. The peer 8100 may also include a configuration module 8150. The configuration module may provide customization settings to the other modules, including thresholds, policies, endpoints, enabled or disabled features, etc. The configuration module 8150 may obtain some or all the customization settings from a remote configuration module 8152, thereby allowing, for example, orchestration of multiple peer 8100 from a common remote configuration module 8152.

[0158] The peer-to-peer network 200 may comprise a storage system 8300 for storing and accessing long-term (i.e., non-transitory) data and may further log data while the peer 8100 is being used. Figure 11 shows examples of the storage system 8300 as a distinct database system 8300A, a distinct module 8300 C of the peer 8100 or a sub-module 8300B of the memory module 8160 of the peer 8100. The storage system 8300 may be distributed over different systems A, B, C. The storage system 8300 may comprise one or more logical or physical as well as local or remote hard disk drive (HDD) (or an array thereof). The storage system 8300 may further comprise a local or remote database made accessible to the peer 8100 by a standardized or proprietary interface or via the network interface module 8170.

[0159] The network interface module 8170 represents at least one physical interface that can be used to communicate with other peers. The network interface module 8170 may be made visible to the other modules of the peer 8100 through one or more logical interfaces. The actual stacks of protocols used by the physical network interface(s) and / or logical network interface(s) 8172, 8174, 8176 and 8178 of the network interface module 8170 do not affect the teachings of the present invention.

[0160] The processor module 8120 may represent a single processor with one or more processor cores or an array of processors, each comprising one or more processor cores. The memory module 8160 may comprise various types of memory(different standardized or kinds of Random Access Memory (RAM) modules, memory cards, Read-Only Memory (ROM) modules, programmable ROM, etc.).

[0161] A bus 8180 is depicted as an example of means for exchanging data between the different modules of the peer 8100. The teachings presented herein are not affected by the way the different modules exchange information. For instance, the memory module 8160 and the processor module 8120 could be connected by a parallel bus, but could also be connected by a serial connection or involve an intermediate module (not shown) without affecting the teachings of the present invention.

[0162] A functionality module 8130 provides specialized services to the peer 8100, which have been described in more details hereinabove. In the first set of embodiments, the functionality module 8130 may be a relaying module, for relaying a media stream in a peer-to-peer network. In the second set of embodiments, the functionality module 8130 may be a monitoring module 6010, for real-time monitoring and detection of problems in a peer-to-peer network. In the third set of embodiments, the functionality module 8130 may be a self-healing module, for real-time problem resolution in a peer- to-peer network. In the fourth set of embodiments, the functionality module 8130 may be a dual-connection module, for establishing a stable connection in a peer-to-peer network.

[0163] The variants of processor module 8120, memory module 8160 and network interface module 8170 usable in the context of the present invention will be readily apparent to persons skilled in the art. Likewise, even though explicit mentions of the functionality module 8130, the memory module 8160, the configuration module 8150 and / or the processor module 8120 are not made throughout the description of the present examples, persons skilled in the art will readily recognize when such modules are used in conjunction with other modules of the peer 8100 to perform routine as well as innovative elements presented herein.

[0164] Various network links may be implicitly or explicitly used in the context of the present invention. While a link may be depicted as a wireless link, it could also be embodied as a wired link using a coaxial cable, an optical fiber, a category 5 cable, and the like. A wired or wireless access point (not shown) may be present on the link between. Likewise, any number of routers (not shown) may be present and part of the link, which may further pass through the Internet.

[0165] The present invention is not affected by the way the different modules exchange information between them. For instance, the memory module and the processor module could be connected by a parallel bus, but could also be connected by a serial connection or involve an intermediate module (not shown) without affecting the teachings of the present invention.

[0166] A method is generally conceived to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic / electromagnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, parameters, items, elements, objects, symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these terms and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The description of the present invention has been presented for purposes of illustration but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments were chosen to explain the principles of the invention and its practical applications and to enable others of ordinary skill in the art to understand the invention in order to implement various embodiments with various modifications as might be suited to other contemplated uses.

[0167] A system may generally be conceived as an arrangement of multiple components that work together to achieve a particular function or result. These components each may have distinct roles and contribute to the overall operation of the system. It is convenient to describe these components as units, modules, parts, or elements. Components may be physical or logical in nature. In practice, systems may be implemented in various forms. While systems are typically comprised of multiple distinct components, in some embodiments, all or some components may coexist within a single device. This integration does not alter the fundamental understanding of the operation but rather represents an embodiment where functionality is consolidated. Such a configuration may be advantageous for specific applications where space, efficiency, or other considerations are paramount. Regardless of configuration, systems are understood to operate through physical interactions, which may, in someembodiments, be achieved through electronic components such as RAM, buses and processors.

[0168] As used in this specification, the expression “at least one of” followed by a set of elements suggests that any combination of the elements from the set is being considered, including a single element from the set, and all elements from the set. For clarity, “at least one of” followed by a set does not strictly refer to having at least the whole set once, and possibly multiple times.

[0169] As used in this specification and claim(s), the words “comprising” (and any form of comprising, such as “comprise” and “comprises”), “having” (and any form of having, such as “have” and “has”), “including” (and any form of including, such as “include” and “includes”) or “containing” (and any form of containing, such as “contain” and “contains”), are inclusive or open-ended and do not exclude additional, unrecited elements or process steps.

[0170] As will be understood by a skilled person, other variations and combinations may be made to the various embodiments of the invention as described herein above. The scope of the claims should not be limited by the preferred embodiments set forth; but should be given the broadest interpretation consistent with the description as a whole.

Claims

CLAIMSWhat is claimed is:

1. A method for relaying a media stream in a peer-to-peer network, the method comprising:- at a relaying peer, producing an upstream media stream from a downstream media stream during an ongoing peer-to-peer session between a plurality of peers in the peer-to-peer network:- computing a media profile of the downstream media stream and peer profiles into an upstream media profile of the upstream media stream, the peer profiles comprising:- limiting constraints according to capabilities of one or more subscribing peers participating in the ongoing peer-to-peer session through the relaying peer; and- limiting constraints according to the capabilities of the relaying peer;- receiving the downstream media stream of the ongoing peer- to-peer session;- decoding the downstream media stream, at least partially, into a decoded media stream;- computing the decoded media stream into an upstream media stream in accordance with the upstream media profile; and- transmitting the upstream media stream from the relaying peer to the one or more subscribing peers.

2. The method of claim 1 , wherein the upstream media profile is configured to reduce a bitrate of the upstream media stream when compared to the downstream media stream.

3. The method of claim 1 or claim 2, wherein the ongoing peer-to-peer session is any one of a conferencing session, a live stream and an on-demand source.

4. The method of any one of claims 1 to 3, wherein the downstream media stream comprises at least one of a text stream, video stream and an audio stream.

5. The method of any one of claims 1 to 4, wherein the computing into an upstream media stream comprises at least one lossy transformation.

6. The method of claim 5 wherein the lossy transformation comprises at least one of a spatial re-sampling, a temporal re-sampling and a re-quantization operation.

7. The method of any one of claims 1 to 6, wherein computing the decoded media stream into the upstream media stream further comprises:- transforming the decoded media stream into a transformed media stream in accordance with the upstream media profile; and- encoding the transformed media stream into the upstream media stream in accordance with the upstream media profile.

8. The method of any one of claims 1 to 6, executed within a web browser, wherein:- the downstream media stream comprises a video stream;- decoding the downstream media stream comprises:- rendering the video stream into a content of a source canvas;- computing the decoded media stream into the upstream media stream comprises:- encoding the content from the source canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile; and transmitting the upstream media stream is configured to transmit the upstream media stream using a stream publisher over a RTC connection.

9. The method of claim 7, wherein:- the downstream media stream comprises a video stream;- decoding the downstream media stream comprises:- rendering the video stream into a source canvas;- transforming the decoded media stream into a transformed media stream is configured to transform a source content from the source canvas into a destination content in a destination canvas in accordance with the media profile;- encoding the transformed media stream into the upstream media stream is configured to encode the destination content from the destination canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile; and- transmitting the upstream media stream is configured to transmit the upstream media stream using a stream publisher over a RTC connection.

10. The method of claim 8 or claim 9, wherein the stream encoder uses at least one of a Mediastream API and a WebCodecs API provided by the web browser.11 . The method of any one of claims 8 to 10, wherein the stream publisher uses at least one of a WebRTC framework and a QUIC framework provided by the web browser.

12. The method of any one of claims 8 to 11 , wherein a framerate of the video stream is decreased by executing the transforming at a lower frequency than the rendering.

13. The method of any one of claims 8 to 12, wherein a resolution of the video stream is decreased by using a destination canvas with a lower resolution than the source canvas.

14. The method of any one of claims 8 to 13, where in the source canvas and the destination canvas are HTML canvases.

15. A system for real-time monitoring and detection of problems in a peer-to-peer network, comprising:a monitoring module configured, considering a plurality of connections between two or more participants of the peer-to-peer network in realtime, to:- collect connection statistics from at least two connections from the plurality of connections using a weighted round-robin collection mechanism, wherein the connection statistics comprise one or more of latency, packet loss, jitter, and bandwidth; and- generate an alert when one or more of the collected connection statistics is outside of one or more of a plurality of statistics thresholds.

16. The system of claim 15, wherein the connection statistics are collected during a transmission of a media stream between a content peer and a subscribing peer over one of the plurality of connections.

17. The system of claim 16, wherein the content peer and the subscribing peer are participating in a conferencing session over the peer-to-peer network.

18. The system of any one of claims 15 to 17, wherein the monitoring module is further configured to sequentially arrange the at least two connections for the weighted round-robin collection mechanism by prioritizing weaker connections.

19. The system of any one of claims 15 to 18, wherein the monitoring module is further configured for dynamically adjusting one or more statistics thresholds from the plurality of statistics thresholds for detecting problems based on the connection statistics, wherein the adjusting is based on a number of the connections between participants.

20. The system of claim 19, wherein the monitoring module is configured to continuously collect connection statistics and continuously compare the connection statistics against one or more dynamically adjusted thresholds for each connection.

21. A method for real-time monitoring and detection of problems in a peer-to-peer network, comprising:- collecting, in real-time, a plurality of connection statistics from at least two connections from a plurality of connections using weighted roundrobin collection mechanism, wherein the plurality of connections is defined between two or more participants of the peer-to-peer network and wherein the plurality of connection statistics comprises at least one of a latency, a packet loss, a jitter, and a bandwidth; and- generating an alert when one or more connection statistics of the plurality of connection statistics is outside of one or more of a plurality of statistics thresholds.

22. The method of claim 21 , wherein the connection statistics are collected during a transmission of a media stream between a content peer and a subscribing peer over one of the plurality of connections.

23. The method of claim 22, wherein the content peer and the subscribing peer are participating in a video conferencing session over the peer-to-peer network.

24. The method of any one of claims 21 to 23, further comprising sequentially arranging the at least two connections for the weighted round-robin collection mechanism by prioritizing weaker connections.

25. The method of any one of claims 21 to 24, further comprising:- dynamically adjusting one or more statistics thresholds from the plurality of statistics thresholds for detecting problems; and wherein the statistics thresholds are adjusted based on a number of connections between participants.

26. The method of claim 25, wherein collecting in real-time connection statistics is performed continuously, and the method further comprises:- comparing the connection statistics to one or more dynamically adjusted thresholds for at one connection for the plurality of connections.

27. A system for real-time problem resolution in a peer-to-peer network, comprising:- a first peer transmitting a media stream to a second peer over a connection between the first peer and the second peer within the peer- to-peer network, wherein the media stream is configured according to a peer profile of the second peer providing limiting constraints over the media stream; and- a monitoring module configured, considering the connection between the first peer and the second peer in real-time, to:- considering an alert indicating that a collected connection statistics is outside of one or more of a plurality of statistics thresholds for the connection between the first and the second peer, adjust the peer profile of the second peer for the first peer.

28. The system of claim 27, wherein:- the second peer transmits a second media stream to a first peer over a connection between the second peer and the first peer within the peer- to-peer network and wherein the second media stream is configured according to a peer profile of the first peer providing limiting constraints over the media stream; and- the monitoring module is further configured, considering the alert, to adjust the peer profile of the first peer for the second peer.

29. The system of claim 27 or claim 28, wherein the media stream is a video stream, and the limiting constraints of the peer profile comprise at least one of a maximum resolution, a maximum frame rate, and a maximum bitrate.

30. A method for real-time problem resolution in a peer-to-peer network, comprising:- transmitting a media stream from a first peer to a second peer over a connection between the first peer and the second peer within the peer- to-peer network, wherein the media stream is configured according to a peer profile of the second peer providing limiting constraints over the media stream;- receiving an alert indicating that a collected connection statistics is outside of one or more of a plurality of statistics thresholds for the connection between the first peer and the second peer; and- adjusting the peer profile of the second peer for the first peer.31 . The method of claim 30 further comprises:- transmitting a second media stream from the second peer to the first peer over a connection between the second peer and the first peer within the peer-to-peer network wherein the second media stream is configured according to a peer profile of the first peer providing limiting constraints over the media stream; and- adjusting, considering the alert, the peer profile of the second peer for the first peer.

32. The method of claim 30 or 31 , wherein the media stream is a video stream, and the limiting constraints of the peer profile comprise at least one of a maximum resolution, a maximum frame rate, and a maximum bitrate.

33. A system for establishing a stable connection in a peer-to-peer network, comprising:- a first peer having a first one-way connection for sending media streams to a second peer and a second one-way connection for receiving media streams from the second peer.

34. The system of claim 33, wherein the first and second one-way connections are established using a PeerConnection API or a WebSocket.

35. The system of claim 33 or claim 34, wherein the first peer and the second peer implement a heartbeat mechanism therebetween to monitor a stability of the first one-way connection.

36. The system of any one of claims 33 to 35, wherein the first peer and the second peer implement a collision prevention protocol for establishing the first one-way connection and the second one-way connection without collision.

37. A method for establishing a stable connection in a peer-to-peer network:- establishing, from a first peer, a first one-way connection for sending media streams from the first peer to a second peer; and- establishing, from a second peer, a second one-way connection for sending media streams from the second peer to the first peer.

38. The method of claim 37, wherein establishing the first one-way connection and establishing the second one-way connection are performed using a PeerConnection API or a WebSocket.

39. The method of claim 37 or claim 38, further comprising implementing a heartbeat mechanism between the first and second peers to monitor a stability of the first oneway connection.

40. The method of any one of claims 37 to 39, further comprising implementing a collision prevention protocol for establishing the first one-way connection and the second one-way connection without collision.41 .A system for relaying a media stream in a peer-to-peer network comprising:- a plurality of peers participating in an ongoing peer-to-peer session;- a relaying peer comprising one or more processors configured to:- produce an upstream media stream from a downstream media stream during the ongoing peer-to-peer session between the plurality of peers in the peer-to-peer network;- compute a media profile of the downstream media stream and peer profiles into an upstream media profile of the upstream media stream, the peer profiles comprising:- limit constraints according to capabilities of one or more subscribing peers participating in the ongoing peer-to-peer session through the relaying peer; and limit constraints according to the capabilities of the relaying peer;receive the downstream media stream of the ongoing peer- to-peer session;- decode the downstream media stream, at least partially, into a decoded media stream;- compute the decoded media stream into an upstream media stream in accordance with the upstream media profile; and- transmit the upstream media stream from the relaying peer to the one or more subscribing peers.

42. The system of claim 41 , wherein the upstream media profile is configured to reduce a bitrate of the upstream media stream when compared to the downstream media stream.

43. The system of claim 41 or claim 42, wherein the ongoing peer-to-peer session is any one of a video conferencing session, a live stream and an on-demand source.

44. The system of claim 41 to 43, wherein the downstream media stream comprises at least one of a text stream, a video stream and an audio stream.

45. The system of claim 41 to 44, wherein the computing into an upstream media stream comprises at least one lossy transformation.

46. The system of claim 45 wherein the lossy transformation comprises at least one of a spatial re-sampling, a temporal re-sampling and a re-quantization operation.

47. The system of any one of claims 41 to 46, wherein the one or more processors are further configured to:- transform the decoded media stream into a transformed media stream in accordance with the upstream media profile and encoding the transformed media stream into the upstream media stream in accordance with the upstream media profile.

48. The system of any one of claims 41 to 46, wherein the one or more processors are further configured to:- process a web browser configured toproduce the upstream media stream;- compute the media profile;- limit constraints according to:- capabilities of the one or more subscribing peers; and- the capabilities of the relaying peer;- receive the downstream media stream;- decode the downstream media stream;- compute the decoded media stream; and- transmit the upstream media stream; wherein the downstream media stream comprises a video stream; and wherein the one or more processors are further configured to:- render the video stream into a content of a source canvas;- encode the content from the source canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile; and- transmit the upstream media stream using a stream publisher over a RTC connection.

49. The system of claim 47, wherein:- the downstream media stream comprises a video stream; wherein the one or more processors are further configured to:- render the video stream into a source content of a source canvas;- transform the source content from the source canvas into a destination content in a destination canvas in accordance with the media profile;- encode the destination content from the destination canvas into the upstream media stream using a stream encoder in accordance with the upstream media profile; and- transmit the upstream media stream using a stream publisher over a RTC connection.

50. The system of claim 48 or claim 49, wherein the stream encoder uses at least one of a Mediastream API and a WebCodecs API provided by the web browser.

51. The system of any one of claims 48 to 50, wherein the stream publisher uses at least one of a WebRTC framework and a QUIC framework provided by the web browser.

52. The system of any one of claims 48 to 51 , wherein a framerate of the video stream is decreased by executing the transforming at a lower frequency than the rendering.

53. The system of any one of claims 48 to 52, wherein a resolution of the video stream is decreased by using a destination canvas with a lower resolution than the source canvas.

54. The system of any one of claims 48 to 53, where in the source canvas and the destination canvas are HTML canvases.

Images