A method and apparatus for reducing transmission bandwidth of a video conference network

Abstract

The application discloses a transmission method and device for reducing bandwidth of a video conference network, relates to the technical field of computer communication, and mainly aims to reduce the uplink and downlink bandwidth pressure of a device end and flexibly adapt to mainstream mobile terminal conference requirements. The main technical scheme of the application is as follows: when a push client and a receiving client are in video conference communication, the push client pushes an audio stream, a static video stream and a dynamic video stream to a media forwarding server of an SFU architecture; when the receiving client enlarges a video conference picture, the receiving client switches the pulled static video stream to the dynamic video stream, and notifies the push client by using the media forwarding server; based on the dynamic video stream pulling event, the push client modifies the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth due to the preset minimum bandwidth. The application is used for a video conference network.

Description

Technical Field

[0001] This invention relates to the field of computer information technology, and in particular to a transmission method and apparatus for reducing network bandwidth in video conferencing. Background Technology

[0002] The most basic video conferencing architectures currently available based on WebRTC technology are Mesh, MCU, and SFU. Among these, the SFU architecture is commonly used to balance client bandwidth usage and server processing power requirements when enterprises cannot afford high server costs. Therefore, optimization of the uplink and downlink bandwidth requirements of SFU architecture devices is necessary.

[0003] Currently, analysis of mainstream video conferencing software reveals the following basic strategies for video conferencing streaming: Technique 1: Simulcast technology. This involves the publisher pushing several different streams to the server, which then forwards the appropriate stream to the receiving user based on the recipient's network status. Technique 2: A secondary stream strategy. This involves simultaneously pushing two different streams to the server, with the client retrieving one or both streams as needed. Compared to the first method, this approach avoids the issue of the playback end missing video streams due to actively reducing the bitrate when network fluctuations occur.

[0004] However, Technique 1 has a significant drawback: the playback end is affected by the uplink client. For the uplink client, if the Simulcast layer is reduced to one for some reason, the playback end will not receive the video stream without special handling. Manually switching will result in a choppy experience, impacting the user experience. While Technique 2's small-stream / small-stream strategy avoids the problem of the playback end not receiving the video stream due to the active reduction in bitrate, it increases the bandwidth requirement for the small stream compared to the single-stream push strategy, and the repeated audio push causes bandwidth waste. Summary of the Invention

[0005] In view of the above problems, the present invention provides a transmission method and apparatus for reducing network bandwidth in video conferencing. The main purpose is to reduce the uplink and downlink bandwidth pressure on the device end and flexibly adapt to the mainstream mobile conferencing needs.

[0006] To solve the above-mentioned technical problems, the present invention proposes the following solution:

[0007] In a first aspect, the present invention provides a method for reducing network bandwidth in video conferencing, the method comprising:

[0008] When the push client and the receiving client are connected in a video conference, the push client pushes the audio stream, the static video stream and the dynamic video stream to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream and the dynamic video stream respectively. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth.

[0009] The receiving client retrieves the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback;

[0010] When the receiving client zooms in on the video conference screen, the receiving client switches the retrieved static video stream to the dynamic video stream, generates a dynamic video stream retrieval event, and uses the media forwarding server to notify the push client of the dynamic video stream retrieval event.

[0011] Based on the event of the dynamic video stream being pulled, the push client modifies the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth, based on the preset minimum bandwidth.

[0012] Secondly, the present invention provides a transmission device for reducing video conferencing network bandwidth, the device comprising:

[0013] The push unit is used to push the audio stream, the static video stream, and the dynamic video stream to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively, when the push client and the receiving client are connected in a video conference. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth.

[0014] The pull unit is used to allow the receiving client to pull the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback when the receiving client is not zooming in on the video conference screen.

[0015] The switching unit is used to switch the static video stream to the dynamic video stream when the receiving client zooms in on the video conference screen, generate a dynamic video stream being pulled event, and use the media forwarding server to notify the push client of the dynamic video stream being pulled event.

[0016] The modification unit is used to modify the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth based on the dynamic video stream being pulled event.

[0017] To achieve the above objectives, according to a third aspect of the present invention, a storage medium is provided, the storage medium including a stored program, wherein, when the program is executed, the device where the storage medium is located is controlled to perform the transmission method for reducing video conferencing network bandwidth described in the first aspect.

[0018] To achieve the above objectives, according to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements all or part of the steps of the transmission apparatus for reducing bandwidth of a video conferencing network as described in the second aspect.

[0019] By employing the above technical solution, the transmission method and apparatus for reducing video conferencing network bandwidth provided by the present invention address the problem of bandwidth waste caused by repeated audio streaming in current methods. To address this, the present invention, when the pushing client and the receiving client are connected in a video conference, pushes the audio stream, the static video stream, and the dynamic video stream to a media forwarding server of an SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth. The receiving client retrieves the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback. When the receiving client zooms in on the video conference screen, the receiving client switches the retrieved static video stream to the dynamic video stream, generating a dynamic video stream retrieval event, and uses the media forwarding server to notify the pushing client of the dynamic video stream retrieval event. Based on the dynamic video stream retrieval event, the pushing client modifies the bandwidth corresponding to the dynamic video stream from the preset minimum bandwidth to a preset low-definition bandwidth or a preset high-definition bandwidth. This invention reduces video uplink bandwidth to the minimum hardware-supported standard while ensuring normal audio transmission during video conferencing. Simultaneously, it allows streaming clients to retrieve video data on demand based on the primary and secondary video content, thus saving bandwidth. This further reduces the uplink and downlink bandwidth pressure on the device and flexibly adapts to the mainstream mobile conferencing needs.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0022] Figure 1 A flowchart of a transmission method for reducing network bandwidth in video conferencing, provided by an embodiment of the present invention, is shown.

[0023] Figure 2 A flowchart of another method for reducing network bandwidth in video conferencing, provided by an embodiment of the present invention, is shown.

[0024] Figure 3 This diagram illustrates a block diagram of a transmission device for reducing network bandwidth in video conferencing, as provided in an embodiment of the present invention.

[0025] Figure 4 This diagram illustrates a block diagram of another transmission device for reducing network bandwidth in video conferencing, provided by an embodiment of the present invention. Detailed Implementation

[0026] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0027] Terminology Explanation:

[0028] WebRTC: Web Real-Time Communications is a real-time communication technology that allows web applications or websites to establish peer-to-peer connections between browsers without the need for an intermediary, enabling the transmission of video and / or audio streams or other arbitrary data. The standards included in WebRTC make it possible for users to create peer-to-peer data sharing and teleconferencing without installing any plugins or third-party software.

[0029] SRS: SRS is a simple and efficient real-time video server that supports video streams using RTMP / WebRTC / HLS / HTTP-FLV / SRT / GB28181 protocols.

[0030] Uplink bandwidth: refers to the data traffic generated by a client device sending data to the outside world. For example, when two people are having an online video call, each person needs to upload their video stream to the Internet, and each person generates 300KB of traffic per second. When converted to the standard communication bandwidth unit Mbps, the client's uplink bandwidth is 3Mbps.

[0031] Downlink bandwidth: refers to the traffic generated by a client device receiving data sent by a server. For example, when two people are having an online video call, each person needs to pull the other's video stream from the server to the client. Each person downloads 300KB of traffic per second. When converted to the standard communication bandwidth unit Mbps, the client's downlink bandwidth is 3Mbps.

[0032] The most basic architectures in existing WebRTC-based video conferencing solutions are Mesh, MCU, and SFU. Assuming four participants, each uploading approximately 1Mbps of video stream bandwidth. In the Mesh solution, each device needs to send its own stream to the other three, requiring a total of 3Mbps uplink bandwidth, and simultaneously retrieve video streams from the other three, requiring a total of 3Mbps downlink bandwidth, with each device handling 6Mbps of uplink and downlink bandwidth. In the MCU solution, each device uploads its own video stream to the server and retrieves the mixed video stream from the server, requiring 2Mbps of uplink and downlink bandwidth per device. Finally, in the SFU solution, each device uploads its own video stream to the server and then retrieves video streams from the other three devices, requiring 4Mbps of uplink and downlink bandwidth per device. The comparison results of video conferencing architecture solutions are as follows: 1. Device-side bandwidth requirements: Mesh (3M downlink + 3M uplink) > SFU (1M uplink + 3M downlink) > MCU (1M uplink + 1M downlink); 2. Device-side processing power requirements: Mesh = SFU > MCU; 3. Server-side processing power requirements: MCU > SFU > 0 = Mesh (no server). Therefore, the SFU architecture is the best choice for enterprises that cannot afford high server costs, balancing client bandwidth usage and server-side processing power requirements. This necessitates two optimizations for the SFU architecture: firstly, reducing device-side uplink and downlink bandwidth requirements; and secondly, reducing device-side processing power requirements.

[0033] Currently, analysis of mainstream video conferencing software reveals the following basic strategies for video conferencing streaming: Technique 1: Simulcast technology. This involves the publisher pushing several different streams to the server, which then forwards the appropriate stream to the receiving user based on the recipient's network status. Technique 2: A secondary stream strategy. This involves simultaneously pushing two different streams to the server, with the client retrieving one or both streams as needed. Compared to the first method, this approach avoids the issue of the playback end missing video streams due to actively reducing the bitrate when network fluctuations occur.

[0034] However, Technique 1 has a significant drawback: the playback end is affected by the uplink client. For the uplink client, if the Simulcast layer is reduced to one for some reason, the playback end will not receive the video stream without special handling. Manually switching will result in a choppy experience, impacting the user experience. While Technique 2's small-stream / small-stream strategy avoids the problem of the playback end not receiving the video stream due to the active reduction in bitrate, it increases the bandwidth requirement for the small stream compared to the single-stream push strategy, and the repeated audio push causes bandwidth waste.

[0035] To address the aforementioned problems, the inventors devised a strategy to adjust the bandwidth of the streamer based on the audience's playback timing in order to reduce unnecessary bandwidth waste.

[0036] To address this, this invention provides a method for reducing network bandwidth in video conferencing. This method reduces video uplink bandwidth to the minimum supported by the hardware while ensuring normal audio transmission. Simultaneously, it allows streaming clients to retrieve video data only as needed based on the primary and secondary requirements of the video feed, thus saving bandwidth. This further reduces the uplink and downlink bandwidth pressure on the device, flexibly adapting to the needs of mainstream mobile conferencing. The specific execution steps are as follows: Figure 1 As shown, it includes:

[0037] 101. When the push client and the receiving client are connected in a video conference, the push client pushes the audio stream, static video stream and dynamic video stream to the media forwarding server of the SFU architecture through the PeerConnection links corresponding to the audio stream, static video stream and dynamic video stream respectively.

[0038] The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth, for example, the preset minimum bandwidth can be set to 32Kbps; this embodiment does not make a specific limitation. The push client is the client that initiates the video conference, and the receiving client is the client that is invited to join the video conference and is also the playback client. The audio stream, the static video stream, and the dynamic video stream each correspond to a PeerConnection link, for example, the audio stream corresponds to the first PeerConnection link, the static video stream corresponds to the second PeerConnection link, and the dynamic video stream corresponds to the third PeerConnection link; this embodiment does not make a specific limitation. The media forwarding server can be an SRS media forwarding server.

[0039] 102. The receiving client pulls audio streams and static video streams from the SFU architecture media forwarding server for video playback.

[0040] Through step 101, the SFU architecture media forwarding server can obtain the audio stream, static video stream, and dynamic video stream pushed by the push client. The receiving client retrieves the audio stream and the static video stream from the SFU architecture media forwarding server for video playback according to its business needs. At this time, the bandwidth of the static video stream is the preset minimum bandwidth, typically 32Kbps. The static video stream is a video stream composed of the minimum resolution and low frame rate supported by the hardware, suitable for retrieval and playback during video conference previews by the receiving client. In this case, the receiving client only needs to bear the downlink bandwidth pressure of the audio stream and the static video stream.

[0041] 103. When the receiving client zooms in on the video conference screen, the receiving client switches the pulled static video stream to a dynamic video stream, generates a dynamic video stream being pulled event, and uses the media forwarding server to notify the pushing client of the dynamic video stream being pulled event.

[0042] The dynamic video stream is a video stream whose resolution and frame rate are affected by the bandwidth limit dynamically adjusted by the push client. When the dynamic video stream starts to push, the parameters are set to the minimum resolution and a low frame rate by default, and empty data packets are used instead of video data packets.

[0043] From step 102, it can be seen that the receiving client has pulled the audio stream and the static video stream. At this time, the receiving client connects the peerConnection links corresponding to the audio stream and the static video stream respectively. When the receiving client zooms in on the video conference screen, it disconnects the peerConnection link corresponding to the static video stream and connects the peerConnection link corresponding to the dynamic video stream. At this time, the SRS media forwarding server detects that the dynamic video stream of the pushing client has been pulled, that is, the dynamic video stream is pulled event, and notifies the pushing client of the dynamic video stream being pulled event.

[0044] 104. Based on the dynamic video stream being pulled event, the push client will modify the bandwidth corresponding to the dynamic video stream from the preset minimum bandwidth to the preset low-definition bandwidth or the preset high-definition bandwidth.

[0045] From step 103, the dynamic video stream is retrieved. After receiving the dynamic video stream retrieval event, the push client dynamically modifies the parameters of the dynamic video stream from a preset minimum bandwidth to a preset low-definition bandwidth and restores normal video data packets. The preset low-definition bandwidth can be 90Kbps, but this embodiment does not specify a particular bandwidth. At this time, the push client and the receiving client transmit the dynamic video stream with the preset low-definition bandwidth requirement. It should be noted that when the receiving client switches back to the static video stream due to business needs and no longer zooms in to view the video conference screen, the SRS media forwarding server senses that the dynamic video stream of the push client has ended playback and notifies the push client to reduce the dynamic video stream to the minimum resolution and a lower frame rate, i.e., the preset minimum bandwidth, and replaces the video data packets with empty data packets.

[0046] Based on the above Figure 1 As can be seen from the implementation of the embodiments, the present invention provides a method for reducing network bandwidth in video conferencing. When a push client and a receiving client are connected in a video conference, the push client pushes the audio stream, the static video stream, and the dynamic video stream to a media forwarding server of an SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth. The receiving client retrieves the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback. When the receiving client zooms in on the video conference screen, the receiving client switches the retrieved static video stream to the dynamic video stream, generating a dynamic video stream retrieval event, and uses the media forwarding server to notify the push client of the dynamic video stream retrieval event. Based on the dynamic video stream retrieval event, the push client modifies the bandwidth corresponding to the dynamic video stream from the preset minimum bandwidth to a preset low-definition bandwidth or a preset high-definition bandwidth. This invention reduces video uplink bandwidth to the minimum hardware-supported standard while ensuring normal audio transmission during video conferencing. Simultaneously, it allows streaming clients to retrieve video data on demand based on the primary and secondary video content, thus saving bandwidth. This further reduces the uplink and downlink bandwidth pressure on the device and flexibly adapts to the mainstream mobile conferencing needs.

[0047] Furthermore, as a response to Figure 1 Further refinement and extension of the illustrated embodiments, this invention also provides another method for reducing video conferencing network bandwidth, such as... Figure 2 As shown, the specific steps are as follows:

[0048] 201. When the push client and the receiving client are connected in a video conference, the push client pushes the audio stream, static video stream and dynamic video stream to the media forwarding server of the SFU architecture through the PeerConnection links corresponding to the audio stream, static video stream and dynamic video stream respectively.

[0049] This step combines the description of step 101 in the above method, and the same content will not be repeated here.

[0050] The push client divides the audio and video streams required for the video conference into independent audio streams, static video streams, and dynamic video streams, and pushes them to the SRS media forwarding server of the SFU architecture through PeerConnection links. The receiving client pulls the audio stream, static video stream, and dynamic video stream from the SRS media forwarding server according to the business logic requirements.

[0051] The online video conferencing product architecture in which this invention is applied includes an SRS media forwarding server, a signaling server, several PC browser clients, and several Android clients; wherein, the SRS media forwarding server and the signaling server can perform SRS callbacks; the signaling server has a signaling channel with each browser client and Android client, and the SRS media forwarding server can pull two streams and push three streams with each browser client and Android client;

[0052] The term "pull two streams" refers to a single audio stream and a single static video stream or a single dynamic video stream. The term "push three streams" refers to a single audio stream, a single static video stream, and a single dynamic video stream. The signaling channel includes signaling to notify the streamer to increase or decrease bandwidth limits. The SRS callback refers to the two callback notifications sent to the signaling server when the SRS forwards the media stream. The callback content includes the streamer ID, the media stream type (audio, static video, dynamic video), and other information.

[0053] For example: User A (name: Zhang Yi, ID: A) sends a meeting invitation to User B (name: Zhang San, ID: B). User B clicks to accept the invitation. At this time, User A pushes audio stream, static video stream (default upload bandwidth limit 32Kbps) and dynamic video stream (default upload bandwidth limit 32Kbps).

[0054] Furthermore, the initial default bandwidth of the static video stream and the dynamic video stream is preset to a minimum bandwidth of 32Kbps. At the same time, the preset low-definition bandwidth of the dynamic video stream is set to 90Kbps and the preset high-definition bandwidth is set to 300Kbps. The bandwidth of the dynamic video stream can be dynamically adjusted between the preset minimum bandwidth, the preset low-definition bandwidth and the preset high-definition bandwidth as needed.

[0055] 202. The receiving client pulls audio streams and static video streams from the SFU architecture media forwarding server for video playback.

[0056] This step combines the description of step 102 in the above method, and the same content will not be repeated here.

[0057] 203. When the receiving client zooms in on the video conference screen, the receiving client switches the pulled static video stream to a dynamic video stream, generates a dynamic video stream being pulled event, and uses the media forwarding server to notify the pushing client of the dynamic video stream being pulled event.

[0058] This step combines the description of step 103 in the above method, and the same content will not be repeated here.

[0059] When the receiving client zooms in on the video conference screen using a preset zoom-in action, the receiving client uses preset pull rules to pull the audio stream and dynamic video stream from the pushing client, generating a dynamic video stream pull event. The preset zoom-in action executed by the receiving client triggers the start playback callback of the media forwarding server in the SFU architecture, and notifies the pushing client of the dynamic video stream pull event through the signaling service, thereby increasing the bandwidth limit corresponding to the dynamic video stream.

[0060] Specifically, when the receiving client zooms in on the video conference screen using a preset zoom-in action, the receiving client uses a preset pull rule to pull the audio stream and dynamic video stream from the pushing client, generating a dynamic video stream pull event. This includes: when the receiving client zooms in on the video conference screen using the preset zoom-in action, the receiving client disconnects the peerConnection link corresponding to the static video stream and connects the peerConnection link corresponding to the dynamic video stream, where the video data packet corresponding to the dynamic video stream is an empty data packet; and based on the receiving client connecting the peerConnection link corresponding to the dynamic video stream, a dynamic video stream pull event is generated.

[0061] For example:

[0062] After accepting the invitation, User B enters the meeting and begins to pull User A's audio and video streams. The pulling action triggers the SRS start playback callback, which notifies User A to increase the bandwidth limit via signaling service.

[0063] 204. Based on the dynamic video stream being pulled event, the push client will modify the bandwidth corresponding to the dynamic video stream from the preset minimum bandwidth to the preset low-definition bandwidth or the preset high-definition bandwidth.

[0064] This step combines the description of step 104 in the above method, and the same content will not be repeated here.

[0065] For example:

[0066] Based on the dynamic video stream being pulled event, user A changed the bandwidth corresponding to the dynamic video stream from 32Kbps to 90Kbps; according to the bandwidth statistics of user B's pull stream, the actual downlink bandwidth was approximately 90Kbps.

[0067] Furthermore, this embodiment also includes: based on the dynamic video stream being pulled event, the push client restores the video data packet corresponding to the dynamic video stream from an empty data packet to a normal video data packet.

[0068] 205. When the receiving client performs a video conference zoom-in operation to restore the video conference screen, the receiving client disconnects the peerConnection connection corresponding to the dynamic video stream and connects the peerConnection connection corresponding to the static video stream, generating a static video stream being pulled event.

[0069] The video data packets corresponding to the dynamic video stream are normal video data packets;

[0070] For example:

[0071] In user B's grid view, user A's view is turned off. The turning-off action triggers the SRS end playback callback, and user A is notified to reduce the bandwidth limit via signaling service.

[0072] 206. Based on the static video stream being pulled event, the push client modifies the video data packet corresponding to the dynamic video stream from a normal video data packet to an empty data packet.

[0073] It should be noted that when the push client ends the push, it needs to disconnect the three Peerconnection connections and reset the number of viewers.

[0074] Furthermore, in another embodiment of the present invention, when the push client conducts a video conference with multiple receiving clients, the SFU architecture media server stores multiple dynamic video stream retrieval events corresponding to the multiple receiving clients in a Redis service; when the number of dynamic video stream retrieval events in the Redis service increases from 0 to 1, the push client is notified to increase the uplink bandwidth; when the number of dynamic video stream retrieval events in the Redis service decreases from n to 0, the push client is notified to reduce the uplink bandwidth, wherein the value of n is not less than 1.

[0075] For example:

[0076] During multi-person video conferencing, the SRS media forwarding server stores the dynamic video stream fetch events it detects in the Redis service. The number of detected events indicates the number of viewers on the push client. When the number of viewers increases from 0 to 1, the push client is notified to increase the uplink bandwidth. Repeated detected events will not notify the push client. Similarly, when the number of viewers decreases from 1 to 0, the push client is notified to decrease the uplink bandwidth.

[0077] Based on the above Figure 2 As can be seen from the implementation, this invention provides a method for reducing network bandwidth in video conferencing. In online video conferencing under the SFU architecture, this invention reduces the bandwidth of the streaming video when there are no viewers. In applications such as online classrooms where the ratio of speakers to non-speakers is significantly different, this invention effectively saves uplink bandwidth for student clients who are not speakers. Furthermore, in applications such as multi-person video chats where the playback end can freely control the zoom in / out, show, and hide of the screen, this solution can notify the streaming end to change the bandwidth limit when controlling the screen, thereby saving uplink bandwidth.

[0078] Furthermore, as a response to the above Figure 1 In addition to the implementation of the method shown, this embodiment of the invention also provides a transmission device for reducing video conferencing network bandwidth, used for the above-mentioned... Figure 1 The method shown is implemented accordingly. This device embodiment corresponds to the foregoing method embodiment. For ease of reading, this device embodiment will not repeat the details of the foregoing method embodiment, but it should be clear that the device in this embodiment can implement all the contents of the foregoing method embodiment. Figure 3 As shown, the device includes:

[0079] The push unit 31 is used to push the audio stream, the static video stream and the dynamic video stream to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream and the dynamic video stream respectively when the push client and the receiving client are connected in a video conference. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth.

[0080] The pull unit 32 is used to pull the audio stream and the static video stream obtained from the push unit 31 from the media forwarding server of the SFU architecture for video playback when the receiving client does not zoom in on the video conference screen.

[0081] The switching unit 33 is used to switch the static video stream pulled from the pulling unit 32 to the dynamic video stream obtained from the pushing unit 31 when the receiving client zooms in on the video conference screen, generate a dynamic video stream being pulled event, and use the media forwarding server to notify the pushing client of the dynamic video stream being pulled event.

[0082] Modification unit 34 is used to modify the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth based on the dynamic video stream being pulled event obtained from the switching unit 33.

[0083] Furthermore, as a response to the above Figure 2 In addition to the method shown, this embodiment of the invention also provides another transmission device for reducing video conferencing network bandwidth, used for the above-mentioned... Figure 2 The method shown is implemented accordingly. This device embodiment corresponds to the foregoing method embodiment. For ease of reading, this device embodiment will not repeat the details of the foregoing method embodiment, but it should be clear that the device in this embodiment can implement all the contents of the foregoing method embodiment. Figure 4 As shown, the device includes:

[0084] Setting unit 36 ​​is used to set the initial default bandwidth of the static video stream and the dynamic video stream to a preset minimum bandwidth of 32Kbps, and at the same time set the preset low-definition bandwidth of the dynamic video stream to 90Kbps and the preset high-definition bandwidth to 300Kbps.

[0085] The push unit 31 is used to push the audio stream, the static video stream, and the dynamic video stream obtained from the setting unit 36 ​​to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively, when the push client and the receiving client are connected in a video conference. The initial bandwidth corresponding to the static video stream and the dynamic video stream is a preset minimum bandwidth.

[0086] The pull unit 32 is used to pull the audio stream and the static video stream obtained from the push unit 31 from the media forwarding server of the SFU architecture for video playback when the receiving client does not zoom in on the video conference screen.

[0087] The switching unit 33 is used to switch the static video stream pulled from the pulling unit 32 to the dynamic video stream obtained from the pushing unit 31 when the receiving client zooms in on the video conference screen, generate a dynamic video stream being pulled event, and use the media forwarding server to notify the pushing client of the dynamic video stream being pulled event.

[0088] Modification unit 34 is used to modify the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth based on the dynamic video stream being pulled event obtained from the switching unit 33.

[0089] The first adjustment unit 35 is used to, based on the dynamic video stream being pulled event obtained from the switching unit 33, restore the video data packet corresponding to the dynamic video stream from an empty data packet to a normal video data packet in the push client;

[0090] Storage unit 37 is used to store the events of multiple dynamic video streams being pulled by multiple receiving clients in a Redis service when the push client conducts a video conference with multiple receiving clients;

[0091] The second adjustment unit 38 is used to notify the push client to increase the uplink bandwidth when the number of events in which the dynamic video stream is pulled from the Redis service obtained from the storage unit 37 increases from 0 to 1.

[0092] The third adjustment unit 39 is used to notify the push client to reduce the uplink bandwidth when the number of events in which the dynamic video stream is pulled from the Redis service obtained from the storage unit 37 decreases from n to 0, wherein the value of n is not less than 1.

[0093] The switching unit 33 is further configured to, when the receiving client performs a video conference screen zoom-in operation to restore the video conference screen zoom-in, disconnect the peerConnection link corresponding to the dynamic video stream and connect the peerConnection link corresponding to the static video stream, and generate a static video stream being pulled event, wherein the video data packet corresponding to the dynamic video stream is a normal video data packet;

[0094] The first adjustment unit 35 is used to modify the video data packet corresponding to the dynamic video stream from a normal video data packet to an empty data packet based on the static video stream being pulled event obtained from the switching unit 33.

[0095] Furthermore, the switching unit 33 includes:

[0096] The generation module 331 is used to generate a dynamic video stream being pulled event when the receiving client uses a preset screen zoom-in action to zoom in on the video conference screen, and the receiving client uses a preset pull-in rule to pull in the audio stream and dynamic video stream of the pushing client.

[0097] The notification module 332 is used to trigger the start playback callback of the media forwarding server of the SFU architecture when the preset screen zoom-in action is executed by the receiving client. It notifies the streaming client of the event that the dynamic video stream obtained from the generation module 331 is pulled through the signaling service, thereby increasing the bandwidth limit corresponding to the dynamic video stream.

[0098] Furthermore, the generation module 331 includes:

[0099] The switching submodule 3311 is used to disconnect the peerConnection connection corresponding to the static video stream and connect the peerConnection connection corresponding to the dynamic video stream when the receiving client uses a preset screen zoom-in action to zoom in on the video conference screen. The video data packets corresponding to the dynamic video stream are empty data packets.

[0100] The generation submodule 3312 is used to generate a dynamic video stream being pulled event based on the peerConnection link corresponding to the dynamic video stream obtained from the receiving client linking the dynamic video stream from the switching submodule 3311.

[0101] Furthermore, embodiments of the present invention also provide a processor for running a program, wherein the program executes the above-described... Figure 1-2 The method for reducing network bandwidth in video conferencing as described in the document.

[0102] Furthermore, embodiments of the present invention also provide a storage medium for storing a computer program, wherein the computer program, when running, controls the device where the storage medium is located to execute the above-described... Figure 1-2 The method for reducing network bandwidth in video conferencing as described in the document.

[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0104] It is understood that the relevant features in the above methods and apparatus can be referenced interchangeably. Furthermore, the terms "first," "second," etc., in the above embodiments are used to distinguish between embodiments and do not represent the superiority or inferiority of any particular embodiment.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0106] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0107] In addition, the memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0108] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0114] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0116] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for reducing network bandwidth in video conferencing, characterized in that, The method includes: When the push client and the receiving client are connected in a video conference, the push client pushes the audio stream, the static video stream, and the dynamic video stream to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively. The initial bandwidth of the static video stream and the dynamic video stream is the preset minimum bandwidth. The static video stream is a video stream composed of the minimum resolution and a low frame rate supported by the hardware. The dynamic video stream is a video stream whose resolution and frame rate are affected by the bandwidth limit dynamically adjusted by the push client. The receiving client retrieves the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback; When the receiving client zooms in on the video conference screen, the receiving client switches the retrieved static video stream to the dynamic video stream, generates a dynamic video stream retrieval event, and uses the media forwarding server to notify the push client of the dynamic video stream retrieval event. Based on the event of the dynamic video stream being pulled, the push client modifies the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth, based on the preset minimum bandwidth.

2. The method according to claim 1, characterized in that, When the receiving client zooms in on the video conference screen, the receiving client switches the retrieved static video stream to the dynamic video stream, generates a dynamic video stream retrieval event, and uses the media forwarding server to notify the push client of the dynamic video stream retrieval event, including: When the receiving client zooms in on the video conference screen using a preset zoom-in action, the receiving client uses preset pull rules to pull the audio stream and dynamic video stream from the streaming client, generating a dynamic video stream pull event. The preset screen zoom-in action performed by the receiving client triggers the start playback callback of the media forwarding server of the SFU architecture. The signaling service then notifies the streaming client that the dynamic video stream has been pulled, thereby increasing the bandwidth limit corresponding to the dynamic video stream.

3. The method according to claim 2, characterized in that, When the receiving client zooms in on the video conference screen using a preset zoom-in action, the receiving client uses preset pull rules to pull the audio stream and dynamic video stream from the pushing client, generating a dynamic video stream pull event, including: When the receiving client zooms in on the video conference screen using a preset zoom-in action, the receiving client disconnects the peerConnection connection corresponding to the static video stream and connects the peerConnection connection corresponding to the dynamic video stream. The video data packets corresponding to the dynamic video stream are empty data packets. Based on the peerConnection link corresponding to the dynamic video stream linked by the receiving client, a dynamic video stream pull event is generated.

4. The method according to claim 3, characterized in that, After the push client modifies the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth based on the preset minimum bandwidth, the method further includes: Based on the event of the dynamic video stream being pulled, the push client restores the video data packet corresponding to the dynamic video stream from an empty data packet to a normal video data packet.

5. The method according to any one of claims 1-4, characterized in that, Before the push client pushes the audio stream, the static video stream, and the dynamic video stream to the SFU architecture media forwarding server via PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively, when the push client and the receiving client are connected in a video conference, the method further includes: The initial default bandwidth for both the static and dynamic video streams is set to a preset minimum bandwidth of 32Kbps. Simultaneously, the preset low-definition bandwidth for the dynamic video stream is set to 90Kbps and the preset high-definition bandwidth is set to 300Kbps.

6. The method according to claim 1, characterized in that, The method further includes: When the push client conducts a video conference with multiple receiving clients, the SFU architecture media server stores the events of multiple dynamic video streams being pulled by the multiple receiving clients in the Redis service; When the number of events in the Redis service that the dynamic video stream is pulled increases from 0 to 1, the push client is notified to increase the uplink bandwidth. When the number of events in which the dynamic video stream is pulled in the Redis service decreases from n to 0, the push client is notified to reduce the uplink bandwidth, where the value of n is not less than 1.

7. The method according to claim 1, characterized in that, The method further includes: When the receiving client performs a video conference image zoom-in operation to restore the video conference image, the receiving client disconnects the peerConnection connection corresponding to the dynamic video stream and connects the peerConnection connection corresponding to the static video stream, generating a static video stream being pulled event, wherein the video data packets corresponding to the dynamic video stream are normal video data packets; Based on the static video stream being pulled event, the push client modifies the video data packet corresponding to the dynamic video stream from a normal video data packet to an empty data packet.

8. A transmission device for reducing network bandwidth in video conferencing, characterized in that, include: The push unit is used to push the audio stream, the static video stream, and the dynamic video stream to the media forwarding server of the SFU architecture through PeerConnection links corresponding to the audio stream, the static video stream, and the dynamic video stream, respectively, when the push client and the receiving client are connected in a video conference. The initial bandwidth of the static video stream and the dynamic video stream is a preset minimum bandwidth. The static video stream is a video stream composed of the minimum resolution and a low frame rate supported by the hardware. The dynamic video stream is a video stream whose resolution and frame rate are affected by the bandwidth limit dynamically adjusted by the push client. The fetch unit is used to fetch the audio stream and the static video stream from the media forwarding server of the SFU architecture for video playback when the receiving client does not zoom in on the video conference screen; The switching unit is used to switch the static video stream to the dynamic video stream when the receiving client zooms in on the video conference screen, generate a dynamic video stream being pulled event, and use the media forwarding server to notify the push client of the dynamic video stream being pulled event. The modification unit is used to modify the bandwidth corresponding to the dynamic video stream to a preset low-definition bandwidth or a preset high-definition bandwidth based on the dynamic video stream being pulled event.

9. A storage medium comprising a stored program, characterized in that, When the program is running, it controls the device containing the storage medium to execute the transmission method for reducing video conferencing network bandwidth as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the transmission method for reducing video conferencing network bandwidth as described in any one of claims 1 to 7.

Images