SVC-based conference concatenation method and system

Abstract

The application provides a conference concatenation method and system based on SVC, which comprises the following steps: when a first conference is concatenated with a second conference, a concatenation controller assigns and stores the access information of a virtual terminal; the concatenation controller transmits the access information of the virtual terminal to a concatenation resource server, and drives the virtual terminal to initiate an access call; a first virtual terminal enters the first conference as a substitute of one terminal of the second conference; a second virtual terminal enters the second conference as a substitute of one terminal of the first conference; the two conferences exchange data through the virtual terminal, wherein the data comprises audio and video data of various resolutions based on SVC, and each participating terminal decodes and combines the audio and video data of appropriate resolution according to the requirement. The conference concatenation method of the application effectively realizes the concatenation of multiple scenes of public cloud, private cloud and hybrid cloud, supports the concatenation of audio and video pictures and shared contents, and has super large scale concurrency, super low time delay and anti-network packet loss.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication technology, in particular, to a conference concatenation method based on SVC and a system thereof. BACKGROUND

[0002] Internet-based video communication technology is widely used in video conference scenarios in work and life. With the increase of conference capacity and the complexity of networking, concatenated conferences emerge as the times require. Concatenated conferences realize the exchange of audio and video pictures and shared content of different conference terminals, greatly expanding the application of video conferences.

[0003] The conference concatenation methods in the prior art are all based on traditional AVC (Advanced Video Coding), such as patent CN 1735190A "A method for centrally managing concatenated conferences on a video operation platform" and patent CN 102404547 A "Processing method and device for concatenated conference sites in a concatenated conference and conference concatenation system". AVC has two sets of standards for video coding and decoding technology, namely, the standards H.261, H.263, H.263+ of the International Telecommunication Union (ITU-T), and the MPEG standards MPEG-1, MPEG-2, MPEG-4 of ISO.

[0004] For ease of understanding, the architecture principle of video conference AVC will be described below in combination with Figure 1 In the figure, three conference terminals A-C respectively transmit fixed-resolution video pictures of blue people, green people and purple people to the MCU (Multi Control Unit, control unit) in the background for picture rendering, and then the MCU encodes and combines the received fixed-resolution video pictures to obtain pictures respectively presented on the terminals A-C, i.e., the three white boxes in the figure, purple+green, purple+blue and green+blue. Finally, the MCU sends the completed three video encodings, i.e., the spliced pictures, to the terminals A-C. Traditional AVC: first, the MCU splices the pictures, which requires the MCU to stack the pictures layer by layer and encode multiple times, and all encoding tasks are borne by the MCU, which has high requirements for the MCU; second, the raw material for encoding is the fixed-resolution pictures sent by each terminal, and once there is a packet loss, the picture will be directly distorted. Therefore, the video quality after traditional AVC encoding is low, the network delay is high, the network adaptability is small, the terminal capacity that can be handled is small, the expansibility is poor, and different resolution pictures cannot be flexibly selected, and the superposition of shared content cannot be supported.

[0005] And the MCU background must be a cloud, in the conference cascading method based on the traditional AVC, the conference participating in the cascade can only come from one of the public cloud, the private cloud and the hybrid cloud, and cannot come from any combination of the three clouds. In addition, using the MCU to encode the synthesized picture can only support the same conference secret key and encryption mode, and the prior art does not support the conference with different conference secret keys or encryption modes to form a cascade.

[0006] In summary, due to the limitation of the AVC technology and the strong dependence on the MCU, the conference cascading scene based on the traditional AVC is limited, the concurrency capability is limited, and the support function is simple and rigid. SUMMARY

[0007] The application provides a conference cascading method and system based on SVC. The SVC, i.e. Scaled Video Coding, is a technology capable of dividing a video stream into multiple resolution, quality and frame rate layers, and is an extension of the H.264 video coding standard adopted by most video conference devices today. The conference cascading method and system based on SVC provided by the application support the multi-scene cascade of the public cloud, the private cloud and the hybrid cloud. The two conferences cascaded can come from any combination of the public cloud, the private cloud and the hybrid cloud, for example, public cloud-public cloud, public cloud-private cloud, private cloud A-private cloud B. The function supports the cascade of the basic functions of the video conference such as the audio and video pictures and shared content. And due to the adoption of the SVC scheme, the application can inherit the characteristics of SVC, support super large-scale concurrency, ultra-low latency, resistance to network packet loss and jitter, and other advantages.

[0008] In a first aspect, the application provides a conference cascading method based on SVC, characterized in that the method comprises:

[0009] When the first conference and the second conference are cascaded, the second cascade controller allocates the entry information of the second virtual terminal and transmits it to the first cascade controller, and the first cascade controller allocates and stores the entry information of the first virtual terminal;

[0010] The first cascade controller transmits the entry information of the second virtual terminal and the entry information of the first virtual terminal to the first cascade resource server, and drives the first virtual terminal and the second virtual terminal to respectively initiate the entry call to the first conference and the second conference;

[0011] The first virtual terminal enters the first conference as a participating terminal of the first conference as a substitute for one of the terminals of the second conference;

[0012] The second virtual terminal enters the second conference as a participant terminal of the second conference as a stand-in of one of the terminals of the first conference;

[0013] One of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals thereof;

[0014] The data includes audio and video data of various resolutions based on SVC, and each participant terminal selects audio and video data of appropriate resolution according to requirements for decoding and combination.

[0015] In a second aspect, the application further provides a conference-level connection system based on SVC, characterized in that the system comprises a first cascade controller and a first cascade resource server, wherein the first cascade resource server comprises a first virtual terminal and a second virtual terminal.

[0016] When the first conference is cascaded with the second conference:

[0017] The first cascade controller is configured to receive entry information of the second virtual terminal, assign and store entry information of the first virtual terminal, and deliver the entry information of the second virtual terminal and the entry information of the first virtual terminal to the first cascade resource server.

[0018] The first virtual terminal is configured to initiate an entry call to the first conference, and enter the first conference as a participant terminal of the first conference as a stand-in of one of the terminals of the second conference.

[0019] The second virtual terminal is configured to initiate an entry call to the second conference, and enter the second conference as a participant terminal of the second conference as a stand-in of one of the terminals of the first conference.

[0020] The first virtual terminal and the second virtual terminal enable one of the terminals of the second conference and one of the terminals of the first conference to exchange data in the first conference and the second conference through the corresponding virtual terminals thereof.

[0021] The data includes audio and video data of various resolutions based on SVC, and each participant terminal selects audio and video data of appropriate resolution according to requirements for decoding and combination.

[0022] The application provides a conference cascading method and system based on SVC, which has the following advantages: 1. supporting multi-level cascading between different conferences, supporting super large-scale concurrency, super low latency, anti-network packet loss and jitter; 2. supporting cascading of conferences with different conference keys or encryption methods and ensuring the security; 3. supporting multi-channel cascading between conferences, and the number of channels and virtual terminals in the channels can be adjusted in real time; 4. supporting audio and video pictures and shared content transmission in different level conferences; 5. supporting automatic speaker function realized by automatically calculating sound energy; 6. supporting global main conference, global shared content, realizing main conference picture and shared picture of any level, and spreading and broadcasting in the whole cascading group; 7. the two cascaded conferences can come from any combination of public cloud, private cloud and hybrid cloud; 8. the external cascading system can be deployed, expanded and cut according to needs. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a schematic diagram of the architecture principle of the prior art video conference AVC;

[0025] Figure 2 is a flowchart of the conference cascading method based on SVC provided by the embodiment of the application;

[0026] Figure 3 is a logical structure diagram of the prior art video conference cascading;

[0027] Figure 4 is a schematic diagram of the conference cascading system based on SVC provided by the embodiment of the application;

[0028] Figure 5 is a flowchart of the cascading data transmission method for different conferences through a virtual terminal provided by the embodiment of the application;

[0029] Figure 6 is a flowchart of the shared content transmission method for different conferences through a virtual terminal provided by the embodiment of the application;

[0030] Figure 7 is a schematic diagram of the architecture principle of the prior art video conference SVC;

[0031] Figure 8 is a schematic diagram of the regular conference main conference provided by the embodiment of the application;

[0032] Figure 9 is a multi-level conference cascade schematic diagram provided by an embodiment of the present application;

[0033] Figure 10 is a global main conference hall schematic diagram provided by an embodiment of the present application;

[0034] Figure 11 is a general conference sharing schematic diagram provided by an embodiment of the present application;

[0035] Figure 12 is a global sharing schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions of the present application are described in further detail below by means of the accompanying drawings and embodiments. SUMMARY

[0038] As described above, the present application provides a conference cascade method and system based on SVC, which realizes multi-scene cascade of public cloud, private cloud and hybrid cloud, supports cascade of video conference basic functions such as audio and video pictures and shared content, and supports super large-scale concurrency, ultra-low latency, anti-network packet loss and jitter.

[0039] Exemplary method

[0040] Figure 2 is a conference cascade method flowchart based on SVC provided by an embodiment of the present application, and the method comprises the following steps:

[0041] S201: When a first conference is cascaded with a second conference, a second cascade controller allocates the entry information of a second virtual terminal and transmits it to the first cascade controller, and the first cascade controller allocates and stores the entry information of a first virtual terminal.

[0042] Wherein, the cascade is a conference cascade, which occurs between two conferences, and realizes mutual exchange of audio and video pictures and shared content of participating terminals of two-end conferences, for example Figure 3 Conf-A and Conf-X on the right, each circle in the figure represents an independent conference, and the arrow between the two independent conferences represents a cascade relationship.

[0043] Further, two conferences forming a cascade relationship can also each cascade with other conferences, as shown in the right side of the figure, where Conf-B cascades with Conf-C while also forming a cascade relationship with Conf-Y. Along the cascade relationship, all the conferences connected together form a cascade group, for example, all the conferences on the left side of the figure form a cascade group, Conf-A and Conf-X on the right side of the figure form a cascade group, and Conf-B, Conf-C and Conf-Y form a cascade group.

[0044] There is a distinction between a superior and an inferior in the cascade relationship, for example, in step S201, the first conference cascades with the second conference, the first conference is the superior conference, and the second conference is the inferior conference; in the cascade relationship between Conf-B and Conf-C in the figure, Conf-B is the superior conference, and Conf-C is the inferior conference. A cascade group can have at most one superior and can have multiple inferiors, and multiple conferences cascading with each other present a tree structure. The superior and the inferior are defined by the user when initiating, for example, the first conference cascades with the second conference, the first conference is the superior conference, and the second conference is the inferior conference; or the second conference can be the superior conference, and the first conference can be the inferior conference. There is no absolute superior and inferior, and the relationship between the superior and the inferior is determined by the user in actual business.

[0045] Specifically, the entry information of the second virtual terminal is a token, a number, an access address, and related information of one of the terminals of the first conference being impersonated, and the like, which are required for the second virtual terminal to enter a conference. The related information of one of the terminals of the first conference being impersonated is an entry token, an entry number, and an entry access address, and the like, of the terminal.

[0046] The entry information of the first virtual terminal is a token, a number, an access address, and related information of one of the terminals of the second conference being impersonated, and the like, which are required for the first virtual terminal to enter a conference. The related information of one of the terminals of the second conference being impersonated is an entry token, an entry number, and an entry access address, and the like, of the terminal.

[0047] The cascade controller (Cas-Mgr) is responsible for the promotion of cascade business and the management of cascade resources. The first cascade controller is the cascade controller of the first conference, and the second cascade controller is the cascade controller of the second conference.

[0048] The cascade resource server (Cas-Res) is composed of cascade channels (TNL), and each cascade channel is composed of a pair of virtual terminals (VEP). For example, Figure 4As shown, the cascade resource server Cas-Res-1 includes one or more cascade channels TNL-1, TNL-2,..., TNL-n, each of which is used to realize the transmission of cascade data between two conferences, for example, the cascade channel TNL-1 is used for the transmission of cascade data between the conferences Conf-A and Conf-B, and the cascade channel TNL-2 can be used for the transmission of cascade data between the conferences Conf-A and Conf-C or between the conferences Conf-A and Conf-B.

[0049] The services among the plurality of cascade channels do not interfere with each other, for example, the data transmission lines L5-L18 in the cascade channel TNL-1 do not interfere with the data transmission of other cascade channels, such as the cascade channel TNL-2.

[0050] In addition, the number of cascade channels and the virtual terminals in the cascade channels are real-time adjustable, for example, the virtual terminals in the cascade channel TNL-1 are adjusted from VEP-A1 and VEP-B1 to VEP-A2 and VEP-C3, then the two conferences in the cascade are changed from the conference A and the conference B to the conference A and the conference C, and the conference terminal data in the cascade transmission is changed.

[0051] The cascade channel supports horizontal expansion and is arbitrarily scalable, thereby realizing the expansion of the cascade resource server and supporting super-large concurrent conference cascades.

[0052] The first cascade resource server is a cascade resource server of a first conference.

[0053] The cascade controller and the cascade resource server are independent of each other and independent of each conference system participating in the cascade, that is, the cascade between any independent conferences is realized in an external hanging form, for example, when the conference A needs to be cascaded with other conferences, then the cascade controller and the cascade resource server are deployed as needed, or only the cascade controller is deployed; when the conference A does not need to be cascaded with other conferences, then the cascade controller and the cascade resource server are unloaded.

[0054] In summary, the cascade controller and the cascade resource server in the application can support horizontal expansion, flexible deployment, expansion and trimming. The cascade controller and the cascade resource server in the application can be used as needed, unloaded according to functions, and theoretically have no upper limit through the expansion of the cascade channel in the cascade resource server, thereby realizing super-large conference cascade.

[0055] In one embodiment, before the first conference is cascaded with the second conference in step S201, the method further includes:

[0056] The conference control system of the first conference initiates a cascade request for a second conference to the first cascade controller;

[0057] The first cascade controller sends the cascade request to the conference control system of the second conference;

[0058] The conference control system of the second conference agrees to cascade and sends an agreement command to the first cascade controller.

[0059] The conference control system is an administrator who performs conference control and operation and maintenance on a conference.

[0060] S202: The first cascade controller transmits the entry information of the second virtual terminal and the entry information of the first virtual terminal to a first cascade resource server, and drives the first virtual terminal and the second virtual terminal to respectively initiate entry call for the first conference and the second conference.

[0061] The first cascade resource server is a cascade resource server of the first conference.

[0062] Specifically, the entry call of the first virtual terminal and the second virtual terminal both go through a signaling channel, and the entry process of the virtual terminal and the real terminal is consistent, that is, the signaling system SIG of the conference initiates signaling control on the participating terminal and the control unit MCU corresponding to the terminal, and establishes contact between the terminal and the control unit through the signaling system for media data interaction.

[0063] In the cascade relationship establishment process, the cascade controllers of the upper and lower conferences are used, but only the cascade resource server of the upper conference is needed. That is, in a cascade relationship, the cascade resource server of the upper conference is actually used. For example, in steps S201 and S202, the first conference is the upper conference and the second conference is the lower conference, and only the first cascade controller, the first cascade resource server and the second cascade controller are needed in the cascade relationship establishment.

[0064] S203: The first virtual terminal enters the first conference as a participating terminal of the first conference as a stand-in of one of the terminals of the second conference.

[0065] S204: The second virtual terminal enters the second conference as a participating terminal of the second conference as a stand-in of one of the terminals of the first conference.

[0066] Specifically, one of the terminals of the second conference and one of the terminals of the first conference can be a real terminal or a virtual terminal. For example, the virtual terminal VEP-B1 of the second conference can be a stand-in of the real terminal EP-A1 of the first conference, or a stand-in of the virtual terminal VEP-A2 of the first conference.

[0067] The virtual terminal enters the conference, and the real-time variable of the cascaded participating terminal is realized by changing one of the terminals which are the avatars. That is, the virtual terminal is similar to a container, and different contents can be changed in real time. For example, when the first virtual terminal is the avatar of terminal A in the second conference, terminal A participates in the first conference through the first virtual terminal; when the first virtual terminal is changed to the avatar of terminal X in the third conference, terminal X participates in the first conference through the first virtual terminal.

[0068] When the participating terminal of the cascade is changed, the corresponding relationship between the virtual terminal in each conference and the participating terminal in the cascaded conference is changed, so that the participating terminal of the cascade can be changed in real time. The terminal which is the avatar can be replaced in the original cascade channel, or the corresponding relationship between the new virtual terminal and the terminal which is the avatar can be established in the new cascade channel.

[0069] S205: One of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals.

[0070] The virtual terminal has the same function as the real terminal, and has the functions of entering the conference and media data stream encryption and decryption. The first virtual terminal can receive data of each participating terminal in the first conference, and the second virtual terminal can receive data of each participating terminal in the second conference.

[0071] Unlike the real terminal, the virtual terminal does not need to display and output the media content such as audio and video and sharing, but transmits the content to the virtual terminal paired in the cascade channel, forming a back-to-back media data forwarding. The cascade of different conferences is realized by the virtual terminal.

[0072] The specific method of cascade data transmission of different conferences by the virtual terminal will be described in detail below, as shown in the following table: Figure 5

[0073] S501: Data of one of the terminals of the first conference which is the avatar of the second virtual terminal is transmitted to the first virtual terminal, decrypted according to the secret key of the first conference, and then transmitted to the second virtual terminal;

[0074] S502: The second virtual terminal encrypts the decrypted data of the avatar of the first conference according to the secret key of the second conference, and forwards the data to each participating terminal of the second conference;

[0075] S503: Data of one of the terminals of the second conference which is the avatar of the first virtual terminal is transmitted to the second virtual terminal, decrypted according to the secret key of the second conference, and then transmitted to the first virtual terminal;​

[0076] S504: The first virtual terminal encrypts the decrypted data of the second conference of the avatar terminal according to the key of the first conference, and forwards to each terminal of the first conference.

[0077] The input source of the audio and video of the virtual terminal is the copy of the data stream of one of the terminals of the avatar, not directly from the camera and / or microphone.

[0078] As a specific embodiment, the cascade channel TNL-1 is composed of the virtual terminal VEP-A1 of conference A and the virtual terminal VEP-B1 of conference B back to back. The virtual terminal VEP-A1 is the avatar of the real terminal EP-B1 in conference B in conference A, and the virtual terminal VEP-B1 is the avatar of the real terminal EP-A1 in conference A in conference B.

[0079] The content receiver of the virtual terminal VEP-A1 receives the data of the real terminal EP-A1 in conference A, which includes but is not limited to audio and video data and shared content; then decrypts the received data according to the key of conference A, and transmits the decrypted data to the content transmitter of the virtual terminal VEP-B1; the content transmitter of the virtual terminal VEP-B1 encrypts the data it receives according to the key of conference B, and then sends it to the control unit of conference B; the control unit of conference B forwards the data it receives to each terminal of conference B, at this time each terminal of conference B can display the data content input by the virtual terminal VEP-B1, which comes from the real terminal EP-A1 in conference A.

[0080] Correspondingly, the content receiver of the virtual terminal VEP-B1 receives the data of the real terminal EP-B1 in conference B, which includes but is not limited to audio and video data and shared content; then decrypts the received data according to the key of conference B, and transmits the decrypted data to the content transmitter of the virtual terminal VEP-A1; the content transmitter of the virtual terminal VEP-A1 encrypts the data it receives according to the key of conference A, and then sends it to the control unit of conference A; the control unit of conference A forwards the data it receives to each terminal of conference A, at this time each terminal of conference B can display the data content input by the virtual terminal VEP-A1, which comes from the real terminal EP-B1 in conference B.

[0081] The specific method of transmitting shared content of different conferences by virtual terminal is described in detail as follows, such as Figure 6

[0082] ​S601: transmit the first shared content in the first conference to the first virtual terminal, and decrypt the first shared content according to the key of the first conference and then transmit the decrypted first shared content to the second virtual terminal;

[0083] S602: encrypt the decrypted first shared content according to the key of the second conference, so as to be forwarded to each terminal in the second conference;

[0084] S603: transmit the second shared content in the second conference to the second virtual terminal, and decrypt the second shared content according to the key of the second conference and then transmit the decrypted second shared content to the first virtual terminal;

[0085] S604: encrypt the decrypted second shared content according to the key of the first conference, so as to be forwarded to each terminal in the first conference.

[0086] It should be noted that the first shared content in the first conference can be the shared content of any terminal in the first conference, that is, the first shared content in the first conference can be the shared content of one of the terminals in the first conference which is replaced by the second virtual terminal VEP-B1, or can be the shared content of another terminal in the first conference.

[0087] Correspondingly, the second shared content in the second conference can be the shared content of any terminal in the second conference.

[0088] As a specific example, the cascade channel TNL-1 is composed of the virtual terminal VEP-A1 of the conference A and the virtual terminal VEP-B1 of the conference B back to back. The virtual terminal VEP-A1 is a replacement of the real terminal EP-B1 in the conference B in the conference A, and the virtual terminal VEP-B1 is a replacement of the real terminal EP-A1 in the conference A in the conference B.

[0089] The content receiver of the virtual terminal VEP-A1 receives the shared content of the real terminal EP-A2 in the conference A; then decrypts the received shared content according to the key of the conference A, and transmits the decrypted shared content to the content transmitter of the virtual terminal VEP-B1; the content transmitter of the virtual terminal VEP-B1 encrypts the shared content received by it according to the key of the conference B, and then sends it to the control unit of the conference B; the control unit of the conference B forwards the shared content received by it to each terminal in the conference B, at this time each terminal in the conference B can display the shared content input by the virtual terminal VEP-B1, which is from the real terminal EP-A2 in the conference A.

[0090] Correspondingly, the content receiver of the virtual terminal VEP-B1 receives the shared content of the real terminal EP-B3 in the conference B; then decrypts the received shared content according to the secret key of the conference B, and delivers the decrypted shared content to the content transmitter of the virtual terminal VEP-A1; the content transmitter of the virtual terminal VEP-A1 encrypts the shared content received by it according to the secret key of the conference A, and then sends it to the control unit of the conference A; the control unit of the conference A forwards the shared content received by it to each terminal of the conference A, at this time each terminal of the conference A can display the shared content input by the virtual terminal VEP-A1 and from the real terminal EP-B3 in the conference B.

[0091] In combination with the above two embodiments, another specific embodiment is listed as shown in the figure, the cascade channel TNL-1 is composed of the virtual terminal VEP-A1 of Conf-A and the virtual terminal VEP-B1 of Conf-B back to back. The virtual terminal VEP-A1 is the double of the real terminal EP-B1 in Conf-B in Conf-A, and the virtual terminal VEP-B1 is the double of the real terminal EP-A1 in Conf-A in Conf-B. Figure 4

[0092] The audio and video media content receiver PRX of the virtual terminal VEP-A1 receives the audio and video data of the real terminal EP-A1 in Conf-A through the line L6, and the shared content receiver CRX of the virtual terminal VEP-A1 receives the shared content of the real terminal EP-A2 in Conf-A through the line L8; then the audio and video media content receiver PRX of the virtual terminal VEP-A1 decrypts the received audio and video data according to the secret key of Conf-A, and delivers the decrypted audio and video data to the audio and video media content transmitter PTX of the virtual terminal VEP-B1 through the line L10; the shared content receiver CRX of the virtual terminal VEP-A1 decrypts the received shared content according to the secret key of Conf-A, and delivers the decrypted shared content to the shared content transmitter CTX of the virtual terminal VEP-B1 through the line L12;

[0093] The audio and video media content transmitter PTX of the virtual terminal VEP-B1 encrypts the audio and video data received by it according to the secret key of Conf-B, and then sends it to the control unit MCU-2 of Conf-B through the line L13; the shared content transmitter CTX of the virtual terminal VEP-B1 encrypts the shared content received by it according to the secret key of Conf-B, and then sends it to the control unit MCU-2 of Conf-B through the line L15;

[0094] ​The control unit MCU-2 of Conf-B forwards the audio and video data and the shared content received by it to each terminal of Conf-B, at this time, each terminal of Conf-B can display the audio and video data from the real terminal EP-A1 in Conf-A and the shared content of the real terminal EP-A2 input by the virtual terminal VEP-B1.

[0095] Similarly, each terminal of Conf-A can display the audio and video data from the real terminal EP-B1 in Conf-B and the shared content of the real terminal EP-B3 input by the virtual terminal VEP-A1.

[0096] In summary, the conference cascading method based on SVC provided by the present application supports forming a cascade of conferences with different conference keys or encryption methods, and the encryption and decryption are all memory-level calculations, which occur in the server and there is no situation of transmitting the conference key, the encryption method and the decrypted data through a public network, thus the security of the data can be ensured. The conference cascading method based on AVC cannot handle the cascade between multiple conferences with different encryption and decryption methods or different conference keys, because it needs to realize the encryption and decryption of data and the composition of pictures of each terminal in the MCU. If the unencrypted data is transmitted through a network for cascading, there is a risk of data leakage.

[0097] S206: The data includes audio and video data of various resolutions based on SVC, and each terminal selects appropriate resolution of the audio and video data for decoding and composition according to the requirement.

[0098] SVC (Scaled Video Coding, scalable video coding) is a technology that can divide a video stream into multiple resolution, quality and frame rate layers, and is an extension of the H.264 video coding standard adopted by most video conference devices today. The video conference device uses the SVC technology to receive and transmit a multi-layer video stream composed of a small base layer and multiple optional layers that can improve the resolution, frame rate and quality. This layered mode can greatly improve the error resilience and video quality, and has no high requirement on the bandwidth. In addition, a multi-layer SVC video stream can support multiple devices and networks.

[0099] The main advantage of using the SVC technology to hold a video conference is to improve the error resilience and thus improve the video quality in the transmission between networks prone to packet loss. This is particularly important in places where the available bandwidth cannot be guaranteed or in unmanaged networks such as public Internet.

[0100] For the convenience of understanding, the architecture principle of the video conference SVC will be described below in combination with Figure 7 .

[0101] In the figure, three conference terminals A-C respectively transmit video pictures of multiple resolutions, blue man, green man and purple man, to the background MCU, the MCU forwards video pictures of certain resolution to the conference terminal according to the actual demand of the conference terminal, the three white boxes in the conference terminals A-C are the display pictures corresponding thereto, the conference terminal A selects the large green man of high resolution to encode and display, the conference terminal B selects the small blue man and small purple man of low resolution to encode and display, and the conference terminal C selects the medium green man and small blue man of medium resolution to encode and display.

[0102] SVC: first, the conference terminal directly performs picture splicing, each conference terminal is responsible for picture encoding of the terminal itself, the background MCU only forwards and does not undertake picture splicing, and the MCU does not need to perform multiple encoding, and the requirement for the MCU is low; second, the encoding raw material is the pictures of various resolutions sent by the conference terminals, the conference terminal wants a large window to directly select a large picture of high resolution, and wants a small window to directly select a small picture of low resolution, which is real-time variable and has strong anti-packet loss capability, and when packet loss occurs, the resolution is automatically reduced to ensure smooth picture. Therefore, using SVC once encoding, the encoded video quality is high, the network delay is low, the network adaptability is high, the terminal capacity that can be coped with is large, the expansibility is high, different resolution pictures can be flexibly selected, not only the superposition of video pictures is supported, but also the superposition of shared content is supported.

[0103] And the SCV supports multi-scene cascading of public cloud, private cloud and hybrid cloud, and each cascaded conference can be implemented by different cloud services. The two cascaded conferences can be from any combination of public cloud, private cloud and hybrid cloud, public cloud-public cloud, public cloud-private cloud, private cloud A-private cloud B. Functionally, it supports the cascading of video conference basic functions such as audio and video pictures and shared content. And because of the use of the SVC scheme, the application can inherit the characteristics of the SVC, support super large scale concurrency, ultra-low latency, anti-network packet loss and jitter, and other advantages.

[0104] In the cascaded conference described above, each virtual terminal can be a stand-in for a designated conference participant, or an automatic speaker.

[0105] Specifically, if the virtual terminal in the cascaded conference is set as a stand-in for a speaker of another conference, the voice energy of each real terminal in the other conference is calculated, the conference participant with high energy is bound to the virtual terminal as the stand-in for the speaker.

[0106] It should be noted that any level of virtual terminal does not participate in the calculation of voice energy to prevent loopback.

[0107] In the cascading group described in this invention, any global operation specified for any meeting will take effect for every meeting within the cascading group. The following section elaborates on the main meeting room and global main meeting room, as well as shared screen and global sharing in the meeting cascading method:

[0108] The SVC-based conference cascading method also includes:

[0109] 1. Main venue and overall main venue

[0110] like Figure 8 As shown, the real terminal EP-A1 in Meeting A is the main venue for this meeting, and all participating terminals in Meeting A view the audio and video of the real terminal EP-A1 in full screen.

[0111] like Figure 9 As shown, multi-level conference cascading works as follows: Conference A is cascaded with Conference B, where Conference A transmits data from real terminal EP-A1 to Conference B, and Conference B transmits data from real terminal EP-B1 to Conference A; Conference A is cascaded with Conference C, where Conference A transmits data from real terminal EP-A1 to Conference C, and Conference C transmits data from real terminal EP-C1 to Conference A; Conference C is cascaded with Conference D, where Conference C transmits data from real terminal EP-C1 to Conference D, and Conference D transmits data from real terminal EP-D1 to Conference C.

[0112] like Figure 10 As shown, when EP-C3, a participating terminal in conference C of a cascading conference, is set as the global master venue, EP-C3 is designated as the master venue for conference C, and all participating terminals in conference C display the audio and video feed of EP-C3 in full screen. Then, all cascading conferences sequentially transmit the audio and video feed of EP-C3. Meeting C transmits the audio and video feed from participant terminal EP-C3 to Meeting A, binding it to virtual terminal VEP-A3. Meeting A sets virtual terminal VEP-A3 as the main meeting room, and all participant terminals in Meeting A display the audio and video feed from virtual terminal VEP-A3 in full screen, which is the same as the audio and video feed from participant terminal EP-C3. Meeting C then transmits the audio and video feed from participant terminal EP-C3 to Meeting D, binding it to virtual terminal VEP-D2. Meeting D sets virtual terminal VEP-D2 as the main meeting room, and all participant terminals in Meeting D display the audio and video feed from virtual terminal VEP-D2 in full screen, which is the same as the audio and video feed from participant terminal EP-C3. Meeting A then transmits the audio and video feed from virtual terminal VEP-A3 to Meeting B, binding it to virtual terminal VEP-B2. Meeting B sets VEP-B2 as the main meeting room, and all participant terminals in Meeting B display the audio and video feed from virtual terminal VEP-B2 in full screen, which is the same as the audio and video feed from participant terminal EP-C3.

[0113] As can be seen from the above embodiments, the terminal being replaced can be a real terminal or a virtual terminal. For example, both the virtual terminal VEP-A3 and the real terminal EP-C3 can be used as the terminal being replaced.

[0114] The above steps involve global business control via the cascade controller, which is equivalent to a one-click activation for the user. After the global main meeting room is expanded through the steps shown in the diagram, each level of the meeting will view the main meeting room terminal in full screen. However, the content all originates from the same real participating terminal, EP-C3.

[0115] In summary, when one participating terminal in a cascading conference is set as the global main venue, other cascading conferences use their participating virtual terminals as substitutes for the global main venue, and then set these substitutes as the main venues for the other cascading conferences. This allows all participating terminals in the cascading conferences to receive audio and video data of various resolutions based on SVC from the global main venue, and each participating terminal can select the appropriate resolution of audio and video data for decoding and combination according to its needs.

[0116] It is important to note that during the transmission process, audio and video sources that are propagated to the main conference room are automatically excluded. For example, conference A will not propagate to conference C, and conference B will not propagate to conference A.

[0117] Additionally, if a virtual terminal in a cascading conference is configured to act as a substitute for the main venue of another conference, then when the main venue of that other conference changes, the virtual terminal acting as the substitute will automatically bind to the new main venue and become its substitute. For example, if the main venue changes from the aforementioned participating terminal EP-C3 to EP-A1, then the virtual terminal in the cascading conference will act as a substitute for EP-A1 or a substitute for a substitute of EP-A1.

[0118] 2. Shared screen and global sharing

[0119] like Figure 11 As shown, the real terminal EP-A1 in meeting A is the shared content of this meeting, and all participating terminals in meeting A can view the shared content of the real terminal EP-A1.

[0120] like Figure 12As shown, in the multi-level conference cascade, when one participating terminal EP-C3 in the cascade conference is set as global sharing, all participating terminals of the conference C can view the shared content of the participating terminal EP-C3; the participating terminal EP-C3 transmits the shared content to the cascade channel of the virtual terminal VEP-A1, shares a virtual terminal with the audio and video data, the virtual terminal VEP-A1 opens the sharing, and all participating terminals of the conference A can view the shared content of the virtual terminal VEP-A1, that is, the shared content of the participating terminal EP-C3; the participating terminal EP-C3 transmits the shared content to the cascade channel of the virtual terminal VEP-D1, shares a virtual terminal with the audio and video data, the virtual terminal VEP-D1 opens the sharing, and all participating terminals of the conference D can view the shared content of the virtual terminal VEP-D1, that is, the shared content of the participating terminal EP-C3; the conference A transmits the shared content of the virtual terminal VEP-A1 to the cascade channel of the virtual terminal VEP-B1, shares a virtual terminal with the audio and video data, the virtual terminal VEP-B1 opens the sharing, and all participating terminals of the conference B can view the shared content of the virtual terminal VEP-B1, that is, the shared content of the participating terminal EP-C3.

[0121] The above steps are global service control performed by the cascade controller, and are equivalent to one-key effect for the user. After the global sharing is diffused through the steps in the above figure, the conference of each level can schedule and watch the sharing of the level. However, the content is from the same real participant EP-C3.

[0122] In summary, when one shared content in the cascade conference is set as global shared content, the shared content of the virtual terminal of the other cascade conference is replaced with the global shared content level by level.

[0123] It should be noted that the source of the shared content diffuses to the global main conference is automatically excluded in the transmission process, for example, the conference A does not diffuse to the conference C, and the conference B does not diffuse to the conference A.

[0124] Exemplary system

[0125] Correspondingly, the embodiment of the present application also provides a conference cascade system based on SVC. Figure 4 is a schematic diagram of the conference cascade system based on SVC provided by the embodiment of the present application, as shown in the figure, Figure 4 As shown, the system provided by the embodiment includes a first cascade controller Cas-Mgr-1 and a first cascade resource server Cas-Res-1, and the first cascade resource server Cas-Res-1 includes a first virtual terminal VEP-A1 and a second virtual terminal VEP-B1.

[0126] When the first conference Conf-A is cascaded with the second conference Conf-B:

[0127] The first cascade controller Cas-Mgr-1 is configured to receive the entry information of the second virtual terminal VEP-B1, allocate and store the entry information of the first virtual terminal VEP-A1, and deliver the entry information of the second virtual terminal VEP-B1 and the entry information of the first virtual terminal VEP-A1 to the first cascade resource server Cas-Res-1.

[0128] The first virtual terminal VEP-A1 is configured to initiate an entry call to the first conference Conf-A, and enter the first conference Conf-A as a participant terminal of the first conference Conf-A as a proxy of one of the terminals of the first conference Conf-A.

[0129] The second virtual terminal VEP-B1 is configured to initiate an entry call to the second conference Conf-B, and enter the second conference Conf-B as a participant terminal of the second conference Conf-B as a proxy of one of the terminals of the second conference Conf-B.

[0130] The first virtual terminal VEP-A1 and the second virtual terminal VEP-B1 enable one of the terminals of the second conference Conf-B and one of the terminals of the first conference Conf-A to exchange data in the first conference Conf-A and the second conference Conf-B through the corresponding virtual terminals.

[0131] The data includes audio and video data of various resolutions based on SVC, and each participant terminal selects appropriate resolution of audio and video data for decoding and combination according to requirements.

[0132] The one of the terminals of the second conference Conf-B and the one of the terminals of the first conference Conf-A are real terminals or virtual terminals.

[0133] The entry information of the second virtual terminal VEP-B1 includes tokens, numbers, access addresses, and related information of the one of the terminals of the first conference Conf-A as the proxy.

[0134] The entry information of the first virtual terminal VEP-A1 includes tokens, numbers, access addresses, and related information of the one of the terminals of the second conference Conf-B as the proxy.

[0135] The entry call of the first virtual terminal VEP-A1 and the second virtual terminal VEP-B1 are both through a signaling channel.

[0136] The first virtual terminal VEP-A1 is further configured to receive data of one of the terminals of the first conference Conf-A impersonated by the second virtual terminal VEP-B1, and decrypt the received data according to the key of the first conference Conf-A and then pass to the second virtual terminal VEP-B1;

[0137] The second virtual terminal VEP-B1 is further configured to encrypt the decrypted data of one of the terminals of the first conference Conf-A impersonated according to the key of the second conference Conf-B, and then forward to each terminal of the second conference Conf-B;

[0138] The second virtual terminal VEP-B1 is further configured to receive data of one of the terminals of the second conference Conf-B impersonated by the first virtual terminal VEP-A1, and decrypt the received data according to the key of the second conference Conf-B and then pass to the first virtual terminal VEP-A1;

[0139] The first virtual terminal VEP-A1 is further configured to encrypt the decrypted data of one of the terminals of the second conference Conf-B impersonated according to the key of the first conference Conf-A, and then forward to each terminal of the first conference Conf-A.

[0140] The first virtual terminal VEP-A1 is further configured to receive first shared content in the first conference Conf-A, and decrypt the received content according to the key of the first conference Conf-A and then pass to the second virtual terminal VEP-B1;

[0141] The second virtual terminal VEP-B1 is further configured to encrypt the decrypted first shared content according to the key of the second conference Conf-B, and then forward to each terminal of the second conference Conf-B;

[0142] The second virtual terminal VEP-B1 is further configured to receive second shared content in the second conference Conf-B, and decrypt the received content according to the key of the second conference Conf-B and then pass to the first virtual terminal VEP-A1;

[0143] The first virtual terminal VEP-A1 is further configured to encrypt the decrypted second shared content according to the key of the first conference Conf-A, and then forward to each terminal of the first conference Conf-A.

[0144] The first virtual terminal VEP-A1 is further configured to receive data of each terminal of the first conference Conf-A;

[0145] The second virtual terminal VEP-B1 is also configured to receive data from each terminal participating in the second conference Conf-B.

[0146] The first cascade controller Cas-Mgr-1 is also configured to receive a cascade request initiated by a conference control system of the first conference Conf-A to a conference control system of the second conference Conf-B, and send the cascade request to the conference control system MMS-2 of the second conference and receive a cascade command.

[0147] After the virtual terminal VEP enters a conference, the virtual terminal VEP can change the real-time variable number of terminals participating in the cascade by changing one of the terminals acting as a dummy.

[0148] The audio and video input of the virtual terminal VEP comes from a copy of the data stream of one of the terminals acting as a dummy.

[0149] The cascade resource server Cas-Res includes a plurality of cascade channels TNL, each of which is configured to realize the transmission of cascade data between two conferences, and the services of the plurality of cascade channels TNL do not interfere with each other.

[0150] The plurality of conferences cascaded with each other form a tree structure.

[0151] The system is also configured to set one of the terminals participating in the cascade conference as a global main conference, and other cascade conferences set the virtual terminal VEP participating in the cascade conference as a dummy of the global main conference, and set the dummy as the main conference of the other cascade conferences, so that each terminal participating in the cascade conference can receive the audio and video data of the global main conference in various resolutions based on SVC, and each terminal selects appropriate resolution of the audio and video data for decoding and combination according to the needs.

[0152] The system is also configured to set the virtual terminal VEP in the cascade conference as a dummy of the main conference of the other conference, and when the main conference of the other conference is changed, the virtual terminal VEP acting as a dummy of the main conference automatically binds to the new main conference after the change and acts as a dummy of the new main conference.

[0153] The system is also configured to set one of the shared contents in the cascade conference as a global shared content, and other cascade conferences replace the shared content of the virtual terminal VEP participating in the cascade conference with the global shared content.

[0154] The system is also configured to set the virtual terminal VEP in the cascade conference as a dummy of the speaker of the other conference, and by calculating the sound energy of each real terminal in the other conference, the terminal with high energy is bound to the virtual terminal VEP as a dummy of the speaker.

[0155] The respective cascaded conferences are implemented by different cloud services.

[0156] The cascaded controller Cas-Mgr and the cascaded resource server Cas-Res are independent of the respective conference systems participating in the cascading.

[0157] It should be noted that, although the operations of the SVC-based conference cascading method of the present application are described in a particular order in the accompanying drawings, this is not required or implied that the operations must be performed in this particular order, or that all of the illustrated operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0158] Furthermore, although several means, units, or modules of the SVC-based conference cascading system are mentioned in the above detailed description, this division is merely exemplary and not mandatory. Indeed, according to an embodiment of the present application, the features and functionalities of two or more modules described above can be embodied in one module. Conversely, the features and functionalities of one module described above can be further divided into several modules.

[0159] While the spirit and principles of the present application have been described with reference to several specific embodiments, it is to be understood that the present application is not limited to the disclosed specific embodiments, and that the division of aspects is not meant to imply that features from these aspects cannot be combined to benefit. This division is merely for ease of presentation. The present application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0160] The present application provides:

[0161] 1. A SVC-based conference cascading method, characterized in that the method comprises:

[0162] When a first conference is cascaded with a second conference, a second cascaded controller allocates and transmits the entry information of a second virtual terminal to the first cascaded controller, and the first cascaded controller allocates and stores the entry information of a first virtual terminal;

[0163] The first cascaded controller transmits the entry information of the second virtual terminal and the entry information of the first virtual terminal to a first cascaded resource server, and drives the first virtual terminal and the second virtual terminal to respectively initiate entry calls to the first conference and the second conference;

[0164] The first virtual terminal enters the first conference as a participant terminal of the first conference as a stand-in of one of the terminals of the second conference;

[0165] The second virtual terminal enters the second conference as a participant terminal of the second conference as a stand-in of one of the terminals of the first conference;

[0166] One of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals thereof;

[0167] The data includes audio and video data of various resolutions based on SVC, and each participant terminal selects audio and video data of an appropriate resolution according to a requirement to decode and combine.

[0168] 2. The conference concatenation method according to item 1, wherein one of the terminals of the second conference and one of the terminals of the first conference are real terminals or virtual terminals.

[0169] 3. The conference concatenation method according to item 1 or item 2, wherein the entry information of the second virtual terminal is token, number, access address required for entry of the second virtual terminal, and relevant information of one of the terminals of the first conference which is the stand-in.

[0170] The entry information of the first virtual terminal is token, number, access address required for entry of the first virtual terminal, and relevant information of one of the terminals of the second conference which is the stand-in.

[0171] 4. The conference concatenation method according to any one of items 1 to 3, wherein the entry call of the first virtual terminal and the second virtual terminal both go through a signaling channel.

[0172] 5. The conference concatenation method according to any one of items 1 to 4, wherein the specific steps that one of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals thereof include:

[0173] transmitting data of one of the terminals of the first conference which is the stand-in of the second virtual terminal to the first virtual terminal, decrypting the data according to a secret key of the first conference, and then transmitting the data to the second virtual terminal;

[0174] encrypting the decrypted data of one of the terminals of the first conference which is the stand-in according to a secret key of the second conference, so as to forward the data to each participant terminal of the second conference;

[0175] transmitting data of one of the terminals of the second conference which is the stand-in of the first virtual terminal to the second virtual terminal, decrypting the data according to a secret key of the second conference, and then transmitting the data to the first virtual terminal;

[0176] The first virtual terminal encrypts the decrypted data of the impersonated terminal of the second conference according to the key of the first conference, to forward to each terminal of the first conference.

[0177] 6. The conference concatenation method according to any one of the preceding items, wherein the specific steps of exchanging data between the first conference and the second conference by the corresponding virtual terminal of the one terminal of the second conference and the one terminal of the first conference comprise:

[0178] transmitting a first shared content in the first conference to the first virtual terminal, and decrypting the first shared content according to the key of the first conference and then transmitting to the second virtual terminal;

[0179] encrypting the decrypted first shared content according to the key of the second conference by the second virtual terminal, to forward to each terminal of the second conference;

[0180] transmitting a second shared content in the second conference to the second virtual terminal, and decrypting the second shared content according to the key of the second conference and then transmitting to the first virtual terminal;

[0181] encrypting the decrypted second shared content according to the key of the first conference by the first virtual terminal, to forward to each terminal of the first conference.

[0182] 7. The conference concatenation method according to any one of the preceding items, wherein the first virtual terminal can receive data of each terminal of the first conference;

[0183] the second virtual terminal can receive data of each terminal of the second conference.

[0184] 8. The conference concatenation method according to any one of the preceding items, wherein before the concatenation step of the first conference and the second conference, further comprising:

[0185] the conference control system of the first conference initiates a concatenation request of the second conference to the first concatenation controller;

[0186] the first concatenation controller sends the concatenation request to the conference control system of the second conference;

[0187] the conference control system of the second conference agrees to the concatenation, and sends an agreement command to the first concatenation controller.

[0188] 9. The conference concatenation method according to any one of the preceding items, wherein after the virtual terminal enters the conference, the real-time variable of the concatenation terminal is realized by changing the impersonated one terminal.

[0189] 10. The conference cascading method according to any one of the preceding claims, wherein the input source of the virtual terminal audio and video is a copy of the data stream of one of the terminals whose avatar it is.

[0190] 11. The conference cascading method according to any one of the preceding claims, wherein the cascading resource server comprises a plurality of cascading channels, each cascading channel being configured to implement the transmission of cascading data between two conferences, the plurality of cascading channels being mutually independent.

[0191] 12. The conference cascading method according to any one of the preceding claims, wherein the plurality of conferences cascaded with each other form a tree structure.

[0192] 13. The conference cascading method according to any one of the preceding claims, further comprising:

[0193] when one of the participating terminals in a cascaded conference is set as a global main conference, the virtual terminals participating in other cascaded conferences are set as the avatar of the global main conference, and the avatar is set as the main conference of the other cascaded conferences, so that all participating terminals in the cascaded conferences can receive the audio and video data of the global main conference in various resolutions based on SVC, and the participating terminals select appropriate resolution audio and video data for decoding and combination according to the needs.

[0194] 14. The conference cascading method according to any one of the preceding claims, wherein if a virtual terminal in a cascaded conference is set as the avatar of the main conference of other conferences, when the main conference of the other conferences is changed, the virtual terminal as the avatar of the main conference automatically binds to the new main conference after the change and becomes the avatar of the new main conference.

[0195] 15. The conference cascading method according to any one of the preceding claims, further comprising:

[0196] when one of the shared contents in a cascaded conference is set as a global shared content, the shared content of the virtual terminals participating in other cascaded conferences is replaced with the global shared content.

[0197] 16. The conference cascading method according to any one of the preceding claims, further comprising:

[0198] if a virtual terminal in a cascaded conference is set as the avatar of the speaker of other conferences, the participating terminal with high energy is bound to the virtual terminal as the avatar of the speaker by calculating the sound energy of each real terminal in the other conferences.

[0199] 17. The conference cascading method according to any one of items 1-16, wherein each cascaded conference is implemented by a different cloud service.

[0200] 18. The conference cascading method according to any one of items 1-17, wherein the cascading controller and the cascading resource server are independent of each conference system participating in the cascading.

[0201] 19. An SVC-based conference cascading system, comprising: a first cascading controller and a first cascading resource server, wherein the first cascading resource server comprises a first virtual terminal and a second virtual terminal.

[0202] When a first conference is cascaded with a second conference, the first cascading controller is configured to receive entry information of the second virtual terminal, allocate and store entry information of the first virtual terminal, and pass the entry information of the second virtual terminal and the entry information of the first virtual terminal to the first cascading resource server.

[0203] The first virtual terminal is configured to initiate an entry call to the first conference, and enter the first conference as a participant terminal of the first conference as a proxy of one of the terminals of the second conference.

[0204] The second virtual terminal is configured to initiate an entry call to the second conference, and enter the second conference as a participant terminal of the second conference as a proxy of one of the terminals of the first conference.

[0205] The first virtual terminal and the second virtual terminal enable one of the terminals of the second conference and one of the terminals of the first conference to exchange data in the first conference and the second conference through the corresponding virtual terminal.

[0206] The data comprises audio and video data of various resolutions based on SVC, and each participant terminal selects audio and video data of appropriate resolution for decoding and combination according to requirements.

[0207] The data comprises audio and video data of various resolutions based on SVC, and each participant terminal selects audio and video data of appropriate resolution for decoding and combination according to requirements.

[0208] 20. The conference cascading system according to item 19, wherein one of the terminals of the second conference and one of the terminals of the first conference are real terminals or virtual terminals.

[0209] 21. The conference cascading system according to item 19 or item 20, wherein the entry information of the second virtual terminal comprises a token, a number, an access address required for entry of the second virtual terminal, and relevant information of one of the terminals of the first conference which is a proxy.

[0210] The entry information of the first virtual terminal is a token, a number, an access address, and information about one of the terminals of the second conference impersonated by the first virtual terminal.

[0211] 22. The conference level contact system according to any one of items 19-21, wherein the entry call of the first virtual terminal and the second virtual terminal are both through a signaling channel.

[0212] 23. The conference level contact system according to any one of items 19-22, wherein:

[0213] The first virtual terminal is further configured to receive data of one of the terminals of the first conference impersonated by the second virtual terminal, decrypt the received data according to the secret key of the first conference, and then transmit the decrypted data to the second virtual terminal.

[0214] The second virtual terminal is further configured to encrypt the decrypted data of one of the terminals of the first conference impersonated by the second virtual terminal according to the secret key of the second conference, and then transmit the encrypted data to each terminal of the second conference.

[0215] The second virtual terminal is further configured to receive data of one of the terminals of the second conference impersonated by the first virtual terminal, decrypt the received data according to the secret key of the second conference, and then transmit the decrypted data to the first virtual terminal.

[0216] The first virtual terminal is further configured to encrypt the decrypted data of one of the terminals of the second conference impersonated by the first virtual terminal according to the secret key of the first conference, and then transmit the encrypted data to each terminal of the first conference.

[0217] 24. The conference level contact system according to any one of items 19-23, wherein:

[0218] The first virtual terminal is further configured to receive first shared content in the first conference, decrypt the received content according to the secret key of the first conference, and then transmit the decrypted content to the second virtual terminal.

[0219] The second virtual terminal is further configured to encrypt the decrypted first shared content according to the secret key of the second conference, and then transmit the encrypted content to each terminal of the second conference.

[0220] The second virtual terminal is further configured to receive second shared content in the second conference, decrypt the received content according to the secret key of the second conference, and then transmit the decrypted content to the first virtual terminal.

[0221] The first virtual terminal is further configured to encrypt the decrypted second shared content according to a key of the first conference, and forward the encrypted second shared content to each terminal of the first conference.

[0222] 25. The conference level connection system according to any one of items 19-24, wherein the first virtual terminal is further configured to receive data from each terminal of the first conference.

[0223] The second virtual terminal is further configured to receive data from each terminal of the second conference.

[0224] 26. The conference level connection system according to any one of items 19-25, wherein:

[0225] The first cascade controller is further configured to receive a cascade request from a conference control system of the first conference, and send the cascade request to a conference control system of the second conference and receive a cascade command.

[0226] 27. The conference level connection system according to any one of items 19-26, wherein the virtual terminal is capable of changing the number of terminals acting as its avatar in real time after entering a conference.

[0227] 28. The conference level connection system according to any one of items 19-27, wherein the audio and video input of the virtual terminal comes from a copy of the data stream of one of the terminals acting as its avatar.

[0228] 29. The conference level connection system according to any one of items 19-28, wherein the cascade resource server comprises a plurality of cascade channels, each cascade channel is configured to realize the transmission of cascade data between two conferences, and the services of the plurality of cascade channels do not interfere with each other.

[0229] 30. The conference level connection system according to any one of items 19-29, wherein a plurality of conferences that are cascaded form a tree structure.

[0230] 31. The conference level connection system according to any one of items 19-30, wherein the system is further configured to set one of the terminals of a cascade conference as a global main conference hall, and the virtual terminals of other cascade conferences act as the avatar of the global main conference hall, and the avatar is set as the main conference hall of the other cascade conferences in cascade, so that each terminal of the conferences participating in the cascade can receive the audio and video data of the global main conference hall in various resolutions based on SVC, and each terminal selects appropriate resolution of the audio and video data for decoding and combination according to the requirement.

[0231] 32. The conference level contact system according to any one of the preceding claims 19-31, wherein the system is further configured to, if a virtual terminal in a cascade conference is set as a stand-in of a main conference of other conferences, automatically bind the virtual terminal as the stand-in of the main conference to a new main conference after a change of the main conference of the other conferences and as a stand-in of the new main conference.

[0232] 33. The conference level contact system according to any one of the preceding claims 19-32, wherein the system is further configured to, if a shared content in a cascade conference is set as a global shared content, replace shared contents of virtual terminals participating in other cascade conferences with the global shared content.

[0233] 34. The conference level contact system according to any one of the preceding claims 19-33, wherein the system is further configured to, if a virtual terminal in a cascade conference is set as a stand-in of a speaker of other conferences, bind a virtual terminal as the stand-in of the speaker to a virtual terminal with high energy calculated from sounds of real terminals in the other conferences.

[0234] 35. The conference level contact system according to any one of the preceding claims 19-34, wherein the cascade conferences are respectively implemented by different cloud services.

[0235] 36. The conference level contact system according to any one of the preceding claims 19-35, wherein the cascade controller and the cascade resource server are respectively independent of conference systems participating in the cascade.

Claims

1. A method for SVC-based conference concatenation, characterized in that, The method comprises: When the first conference and the second conference are cascaded, the second cascaded controller allocates and transmits the access information of the second virtual terminal to the first cascaded controller, and the first cascaded controller allocates and stores the access information of the first virtual terminal; The first cascaded controller transmits the access information of the second virtual terminal and the access information of the first virtual terminal to the first cascaded resource server, and drives the first virtual terminal and the second virtual terminal to respectively initiate the access call to the first conference and the second conference; The first virtual terminal enters the first conference as a participant terminal of the first conference as a dummy of one of the terminals of the second conference; The second virtual terminal enters the second conference as a participant terminal of the second conference as a dummy of one of the terminals of the first conference; One of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals; The data includes audio and video data of various resolutions based on SVC, and each participant terminal selects appropriate resolution of audio and video data for decoding and combination according to the requirement; The specific steps that one of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals comprise: Data of one of the terminals of the first conference which is replaced by the second virtual terminal is transmitted to the first virtual terminal, decrypted according to the secret key of the first conference, and then transmitted to the second virtual terminal; The second virtual terminal encrypts the decrypted data of one of the terminals of the first conference which is replaced according to the secret key of the second conference, and forwards to each participant terminal of the second conference; Data of one of the terminals of the second conference which is replaced by the first virtual terminal is transmitted to the second virtual terminal, decrypted according to the secret key of the second conference, and then transmitted to the first virtual terminal; The first virtual terminal encrypts the decrypted data of one of the terminals of the second conference which is replaced according to the secret key of the first conference, and forwards to each participant terminal of the first conference.

2. The conferencing cascade method of claim 1, wherein, One of the terminals of the second conference and one of the terminals of the first conference are real terminals or virtual terminals.

3. The conference cascading method according to claim 1 or 2, characterized in that, The access information of the second virtual terminal is a token, a number, an access address, and related information of one of the terminals of the first conference which is replaced, required by the second virtual terminal for access; The access information of the first virtual terminal is a token, a number, an access address, and related information of one of the terminals of the second conference which is replaced, required by the first virtual terminal for access.

4. The conferencing cascading method according to any of claims 1-3, characterized by, The access call of the first virtual terminal and the second virtual terminal both go through a signaling channel.

5. The conferencing cascading method according to any of claims 1-4, characterized by, The specific steps that one of the terminals of the second conference and one of the terminals of the first conference exchange data in the first conference and the second conference through the corresponding virtual terminals comprise: The first shared content in the first conference is transmitted to the first virtual terminal, decrypted according to the key of the first conference, and then transmitted to the second virtual terminal; The second virtual terminal encrypts the decrypted first shared content according to the key of the second conference, and forwards it to each terminal of the second conference; The second shared content in the second conference is transmitted to the second virtual terminal, decrypted according to the key of the second conference, and then transmitted to the first virtual terminal; The first virtual terminal encrypts the decrypted second shared content according to the key of the first conference, and forwards it to each terminal of the first conference.

6. The conferencing cascading method according to any of claims 1-5, characterized by, The first virtual terminal can receive data from each terminal of the first conference; The second virtual terminal can receive data from each terminal of the second conference.

7. The conferencing cascading method according to any of claims 1-6, characterized by, The first conference and the second conference further include the following steps before cascading: The conference control system of the first conference initiates a cascading request to the second conference to the first cascading controller; The first cascading controller sends the cascading request to the conference control system of the second conference; The conference control system of the second conference agrees to cascade and sends an agreement command to the first cascading controller.

8. The conferencing cascading method according to any of claims 1-7, characterized by, After the virtual terminal enters the conference, the number of real-time variable cascading terminals is changed by changing one of the terminals it impersonates.

9. The conferencing cascading method according to any of claims 1-8, characterized by, The audio and video input of the virtual terminal comes from a copy of the data stream of one of the terminals it impersonates.

10. The conferencing cascading method according to any of claims 1-9, characterized by, The cascading resource server includes multiple cascading channels, each of which is used to realize the transmission of cascading data between two conferences, and the services between the multiple cascading channels do not interfere with each other.

11. The conferencing cascading method according to any of claims 1-10, characterized by, The multiple conferences cascaded with each other form a tree structure.

12. The conferencing cascading method according to any of claims 1-11, characterized by, The method further includes: When a terminal in a cascaded conference is set as a global main conference, the virtual terminals of other cascaded conferences are set as the impersonation of the global main conference, and the impersonation is set as the main conference of the other cascaded conferences, so that each terminal in all cascaded conferences can receive the audio and video data of the global main conference in various resolutions based on SVC, and each terminal selects appropriate resolution audio and video data for decoding and combination according to demand.

13. The conferencing cascading method according to any of claims 1-12, characterized by, If a virtual terminal in a cascaded conference is set as an impersonation of the main conference of another conference, when the main conference of the other conference changes, the virtual terminal as the impersonation of the main conference automatically binds to the new main conference after the change and becomes the impersonation of the new main conference.

14. The conferencing cascading method according to any of claims 1-13, characterized by, The method further includes: When a shared content in a cascaded conference is set as a global shared content, the shared content of the virtual terminals of other cascaded conferences is replaced with the global shared content.

15. The conferencing cascading method according to any of claims 1-14, characterized by, The method further includes: If a virtual terminal in a cascaded conference is set as an impersonation of a speaker in another conference, the speaker is bound to the virtual terminal as the impersonation of the speaker by calculating the sound energy of each real terminal in the other conference, and the terminal with high energy is set as the speaker.

16. The conferencing cascading method according to any of claims 1-15, characterized by, Each cascaded conference is implemented using a different cloud service.

17. The conferencing cascading method according to any of claims 1-16, characterized by, The cascading controller and the cascading resource server are independent of each conference system participating in cascading.

18. A SVC-based conference level contact system, characterized by, The system comprises a first cascade controller and a first cascade resource server, the first cascade resource server comprising a first virtual terminal and a second virtual terminal; When the first conference and the second conference are cascaded: The first cascade controller is configured to receive the access information of the second virtual terminal, allocate and store the access information of the first virtual terminal, and deliver the access information of the second virtual terminal and the access information of the first virtual terminal to the first cascade resource server; The first virtual terminal is configured to initiate an access call to the first conference, and enter the first conference as a participant terminal of the first conference as a dummy of one of the terminals of the second conference; The second virtual terminal is configured to initiate an access call to the second conference, and enter the second conference as a participant terminal of the second conference as a dummy of one of the terminals of the first conference; The first virtual terminal and the second virtual terminal enable one of the terminals of the second conference and one of the terminals of the first conference to exchange data in the first conference and the second conference through the corresponding virtual terminal; The data comprises audio and video data of various resolutions based on SVC, and each participant terminal selects appropriate resolution of audio and video data for decoding and combination according to requirements; The first virtual terminal is further configured to receive data of one of the terminals of the first conference which is a dummy of the second virtual terminal, decrypt the received data according to a key of the first conference, and then deliver the decrypted data to the second virtual terminal; The second virtual terminal is further configured to encrypt the decrypted data of one of the terminals of the first conference which is a dummy according to a key of the second conference, and then forward the encrypted data to each participant terminal of the second conference; The second virtual terminal is further configured to receive data of one of the terminals of the second conference which is a dummy of the first virtual terminal, decrypt the received data according to a key of the second conference, and then deliver the decrypted data to the first virtual terminal; The first virtual terminal is further configured to encrypt the decrypted data of one of the terminals of the second conference which is a dummy according to a key of the first conference, and then forward the encrypted data to each participant terminal of the first conference.

19. The conference level contact system according to claim 18, characterized in that, One of the terminals of the second conference and one of the terminals of the first conference are real terminals or virtual terminals.

20. The conference level contact system according to claim 18 or 19, characterized in that, The access information of the second virtual terminal comprises a token, a number, an access address, and related information of one of the terminals of the first conference which is a dummy, required for the second virtual terminal to access the second conference; The access information of the first virtual terminal comprises a token, a number, an access address, and related information of one of the terminals of the second conference which is a dummy, required for the first virtual terminal to access the first conference.

21. The conference level contact system according to any of claims 18-20, characterized by The access call of the first virtual terminal and the second virtual terminal both go through a signaling channel.

22. The conference cascade system according to any one of claims 18-21, wherein: The first virtual terminal is further configured to receive a first shared content in the first conference, decrypt the received content according to a key of the first conference, and then deliver the decrypted content to the second virtual terminal; The second virtual terminal is further configured to encrypt the decrypted first shared content according to a secret key of the second conference, and forward the encrypted first shared content to each terminal of the second conference; The second virtual terminal is further configured to receive second shared content of the second conference, decrypt the received content according to the secret key of the second conference, and then transmit the decrypted content to the first virtual terminal; The first virtual terminal is further configured to encrypt the decrypted second shared content according to the secret key of the first conference, and forward the encrypted second shared content to each terminal of the first conference.

23. A conference level contact system according to any of claims 18-22, characterized by The first virtual terminal is further configured to receive data from each terminal of the first conference. The second virtual terminal is further configured to receive data from each terminal of the second conference.

24. The conference cascade system according to any one of claims 19-23, wherein: The first cascade controller is further configured to receive a cascade request initiated by a conference control system of the first conference to a conference control system of the second conference, and send the cascade request to the conference control system of the second conference and receive a cascade command.

25. The conference level contact system according to any of claims 18-24, characterized by The virtual terminal can change the number of real-time terminals participating in the cascade by changing one of the terminals acting as a dummy.

26. The conference level contact system according to any of claims 18-25, characterized by The audio and video input source of the virtual terminal comes from a copy of the data stream of one of the terminals acting as a dummy.

27. The conference level contact system according to any of claims 18-26, characterized by The cascade resource server includes a plurality of cascade channels, each of which is configured to implement the transmission of cascade data between two conferences, and the services between the plurality of cascade channels do not interfere with each other.

28. The conference level contact system according to any of claims 18-27, characterized by The plurality of conferences cascaded with each other form a tree structure.

29. The conference level contact system according to any of claims 18-28, characterized by The system is further configured to set one of the terminals participating in the cascade conference as a global main conference, and the virtual terminals participating in other cascade conferences act as dummies of the global main conference, and the dummies are set as main conferences of the other cascade conferences, so that each terminal participating in the cascade conferences can receive audio and video data of various resolutions based on SVC of the global main conference, and each terminal selects appropriate resolution of audio and video data for decoding and combination according to needs.

30. The conference level contact system according to any of claims 18-29, characterized by The system is further configured to set a virtual terminal in a cascade conference as a dummy of a main conference of another conference, and when the main conference of the other conference is changed, the virtual terminal acting as the dummy of the main conference automatically binds to the new main conference after the change and acts as a dummy of the new main conference.

31. The conference level contact system according to any of claims 18-30, characterized by The system is further configured to set one of the shared contents in the cascade conference as a global shared content, and the virtual terminals participating in other cascade conferences replace the shared contents of the virtual terminals with the global shared content.

32. The conference level contact system according to any of claims 18-31, characterized by The system is further configured to set a virtual terminal in a cascade conference as a dummy of a speaker of another conference, and by calculating the sound energy of each real terminal in the other conference, the terminal with high energy is bound to the virtual terminal as the dummy of the speaker.

33. The conference level contact system according to any of claims 18-32, characterized by Each cascade conference is implemented by a different cloud service.

34. The conference level contact system according to any of claims 18-33, characterized by The cascade controller and the cascade resource server are independent of each conference system participating in the cascade.

Images