Conference data transmission method and conference system

Abstract

The present application provides a conference data transmission method and a conference system, in which a user control platform controls the number of first clients and second clients according to pre-stored user permissions, and further controls the network bandwidth occupied by transmitting conference data. The media cloud server sends conference data to each first client and the target CDN, wherein the first client can directly display the received conference data, and each second client can obtain the conference data from the target CDN, so that the second client can also listen in on the conference data. At the same time, the way the second client obtains the conference data does not occupy the bandwidth resources of the communication network between the first client and the media cloud server, and therefore, more second clients can be supported to listen in on the conference, thereby achieving the effect of thousands of people participating in the conference.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a conference data transmission method and a conference system. Background Art

[0002] At work, multiple users are often required to participate in online meetings at the same time to exchange information with each other. At this time, each user participating in the meeting needs to upload their own audio and video data to the media cloud server in real time. The media cloud server then processes the audio and video data uploaded by each user and sends the processed data to other users participating in the meeting, thus realizing the transmission of speech data between each user participating in the meeting.

[0003] However, this conference data transmission method, where users upload audio and video data in real time and the cloud server distributes it to other users, continuously consumes the conference network's bandwidth once a data transmission channel is established between the participating users and the media cloud server. Due to the limited total available bandwidth of the conference network, the number of participating users must be strictly limited to ensure smooth audio and video playback. This means that it's impossible to achieve the effect of thousands of people participating in a conference together. Summary of the Invention

[0004] The present application provides a conference data transmission method and a conference system to effectively increase the number of users participating in a conference.

[0005] In a first aspect, the present application provides a conference data transmission method, applied to a user control platform, the method comprising:

[0006] Receiving a conference access request sent by a client through a signaling channel, and determining the category of the client according to pre-stored user permissions;

[0007] If the client is a first client, connecting the client to a data transmission channel so that the client receives conference data sent by the media cloud server through the data transmission channel, and transmitting speech data to the media cloud server through the data transmission channel when the client meets the preset speech rules, wherein the conference data is speech data processed by the media cloud server;

[0008] If the client is a second client, a streaming address corresponding to the conference is sent to the client, so that the client streams the conference data from a target CDN according to the streaming address so as to attend the conference.

[0009] Optionally, when the client satisfies a preset speaking rule, transmitting the speaking data to the media cloud server through the data transmission channel includes:

[0010] Detecting whether the decibel value of the speech data meets a preset decibel threshold;

[0011] If the decibel value of the speech data is greater than or equal to the preset decibel threshold, the speech data is transmitted to the media cloud server through the data transmission channel.

[0012] Optionally, when the client satisfies a preset speaking rule, transmitting the speaking data to the media cloud server through the data transmission channel further includes:

[0013] If the decibel value of the speech data is less than the preset decibel threshold, the decibel value of the speech data is increased by using a sound excitation technology so that the decibel value after excitation is greater than or equal to the preset decibel threshold.

[0014] Optionally, the preset speaking rules are one or more combinations of the following: the user identity corresponding to the client conforms to the preset user identity, the user identity weight corresponding to the client conforms to the preset user identity weight, and the user identity corresponding to the client conforms to the host's management strategy for speaking users, wherein the host has user speech management authority.

[0015] Optionally, the method further includes:

[0016] receiving, through the signaling channel, an identity switching request sent by a client to be switched, where the client to be switched is a first client;

[0017] Determine whether the client to be switched meets the preset identity switching conditions according to the pre-stored user permissions;

[0018] If the client to be switched meets the preset identity switching condition, an identity switching instruction is generated and sent to the client to be switched via the signaling channel, so that the client to be switched switches to a target identity, where the target identity includes a second client and a leaving identity.

[0019] Optionally, the method further includes:

[0020] receiving, via the signaling channel, an identity switching request sent by the client to be switched;

[0021] Determine whether the client to be switched meets the preset identity switching conditions according to the pre-stored user permissions;

[0022] If the client to be switched meets the preset identity switching condition, an identity switching instruction is generated and sent to the client to be switched via the signaling channel, so that the client to be switched switches to a target identity, where the target identity includes the first client and the leaving identity.

[0023] Optionally, the preset identity switching conditions include the total number of clients corresponding to the client type after switching being less than or equal to a preset number threshold, the user identity corresponding to the client to be switched conforming to the preset user identity, the user identity weight corresponding to the client to be switched conforming to the preset user identity weight, or the user identity corresponding to the client to be switched conforming to the host's user identity management strategy, one or more of the combinations thereof, wherein the host has user identity management authority.

[0024] Optionally, the method further includes: using the MQTT protocol to transmit instructions between the user control platform and the first client and the second client in the signaling channel.

[0025] Optionally, the conference data is pushed to a target CDN in real time via RTMP, so that the second client obtains the conference data from the target CDN.

[0026] In a second aspect, the present application provides a conference system, comprising: a user control platform, multiple clients, a media cloud server, and a target CDN;

[0027] The user control platform is used to receive a conference access request sent by a client through a signaling channel and determine the category of the client according to pre-stored user rights;

[0028] If the client is a first client, connecting the client to a data transmission channel so that the client receives conference data sent by the cloud server through the data transmission channel, and transmitting speech data to the cloud server through the data transmission channel when the client meets the preset speech rules, wherein the conference data is speech data processed by the media cloud server;

[0029] If the client is a second client, sending a streaming address corresponding to the conference to the client, so that the client streams the conference data from the target CDN according to the streaming address to attend the conference;

[0030] The media cloud server is used to receive the speech data uploaded by the first client through the data transmission channel, and perform audio and video synchronization processing, and send the processed conference data to each first client through the data channel respectively, and push it to the target CDN in real time through the RTMP protocol;

[0031] The target CDN is used to receive the conference data pushed by the media cloud server, so that each second client pulls the stream from the target CDN and listens in.

[0032] From the above technology, it can be seen that the present application provides a conference data transmission method and a conference system, in which the user control platform controls the number of first clients and second clients according to pre-stored user permissions, and further controls the network bandwidth occupied by transmitting speech data. The media cloud server sends conference data to each first client and the target CDN, wherein the first client can directly display the received conference data, and each second client can obtain the conference data from the target CDN, so that the second client can also listen to the conference data. At the same time, the way the second client obtains the conference data does not occupy the bandwidth resources of the communication network between the first client and the media cloud server. Therefore, more second clients can be supported to listen in on the conference, thereby achieving the effect of thousands of people participating in the conference. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 A schematic diagram of the structure of a conference system provided by an embodiment of the present invention;

[0035] Figure 2 A flowchart of a conference data transmission method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a process for determining a target first client provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a process for uploading speech data by a first client provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of a process for processing speech data provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of a process for determining a target second client provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a process for determining display interface data provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of a process of switching identities of a first client provided in an embodiment of the present application;

[0042] Figure 9 A schematic diagram of a process for switching identities of a client provided in an embodiment of the present application.

[0043] Illustration:

[0044] Among them, 1-user control platform, 2-first client, 3-media cloud server, 4-target CDN, 5-second client. DETAILED DESCRIPTION

[0045] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] Figure 1 A schematic diagram of the structure of a conference system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the conference system includes a user control platform 1, a first client 2, a media cloud server 3, a target CDN 4, and a second client 5. The user control platform 1 can be a server, computing platform, computer, or other device with signaling, signaling processing, and data processing capabilities. A participating user is a user who joins a target conference and has permissions to upload speech data and receive conference data. The target conference is the conference joined by the participating user. The first client 2 and the second client 5 can be devices with signaling, data, and data processing capabilities, such as mobile phones, computers, tablets, and virtual reality devices. Furthermore, since the first client 2 and the second client 5 can subsequently switch identities, the terms "first" and "second" are used only to distinguish the identities corresponding to the clients under certain conditions and do not limit the number of clients or their specific functions. The media cloud server 3 can be a server, computer, computing platform, or other device with data transmission, reception, and data processing capabilities. The target CDN 4 is a virtual network with data transmission, reception, and data storage capabilities. In this embodiment, the target CDN 4 refers to the CDN corresponding to the target conference.

[0047] Figure 2 A flow chart of a conference data transmission method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the specific process is as follows:

[0048] S101: Receive a conference access request sent by a client through a signaling channel, and determine the category of the client according to pre-stored user rights.

[0049] The conference data transmission in the conference system can be divided into two parts, namely, conference data transmission for the first client 2 and conference data transmission for the second client 5 . The conference data transmission process for the first client 2 will now be described.

[0050] The user control platform 1 is provided with a login registration center (register). Before joining a meeting, the user needs to register personal information in the user control platform 1 through the client. For example, the user sends registration information to the user control platform 1 through the client. The registration information may include: user information, user account, user password, target meeting information, device identification, etc. In this way, after receiving the registration information, the user control platform 1 reviews the identity of each user through the internal authentication center (auth), and if the user's identity is legal, determines that the user is eligible to participate in the meeting. In this application, the qualification to participate in the meeting can refer to two situations: the qualification to speak on the microphone and receive conference data at the same time, and the qualification to only receive conference data in the meeting. According to the different needs of users, users can send corresponding requests to the user control platform 1 through the client. In one case, if the user wants to have the qualification to speak on the microphone, it is necessary to send a first conference participation request to the user control platform 1 through the client. The first conference participation request is used to apply for the right to speak on the microphone in the meeting. When the user control platform 1 agrees that these users have the right to speak on the microphone in the meeting, the clients corresponding to these users can be called the first client 2 (since there is a mapping relationship between the user and the client, in order to facilitate subsequent expression, the client is directly used instead of the user). At this time, whether the first client 2 can actually speak on the microphone (send speech data to the cloud server 3) still needs further judgment by the user control platform 1. In another case, if the user only wants to watch and listen to the meeting without using the microphone, that is, to attend the meeting as an observer, they need to send a second meeting participation request to the user control platform 1 through the client. The second meeting participation request is used to apply for the right to attend the meeting as an observer. The client corresponding to these users can be called the second client 5. At this time, whether the second client 5 can actually attend the meeting as an observer still needs to be further determined by the user control platform 1. It should be noted that the first client 2's eligibility to participate in the meeting means that these users of the first client 2 currently only have the qualification to participate in the meeting, that is, the right to use the microphone (upload speech data). However, whether they can use the microphone during the meeting still needs further management and control by the user control platform 1.

[0051] Figure 3 A flow chart of determining a target first client provided in an embodiment of the present application is as follows: Figure 3 As shown, the method includes:

[0052] S201: The user control platform receives a speaking request sent by the first client through the signaling channel;

[0053] S202: The user control platform determines whether the first client complies with preset speaking rules;

[0054] S203: If the first client complies with the preset speaking rule, determine the first client as a target first client and generate a speaking data uploading instruction, wherein the speaking data uploading instruction includes a device identifier corresponding to the target first client.

[0055] Before a meeting begins, each first client 2 can send a speaking request to the user control platform 1 via a signaling channel to apply for permission to upload speaking data during the meeting. The speaking request can include a device identifier and user information. At this point, the user control platform 1 can review the first client 2 through its internal conference control management center (conference). The user control platform 1 can pre-set speaking rules to prevent bandwidth usage caused by an excessive number of speakers.

[0056] In one implementation, if bandwidth resources are abundant enough and will not affect the audio-visual quality of the conference (such as multiple people speaking at the same time), the user control platform 1 can set all first clients 2 as target first clients, where the target first client refers to the first client that actually uploads the speech data to the media cloud server 3 in the conference.

[0057] In one implementation, if bandwidth resources are relatively tight and cannot accommodate too many first clients uploading speech data, or if the speech data displayed simultaneously by too many first clients will interfere with each other (e.g., multiple people speaking at the same time, disrupting the audio and video), etc., the user control platform 1 needs to determine the target first client according to the preset speech rules, thereby reducing bandwidth pressure and ensuring the quality of the meeting. For example, the user control platform 1 receives speech requests sent by multiple first clients 1 at the same time, such as receiving speech requests sent by 10 first clients 1 at the same time. If the preset number of speakers set by the user control platform 1 is 4, it is necessary to select 4 from the above 10 first clients 1 as the target first clients, thereby controlling the bandwidth resources occupied by the target first clients uploading speech data.

[0058] The preset speaking rules can be preset speaking rules such that the user identity corresponding to the client conforms to the preset user identity, the user identity weight corresponding to the client conforms to the preset user identity weight, and the user identity corresponding to the client conforms to the host's management strategy for speaking users, or a combination of the two, wherein the host has user speech management authority.

[0059] Specifically, the user identity corresponding to the first client in the preset speaking rule matches the user identity of the preset speaking user. Since some users with special identities are usually required to speak in a meeting, such as guests, hosts, etc., in order to prevent the situation where a user with a special identity enters the meeting after the target first client is determined, and the number of speakers reaches the upper limit and is unable to speak, the user control center 1 can adopt a reservation rule, that is, only a certain number of speakers are opened, allowing users with ordinary identities (users who do not have the above-mentioned special identities to speak) to speak. However, when the number of speakers for users with ordinary identities reaches the open number of speakers, no more users with ordinary identities are allowed to speak. In this case, users with special identities can be directly designated to speak. For example, if the preset number of speakers is 4, and according to the reservation rule, the number of speakers is set to 2, the maximum number of users with ordinary identities allowed to speak is 2. When the number of users with ordinary identities speaking reaches 2, other users with ordinary identities are prohibited from speaking. In this case, users with special identities can be directly designated to speak.

[0060] The preset speaking rule specifies that the identity weight of the user corresponding to the first client 1 matches the preset speaking user identity weights of the user control platform 1. The user control platform 1 sequentially receives speaking requests from the first client 1. When the number of speaking requests received by the user control platform 1 exceeds the number of remaining speakers, the user control platform 1 identifies the identity weights of the users corresponding to the speaking requests and determines the first client 1 corresponding to the higher identity weight as the target first client. Identity weights can be calculated based on preset criteria, such as whether the user is a member, whether the user is a special invite to the meeting, or whether the user has a bad record. For example, if the preset number of speakers is 4 and the number of confirmed speakers is 2, the number of remaining speakers is 2. If there are 10 speaking requests, the identity weights of the users corresponding to these 10 speaking requests are obtained. For example, if the identity weights of the 10 users are 85, 60, 80, 90, 91, 92, 70, 74, 78, and 80, respectively, the first clients 1 corresponding to the users with identity weights 91 and 92 can be determined as the target first clients.

[0061] In one implementation, the preset speaking condition is that the user information corresponding to the participating user to be spoken complies with the host management policy, the host is a participating user with user management authority, and the host management policy is the user authority set by the host. There will be participating users with management authority in the meeting, who can be called hosts, such as Figure 1The black person shown in the figure is the host who has certain user management permissions, for example, controlling the user to go on the microphone (can upload speech data), leave the microphone (prohibit uploading speech data), etc. After the user control platform 1 receives the speech data upload request, it identifies whether the user information in the speech data upload request conflicts with the host's management policy, and determines that the speech data upload request corresponding to the non-conflicting user identifier is the target speech data upload request. For example, if the host's management policy is to prohibit ordinary participating users from going on the microphone, the user control platform 1 will generate a prohibition on uploading speech data instructions for all speech data upload requests; if the host's management policy is to prohibit the participating users to be spoken whose identity weight is less than 90 from uploading speech data, if the identity weight of the participating user to be spoken corresponding to the speech data upload request is 80, then a prohibition on uploading speech data instruction is generated; if the identity weight of the participating user to be spoken corresponding to the speech data upload request is 95, then the speech data upload request is the target speech data upload request.

[0062] The preset speech rule is that the user identity corresponding to the first client 1 complies with the host's control policy. The host is the first client 1 with user management authority, wherein the host control policy is the user authority set by the host. There are usually users with management authority in the meeting, which can be called hosts, such as Figure 1 The black person shown in the figure is the host who has certain user management permissions, for example, controlling the user to go on the microphone (can upload speech data), to leave the microphone (prohibit uploading speech data), etc. When the user control platform 1 receives a speech request, it identifies whether the speech request conflicts with the host's management and control policy, and determines that the first client 1 corresponding to the speech request that does not conflict is the target first client. For example, if the host's management and control policy prohibits users with ordinary identities from going on the microphone, the user control platform 1 will target the first clients corresponding to all users with ordinary identities as the first clients; if the host's management and control policy prohibits users with identity weights lower than 90 from uploading speech data, if the identity weight of the user corresponding to the speech request is 80, a prohibition speech instruction is generated. If the identity weight of the user corresponding to the speech request is 95, the first client 1 corresponding to the speech request is the target first client.

[0063] The preset speaking rule can also be used when the number of current speakers is less than the preset number of speakers. The user control platform 1 obtains the number of current target first clients. If the number of current target first clients reaches the preset number of speakers, the user control platform 1 generates a prohibition of speaking instruction or a wait-for-speaking instruction for the subsequent speech requests received, thereby preventing too many target first clients from occupying bandwidth resources at the same time. For example, the preset number of speakers of the user control platform 1 is 4. When the user control platform 1 receives 10 speech requests, the number of current target first clients is already 4. In this case, the user control platform 1 needs to generate a prohibition of speaking instruction or a wait-for-speaking instruction for these 10 speech requests.

[0064] Of course, you can also set corresponding preset speech rules according to various actual situations, which are not listed here one by one. These preset speech rules can be used individually or in combination. When used in combination, you can set the priority of each preset speech rule as needed, for example, the host's management and control strategy takes precedence over other speech rules set by the user management platform 1.

[0065] Furthermore, the first client 1 can also send a speaking request to the user control platform 1 at any time during the meeting. The node of the speaking request can collect the user's audio data for the first client 1, or collect the instructions issued by the user, etc. At this time, the user control platform 1 needs to determine whether these first clients 1 can become the target first clients according to the above process, thereby realizing dynamic regulation of the conference system.

[0066] It can be seen that through the collaborative management of the user control platform 1 and the host, the number of target first clients can be effectively controlled, thereby controlling the bandwidth resources occupied by uploading speech data. At the same time, the identity of the speaking user can be effectively controlled to meet the identity requirements of the speaker.

[0067] S102. If the client is a first client, connect the client to a data transmission channel so that the client receives speech data sent by the media cloud server through the data transmission channel, and transmits the speech data to the media cloud server through the data transmission channel when the client meets the preset speech rules.

[0068] Specifically, Figure 4 A schematic diagram of a process of uploading speech data by a first client provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method includes:

[0069] S301: The first client collects user audio data through sound pickup technology;

[0070] S302: When the audio data exists, the first client collects the video data of the user;

[0071] S303: The first client transmits the speech data to the media cloud server through the data transmission channel, where the speech data includes the audio data and the video data.

[0072] Speech data typically includes audio data and video data. It is generally believed that audio data is more important in speech data, and only when audio data is generated will the first client be triggered to continue collecting video data. The first client uses audio pickup technology to collect the user's audio data. Only after the first client has collected audio data will the first client continue to collect the user's video data. This avoids the problem of directly collecting excessive amounts of video data for uploading without audio data, resulting in invalid conference data and a large waste of bandwidth resources. After the first client collects audio data and video data, it packages the audio data and video data as speech data and transmits them to the media cloud server 3 via the data transmission channel.

[0073] To ensure the quality of speech data, the uploaded speech data needs to be screened. That is, only clients that meet the preset speech rules can upload their speech data. The details are as follows:

[0074] The user control platform 1 receives the speech data sent by each first client 2 and detects whether the decibel value of the speech data meets the preset decibel threshold. Only when the decibel value of the speech data is greater than or equal to the preset decibel threshold, the first client 2 is allowed to continue uploading the speech data to the media cloud server 3 through the data transmission channel.

[0075] In order to further improve the quality of speech data, when the decibel value corresponding to the audio data is less than the preset decibel threshold, the decibel value corresponding to the audio data is increased by sound excitation technology so that the decibel value of the excited audio data is greater than or equal to the preset decibel threshold, and then the excited audio data is continued to be uploaded to the media cloud server 3 through the data transmission channel.

[0076] For example, if the first client 2 detects that the voice quality of a participating user is relatively poor, such as low volume, poor timbre, or poor spatial perception, it can use sound excitation technology to track the detected sound source and appropriately optimize the audio data to improve the sound quality. The optimized audio data can then be uploaded, effectively improving the quality of the audio data.

[0077] Furthermore, when the first client collects the user's audio data, it is necessary to remove noise from the user's audio data, for example, by setting a more refined decibel threshold, such as a first decibel value and a second decibel value. When the collected audio data is less than the first decibel value, it means that the audio data is not a human voice, but may be more subtle audio data such as flipping a book, which is noise and needs to be removed. When the audio data is greater than or equal to the first decibel value, the audio data may be a human voice. At this time, the collected audio data needs to be further processed. Since the first decibel value is less than the second decibel value, when the audio data is between the first decibel value and the second decibel value, it means that the volume is low. At this time, it is necessary to use sound excitation technology to increase the decibel value of the audio data. In actual applications, other noise removal methods can be applied and are not limited here.

[0078] S103: If the client is a second client, a streaming address corresponding to the conference is sent to the client, so that the client streams the conference data from a target CDN according to the streaming address to listen in on the conference.

[0079] After the media cloud server 3 receives the speech data sent by the first client, if the speech data only includes audio data, the media cloud server 3 can directly send the audio data as conference data to each first client 1 and the target CDN 4; if the speech data includes both audio data and video data, the media cloud server 3 needs to perform audio and video synchronization processing on the speech data, and then distribute it after obtaining the conference data. Figure 5 A flow chart of processing speech data provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the method includes:

[0080] S401: The media cloud server identifies the device identifier carried in each speech data;

[0081] S402: The media cloud server obtains target speech data, where the speech conference data includes audio data and video data having the same device identifier.

[0082] S403: The media cloud server performs synchronous synthesis processing on the audio data and video data in the target speech data to obtain conference data.

[0083] Each piece of speech data carries a corresponding device identifier, which effectively distinguishes conference data uploaded by different first clients and avoids confusion and interference between different speech data. Media cloud server 3 performs the above-mentioned synthesis processing on each piece of speech data received, thus obtaining conference data with synchronized audio and video, thereby ensuring the presentation effect of the conference data.

[0084] The media cloud server 3 sends the conference data to each first client 2 and the target CDN 4 .

[0085] Media Cloud Server 3 can stream conference data to the target CDN 4, an intelligent virtual network built on top of the conference's network. Relying on edge servers deployed across various locations, Target CDN 4 leverages the central platform's load balancing, content distribution, and scheduling functions to enable users to access presentation data locally. This effectively reduces network congestion, improving user access response speed and user experience. This is particularly useful in scenarios requiring large amounts of streaming media data from diverse locations and networks. Typically, Media Cloud Server 3 streams the conference data to be displayed to Target CDN 4 in real time via the RTMP protocol.

[0086] Regarding data transmission from second client 5, as mentioned above, second client 5 only has permission to access conference data, not upload speech data. However, due to considerations such as target CDN security, conference security, user security, conference attendee capacity, and conference quality, user control platform 1 may need to further screen second clients 5 to identify the target second client that can actually access conference data during the meeting.

[0087] Figure 6 A schematic diagram of a process for determining a target second client provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the method includes:

[0088] S501: The user control platform receives a listening request sent by the second client through the signaling channel;

[0089] S502: The user control platform determines whether the second client complies with the preset audit rule;

[0090] S503: If the second client meets the preset audit rule, determine the second client as a target second client and generate a conference data acquisition instruction, wherein the conference data acquisition instruction includes the address of the target CDN.

[0091] The user control platform 1 can control the quantity and quality of the target second client by pre-setting the audit rules. For example, the preset audit rule can be that the number of attendees is prohibited from exceeding the preset number of attendees, or that only users with preset user identities and preset identity weights are allowed to attend, or the number of attendees can be controlled according to the host management strategy, etc. The specific process can refer to the process of the user control platform 1 controlling the number of speakers, which will not be repeated here. If the user control platform 1 determines that the second client 5 meets the preset audit rules, the user control platform 1 will determine the second client 5 as the target second client and generate the corresponding conference data acquisition instruction. The user control platform 1 will send the conference data acquisition instruction to the corresponding target second client through the signaling channel so that the target second client can obtain the conference data.

[0092] Generally, the user control platform 1 can control multiple conferences at the same time. The conference corresponding to the conference data required by the target second client can be called the target conference. Different conferences correspond to different target CDNs. Therefore, in order for the target second client to accurately obtain the required conference data, and also to avoid the conference data from being leaked to the outside, the conference data acquisition instruction will contain the address of the target CDN. In this way, the target second client can accurately obtain the required conference data from the corresponding target CDN according to the address of the target CDN.

[0093] The embodiment of the present application can greatly increase the number of people who can obtain conference data through the target CDN4 without occupying the bandwidth resources of the network where the conference is located, thereby ensuring the transmission quality of the conference data in the network where the conference is located.

[0094] After the first client and the target second client obtain the conference data, they need to display it accordingly, such as Figure 7 As shown, the first client is used as an example to illustrate:

[0095] S601: The first client obtains a device identifier in conference data;

[0096] S602: The first client determines target conference data from each conference data according to the device identifier, where the target conference data is the conference data finally displayed on the first client;

[0097] S603. The first client sets the number of sub-interfaces in the display interface according to the number of copies of the target conference data, wherein the sub-interface is used to display each copy of the target conference data;

[0098] S604: The first client displays corresponding target conference data on each of the sub-interfaces.

[0099] The first client 2 will receive multiple copies of conference data. The user can choose to display all the conference data at the same time, or display part of the conference data according to his actual needs. For example, there are 4 copies of conference data in total. If the user chooses to display all the conference data, then there are 4 copies of the target conference data in total. Correspondingly, it is necessary to set the number of sub-interfaces in the display interface to 4, such as 4 grids, and display the corresponding target conference data in each sub-interface. If the user only chooses to display part of the conference data, for example, only display the two conference data of the leader and the guest, then the user identity corresponding to the conference data can be determined by identifying the device identifier carried in the conference data, where the conference data corresponding to the leader and the guest is the target conference data. It can be seen that the number of target conference data is 2, so the number of sub-interfaces in the display interface should also be 2. Of course, the target conference data can be set according to the needs of the user, and other situations will not be repeated here.

[0100] The conference data pushed by the media cloud server 3 to the target CDN 4 can be pre-configured display interface data, for example, the same as the host's display interface. In this case, the conference data obtained by the target second client from the target CDN 4 is directly usable display interface data, eliminating the need for the target second client to process and generate display interface data. The conference data pushed by the media cloud server 3 to the target CDN 4 can also be conference data corresponding to multiple target first clients, without corresponding display interface data. In this case, the target second client can generate display interface data by following the same process as the first client described above.

[0101] Furthermore, the conference system provided in this embodiment is a dynamic system, that is, the identities of the first client and the second client can be adjusted as needed, as follows:

[0102] Figure 8 A schematic diagram of a process of switching identities of a client provided in an embodiment of the present application is shown as follows: Figure 8 As shown, the method includes:

[0103] S701: Receive, through the signaling channel, an identity switching request sent by a client to be switched, where the client to be switched is a first client;

[0104] S702: Determine whether the client to be switched meets the preset identity switching conditions based on pre-stored user permissions;

[0105] S703: If the client to be switched meets the preset identity switching condition, generate an identity switching instruction, and send the identity switching instruction to the client to be switched through the signaling channel, so that the client to be switched switches to a target identity, where the target identity includes a second client and a leaving identity.

[0106] Figure 9 A schematic diagram of a process of switching identities of a client provided in an embodiment of the present application is shown as follows: Figure 9 As shown, the method includes:

[0107] S801: Receive, through the signaling channel, an identity switching request sent by a client to be switched, where the client to be switched is a second client;

[0108] S802: Determine whether the client to be switched meets the preset identity switching conditions based on pre-stored user permissions;

[0109] S803: If the client to be switched meets the preset identity switching condition, generate an identity switching instruction, and send the identity switching instruction to the client to be switched through the signaling channel, so that the client to be switched switches to a target identity, where the target identity includes the first client and the leaving identity.

[0110] The client to be switched includes: a first client 1 and a second client 5 that send an identity switching request to the user control platform 1 through a signaling channel. The user control platform 1 generates a corresponding instruction by determining whether the client to be switched meets the preset identity switching conditions. For example, if the preset identity switching conditions are met, the user control platform 1 generates an identity switching instruction and sends the identity switching instruction to the corresponding client to be switched through a signaling channel. The client to be switched can then switch to the corresponding target identity according to the identity switching instruction. The target identity can be the second client, the first client, the host, the leaving identity, etc. For example, the first client 2 switches to the second client or the leaving identity, the second client 5 switches to the first client or the leaving identity, or even the first client 2 and the second client 5 switch to the host identity.

[0111] In this way, the user control platform 1 can dynamically adjust the identities of the first client 2 and the second client 5 to meet the actual needs of the user. The preset identity switching conditions can be set according to actual conditions. The setting rules can refer to the preset speaking rules and the preset listening rules, which will not be repeated here.

[0112] In this embodiment of the present application, the first and second clients 2 and 5 can use the Message Queuing Telemetry Transport (MQTT) protocol for signaling transmission between the user control platform 1 and the user control platform 1. MQTT is a low-overhead, low-bandwidth instant messaging protocol that can meet the signaling needs of the Internet of Things, small devices, mobile applications, and other applications. Of course, WebSocket or SocketIO can also be used for signaling transmission between the first and second clients 2 and 5 and the user control platform 1.

[0113] The user control platform 1 also has an audio and video recording module that supports persisting conference data to the cloud service so that it can be replayed after the meeting. The user control platform 1 also has a cache module. The user control platform 1 uses a cache mechanism to cache hot data in the Redis Cluster, thereby improving the platform's response speed and supporting high concurrency. The user control platform 1 also has a DB database module, which can persist conference data in MySQL for easy query and statistics. The user control platform 1 can also have other functional modules to provide different functions for the conference system. It can be designed according to actual needs and will not be described here.

[0114] From the above technology, it can be seen that the present application provides a conference data transmission method and a conference system, in which the user control platform controls the number of first clients and second clients according to pre-stored user permissions, and further controls the network bandwidth occupied by transmitting speech data. The media cloud server sends conference data to each first client and the target CDN, wherein the first client can directly display the received conference data, and each second client can obtain the conference data from the target CDN, so that the second client can also listen to the conference data. At the same time, the way the second client obtains the conference data does not occupy the bandwidth resources of the communication network between the first client and the media cloud server. Therefore, more second clients can be supported to listen in on the conference, thereby achieving the effect of thousands of people participating in the conference.

[0115] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0116] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A conference data transmission method, characterized in that: Applied to a user control platform, the method comprises: receiving a conference access request sent by a client through a signaling channel, and determining a category of the client according to pre-stored user rights; If the client is a first client, a data transmission channel is connected to the client so that the client receives conference data sent by the media cloud server through the data transmission channel, and when the client meets the preset speech rule, the speech data is transmitted to the media cloud server through the data transmission channel, and the conference data is speech data processed by the media cloud server, wherein the speech data includes audio data and video data, the audio data is the audio data of the user collected by the client through the sound pickup technology, and the video data is the video data of the user collected by the client when the audio data exists; If the client is a second client, sending a streaming address corresponding to the conference to the client, so that the client streams the conference data from the target CDN according to the streaming address to listen to the conference; receiving, through the signaling channel, an identity switching request sent by a client to be switched, where the client to be switched is a second client; Determining whether the client to be switched meets the preset identity switching condition according to the pre-stored user authority; If the client to be switched meets the preset identity switching condition, an identity switching instruction is generated, and the identity switching instruction is sent to the client to be switched through the signaling channel, so that the client to be switched switches to a target identity, wherein the target identity includes a first client and an off-site identity; receiving a speaking request sent by the first client through the signaling channel; Determining whether the first client complies with the preset speaking rule; If the first client complies with the preset speaking rules, the first client is determined as the target first client, and a speech data upload instruction is generated, wherein the speech data upload instruction includes a device identifier corresponding to the target first client, and the target first client refers to the first client that actually uploads the speech data to the media cloud server in the meeting.

2. The method according to claim 1, characterized in that When the client satisfies the preset speaking rule, transmitting the speaking data to the media cloud server through the data transmission channel includes: Detecting whether the decibel value of the audio data in the speech data meets a preset decibel threshold; If the decibel value of the audio data is greater than or equal to the preset decibel threshold, the speech data is transmitted to the media cloud server through the data transmission channel.

3. The method according to claim 2, characterized in that When the client satisfies the preset speaking rule, transmitting the speaking data to the media cloud server through the data transmission channel further includes: If the decibel value of the audio data is less than the preset decibel threshold, the decibel value of the audio data is increased by using sound excitation technology so that the decibel value of the audio data after excitation is greater than or equal to the preset decibel threshold.

4. The method according to claim 1, characterized in that: The preset speech rules are that the user identity corresponding to the client conforms to the preset user identity, the user identity weight corresponding to the client conforms to the preset user identity weight, and the user identity corresponding to the client conforms to the host's management and control strategy for speaking users, or a combination of the two, wherein the host has user speech management authority.

5. The method according to claim 1, characterized in that The method further comprises: receiving, through the signaling channel, an identity switching request sent by a client to be switched, the client to be switched being a first client; Determining whether the client to be switched meets the preset identity switching condition according to the pre-stored user authority; If the client to be switched meets the preset identity switching condition, an identity switching instruction is generated and sent to the client to be switched through the signaling channel, so that the client to be switched switches to a target identity, wherein the target identity includes a second client and an off-site identity.

6. The method according to claim 1 or 5, characterized in that: The preset identity switching conditions include that the total number of clients corresponding to the client type after switching is less than or equal to a preset number threshold, the user identity corresponding to the client to be switched meets the preset user identity, the user identity weight corresponding to the client to be switched meets the preset user identity weight, or the user identity corresponding to the client to be switched meets the host's user identity management strategy, one or a combination of several of the above, wherein the host has user identity management authority.

7. The method according to claim 1, characterized in that The method further comprises: The user control platform and the first client and the second client use the MQTT protocol to transmit instructions through the signaling channel.

8. The method according to claim 1, characterized in that The conference data is pushed to the target CDN in real time via RTMP, so that the second client obtains the conference data from the target CDN.

9. A conference system, characterized in that: The conference system includes: a user control platform, multiple clients, a media cloud server and a target CDN; The user control platform is used to receive a conference access request sent by a client through a signaling channel, and determine the category of the client according to pre-stored user rights; If the client is a first client, the client is connected to a data transmission channel so that the client receives conference data sent by the cloud server through the data transmission channel, and when the client meets the preset speech rule, the speech data is transmitted to the cloud server through the data transmission channel, and the conference data is speech data processed by the media cloud server, wherein the speech data includes audio data and video data, the audio data is the audio data of the user collected by the client through the sound pickup technology, and the video data is the video data of the user collected by the client when the audio data exists; If the client is a second client, sending a streaming address corresponding to the conference to the client, so that the client streams the conference data from the target CDN according to the streaming address to listen to the conference; receiving, through the signaling channel, an identity switching request sent by a client to be switched, where the client to be switched is a second client; Determining whether the client to be switched meets the preset identity switching condition according to the pre-stored user authority; If the client to be switched meets the preset identity switching condition, an identity switching instruction is generated, and the identity switching instruction is sent to the client to be switched through the signaling channel, so that the client to be switched switches to a target identity, wherein the target identity includes a first client and an off-site identity; receiving a speaking request sent by the first client through the signaling channel; Determining whether the first client complies with the preset speaking rule; If the first client meets the preset speech rule, the first client is determined as a target first client, and a speech data upload instruction is generated, wherein the speech data upload instruction includes a device identifier corresponding to the target first client, and the target first client refers to the first client that actually uploads speech data to the media cloud server in the conference; The media cloud server is used to receive the speech data uploaded by the first client through the data transmission channel, and perform audio and video synchronization processing, and the processed conference data is sent to each first client through the data channel respectively, and pushed to the target CDN in real time through the RTMP protocol; The target CDN is used to receive the conference data pushed by the media cloud server, so that each second client can listen in by pulling the stream from the target CDN.

Images