Real-time online conference access method and device

Abstract

The invention provides a real-time online conference access method and device. The method comprises the following steps: establishing communication connection with an external terminal A, an RTC server and a signaling server; controlling the terminal A to establish data connection with the terminal B based on the incoming call event of the terminal A in combination with the target instruction of the signaling server; or the terminal A is controlled to establish data connection with the terminal B based on the calling-out instruction of the signaling server; and after the terminal A is connected with the terminal B, executing bidirectional transcoding of the audio data so as to access the terminal B to the online conference. When the method is applied, a user can directly join an RTC online conference which can only be accessed through an application end or a webpage end originally through a common terminal, a special line or hardware with high cost does not need to be deployed, the development and operation cost is low, and the method can adapt to various application environments.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of digital information transmission, and particularly relates to a real-time online conference access method and device. BACKGROUND

[0002] With the popularization of Internet technology, online conferences supported by real-time communication (RTC) technology have become the core scenario of office and collaboration, and their efficient and low-cost characteristics meet the multi-party communication needs under most network conditions.

[0003] Among them, the conference inviter can invite participants distributed in different places to participate in the conference through an instant messaging application with online conference function. The invited party can access the online conference after accepting the invitation through the terminal, thereby realizing the conference without being restricted by the conference space.

[0004] Currently, there are two typical pain points in the actual application of online conferences through instant messaging applications: first, the terminal equipment of some users cannot directly access online conferences due to hardware performance limitations; second, in extreme network environments (such as remote areas and weak network scenarios), the terminal network stability is insufficient, resulting in conference lag or interruption.

[0005] Meanwhile, in the mainstream scheme of existing terminal access to the Internet, the current VoIP (Voice over Internet Protocol) architecture is mainly based on SIP (Session Initiation Protocol), which relies on the combination of "SIP protocol stack + dedicated server + operator outbound service", which has high deployment cost and complex operation and maintenance. SUMMARY

[0006] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a real-time online conference access method and device to solve the problems of high deployment cost, limited member participation conditions, high network requirements and delay of traditional online conferences.

[0007] In a first aspect, the present application provides a real-time online conference access method, which comprises the following steps:

[0008] establishing a communication connection with a terminal A of an external device, an RTC server and a signaling server;

[0009] controlling the terminal A to establish a data connection with a terminal B based on an incoming call event of the terminal A and a target instruction of the signaling server; or

[0010] controlling the terminal A to establish a data connection with a terminal B based on an outgoing call instruction of the signaling server;

[0011] When the terminal A and the terminal B establish a connection, bidirectional transcoding of audio data is performed to access the terminal B into the online conference.

[0012] In some embodiments of the first aspect of the application, the terminal A establishes a data connection with the terminal B based on an incoming call event of the terminal A and a target instruction of the signaling server, specifically comprising:

[0013] obtaining the incoming call event sent by the terminal A;

[0014] reporting the incoming call event to the signaling server to obtain a target instruction issued by the signaling server, wherein the target instruction at least includes an access instruction or a termination instruction;

[0015] controlling the terminal A to act based on the target instruction, wherein if the target instruction is the access instruction, the terminal A establishes a data connection with the terminal B.

[0016] In some embodiments of the first aspect of the application, the terminal A establishes a data connection with the terminal B based on an outgoing call instruction of the signaling server, specifically comprising:

[0017] obtaining an outgoing call instruction issued by the signaling server;

[0018] sending a request instruction to the terminal A in combination with the outgoing call instruction;

[0019] controlling the terminal A to establish a data connection with the terminal B based on the request instruction.

[0020] In some embodiments of the first aspect of the application, when the terminal A and the terminal B establish a connection, bidirectional transcoding of audio data is performed to access the terminal B into the online conference, specifically comprising:

[0021] The bidirectional transcoding includes uplink transcoding and downlink transcoding, wherein,

[0022] In the uplink transcoding, the initial audio including the audio of the terminal A and / or the terminal B is decoded by the first protocol to obtain uplink audio, and the uplink audio is re-encoded according to the second protocol and sent to the RTC server to access the online conference.

[0023] In the downlink transcoding, the conference audio is decoded by the second protocol to obtain downlink audio, and the downlink audio is re-encoded according to the first protocol and sent to the terminal A, and the terminal A transmits the encoded downlink audio to the terminal B.

[0024] To achieve the above object and other related objects, the second aspect of the application provides a real-time online conference access device, comprising:

[0025] The communication module and the processing module,

[0026] The communication module is configured to establish a communication connection with a terminal A of an external device, an RTC server, and a signaling server.

[0027] The processing module is configured to perform data conversion, including:

[0028] performing data access of terminal signaling, wherein the data connection between the terminal A and a terminal B is established based on the terminal A and the signaling server, or the data connection between the terminal A and the terminal B is established based on the signaling server; and

[0029] performing bidirectional transcoding of audio data, wherein uplink transcoding is performed based on audio of the terminal A and / or the terminal B, and downlink transcoding is performed based on conference audio.

[0030] In some embodiments of the first aspect of the application, the communication module and the processing module are configured in a discrete manner or an integrated manner.

[0031] In some embodiments of the first aspect of the application, the communication module includes at least a Bluetooth module and a network module, wherein the Bluetooth module is configured to establish a Bluetooth connection with the terminal A, and the network module is configured to establish a communication connection with the RTC server and the signaling server.

[0032] In some embodiments of the first aspect of the application, the communication connection mode of the network module includes wireless communication and / or wired communication.

[0033] In some embodiments of the first aspect of the application, the device further includes an auxiliary module, which includes a power supply, an antenna, and a network interface, wherein the power supply is configured to supply power, the antenna is configured to enhance a Bluetooth transmission signal, and the network interface is configured to establish a network connection for the wired communication.

[0034] In some embodiments of the first aspect of the application, the data access of terminal signaling includes:

[0035] obtaining an incoming call event sent by the terminal A;

[0036] reporting the incoming call event to the signaling server to obtain a target instruction issued by the signaling server, wherein the target instruction includes at least an access instruction or a termination instruction;

[0037] controlling the terminal A to act based on the target instruction, wherein if the target instruction is the access instruction, the terminal A is controlled to establish a data connection with the terminal B; or

[0038] obtaining an outgoing call instruction issued by the signaling server;

[0039] send a request instruction to terminal A in combination with the outgoing instruction;

[0040] control terminal A to establish a data connection with terminal B based on the request instruction.

[0041] In some embodiments of the first aspect of the application, bidirectional transcoding of audio data is performed, specifically including:

[0042] Bidirectional transcoding includes uplink transcoding and downlink transcoding, wherein,

[0043] In uplink transcoding, the initial audio is decoded by the first protocol to obtain uplink audio, and the uplink audio is re-encoded according to the second protocol and then sent to the RTC server to access the online conference, wherein the initial audio includes the audio of terminal A and / or terminal B;

[0044] In downlink transcoding, the conference audio is decoded by the second protocol to obtain downlink audio, and the downlink audio is re-encoded according to the first protocol and then sent to terminal A, and the encoded downlink audio is transmitted to terminal B by terminal A.

[0045] To achieve the above object and other related objects, the third aspect of the application provides a real-time online conference access method, which comprises:

[0046] establishing a data connection with an external terminal, which includes:

[0047] reporting an incoming event to a signaling server to obtain a target instruction issued by the signaling server, wherein the target instruction at least includes an access instruction or a termination instruction; establishing a data connection with the external terminal based on the access instruction; or

[0048] obtaining an outgoing instruction issued by the signaling server, generating a request instruction in combination with the outgoing instruction, and establishing a data connection with the external terminal based on the request instruction;

[0049] When the communication connection with the external terminal is established, bidirectional transcoding of audio data is performed to access the online conference, wherein bidirectional transcoding includes uplink transcoding and downlink transcoding, in uplink transcoding, the uplink audio is encoded and sent to the RTC server to access the online conference, wherein the uplink audio includes the audio of the local terminal and / or the external terminal; in downlink transcoding, the downlink audio is obtained based on the conference audio of the online conference, and the downlink audio is transmitted to the external terminal.

[0050] As described above, the real-time online conference access method and device of the application have the following beneficial effects:

[0051] 1. Cost advantage, the application does not need to deploy SIP server, purchase operator outbound service, and the hardware cost of conversion device is reduced, and small organizations or individual users can deploy at low cost;

[0052] 2. Deployment advantage, plug and play, only three steps of "conversion device power supply→terminal Bluetooth pairing→network connection" are needed, and no professional operation and maintenance is needed;

[0053] 3. Delay advantage, the forwarding link of "SIP gateway→SIP proxy server" is reduced, audio only passes through two layers of transmission of "terminal A→conversion device→RTC server", and the end-to-end delay is ≤150 ms;

[0054] 4. Compatibility advantage, without modifying the hardware / software of the terminal, compatible with all terminal devices supporting Bluetooth or other communication connections, and wide adaptability;

[0055] 5. Flexibility advantage, supporting WIFI / Ethernet dual-mode network access, and suitable for different network environments such as office and outdoor. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A constituent schematic diagram of the real-time online conference access device of the application in an embodiment is shown;

[0057] Figure 2 A network topology schematic diagram of the real-time online conference access device of the application in an embodiment is shown;

[0058] Figure 3 A data conversion flow schematic diagram of the real-time online conference access device of the application in an embodiment is shown;

[0059] Figure 4 A flow schematic diagram of the real-time online conference access method of the application in an embodiment is shown;

[0060] Figure 5 A flow schematic diagram of the real-time online conference access method of the application in an embodiment is shown;

[0061] Figure 6 A flow schematic diagram of the real-time online conference access method of the application in an embodiment is shown;

[0062] Figure 7 A structure schematic diagram of the electronic device of the application in an embodiment is shown.

[0063] ELEMENT NUMBER EXPLANATION 100 Access device 101 Communication module 1011 Bluetooth module 1012 Network module 102 Processing module 103 Auxiliary module 1031 Power supply 1032 Antenna 1033 Network interface 202 Terminal A 204 RTC server 206 Signaling server 208 Terminal B 210 Cloud service 700 Electronic terminal 701 Processor 702 Memory 7021 Operating system 7022 Application program 703 Network interface 704 Bus system 705 User interface DETAILED DESCRIPTION

[0064] The advantages and effects of the present application can be easily understood by those skilled in the art from the above description. The present application can also be implemented or applied through other different embodiments, and various modifications or changes can be made to the details of the present application based on different views and applications without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0065] It should be noted that the diagrams provided in the following examples only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component can be arbitrarily changed in terms of shape, number and ratio, and the layout pattern of the components can be more complex.

[0066] In addition, the description such as "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection claimed in the present application.

[0067] At the same time, those skilled in the art should understand that the embodiments of the present application provided herein are only exemplary and not restrictive. Unless otherwise explicitly stated, all functions disclosed in the specification can be replaced by other functions with the same or similar purposes. Therefore, other embodiments or modifications not exhaustively listed, as long as they belong to the scope of the present application defined herein and their equivalents, should be considered as included in the present application. It should be particularly noted that the imperative and / or sequential language used herein (such as "will", "will not", "should", "should not", "must", "must not", "first", "initially", "then", "subsequently", "before", "after", "finally" and the like) is not used to limit the scope of the present application, because the embodiments listed herein are only exemplary in nature.

[0068] Before further detailing the present application, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations:

[0069] <1> RTC (Real-Time Communication, real-time communication);

[0070] <2> PSTN (Public Switched Telephone Network, Public Switched Telephone Network);

[0071] <3> HFP (Hands-Free Profile, Hands-Free Profile);

[0072] <4> PCM (Pulse Code Modulation, Pulse Code Modulation);

[0073] <5> GSM (Global System for Mobile Communications, Global System for Mobile Communications);

[0074] <6> ACC (Advanced Audio Coding, Advanced Audio Coding);

[0075] <7> RTC SDK (Real-Time Communication Software Development Kit, Real-Time Communication Software Development Kit);

[0076] <8> CVSD (Continuously Variable Slope Delta Modulation, Continuously Variable Slope Delta Modulation);

[0077] <9> mSBC (modified Sub-Band Coding, modified Sub-Band Coding).

[0078] The application provides a real-time online conference access method and device, solves the interconnection and intercommunication problem of RTC online conference and traditional PSTN network, and can enable users to join the online RTC conference originally entered only through an App or a Web end in a "signaling interaction request" manner through a terminal (for example, a tablet computer, an Internet of Things device, and a portable wearable device (a smart watch), a mobile phone, and a fixed phone) with signaling interaction function. The application is suitable for various application scenarios. One is that the hardware of the terminal participating in the conference does not support it, for example, the invited objects of an enterprise internal conference include front-line sales personnel, outsourced personnel, retired rehired personnel, or elderly customers, some users still use feature phones, or the application scenario that the company security policy cannot install a conference App on a private mobile phone. The other is that the network of the terminal does not support it in an extreme / weak network environment, for example, the application scenario that the participants are located in remote mountainous areas, offshore platforms, or are in a GSM (2G) signal coverage area. The technical solutions in the embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0079] As Figure 1 shown in the figure, in an embodiment of the application, the real-time online conference access device 100 of the application comprises:

[0080] a communication module 101, a processing module 102, wherein,

[0081] the communication module 101 is configured to establish a communication connection with the terminal A 202 of the external device, the RTC server 204 and the signaling server 206;

[0082] the processing module is configured to perform data conversion, including:

[0083] performing data access of terminal signaling, wherein the data connection between the terminal A 202 and the terminal B 208 is established based on the terminal A 202 and the signaling server 206, or the data connection between the terminal A 202 and the terminal B 208 is established based on the RTC server 204; and

[0084] performing bidirectional transcoding of audio data, wherein uplink transcoding is performed based on the audio of the terminal A 202 and / or the terminal B 208, and downlink transcoding is performed based on the conference audio.

[0085] It should be noted that in this embodiment, the terminal B 208 has a system signaling interaction function, such as a tablet computer, a smart watch, a mobile phone or a fixed phone, and the terminal A 202 not only has a signaling interaction function, but also has a function of communicating with the device, such as connecting the device through Bluetooth, specifically, Figure 1 and Figure 2 as shown in the figure, the communication module 101 is configured to establish a communication connection with the terminal A 202 of the external device, the RTC server 204 and the signaling server 206, in this embodiment, the communication module 101 at least includes a Bluetooth module 1011 and a network module 1012, the Bluetooth module 1011 is configured to establish a Bluetooth connection with the terminal A 202, accordingly, in actual application, the connection with the terminal A 202 can also be established through other ways, wherein the audio collection and playback of the terminal A 202 are realized through the Classic Bluetooth mode, without relying on the RTC SDK support of the terminal, which can be applied to the application scenario that the hardware of the terminal does not support online conference, at the same time, the network module 1012 is configured to establish a communication connection with the RTC server 204 and the signaling server 206, wherein the communication connection mode of the network module 1012 includes wireless communication and / or wired communication, further, in application, the communication module 101 and the processing module 102 include discrete setting or integrated setting, wherein if integrated setting is adopted, a dual-mode chip can be adopted, specifically a chip integrating Bluetooth and WIFI dual-mode functions.

[0086] Specifically, in the embodiment, the Bluetooth module 1011 works in the Classic mode, supports the HFP 1.5 and above protocol versions, acts as a hands-free component, can complete Bluetooth pairing with the terminal A 202 and establish a stable connection; the network module 1012 supports WIFI (802.11b / g / n) or Ethernet network, integrates a complete TCP / IP protocol stack, and realizes network communication with the RTC server 204 and the signaling server 206, wherein, when the communication module 101 and the processing module 102 are in an integrated setting, a lightweight chip (such as BK7258, ESP32-C3, etc.) supporting Bluetooth + network dual-mode driving is selected, and the chip has basic computing power (main frequency ≥ 100 MHz, RAM ≥ 32 KB) for running a protocol conversion program.

[0087] Further, in the embodiment, as shown in Figure 1 the device further includes an auxiliary module 103, the auxiliary module 103 includes a power supply 1031, an antenna 1032 and a network interface 1033, wherein the power supply 1031 is used for power supply, for example, a “5V” power supply, the antenna 1032 is used for enhancing the Bluetooth transmission signal, for example, a gain “≥2dBi”, and the network interface 1033 is used for establishing the network connection of the wired communication, for example, an RJ45 interface.

[0088] Specifically, in the embodiment, in application, the access device 100 is connected to the “5V” power supply, the terminal A 202 searches and pairs the current device (device name: “RTC-HFP-Adapter”), after successful pairing, the terminal A 202 displays “connected earphone”, and at the same time of network connection, the device can access the Internet through wireless communication (WIFI), and automatically establish a TCP connection with the RTC server 204 (a global node of Agora) and the signaling server 206, wherein the signaling server 206 can be deployed in the cloud service 210, so as to establish a long connection with the access device 100.

[0089] Further, in the embodiment, the processing module 102 is used for executing data conversion, as shown in Figure 3 a data conversion flowchart is shown, wherein the data connection between the terminal A 202 and the terminal B 208 is established based on the terminal A 202 and the signaling server 206, or the data connection between the terminal A 202 and the terminal B 208 is established based on the RTC server 204; and bidirectional transcoding of audio data is performed, wherein uplink transcoding is performed based on the audio of the terminal A 202 and / or the terminal B 208, and downlink transcoding is performed based on conference audio.

[0090] It should be noted that, in the embodiment of the application, the data access of the terminal signaling is executed, specifically including the following steps:

[0091] Acquiring an incoming event sent by the terminal A;

[0092] Reporting the incoming event to the signaling server to acquire a target instruction issued by the signaling server, wherein the target instruction at least includes an access instruction or a termination instruction;

[0093] Controlling the terminal A to act based on the target instruction, wherein if the target instruction is the access instruction, the terminal A is controlled to establish a data connection with the terminal B; or

[0094] Acquiring an outgoing instruction issued by the signaling server;

[0095] Sending a request instruction to the terminal A in combination with the outgoing instruction;

[0096] Controlling the terminal A to establish a data connection with the terminal B based on the request instruction.

[0097] Specifically, in the embodiment, the terminal signaling access includes two modes, one is to respond to the passive incoming event of the terminal A 202 to determine whether to access, and when the target instruction issued by the signaling server 206 is the access instruction, the terminal A 202 is controlled to establish a data connection with the terminal B 208; the other is to acquire the outgoing instruction issued by the signaling server 206 to control the terminal A 202 to actively call out, that is, to send a request instruction to the terminal A 202, so as to control the terminal A 202 to establish a data connection with the terminal B 208 based on the request instruction.

[0098] It should be noted that in the embodiment of the application, the bidirectional transcoding of the audio data is performed, specifically including the following steps:

[0099] The bidirectional transcoding includes uplink transcoding and downlink transcoding, wherein,

[0100] In the uplink transcoding, the uplink audio is obtained by decoding the initial audio through the first protocol, and the uplink audio is re-encoded according to the second protocol and then sent to the RTC server to access the online conference, wherein the initial audio includes the audio of the terminal A and / or the terminal B;

[0101] In the downlink transcoding, the downlink audio is obtained by decoding the conference audio through the second protocol, and the downlink audio is re-encoded according to the first protocol and then sent to the terminal A, and the encoded downlink audio is transmitted to the terminal B by using the terminal A.

[0102] Specifically, in the embodiment, the bi-directional transcoding of the audio data includes uplink transcoding and downlink transcoding, and the first protocol and the second protocol are used, and in the embodiment, the HFP protocol is taken as the first protocol and the OPUS protocol is taken as the second protocol, and in actual application, other protocols can also be included, for example, the second protocol can include the ACC protocol, wherein the HFP protocol is a profile for voice control in the Bluetooth protocol stack, and the OPUS protocol is an audio coding and decoding format suitable for real-time interactive voice and music transmission on the Internet.

[0103] Specifically, in the embodiment, in the uplink transcoding, the initial audio is decoded by the HFP protocol to obtain uplink audio, and the uplink audio is re-encoded according to the OPUS protocol and then sent to the RTC server 204 to access the online conference, wherein the initial audio includes the audio of the terminal A 202 and / or the terminal B 208, and specifically, the initial audio includes CVSD or mSBC encoded audio, so that the participants of the online conference can hear the sound of the user of the terminal A 202 and / or the terminal B 208.

[0104] Specifically, in the embodiment, in the downlink transcoding, the conference audio is decoded by the OPUS protocol to obtain downlink audio, and the downlink audio is re-encoded according to the HFP protocol and then sent to the terminal A 202, and the encoded downlink audio is transmitted to the terminal B 208 by the terminal A 202, so that the sound of the participants of the online conference can be heard by the user of the terminal A 202 and / or the terminal B 208.

[0105] As shown in the embodiment of the application, the real-time online conference access method of the application includes: Figure 4

[0106] establishing a communication connection with the terminal A, the RTC server and the signaling server of the peripheral device;

[0107] controlling the terminal A to establish a data connection with the terminal B based on the incoming call event of the terminal A and the target instruction of the signaling server; or

[0108] controlling the terminal A to establish a data connection with the terminal B based on the outgoing call instruction of the signaling server;

[0109] When the terminal A and the terminal B are connected, bi-directional transcoding of audio data is performed to access the terminal B to the online conference.

[0110] ​It should be noted that the real-time online conference access method in the embodiment is applied to the real-time online conference access device 100 described in any of the above embodiments, and thus the device is taken as an execution subject to access the terminal A 202 and the terminal B 208 into the same online conference (while the conference also has other conference participants). When the method is applied, first, a communication connection with the terminal A 202, the RTC server 204, and the signaling server 206 needs to be established, and then a data connection between the terminal A 202 and the terminal B 208 needs to be established, including controlling the terminal A 202 to establish the data connection with the terminal B 208 based on an incoming event of the terminal A 202 and a target instruction of the signaling server 206; or controlling the terminal A 202 to establish the data connection with the terminal B 208 based on an outgoing instruction of the RTC server 204. The specific access steps will be described in detail in the subsequent description.

[0111] Further, in the embodiment, when the terminal A 202 and the terminal B 208 are connected, the method further includes performing bidirectional transcoding of audio data to access the audio of the terminal A 202 and / or the terminal B 208 into the online conference, so as to realize interconnection and intercommunication between the RTC conference system and the traditional PSTN network. When the bidirectional transcoding of the audio data is performed, the uplink / downlink audio data is transcoded at least twice. The embodiment is applied to the real-time online conference access device 100 described in any of the above embodiments.

[0112] Therefore, in the embodiment, referring to the above embodiment, the initial audio is decoded by the HFP protocol to obtain uplink PCM audio, where the initial audio includes the audio of the terminal A 202 and / or the terminal B 208, and the uplink PCM audio is re-encoded (re-encoded according to the OPUS protocol) to obtain conference audio adapted to the RTC online conference; in the downlink, the real-time conference audio of the RTC online conference is decoded into downlink PCM audio, and the downlink PCM audio is re-encoded according to the HFP protocol and then sent to the terminal A 202, while the terminal A 202 is used to transmit the corresponding audio to the terminal B 208.

[0113] Further, in the embodiment, the data connection between the terminal A and the terminal B is controlled based on an incoming event of the terminal A and a target instruction of the signaling server, and specifically includes the following steps.

[0114] An incoming event sent by the terminal A is acquired;

[0115] The incoming event is reported to the signaling server to acquire a target instruction issued by the signaling server, where the target instruction at least includes an access instruction or a termination instruction;

[0116] Control terminal A to act based on the target instruction, wherein if the target instruction is an access instruction, control terminal A to establish a data connection with terminal B.

[0117] It should be noted that in this embodiment, the incoming event corresponds to terminal B 208 calling terminal A 202, at this time, the incoming event sent by terminal A 202 is obtained, for example, {"type": "incoming", "number": "139XXXX5678"}, and the incoming event is reported to the signaling server 206 to obtain the target instruction issued by the signaling server 206, wherein the target instruction at least includes an access instruction or a termination instruction, for example, the access instruction: {"type": "answer"}, and the termination instruction: {"type": "hangup"}, and terminal A 202 is controlled to act based on the target instruction, wherein if the target instruction is an access instruction, terminal A 202 is automatically accessed to establish a data connection with terminal B 208.

[0118] Further, in an embodiment of the application, terminal A is controlled to establish a data connection with terminal B based on the outgoing instruction of the signaling server, specifically including the following steps:

[0119] Obtain the outgoing instruction issued by the signaling server;

[0120] Send a request instruction to terminal A in combination with the outgoing instruction;

[0121] Control terminal A to establish a data connection with terminal B based on the request instruction.

[0122] It should be noted that in this embodiment, the scenario of controlling terminal A 202 to actively call terminal B 208 is specifically explained, specifically, the signaling server 206 issues a call instruction {"type": "call", "target": "138XXXX1234"}, the call instruction is parsed, a request instruction is sent to terminal A 202 through the HFP protocol, so as to control terminal A 202 to automatically dial "138XXXX1234", and after terminal B 208 is connected, the two parties establish PSTN communication.

[0123] The application also provides a real-time online conference access method, which performs bidirectional transcoding of audio data, specifically including the following steps:

[0124] Bidirectional transcoding includes uplink transcoding and downlink transcoding, wherein,

[0125] In uplink transcoding, the initial audio is decoded by the first protocol to obtain uplink audio, and the uplink audio is re-encoded according to the second protocol and then sent to the RTC server to access the online conference, wherein the initial audio includes the audio of the terminal A and / or the terminal B.

[0126] In downlink transcoding, the conference audio is decoded by the second protocol to obtain downlink audio, and the downlink audio is re-encoded according to the first protocol and then sent to the terminal A, and the encoded downlink audio is transmitted to the terminal B by the terminal A.

[0127] It should be noted that in the embodiment, the execution steps of the bidirectional transcoding of the audio data are specifically described, wherein the bidirectional transcoding of the audio data includes uplink transcoding and downlink transcoding, and the first protocol and the second protocol are used, in the embodiment, the HFP protocol is taken as the first protocol, and the OPUS protocol is taken as the second protocol, and in actual application, other protocols can also be included, for example, the second protocol can include the ACC protocol, wherein the HFP protocol is a configuration file for voice control in the Bluetooth protocol stack, and the OPUS protocol is an audio coding and decoding format suitable for real-time interactive voice and music transmission on the Internet.

[0128] Specifically, as shown in Figure 5 In the embodiment, the initial audio is decoded by the HFP protocol to obtain uplink PCM audio in uplink, wherein the initial audio includes the audio of the terminal A 202 and / or the terminal B 208, and the uplink PCM audio is re-encoded (re-encoded according to the OPUS protocol) to obtain conference audio suitable for the RTC online conference. Specifically, the initial audio includes CVSD or mSBC encoded audio, so that the participants of the online conference can hear the sound of the user of the terminal A 202 and / or the terminal B 208.

[0129] Specifically, in the embodiment, the real-time conference audio of the RTC online conference is decoded into downlink PCM audio in downlink, and the downlink PCM audio is re-encoded according to the HFP protocol and then sent to the terminal A 202, and the corresponding audio is transmitted to the terminal B 208 by the terminal A 202, so that the sound of the participants of the online conference can be heard by the user of the terminal A 202 and / or the terminal B 208, wherein the PCM audio parameter decoded by the HFP protocol is “8kHz sampling rate, 16bit bit width, single channel”, and the OPUS protocol encoding format is “16kHz sampling rate, 20ms frame length”.

[0130] As shown in Figure 6 The application embodiment further provides a real-time online conference access method, and the method comprises the following steps:

[0131] establishing a data connection with the peripheral terminal, which comprises:

[0132] reporting an incoming event to a signaling server to obtain a target instruction issued by the signaling server, wherein the target instruction at least comprises an access instruction or a termination instruction; establishing a data connection with the peripheral terminal based on the access instruction; or

[0133] obtaining an outgoing instruction issued by the signaling server, generating a request instruction in combination with the outgoing instruction, and establishing a data connection with the peripheral terminal based on the request instruction;

[0134] after the communication connection with the peripheral terminal is established, performing bidirectional transcoding of audio data to access the online conference by the peripheral terminal, wherein the bidirectional transcoding comprises uplink transcoding and downlink transcoding; in the uplink transcoding, uplink audio is encoded and sent to an RTC server to access the online conference, wherein the uplink audio comprises audio of the local terminal and / or the peripheral terminal; in the downlink transcoding, downlink audio is obtained based on conference audio of the online conference, and the downlink audio is transmitted to the peripheral terminal.

[0135] It should be noted that in the present embodiment, the real-time online conference access method can be applied to the real-time online conference access device 100 described in the above embodiments, and when the technology matures in the future, the device can be integrated into the local terminal to directly realize the interconnection and intercommunication function of the RTC online conference and the traditional PSTN network based on the local terminal.

[0136] Specifically, in the present embodiment, when the local terminal is taken as the execution subject, first, a data connection with the peripheral terminal needs to be established, an incoming event is reported to a signaling server 206 to obtain a target instruction issued by the signaling server 206, wherein the target instruction at least comprises an access instruction or a termination instruction; a data connection with the peripheral terminal is established based on the access instruction; or an outgoing instruction issued by the signaling server 206 is obtained, a request instruction is generated in combination with the outgoing instruction, and a data connection with the peripheral terminal is established based on the request instruction.

[0137] Further, in the present embodiment, after the communication connection with the peripheral terminal is established, bidirectional transcoding of audio data is performed to access the online conference by the peripheral terminal, wherein the bidirectional transcoding comprises uplink transcoding and downlink transcoding; in the uplink transcoding, uplink audio is encoded and sent to an RTC server 204 to access the online conference, wherein the uplink audio comprises audio of the local terminal and / or the peripheral terminal; in the downlink transcoding, downlink audio is obtained based on conference audio of the online conference, and the downlink audio is transmitted to the peripheral terminal.

[0138] The protection scope of the real-time online conference access method described in the embodiments of the present application is not limited to the step execution order listed in the embodiments, and any scheme realized by increasing, reducing or replacing steps of the prior art according to the principles of the present application is included in the protection scope of the present application.

[0139] In several embodiments provided by the present application, it should be understood that the disclosed apparatus or method can be implemented in other manners. For example, the described embodiments of the apparatus are merely schematic, and the division of modules / units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, and electrical, mechanical or other forms.

[0140] The modules / units described as separated components can or can not be physically separated, and the components displayed as modules / units can or can not be physical modules, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules / units can be selected according to actual needs to achieve the purposes of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application can be integrated in a processing module 102, or can be physically separated modules / units, or two or more modules / units can be integrated in one module / unit.

[0141] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] The embodiments of the present application also provide an electronic device, as shown in the accompanying drawings. Figure 7 The electronic device includes a processor and a memory.

[0143] As shown in the accompanying drawings, the electronic device includes a processor and a memory. Figure 7As shown, the electronic device includes at least one processor 701, a memory 702, at least one network interface 703 and a user interface 705. The various components in the apparatus are coupled together by a bus system 704. It will be appreciated that the bus system 704 is used for communicating data between the components. The bus system 704 includes a data bus, a power bus, a control bus, and a state signal bus. However, for the sake of clarity, only the data bus is shown in Figure 7 the various buses are referred to as the bus system.

[0144] The user interface 705 can include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad or a touch screen, etc.

[0145] It will be appreciated that the memory 702 can be volatile or non-volatile memory, or both. The non-volatile memory can be read only memory (ROM), programmable read only memory (PROM), which is used as an external cache. By way of example, but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable category of memory.

[0146] The memory 702 in the embodiments of the present application is used to store various categories of data to support the operation of the electronic terminal 700. Examples of these data include: any executable program for operating on the electronic terminal 700, such as an operating system 7021 and an application program 7022; the operating system 7021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 7022 can contain various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The method provided by the embodiments of the present application can be included in the application program 7022.

[0147] The method disclosed by the embodiments of the present application can be applied to the processor 701 or implemented by the processor 701. The processor 701 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 701. The processor 701 described above can be a general processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 701 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The processor 701 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiments of the present application, the steps can be directly embodied as hardware decoding processor execution, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory, and the processor reads the information in the memory to complete the steps of the above method in combination with the hardware thereof.

[0148] In the exemplary embodiments, the electronic terminal 700 can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), or the like for executing the above method.

[0149] According to the method provided by the embodiments of the present application, the present application further provides a computer program product, which comprises computer program code, and when the computer program code runs on a computer, the computer executes the method of any one of the above real-time online conference access method embodiments.

[0150] According to the method provided by the embodiments of the present application, the present application further provides a computer readable storage medium, which stores program code, and when the program code runs on a computer, the computer executes the method of any one of the above real-time online conference access method embodiments.

[0151] Those of skill in the art would understand that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. The choice of hardware or software, or combinations of both, would be dependent on the specific application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0152] Those of skill in the art would understand that, for the purposes of description and enabling the claimed application, the specific process described above for the system, apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0153] In several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, for example, the division of units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0154] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0155] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit.

[0156] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0157] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0158] In summary, the present application provides a real-time online conference access method and device, which realizes a low-cost, easy-to-deploy, high-performance RTC and PSTN fusion scheme through lightweight design. The hardware cost is low, and there is no need to purchase outbound services or deploy dedicated servers. When applied, three-step operation can complete plug-and-play, without the need for professional operation and maintenance. Through optimization of the audio transmission path, the end-to-end delay is controlled within 150 ms, ensuring real-time audio. It can be compatible with all Bluetooth-enabled terminals without the need for hardware and software modification. It also supports Wi-Fi and Ethernet dual-mode access, which can be flexibly adapted to different scenarios such as offices and outdoor environments, greatly reducing the docking threshold of real-time communication systems.

[0159] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed by the present application should be covered by the claims of the present application.

Claims

1. A method for accessing a real-time online meeting, the method comprising: The method comprises the following steps: establishing a communication connection with a terminal A of an external device, an RTC server and a signaling server; controlling the terminal A to establish a data connection with a terminal B based on an incoming event of the terminal A in combination with a target instruction of the signaling server; or controlling the terminal A to establish a data connection with the terminal B based on an outgoing instruction of the signaling server; after the terminal A and the terminal B establish a connection, performing bidirectional transcoding of audio data to access the terminal B into an online conference.

2. The real-time online conference access method of claim 1, wherein, The method of controlling the terminal A to establish a data connection with the terminal B based on an incoming event of the terminal A in combination with a target instruction of the signaling server comprises: obtaining an incoming event sent by the terminal A; reporting the incoming event to the signaling server to obtain a target instruction issued by the signaling server, wherein the target instruction at least comprises an access instruction or a termination instruction; controlling the terminal A to act based on the target instruction, wherein if the target instruction is the access instruction, the terminal A is controlled to establish a data connection with the terminal B.

3. The real-time online conference access method of claim 1, wherein, The method of controlling the terminal A to establish a data connection with the terminal B based on an outgoing instruction of the signaling server comprises: obtaining an outgoing instruction issued by the signaling server; sending a request instruction to the terminal A in combination with the outgoing instruction; controlling the terminal A to establish a data connection with the terminal B based on the request instruction.

4. The real-time online conference access method of claim 1, wherein, After the terminal A and the terminal B establish a connection, performing bidirectional transcoding of audio data to access the terminal B into an online conference comprises: The bidirectional transcoding comprises uplink transcoding and downlink transcoding, wherein, in the uplink transcoding, an uplink audio is obtained by decoding an initial audio through a first protocol, and the uplink audio is re-encoded according to a second protocol and then sent to the RTC server to access the online conference, wherein the initial audio comprises audio of the terminal A and / or the terminal B; in the downlink transcoding, a downlink audio is obtained by decoding conference audio through a second protocol, and the downlink audio is re-encoded according to a first protocol and then sent to the terminal A, and the terminal A is used to transmit the encoded downlink audio to the terminal B.

5. A real-time online meeting access device, characterized in that, comprises: a communication module and a processing module, wherein the communication module is configured to establish a communication connection with a terminal A of an external device, an RTC server and a signaling server; the processing module is configured to perform data conversion, comprising: performing data access of terminal signaling, wherein the terminal A and the terminal B establish a data connection based on the terminal A and the signaling server, or the terminal A and the terminal B establish a data connection based on the signaling server; and performing bidirectional transcoding of audio data, wherein uplink transcoding is performed based on audio of the terminal A and / or the terminal B, and downlink transcoding is performed based on conference audio.

6. The real-time online conference access device according to claim 5, characterized in that, The communication module and the processing module comprise a discrete setting or an integrated setting.

7. The real-time online conference access device according to claim 5, characterized in that, The communication module at least comprises a Bluetooth module and a network module, wherein the Bluetooth module is configured to establish a Bluetooth connection with the terminal A, and the network module is configured to establish a communication connection with the RTC server and the signaling server.

8. The real-time online conference access device according to claim 7, characterized in that, The communication connection mode of the network module comprises wireless communication and / or wired communication.

9. The real-time online conference access device according to claim 8, characterized in that, The device also includes an auxiliary module, which includes a power supply, an antenna and a network interface, wherein the power supply is used for power supply, the antenna is used for enhancing Bluetooth transmission signal, and the network interface is used for establishing network connection of the wired communication.

10. The real-time online conference access device according to claim 5, characterized in that, The data access of terminal signaling is executed, specifically including: An incoming call event sent by the terminal A is acquired; The incoming call event is reported to the signaling server to acquire a target instruction issued by the signaling server, wherein the target instruction at least includes an access instruction or a termination instruction; The terminal A is controlled to act based on the target instruction, wherein if the target instruction is the access instruction, the terminal A is controlled to establish data connection with the terminal B; or An outgoing call instruction issued by the signaling server is acquired; A request instruction is sent to the terminal A in combination with the outgoing call instruction; The terminal A is controlled to establish data connection with the terminal B based on the request instruction.

11. The real-time online conference access device according to claim 5, characterized in that, The bidirectional transcoding of audio data is executed, specifically including: The bidirectional transcoding includes uplink transcoding and downlink transcoding, wherein In the uplink transcoding, the initial audio including the audio of the terminal A and / or the terminal B is decoded to obtain uplink audio, and the uplink audio is re-encoded according to the second protocol and then sent to the RTC server to access the online conference; In the downlink transcoding, the conference audio is decoded according to the second protocol to obtain downlink audio, and the downlink audio is re-encoded according to the first protocol and then sent to the terminal A, and the encoded downlink audio is transmitted to the terminal B by the terminal A.

12. A method for accessing a real-time online meeting, the method comprising: The method includes the following steps: Data connection with the peripheral terminal is established, including: The incoming call event is reported to the signaling server to acquire a target instruction issued by the signaling server, wherein the target instruction at least includes an access instruction or a termination instruction; data connection with the peripheral terminal is established based on the access instruction; or An outgoing call instruction issued by the signaling server is acquired, a request instruction is generated in combination with the outgoing call instruction, and data connection with the peripheral terminal is established based on the request instruction; After the communication connection with the peripheral terminal is established, bidirectional transcoding of audio data is executed to access the online conference by the peripheral terminal, wherein the bidirectional transcoding includes uplink transcoding and downlink transcoding, in the uplink transcoding, the uplink audio including the audio of the local terminal and / or the peripheral terminal is encoded and then sent to the RTC server to access the online conference; in the downlink transcoding, the downlink audio is obtained based on the conference audio of the online conference, and the downlink audio is transmitted to the peripheral terminal.

Images