Cross-protocol real-time audio communication method and system, medium and server
Through the status of the transit server simulation end user, obtain the audio data on the type II streaming media protocol server and convert it into the data of the ICCC media protocol, realize the real -time audio communication of cross -protocol, solve the audio data compatibility in the existing technology And performance issues have achieved efficient and real -time audio communication.
Patent Information
- Application Number
- CN202411442899.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-01-21
AI Technical Summary
The prior art is difficult to achieve seamless compatibility of real-time audio data between private protocols and standard streaming protocols, resulting in reduced performance of audio terminals and private streaming servers and limited expansion.
The relay server simulates the identity of the end user, obtains the audio data on the Class II streaming media protocol server, and converts it into the audio data of the Class I streaming media protocol and sends it to the Class I streaming media protocol server to achieve cross-protocol real-time audio communication. .
This method imperceptibly improves the performance of audio terminals and Class II streaming media servers, ensures real-time transmission, is easy to expand, and solves performance problems and limited expansion issues.
Smart Images
Figure CN119342040A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a cross-protocol real-time audio communication method, system, medium and server. Background Art
[0002] Real-time audio transmission is widely used in various applications. However, due to the existence of private protocols, the transmission of real-time audio data is restricted and difficult to be compatible with standard streaming protocol servers (standard streaming protocol servers such as RTMP, RTSP, etc.). Even if the audio data can be forcibly converted into a standard streaming protocol, this will cause performance problems (such as high latency) to the audio terminal or private streaming server.
[0003] The currently known approximate solution is that the audio terminal encapsulates the audio data into the format of the standard streaming media protocol when uploading the audio data and sends it to the server of the standard streaming media protocol, or encapsulates the audio data from each terminal into the audio of the standard streaming media protocol on the private streaming media protocol server, and then sends it to the standard streaming media protocol server. Both of these methods can realize the communication between the private audio data and the standard streaming media protocol server, but both of these solutions have certain performance problems. Originally, if the private protocol audio does not need to communicate with the standard streaming media protocol server, the audio terminal or the private streaming media protocol server only needs to complete the encapsulation and decapsulation of the private audio protocol, but in order to be compatible with the standard streaming media protocol, the original audio data has to be encapsulated into the standard streaming media protocol and sent to the standard streaming media protocol server, which increases the burden for the terminal or the private streaming media protocol server, and the expansion is limited. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a cross-protocol real-time audio communication method, system, medium and server, which simulates the terminal user through a transit server, obtains audio data on a Class II streaming protocol server, and then converts the Class II audio data into audio data of a Class I streaming protocol and sends it to the Class I streaming protocol server. This method is imperceptible to the audio terminal and the Class II streaming server, and will not cause any performance problems for the Class II streaming server and the user terminal, thereby ensuring the real-time transmission and easy expansion.
[0005] In order to achieve the above purpose, the embodiments of the present application are implemented in the following manner:
[0006] In a first aspect, an embodiment of the present application provides a cross-protocol real-time audio communication method, wherein the real-time audio communication system includes a Class I streaming protocol server, a Class II streaming protocol server, and a transit server cluster, wherein the Class I streaming protocol is different from the Class II streaming protocol, and the method is applied to any transit server in the transit server cluster, including: configuring the identities of several virtual terminal users for the transit server, each virtual terminal user having a unique identifier; joining a voice room to be configured through the identity of the virtual terminal user, wherein each voice room includes a virtual seat, and the virtual seat is used to receive the virtual terminal user, and the virtual seat and the virtual terminal user are invisible to all regular terminal users in the voice room; receiving Class II protocol audio data of each user in the voice room pushed by the Class II streaming protocol server in real time; performing Class II streaming protocol decapsulation and decoding on the Class II encapsulated audio data to obtain original audio data, and then performing Class I streaming protocol encoding and encapsulation on the original audio data to obtain Class I protocol audio data, and sending the Class I protocol audio data to the Class I streaming protocol server.
[0007] In combination with the first aspect, in a first possible implementation of the first aspect, the identities of several virtual terminal users are configured for the transit server, including: installing virtualization software, creating several virtual machines, and installing corresponding virtual operating systems in the virtual machines; configuring the identity of the virtual terminal user for each virtual machine and assigning a unique identifier.
[0008] In combination with the first aspect, in a second possible implementation method of the first aspect, joining the voice room to be configured through the identity of a virtual terminal user includes: a Class II streaming media protocol server receives a voice room creation request from any regular terminal user, creates a voice room, sets a virtual seat for the voice room as the voice room to be configured, and sends the voice room information to a relay server cluster, wherein the voice room information includes a voice room number; the relay server in the relay server cluster joins the voice room to be configured corresponding to the voice room number through the identity of the virtual terminal user, and occupies the virtual seat in the voice room.
[0009] In combination with the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, a relay server cluster is deployed in a centralized manner, including a central server and several relay servers, and the relay servers in the relay server cluster join the voice room to be configured corresponding to the voice room number through the identity of a virtual terminal user, including: after the central server receives the voice room information including the voice room number, it obtains the status and available virtual terminal users of each relay server; based on the status and available virtual terminal users of each relay server, it determines the target relay server and sends the voice room information to the target relay server; the target relay server joins the voice room to be configured corresponding to the voice room number through the identity of its available virtual terminal user.
[0010] In combination with the second possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the relay server cluster is deployed in a distributed manner, including several relay servers, and the relay servers in the relay server cluster are joined to the voice room to be configured corresponding to the voice room number through the identity of a virtual terminal user, including: after any relay server receives the voice room information including the voice room number, it verifies and parses the voice room information; determines the target relay server through load balancing, and forwards the voice room information to the target relay server; the target relay server joins the voice room to be configured corresponding to the voice room number through its available virtual terminal user identity.
[0011] In combination with the first aspect, in a fifth possible implementation method of the first aspect, Class II protocol audio data of each user in the voice room pushed by a Class II streaming protocol server is received in real time, including: each regular terminal user in the voice room uploads Class II protocol audio data encoded and encapsulated by the Class II streaming protocol through its user terminal; the Class II streaming protocol server pushes the Class II protocol audio data to the IP and port where the user terminal of the virtual terminal user is located through audio; and the transit server receives the Class II protocol audio data pushed by the Class II streaming protocol server in real time.
[0012] In combination with the first aspect, in a sixth possible implementation method of the first aspect, Class II encapsulated audio data is decapsulated and decoded according to a Class II streaming protocol to obtain original audio data, and then the original audio data is encoded and encapsulated according to a Class I streaming protocol to obtain Class I protocol audio data, including: decapsulating Class II encapsulated audio data according to a Class II streaming protocol; decoding the decapsulated audio stream according to a Class II streaming protocol encoding format to obtain original PCM audio data; encoding the original PCM audio data according to a Class I streaming protocol encoding format; and encapsulating the encoded audio data according to a Class I streaming protocol to obtain Class I protocol audio data.
[0013] In a second aspect, an embodiment of the present application provides a cross-protocol real-time audio communication system, the real-time audio communication system comprising a Class I streaming protocol server, a Class II streaming protocol server and a relay server cluster, each relay server in the relay server cluster being used to run the cross-protocol real-time audio communication method described in the first aspect.
[0014] In a third aspect, an embodiment of the present application provides a storage medium, which is arranged in a server and includes a stored program, wherein when the program is running, the server where the storage medium is located is controlled to execute the cross-protocol real-time audio communication method described in the first aspect or any one of the possible implementation methods of the first aspect.
[0015] In a fourth aspect, an embodiment of the present application provides a server comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the steps of the cross-protocol real-time audio communication method described in the first aspect or any one of the possible implementation methods of the first aspect are implemented.
[0016] Beneficial effects:
[0017] 1. This solution configures the identities of several virtual terminal users for the relay server, and joins the voice room to be configured through the identities of the virtual terminal users. Each voice room contains virtual seats, which are used to receive virtual terminal users. The virtual seats and virtual terminal users are invisible to all regular terminal users in the voice room. In this way, the relay server simulates the terminal user and exists in the form of a virtual terminal user on the Class II streaming media protocol (such as a private streaming media protocol) server, so that it can share the same business logic with normal terminal users. The relay server can receive the Class II protocol audio data of each user in the voice room pushed by the Class II streaming media protocol server in real time, thereby decapsulating and decoding the Class II encapsulated audio data by the Class II streaming media protocol to obtain the original audio data, and then encode and encapsulate the original audio data by the Class I streaming media protocol (such as a standard streaming media protocol, or another private streaming media protocol different from the Class II streaming media protocol) to obtain the Class I protocol audio data, and send the Class I protocol audio data to the Class I streaming media protocol server. As a result, other users can access the audio sound that was originally only heard by users who comply with the Class II audio protocol through the Class I streaming media protocol. In this way, the presence of the relay server on the Class II streaming media protocol server is equivalent to having one more terminal user (virtual terminal user) in use. This has almost no performance loss for the Class II streaming media protocol server. At the same time, the user terminal of the regular terminal user does not need to process the encapsulation work of the Class I streaming media protocol, and there is no performance loss for the user terminal. The virtual terminal user is invisible to the regular terminal user and is imperceptible to the user. In addition, multiple relay servers can be deployed, which greatly improves the forwarding performance of Class II protocol audio data to the Class I streaming media protocol server. At the same time, it is less invasive to the original Class II streaming media protocol server, ensuring the real-time performance of cross-protocol audio communication. Moreover, as the scale expands, the relay server cluster can be easily expanded, and the scalability is very strong. The essence of this solution is to separate the encoding and encapsulation work that should have been done by the terminal or Class II streaming media protocol server in the form of a relay server, which is not only flexible but also greatly improves the performance of the entire system.
[0018] 2. The transit server cluster can be deployed in a centralized manner or in a distributed manner. The more appropriate deployment method can be determined based on the user scale. When the user scale is small, the transit server cluster can adopt a centralized deployment solution, which is simpler, more efficient, and easier to maintain; when the user scale is large, the transit server cluster can adopt a distributed deployment solution, which is more suitable for situations where the user scale is large and the distribution is wide, and can well disperse the processing pressure to each node (transit server), which can be more efficient.
[0019] 3. Since the relay server is added to the Class II streaming media protocol server by simulating a user terminal and exists on the Class II streaming media protocol server as a virtual terminal user, it can share the same business logic with regular terminal users without any adjustment to the user terminals of regular terminal users. The virtual terminal users comply with the Class II streaming media protocol, so the Class II streaming media protocol server can push the Class II protocol audio data (the Class II protocol audio data encoded and encapsulated by the user terminal of the regular terminal user through the Class II streaming media protocol) to the IP and port where the user terminal of the virtual terminal user (i.e., the relay server) is located through audio streaming, thereby realizing real-time transmission of audio. The relay server uses its performance (compared to the user terminal and the Class II streaming media protocol server that needs to process many tasks) to effectively improve the conversion efficiency, without any additional burden on the user terminal and the Class II streaming media protocol server, thereby ensuring the real-time nature of audio communication.
[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0022] Figure 1 A framework diagram of a cross-protocol real-time audio communication system provided in an embodiment of the present application.
[0023] Figure 2 The present invention is a flowchart of a method for real-time audio communication across protocols.
[0024] Figure 3 A flowchart of the conversion of audio data by a relay server.
[0025] Figure 4 The overall framework diagram of an example cross-protocol real-time audio communication system. DETAILED DESCRIPTION
[0026] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0027] See also Figure 1 , Figure 1 A framework diagram of a cross-protocol real-time audio communication system provided in an embodiment of the present application.
[0028] In the present embodiment, the real-time audio communication system includes a class I streaming media protocol server, a class II streaming media protocol server and a relay server cluster, the class I streaming media protocol is different from the class II streaming media protocol (e.g., the class I streaming media protocol refers to a standard streaming media protocol, and the class II streaming media protocol refers to a private streaming media protocol; of course, class II can also be a standard streaming media protocol, then the corresponding class I streaming media protocol must be regarded as a private streaming media protocol different from the standard streaming media protocol; in addition, the class I streaming media protocol can also refer to a private streaming media protocol, and the class II streaming media protocol refers to another private streaming media protocol), and the relay server cluster includes several relay servers (one or more relay servers). Of course, the real-time audio communication system can also be expanded, for example, it also includes a class C streaming media protocol server, a class D streaming media protocol server, etc.
[0029] For the sake of ease of description, this embodiment takes a classic application scenario as an example, uses Class I streaming media protocol to refer to a standard streaming media protocol, uses Class II streaming media protocol to refer to a private streaming media protocol, and uses a transit server as the execution entity to run a cross-protocol real-time audio communication method to introduce this solution. The reference relationship between Class I streaming media protocol and Class II streaming media protocol will not be repeated in the following text.
[0030] See also Figure 2 , Figure 2 The present invention is a flowchart of a method for real-time audio communication across protocols.
[0031] First, step S10 needs to be executed.
[0032] Step S10: configuring the identities of several virtual terminal users for the transit server, each virtual terminal user having a unique identifier.
[0033] In this embodiment, the identities of several virtual terminal users may be configured for each transit server in the transit server cluster, and each virtual terminal user has a unique identifier.
[0034] Exemplarily, for each transit server, virtualization software can be installed (virtualization software is the underlying infrastructure, providing the environment and tools required to create, manage and run virtual machines; for example, VMware Workstation, VirtualBox, Hyper-V, etc. are all common virtualization software), and then several virtual machines are created (a virtual machine is a simulated computer environment that can run on a physical computer and can be regarded as an independent computer; when creating a virtual machine, some configurations need to be specified, such as the number of CPUs allocated to the virtual machine, memory size, hard disk space, and network interface, etc.), and the corresponding virtual operating system is installed in the virtual machine, and the operating environment of the virtual machine is configured. After that, the identity of the virtual terminal user is configured for each virtual machine, and a unique identifier is assigned. In this way, the configuration of the virtual terminal users of the transit server can be realized.
[0035] After configuring the virtual terminal user for each transit server, step S20 may be executed.
[0036] Step S20: Join the voice room to be configured through the identity of the virtual terminal user, wherein each voice room contains a virtual seat, the virtual seat is used to receive the virtual terminal user, and the virtual seat and the virtual terminal user are invisible to all regular terminal users in the voice room.
[0037] Any transit server can join the voice room to be configured through the identity of a virtual terminal user.
[0038] Exemplarily, a Class II streaming media protocol (private streaming media protocol) server can receive a voice room creation request from any regular terminal user, create a voice room, set a virtual seat for the voice room as a voice room to be configured, and send the voice room information to the transit server cluster, wherein the voice room information includes the voice room number. Each voice room includes a virtual seat, which is used to receive a virtual terminal user, and the virtual seat and the virtual terminal user are invisible to all regular terminal users in the voice room.
[0039] The transit servers in the transit server cluster can join the voice room to be configured corresponding to the voice room number through the identity of the virtual terminal user, and occupy the virtual seat in the voice room.
[0040] Specifically, if the relay server cluster is deployed in a centralized manner, then the relay server cluster includes a central server and several relay servers. Then, after the central server receives the voice room information including the voice room number, it can obtain the status of each relay server and the available virtual terminal users, and then determine the target relay server based on the status of each relay server and the available virtual terminal users (the determination method can be a solution that considers load balancing, or other solutions, such as a solution with the lowest estimated network delay), and send the voice room information to the target relay server. The target relay server can then join the voice room to be configured corresponding to the voice room number through the identity of its available virtual terminal users.
[0041] If the relay server cluster is deployed in a distributed manner, the relay server cluster includes several relay servers. After receiving the voice room information including the voice room number, any relay server verifies and parses the voice room information. Through load balancing (it can be an independent load balancer or a load balancing function can be deployed in each relay server. This embodiment takes an independent load balancer as an example), the target relay server is determined and the voice room information is forwarded to the target relay server. The target relay server can then join the voice room to be configured corresponding to the voice room number through the identity of its available virtual terminal user.
[0042] After the target transit server joins the voice room through the identity of the virtual terminal user, step S30 may be executed.
[0043] Step S30: receiving in real time the Class II protocol audio data of each user in the voice room pushed by the Class II streaming media protocol server.
[0044] In this embodiment, the words spoken by each regular terminal user in the voice room can be collected through the microphone of its user terminal, encoded and encapsulated by the Class II streaming media protocol (private streaming media protocol), and then the Class II protocol audio data (i.e., private protocol audio data) encoded and encapsulated by the Class II streaming media protocol is uploaded.
[0045] The Class II streaming media protocol server (private streaming media protocol server) can push each user's Class II protocol audio data (private protocol audio data) to the IP and port of the user terminal of the virtual terminal user (i.e., the transit server) through audio. Then, the transit server can receive the Class II protocol audio data (private protocol audio data) pushed by the Class II streaming media protocol server (private streaming media protocol server) in real time.
[0046] After obtaining the Class II protocol audio data, the relay server may execute step S40.
[0047] Step S40: Decapsulate and decode the Class II encapsulated audio data according to the Class II streaming protocol to obtain the original audio data, then encode and encapsulate the original audio data according to the Class I streaming protocol to obtain Class I protocol audio data, and send the Class I protocol audio data to the Class I streaming protocol server.
[0048] In this embodiment, the transit server can decapsulate and decode Class II encapsulated audio data (private protocol audio data) using Class II streaming media protocol (private streaming media protocol) to obtain original audio data, and then encode and encapsulate the original audio data using Class I streaming media protocol (standard streaming media protocol) to obtain Class I protocol audio data (standard protocol audio data), and send the Class I protocol audio data (standard protocol audio data) to a Class I streaming media protocol server (standard streaming media protocol server).
[0049] For example, see Figure 3 , Figure 3 A flowchart of the conversion of audio data by a relay server.
[0050] The process of converting audio data by the relay server (for completeness, the receiving and sending parts are also shown in the figure) includes private (Class II) protocol audio data - data cache - private (Class II) protocol decapsulation - private (Class II) encoding format decoding - standard (Class I) encoding format encoding - standard (Class I) protocol encapsulation - standard (Class I) protocol audio data sending and other processes.
[0051] Specifically, the relay server receives the Class II encapsulated audio data, can cache the Class II encapsulated audio data, then decapsulate the Class II encapsulated audio data according to the Class II streaming media protocol, and decode the decapsulated audio stream according to the Class II streaming media protocol encoding format to obtain the original PCM audio data; then encode the original PCM audio data according to the Class I streaming media protocol encoding format, and then encapsulate the encoded audio data according to the Class I streaming media protocol to obtain Class I protocol audio data, and then send the Class I protocol audio data to the Class I streaming media protocol server. In this way, the relay server realizes the cross-protocol conversion and real-time communication of audio data.
[0052] Generally speaking, this solution simulates the terminal user through the relay server, obtains the audio data of each user from the private streaming protocol server as a virtual user, and then implements the decapsulation and decoding of the private protocol on the relay server to obtain the original audio data, and then encodes the original audio in the standard encoding format and encapsulates the standard streaming protocol, and finally sends it to the standard streaming protocol server to realize the communication between the private protocol audio and the standard streaming protocol server.
[0053] Here is an example:
[0054] See also Figure 4 , Figure 4 Terminals A, B, and C in the example are private streaming protocol users (corresponding to different voice rooms in social software). The audio uploaded by users in the chat room when speaking complies with the private streaming protocol. The transit server exists in the form of a virtual terminal user on the private streaming protocol server, so that it can share the same business logic with normal terminal users. These virtual terminal users are identified by special user IDs. The private streaming protocol server pushes the private protocol audio to the IP and port where the virtual terminal user is located, so that the corresponding transit server can receive the audio push stream from the private streaming protocol server. After receiving the private protocol audio stream, the transit server decapsulates and decodes the audio stream to obtain the original PCM audio stream, and then encodes the PCM audio stream according to the standard encoding format and the standard streaming protocol, and encapsulates it, and sends it to a standard streaming protocol server, such as SRS (this is an open source real-time streaming server that supports multiple standard streaming protocols, such as RTMP / RTSP / HLS, etc.). In this way, other users can access the audio sounds that were originally only heard by users who comply with the private audio protocol through the standard streaming protocol.
[0055] The presence of a relay server on a private streaming media server is equivalent to having one more terminal user. This causes almost no performance loss to the private streaming media protocol server. At the same time, the terminal no longer needs to handle the encapsulation of the standard streaming media protocol. In addition, multiple relay servers can be deployed in a distributed manner, which greatly improves the forwarding performance of private protocol audio data to the standard streaming media protocol server, while having little invasiveness to the original private streaming media protocol server.
[0056] This embodiment provides a storage medium, which is set in a server and includes a stored program. When the program is running, the server where the storage medium is located is controlled to execute the cross-protocol real-time audio communication method of this embodiment.
[0057] In addition, this embodiment further provides a server, including a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the steps of the cross-protocol real-time audio communication method of this embodiment are implemented.
[0058] In summary, the embodiments of the present application provide a cross-protocol real-time audio communication method, system, medium and server. By configuring the identities of several virtual terminal users for the transit server, the virtual terminal users are added to the voice room to be configured through the identities of the virtual terminal users. Each voice room contains virtual seats, which are used to receive virtual terminal users. The virtual seats and virtual terminal users are invisible to all regular terminal users in the voice room. In this way, the transit server simulates the terminal user and exists in the form of a virtual terminal user on a Class II streaming media protocol (such as a private streaming media protocol) server. In this way, the same business logic can be shared with normal terminal users. The transit server can receive the Class II protocol audio data of each user in the voice room pushed by the Class II streaming media protocol server in real time, thereby decapsulating and decoding the Class II encapsulated audio data using the Class II streaming media protocol to obtain the original audio data, and then encoding and encapsulating the original audio data using the Class I streaming media protocol (such as a standard streaming media protocol, or another private streaming media protocol different from the Class II streaming media protocol) to obtain Class I protocol audio data, and sending the Class I protocol audio data to the Class I streaming media protocol server. Thus, other users can access the audio sound that only users who comply with the Class II audio protocol can hear through the Class I streaming media protocol. In this way, the presence of the transit server on the Class II streaming media protocol server is equivalent to having one more terminal user (virtual terminal user) for use, which has almost no performance loss for the Class II streaming media protocol server. At the same time, the user terminal of the conventional terminal user does not need to process the encapsulation work of the Class I streaming media protocol, and there is no performance loss for the user terminal. The virtual terminal user is invisible to the conventional terminal user and is imperceptible to the user. Moreover, multiple transit servers can be deployed, which greatly improves the forwarding performance of the Class II protocol audio data to the Class I streaming media protocol server, and at the same time, the intrusion to the original Class II streaming media protocol server is very small, ensuring the real-time performance of cross-protocol audio communication. Moreover, as the scale expands, the transit server cluster can be easily expanded, and the scalability is very strong. The essence of this solution is to separate the encoding and encapsulation work that should have been done by the terminal or the Class II streaming media protocol server in the form of a transit server, which is not only flexible and expandable, but also greatly improves the performance of the entire system.
[0059] The transit server cluster can be deployed in a centralized manner or in a distributed manner. The more appropriate deployment method can be determined based on the user scale. When the user scale is small, the transit server cluster can adopt a centralized deployment solution, which is simpler, more efficient and easier to maintain. When the user scale is large, the transit server cluster can adopt a distributed deployment solution, which is more suitable for situations where the user scale is large and the distribution is wide, and can well disperse the processing pressure to each node (transit server), which can be more efficient.
[0060] Since the relay server is added to the Class II streaming media protocol server by simulating a user terminal and exists on the Class II streaming media protocol server as a virtual terminal user, it can share the same business logic with regular terminal users without making any adjustments to the user terminals of regular terminal users. The virtual terminal users comply with the Class II streaming media protocol, so the Class II streaming media protocol server can push the Class II protocol audio data (Class II protocol audio data encoded and encapsulated by the user terminal of the regular terminal user through the Class II streaming media protocol) to the IP and port where the user terminal of the virtual terminal user (i.e., the relay server) is located through audio streaming, thereby realizing real-time transmission of audio. The relay server uses its performance (compared to the user terminal and the Class II streaming media protocol server that needs to process many tasks) to effectively improve the conversion efficiency, without any additional burden on the user terminal and the Class II streaming media protocol server, thereby ensuring the real-time nature of audio communication.
[0061] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0062] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A cross-protocol real-time audio communication method, characterized in that: The real-time audio communication system includes a class I streaming media protocol server, a class II streaming media protocol server and a relay server cluster, the class I streaming media protocol is different from the class II streaming media protocol, and the method is applied to any relay server in the relay server cluster, including: Configure the identities of several virtual terminal users for the transit server, each virtual terminal user having a unique identifier; Join the voice room to be configured as a virtual terminal user, where each voice room contains a virtual seat, which is used to receive the virtual terminal user. The virtual seat and the virtual terminal user are invisible to all regular terminal users in the voice room; Receive in real time the Class II protocol audio data of each user in the voice room pushed by the Class II streaming protocol server; The Class II encapsulated audio data is decapsulated and decoded according to the Class II streaming media protocol to obtain the original audio data, and then the original audio data is encoded and encapsulated according to the Class I streaming media protocol to obtain Class I protocol audio data, and the Class I protocol audio data is sent to the Class I streaming media protocol server.
2. The cross-protocol real-time audio communication method according to claim 1, characterized in that: Configure several virtual terminal user identities for the transit server, including: Install virtualization software, create several virtual machines, and install corresponding virtual operating systems in the virtual machines; Configure the identity of the virtual terminal user for each virtual machine and assign a unique identifier.
3. The cross-protocol real-time audio communication method according to claim 1, characterized in that: Join the voice room to be configured as a virtual terminal user, including: The class II streaming media protocol server receives a voice room creation request from any regular terminal user, creates a voice room, sets a virtual seat for the voice room as the voice room to be configured, and sends the voice room information to the transit server cluster, wherein the voice room information includes the voice room number; The transit server in the transit server cluster joins the voice room to be configured corresponding to the voice room number through the identity of the virtual terminal user, and occupies the virtual seat in the voice room.
4. The cross-protocol real-time audio communication method according to claim 3, characterized in that: The relay server cluster is deployed in a centralized manner, including a central server and several relay servers. The relay servers in the relay server cluster join the voice room to be configured corresponding to the voice room number through the identity of the virtual terminal user, including: After receiving the voice room information including the voice room number, the central server obtains the status of each transit server and the available virtual terminal users; Based on the status of each relay server and the available virtual terminal users, determine the target relay server and send the voice room information to the target relay server; The target transit server joins the voice room to be configured corresponding to the voice room number through the identity of its available virtual terminal user.
5. The cross-protocol real-time audio communication method according to claim 3, characterized in that: The relay server cluster is deployed in a distributed manner and includes several relay servers. The relay servers in the relay server cluster are added to the voice room to be configured corresponding to the voice room number through the identity of the virtual terminal user, including: After any relay server receives the voice room information including the voice room number, it verifies and parses the voice room information; Determine the target relay server through load balancing and forward the voice room information to the target relay server; The target transit server joins the voice room to be configured corresponding to the voice room number through the identity of its available virtual terminal user.
6. The cross-protocol real-time audio communication method according to claim 1, characterized in that: Receives the Class II protocol audio data of each user in the voice room pushed by the Class II streaming protocol server in real time, including: Each regular terminal user in the voice room uploads the Class II protocol audio data encoded and encapsulated by the Class II streaming media protocol through its user terminal; The Class II streaming media protocol server pushes the Class II protocol audio data to the IP and port where the user terminal of the virtual terminal user is located through audio streaming; The relay server receives the Class II protocol audio data pushed by the Class II streaming media protocol server in real time.
7. The cross-protocol real-time audio communication method according to claim 1, characterized in that: Decapsulating and decoding the Class II encapsulated audio data using the Class II streaming media protocol to obtain the original audio data, and then encoding and encapsulating the original audio data using the Class I streaming media protocol to obtain Class I protocol audio data, including: Decapsulate the Class II streaming media protocol for Class II encapsulated audio data; Decode the decapsulated audio stream in the type II streaming media protocol encoding format to obtain the original PCM audio data; Encode the original PCM audio data in the Class I streaming media protocol encoding format; The encoded audio data is encapsulated according to the Class I streaming media protocol to obtain Class I protocol audio data.
8. A cross-protocol real-time audio communication system, characterized in that: The real-time audio communication system includes a Class I streaming media protocol server, a Class II streaming media protocol server and a relay server cluster, wherein each relay server in the relay server cluster is used to run the cross-protocol real-time audio communication method described in claim 1.
9. A storage medium, characterized in that: The storage medium is arranged in a server, and includes a stored program, wherein when the program is running, the server where the storage medium is located is controlled to execute the cross-protocol real-time audio communication method described in any one of claims 1 to 7.
10. A server comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, characterized in that: When the program instructions are loaded and executed by the processor, the steps of the cross-protocol real-time audio communication method described in any one of claims 1 to 7 are implemented.