A conference communication transmission method integrating audio and video

Abstract

The present invention provides a conference communication transmission method integrating audio and video, and the steps include: a server establishes a connection with multiple terminals, and configures the audio and video encoding format of each terminal, including a unified encoding format and an adaptive transcoding encoding format; the server selects the adaptive transcoding encoding format of the terminal to be transcoded and performs a transcoding; then the audio stream is screened and mixed, and at least one audio and video stream is selected for mixing based on a first weighted algorithm; finally, the mixed audio and video stream is pushed to at least part of the subscribed terminals based on a second dynamic weighted algorithm. The present invention reduces the number of audio transcoding times by transcoding the audio separately and combining the application of two types of encoding formats to reduce the amount of calculation and port occupancy and investment; it can also reduce the occupation of network bandwidth by the code stream, and at the same time be compatible with telephones and voice recognition services that require a single stream for docking; further, the stream can be pulled as needed, thereby reducing resource waste, improving the success rate of the session, and providing a high-quality session experience.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and in particular to a conference communication transmission method integrating audio and video. Background Art

[0002] Converged communication is a communication mode that organically integrates multiple communication methods and technologies. It breaks the boundaries between traditional communication methods and realizes the interconnection and collaboration between different communication systems. In today's digital age, converged communication plays a vital role and has many significant advantages, such as:

[0003] 1. Efficient collaborative work: Converged communications allow users to use multiple communication methods such as voice, video, and data on a unified platform. For example, in an enterprise office scenario, employees can use the converged communications platform to conduct real-time voice calls and video conferences, while sharing documents, data and other information, greatly improving the efficiency of teamwork. Different departments can communicate quickly and solve problems in a timely manner without switching between multiple communication tools, saving time and energy and making the workflow smoother.

[0004] 2. Rapid response and decision-making: In the fields of emergency command, public safety, etc., converged communications can achieve real-time coordination and command between the front and rear. When faced with a need, the command center can quickly summon relevant personnel through the converged communication system, obtain on-site video and audio information in real time, accurately grasp the situation, make decisions and issue instructions in a timely manner. This rapid response capability helps to improve the efficiency of responding to incidents, reduce losses, and ensure public safety.

[0005] 3. Wide compatibility and flexibility: Converged communications supports access from multiple devices, including desktop computers, laptops, smartphones, tablets, etc. Users can choose the right device for communication based on their needs and scenarios, without being restricted by time and space. Whether in the office, on the way or at home, they can stay in close contact with the team or the outside world and achieve a seamless communication experience.

[0006] At present, for emergency, disaster, rescue and other scenarios, when an incident occurs, a professional converged communication command and dispatch platform is generally needed to achieve front-end and rear-end coordination and command, collaborative coordination of business deployment, and rapid initiation of audio and video conferences to achieve the sharing and communication of event information by all staff, and the dispatch of converged communication video streams. It is different from the common video conferencing technology solution, which must ensure efficient and stable operation under special circumstances; therefore, a separate server and platform must be set up for converged communication, which can not only ensure normal line communication and access to special equipment, but also be compatible with conventional communication equipment.

[0007] At present, the audio and video transmission solutions on the market for such scenarios are still based on traditional technology using MCU architecture servers for the same transcoding communication, which has the following disadvantages:

[0008] (1) Re-decoding, encoding, and mixing operations require a lot of computing, which consumes a lot of server CPU resources. This not only increases hardware costs, but may also affect the smoothness of communication. For example, in large-scale video conferences, when processing multiple video streams at the same time, the MCU may experience problems such as freezing and delay due to resource constraints, seriously affecting the quality of the conference.

[0009] (2) Re-decoding, encoding, and mixing will also cause significant delays. In application scenarios with high real-time requirements, such as remote command and rescue, delays may cause information to be transmitted untimely, affecting work efficiency.

[0010] (3) The capacity provided by MCU is limited, and generally a dozen or so video channels are the upper limit. In the application scenarios of some large organizations, it may be necessary to connect more terminal devices simultaneously for audio and video communications, but traditional MCU solutions are difficult to meet such needs. This limits the application scope and scalability of the converged communication system, and it cannot adapt to the growing business needs.

[0011] In summary, there is a need to improve existing technologies to obtain a more efficient, stable and cost-effective converged communication solution to meet the growing and diversified communication needs. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a converged audio and video conference communication transmission method which is low in cost, has high efficiency and low latency, and supports multiple communication channels.

[0013] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0014] The present invention provides a conference communication transmission method integrating audio and video, which comprises the following steps:

[0015] The server establishes connections with multiple terminals, and the server configures the audio coding format of each terminal based on the communication negotiation content;

[0016] The audio encoding format of each terminal includes a unified encoding format and an adaptive transcoding encoding format;

[0017] The server lists the audio coding formats supported by each terminal in the plurality of terminals based on the communication negotiation content;

[0018] When there is a terminal that needs to be transcoded that does not support the unified encoding format, the server selects the fastest encoding format for transferring the unified encoding format from the audio encoding format supported by the terminal that needs to be transcoded, and uses it as the adapted transcoding encoding format for the terminal that needs to be transcoded;

[0019] The server receives the terminal audio streams and the terminal video streams sent by the multiple terminals, and transcodes the terminal audio streams adapted to the transcoding encoding format to obtain an audio stream in a unified encoding format;

[0020] The server screens the audio streams in a unified coding format, and selects at least multiple audio streams for confluence to obtain a confluence audio stream;

[0021] The server selects at least one video stream based on a first dynamic weighting algorithm, and mixes the video stream with the confluent audio stream to obtain an audio and video stream; the weight items of the first dynamic weighting algorithm at least include a video user weight and a volume weight of a corresponding terminal audio stream;

[0022] The server pushes the audio and video stream to at least some of the subscribed terminals based on a second dynamic weighted algorithm; the weight items of the second dynamic weighted algorithm at least include audio-visual user weights.

[0023] Optionally, the server is based on SFU architecture.

[0024] Optionally, the server and the multiple terminals are connected via a public network or a private network;

[0025] The data transmission between the server and the multiple terminals is based on the web audio and video real-time communication protocol.

[0026] Optionally, the step of configuring the audio coding format of each terminal by the server based on the communication negotiation content includes:

[0027] The web audio and video real-time communication protocol is configured with a session description protocol, and the text of the session description protocol includes the audio coding format of each terminal;

[0028] The server collects the text of the session description protocol, and obtains the audio coding format of each terminal.

[0029] Optionally, the unified encoding format is a digital audio encoding format of Opus or G.711.

[0030] Optionally, the method for selecting the adaptive transcoding encoding format includes the following steps:

[0031] The server lists the audio encoding formats supported by the terminal requiring transcoding, and determines the fastest encoding format based on an encoding format feature analysis method or a transcoding test method, and uses the fastest encoding format as the adapted transcoding encoding format for the terminal requiring transcoding.

[0032] Optionally, the transcoding test method includes:

[0033] The server collects multiple groups of test audios sent by the terminal that needs to be transcoded, and the formats of the multiple groups of test audios include audio coding formats correspondingly supported by the terminal that needs to be transcoded;

[0034] The server transcodes the multiple groups of test audio into audio with a unified encoding format under the same test environment, and records the transcoding time of each group, and selects the encoding format when sending the group of test audio with the least time as the fastest encoding format.

[0035] Optionally, the server screening the audio stream in the unified encoding format comprises the following steps:

[0036] The server sorts the audio streams in the unified coding format based on the volume of the audio streams in the unified coding format, and selects the first several audio streams with large volume.

[0037] Optionally, the server further performs a preliminary screening before screening the audio stream in the unified coding format, and the preliminary screening includes the following steps:

[0038] The server preliminarily screens the audio stream in the unified coding format based on the audio spectrum characteristics of human voice, and removes non-human voice audio streams.

[0039] Optionally, the server collects characteristic information of each terminal user and assigns an initial weight based on the characteristic information;

[0040] The initial weights also include the video user weight, volume weight, and audio-visual user weight;

[0041] The video user weight is used to adjust the mixing priority of the terminal video stream;

[0042] The audio-visual user weight is used to adjust the priority of the terminal receiving the stream.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention reduces the number of audio transcoding times and the difficulty of mixing streams by transcoding the audio separately and combining a unified encoding format with an adaptive transcoding encoding format, thereby reducing the amount of calculation and port occupancy, thereby reducing the number of paths that need to be transcoded and the investment in the required equipment, and improving the success rate of hole punching; it can also reduce the occupation of network bandwidth by the code stream, and at the same time be compatible with telephones and voice recognition services that require a single stream; further, it can pull the stream according to user needs, thereby reducing unnecessary hardware investment and resource waste in the dispatching and command system or engineering construction, improving the session success rate, and providing a high-quality session experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A connection logic diagram of the system involved in the method of the present invention;

[0047] Figure 2 This is a flow chart of the push-stream and receive-stream process of the present invention.

[0048] In the figure: 1. Media service, 2. Voice recognition service, 3. PC desktop, 4. Special equipment terminal, 5. Smart phone, 6. Phone, 7. Visual terminal. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0051] It is worth noting that the methods used in the present invention are all conventional methods unless otherwise specified; the raw materials and devices used are all conventional commercially available products, and their sources are not specifically limited unless otherwise specified.

[0052] like Figure 1As shown, this embodiment provides a conference communication transmission method integrating audio and video, including the following steps:

[0053] First, the server used in this embodiment is a server based on the SFU architecture, and establishes connections with multiple terminals through the media service 1. In a specific embodiment, the terminal types may include a PC desktop 3, a dedicated device terminal 4, a smart phone 5 and a corresponding mobile phone APP, a speech recognition service 2 for big data processing, a telephone 6, and a visualization terminal 7. Further, the connection between the server and the multiple terminals can be established through a public network or a private network, and the data transmission between the server and the multiple terminals in this embodiment is based on the web audio and video real-time communication protocol (WebRTC, the abbreviation of Web Real-Time Communication), which supports web browsers for real-time voice calls or video chats.

[0054] Afterwards, the server configures the audio coding format of each terminal based on the communication negotiation content; specifically, the step of the server configuring the audio coding format of each terminal based on the communication negotiation content includes:

[0055] When establishing a connection, the server configures a session description protocol in the web audio and video real-time communication protocol, and the text of the session description protocol includes the audio coding format of each terminal;

[0056] Thus, the server can obtain the audio coding format supported by each terminal by collecting the text of the session description protocol;

[0057] In this step, the server divides the audio encoding formats of each terminal into two categories based on preset information, one is a unified encoding format, and the other is an adaptive transcoding encoding format.

[0058] Optionally, in this embodiment, the unified encoding format is the Opus digital audio encoding format, which is a lossy audio compression digital audio encoding format suitable for low-latency instant sound transmission on the network. The standard format is defined in the RFC6716 file.

[0059] Based on the communication negotiation content, the server lists the audio encoding formats supported by each terminal in the multiple terminals. Among them, when there is a terminal that needs to be transcoded that does not support the unified encoding format, such as the aforementioned telephone and visual terminal. Then the server selects the fastest encoding format from the audio encoding format supported by the terminal that needs to be transcoded to the unified encoding format, and uses it as the adaptive transcoding encoding format of the terminal that needs to be transcoded; specifically, the selection method of the adaptive transcoding encoding format includes the following steps:

[0060] The server lists the audio encoding formats of the two terminals according to the text of the session description protocol. If the phone does not support multiple formats, the only audio format it supports is used as the adaptive transcoding format. As for the video terminal, it supports multiple audio encoding formats, including PCM (pulse code modulation, a common uncompressed audio encoding format), G.711 (voice compression standard established by the International Telecommunication Union, including A-law and μ-law algorithms), AAC (Advanced Audio Coding, with high compression efficiency and good sound quality), MP3 (a widely used lossy audio compression format), WMA (an audio encoding format launched by Microsoft, supporting digital copyright management); for example, the video phone using the SV-1001V IP video intercom module can support PCM, G.711, and MP3 audio encoding formats, and the IP network video intercom terminal XC-9133AV also supports the above formats. Therefore, the server needs to determine the fastest encoding format in the audio encoding format support list of such terminals based on the encoding format feature analysis method or transcoding test method, and use it as the adaptive transcoding format for the terminal that needs transcoding.

[0061] Among them, the encoding format feature analysis method is based on expert analysis systems or technical documents to compare the compression methods and data structures of various encoding formats with the Opus digital audio encoding format, so as to infer the fastest encoding format for converting to the Opus digital audio encoding format based on the principle of similarity. The statistical results are made into a queryable list for the server to quickly select, but this method also has disadvantages. On the one hand, the inference may not be reliable in different transcoding environments and data files; on the other hand, different terminal types may have encoding formats outside the query list, which may cause server system selection problems.

[0062] Therefore, this embodiment specifically adopts the transcoding test method, and the steps include: in the initial stage of establishing the connection, the server requests each terminal to perform a very short audio test, and the server collects multiple groups of test audio sent by all the terminals that need to be transcoded, and the format of the multiple groups of test audio includes the audio encoding format supported by the corresponding terminal that needs to be transcoded. As mentioned above, the video terminal equipped with the SV-1001V IP video intercom module sends three groups of short test audios respectively, and the corresponding formats are PCM, G.711, and MP3.

[0063] The server transcodes multiple test audios into audios of a unified encoding format (Opus digital audio encoding format) in the same test environment (same system resource configuration, such as processor occupancy), and records the transcoding time of each group. The encoding format of the test audio that takes the least time to send is selected as the fastest encoding format. If it is found after the test that PCM to Opus is the fastest under the same conditions, PCM is used as the adaptive transcoding format of the visual terminal. Based on the configuration strategy, the audio stream pushed by this terminal to the server during the connection period is in PCM format.

[0064] Combined with this Figure 2 As shown, during the connection period, the server of this embodiment will receive the terminal audio stream and terminal video stream sent by multiple terminals. Among them, based on the previous configuration information, the media service supports the terminal audio stream of the terminal pushing stream in the unified coding format in the unified coding format, which does not need transcoding (no) and therefore directly enters the next link; while the terminal that needs transcoding pushes the terminal audio stream that is adapted to the transcoding format, which needs transcoding (yes), and the server performs a transcoding (converting audio) on the terminal audio stream that is adapted to the transcoding format to obtain an audio stream in the unified coding format, thereby all the terminal audio streams on the server side are in the Opus digital audio coding format, and the server only needs to transcode part of the audio stream once, which reduces the occupation of system resources, improves transcoding efficiency and reduces latency.

[0065] Optionally, since some terminals may be affected by external noise or abnormal sounds from terminal users during the conference, resulting in excessive audio signals and affecting the conference, this embodiment also designs a pre-screening step, that is, the server performs a preliminary screening of the multi-channel audio streams in a unified encoding format before merging, and the preliminary screening includes the following steps:

[0066] The server preliminarily screens the audio streams in a unified encoding format based on the characteristics of the human voice audio spectrum, and removes non-human voice audio streams. Specifically, the spectrum of human voice has specific characteristics, and compared with other sounds, the frequency range of human voice is mainly concentrated in a certain range (usually hundreds of hertz to thousands of hertz), and the spectrum will show some harmonic structures related to the vibration of the vocal organs (such as vocal cords). By analyzing the spectrum of the audio signal, such as calculating the peak value of the spectrum, harmonic distribution and other characteristics, it can be determined whether there is a human voice signal. For example, the Mel Frequency Cepstral Coefficient (MFCC) commonly used in speech recognition systems is an effective representation method based on spectral features, which can better describe the characteristics of human voice and is used for subsequent recognition and classification tasks. This is highly efficient for non-human voice removal, uses less system resources, and can significantly improve the quality of conference communications.

[0067] The server screens the audio streams in a unified coding format, and selects at least multiple audio streams to merge, to obtain a merged audio stream. The server screens the audio streams in a unified coding format, including the following steps:

[0068] The server sorts the audio streams in the unified encoding format based on the volume of the audio streams in the unified encoding format, and selects the first several audio streams with large volume.

[0069] The main purpose of this step is that according to common understanding, a louder volume represents the importance and urgency of the content, and some words that are not wanted to be reflected in the meeting, such as discussions with others, repetitions when taking notes, irrational opinions, etc., are all at a lower volume. Therefore, based on actual scenario considerations, three audio streams can be selected, which generally include the meeting host, the current speaker, and the inserted speech. The number of merged streams should not be too large. When the number of merged streams is too large, it will seriously affect the audio quality after merging.

[0070] Since it is a conference that integrates audio and video, a corresponding video and audio mixing strategy is required. Therefore, the server in this embodiment selects at least one video stream based on the first dynamic weighted algorithm (the scenario in which multiple video streams are applied is to display multi-terminal videos in a single window partition), and mixes the video stream with the combined audio stream to obtain the audio and video stream.

[0071] The server will collect characteristic information of each terminal user during the negotiation phase, including but not limited to: terminal user login ID information, question-and-answer conference registration information, preset participant information and other characteristic information, and assign initial weights based on the above characteristic information. The specific composition of the initial weight can be preset according to different scenarios, but the weight items at least include video user weight, volume weight, and audio-visual user weight.

[0072] The video user weight represents the importance of the terminal user's screen display during the speech, and is used to dynamically adjust the priority of the video stream required for the terminal to push the terminal video stream as the mixed stream;

[0073] The volume weight represents the importance of the terminal user's audio stream being kept during the speech. It is used to cooperate with the aforementioned terminal audio stream filtering and avoid the problem of different audio and video in some scenarios.

[0074] The audio-visual user weight represents the importance of the terminal user's need to listen and watch during the speech of other terminal users, and is used to dynamically adjust the priority of receiving the push stream under network congestion conditions.

[0075] Therefore, the weights in the first dynamic weighting algorithm include at least the video user weight and the volume weight of the corresponding terminal audio stream. For example: The user of the nth terminal speaks during the meeting, and the following scenario will occur. As the volume exceeds the threshold to trigger the weight adjustment, the index value of the video user weight of the terminal user is adjusted to 1, and the volume weight calculation increases with the volume, making the overall calculation of the back row the highest priority. The terminal video stream pushed by the nth terminal and the merged audio stream (including the terminal audio stream of the first terminal) are mixed, and then pushed to the subscribed users by the server. The participants will watch the user screen of the nth terminal and hear the user speak. During this period, as the volume of the first terminal user decreases for a period of time (scenarios such as doubts about speech, doubts about content, non-subjective topic change, etc.), the overall weight of the nth terminal will decrease with the volume weight. At this time, the corresponding weight of the terminal used by the important participants may exceed the weight of the nth terminal. Therefore, the terminal video stream participating in the mixed stream is replaced by the terminal video stream pushed by the important participants using the terminal, and then the video screen or the main partition of the video screen is displayed as the screen of the important participants, thus forming a more natural lens switching, while the speaker's audio is retained under the influence of the volume weight; while when no one is speaking, the index value of the video user weight of all terminals is 0, and there is no corresponding volume weight, so according to the initial weight, the screen will be the terminal video stream pushed by the terminal used by the important participants. The above design is more intelligent and manpower-saving than the existing manual lens switching; and it is more natural than the existing commercial video conferencing system that switches based solely on volume, and it conforms to viewing habits.

[0076] The server pushes audio and video streams to at least some of the subscribed terminals based on the second dynamic weighted algorithm. Among them, the weights of the second dynamic weighted algorithm include at least the audio-visual user weights. When network congestion occurs, the server sorts according to the initial weight calculation situation, and the terminals used by users with higher priority will give priority to ensuring the network bandwidth occupancy rate and achieving better streaming effects. Other users can trigger dynamic weight adjustment according to preset conditions, and adjust the index value of the audio-visual user weight of the terminal used to 1, thereby increasing the overall weight calculation value, improving the priority, and then continuously receiving audio and video streams. In a feasible embodiment, the preset conditions can analyze the current conference speech content according to the aforementioned voice recognition service, and extract keywords and terminal user feature information for similarity comparison. When the similarity exceeds the threshold, the adjustment is triggered, and the index value of the audio-visual user weight of this terminal is adjusted to 1, thereby increasing the overall weight calculation value.

[0077] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A conference communication transmission method integrating audio and video, characterized in that: The steps include: The server establishes connections with multiple terminals, and the server configures the audio coding format of each terminal based on the communication negotiation content; The audio encoding format of each terminal includes a unified encoding format and an adaptive transcoding encoding format; The server lists the audio coding formats supported by each terminal in the plurality of terminals based on the communication negotiation content; When there is a terminal that needs to be transcoded that does not support the unified encoding format, the server selects the fastest encoding format for transferring the unified encoding format from the audio encoding format supported by the terminal that needs to be transcoded, and uses it as the adapted transcoding encoding format for the terminal that needs to be transcoded; The server receives the terminal audio streams and the terminal video streams sent by the multiple terminals, and transcodes the terminal audio streams adapted to the transcoding encoding format to obtain an audio stream in a unified encoding format; The server screens the audio streams in a unified coding format, and selects at least multiple audio streams for confluence to obtain a confluence audio stream; The server selects at least one video stream based on a first dynamic weighting algorithm, and mixes the video stream with the confluent audio stream to obtain an audio and video stream; the weight items of the first dynamic weighting algorithm at least include a video user weight and a volume weight of a corresponding terminal audio stream; The server pushes the audio and video stream to at least some of the subscribed terminals based on a second dynamic weighted algorithm; the weight items of the second dynamic weighted algorithm at least include audio-visual user weights.

2. The conference communication transmission method integrating audio and video according to claim 1 is characterized in that: The server is based on the SFU architecture.

3. The conference communication transmission method integrating audio and video according to claim 1 or 2 is characterized in that: The server and the plurality of terminals are connected via a public network or a private network; The data transmission between the server and the multiple terminals is based on the web audio and video real-time communication protocol.

4. The conference communication transmission method integrating audio and video according to claim 3 is characterized in that: The step of configuring the audio coding format of each terminal based on the communication negotiation content by the server includes: The web audio and video real-time communication protocol is configured with a session description protocol, and the text of the session description protocol includes the audio coding format of each terminal; The server collects the text of the session description protocol, and obtains the audio coding format of each terminal.

5. The conference communication transmission method integrating audio and video according to claim 1 is characterized in that: The unified coding format is the digital audio coding format of Opus or G.

711.

6. The conference communication transmission method integrating audio and video according to claim 1 or 5, characterized in that: The method for selecting the adaptive transcoding encoding format comprises the following steps: The server lists the audio encoding formats supported by the terminal requiring transcoding, and determines the fastest encoding format based on an encoding format feature analysis method or a transcoding test method, and uses the fastest encoding format as the adapted transcoding encoding format for the terminal requiring transcoding.

7. The conference communication transmission method integrating audio and video according to claim 6 is characterized in that: The transcoding test method comprises: The server collects multiple groups of test audios sent by the terminal that needs to be transcoded, and the formats of the multiple groups of test audios include audio coding formats correspondingly supported by the terminal that needs to be transcoded; The server transcodes the multiple groups of test audio into audio with a unified encoding format under the same test environment, and records the transcoding time of each group, and selects the encoding format when sending the group of test audio with the least time as the fastest encoding format.

8. The conference communication transmission method integrating audio and video according to claim 1 is characterized in that: The server screens the audio stream in the unified coding format, comprising the following steps: The server sorts the audio streams in the unified coding format based on the volume of the audio streams in the unified coding format, and selects the first several audio streams with large volume.

9. The conference communication transmission method integrating audio and video according to claim 1 is characterized in that: The server also performs a preliminary screening before screening the audio stream in the unified coding format, and the preliminary screening includes the following steps: The server preliminarily screens the audio stream in the unified coding format based on the audio spectrum characteristics of human voice, and removes the non-human voice audio stream.

10. The conference communication transmission method integrating audio and video according to claim 1 is characterized in that: The server collects characteristic information of each terminal user and assigns an initial weight based on the characteristic information; The initial weights also include the video user weight, volume weight, and audio-visual user weight; The video user weight is used to adjust the mixing priority of the terminal video stream; The audio-visual user weight is used to adjust the priority of the terminal receiving the stream.

Images