Video conference processing method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video conference processing method and device which are applied to a conference terminal, and the method comprises the steps: obtaining audio data packets of a plurality of speaking party terminals in a video conference; analyzing volume information of each audio data packet; a target speaking party terminal is screened out from the speaking party terminals according to the volume information, speaking party evaluation notification information is sent to a conference server, so that the conference server evaluates the target speaking party terminal according to the speaking party evaluation notification information to obtain a state evaluation result, and the target speaking party terminal is determined; and if the state evaluation result shows that the target speaking party terminal meets the conference speaking condition, transmitting the conference data of the target speaking party terminal to other conference terminals of the video conference, so that the other conference terminals display the conference data. According to the embodiment of the invention, the speaker identification accuracy and the communication real-time performance are effectively improved, the interference of background noise is reduced, the conference management is simplified, and more efficient and intelligent conference experience is provided for users.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to a method for processing a video conference, a device for processing a video conference, an electronic device, and a computer-readable storage medium. Background Art

[0002] In modern video conferencing systems, with the popularity of remote collaboration, more and more companies and organizations rely on video conferencing for communication and decision-making. However, as the number of participants in the meeting increases, especially when the meeting involves multiple members speaking at the same time, traditional video conferencing systems face many challenges.

[0003] Traditional video conferencing systems usually rely on members to manually activate microphones or use other means to identify speakers, which may be feasible in small-scale meetings. However, when there are a large number of meeting members, manual operations are not only cumbersome, but also prone to inaccurate speaker identification. For example, in a large meeting with dozens of members, multiple members may try to speak at the same time, or some members' speeches may be accompanied by background noise. These situations may make it difficult for the conference system to effectively identify which members are actively speaking.

[0004] When dealing with scenarios where multiple members speak at the same time, current video conferencing systems may cause delays and misunderstandings in information transmission, which in turn affects the accuracy and timeliness of decision-making. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a method for processing a video conference, a device for processing a video conference, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In order to solve the above problems, an embodiment of the present invention discloses a method for processing a video conference, which is applied to a conference terminal, and the method includes: obtaining audio data packets of multiple speaker terminals in the video conference; parsing the volume information of each of the audio data packets; screening out a target speaker terminal from the multiple speaker terminals according to the volume information, and sending speaker evaluation notification information to a conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal; if the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, the conference data of the target speaker terminal is transmitted to other conference terminals of the video conference, so that the other conference terminals display the conference data.

[0007] Optionally, parsing the volume information of each of the audio data packets includes: parsing the volume information from a packet header field of each of the audio data packets; wherein the volume information is calculated by the speaker terminal of each of the audio data packets.

[0008] Optionally, the method of filtering out a target speaker terminal from the multiple speaker terminals according to the volume information includes: storing each volume information in a volume data table and updating the volume data table; arranging the volume information in the volume data table in descending order; and determining the speaker terminals corresponding to a preset number of the volume information that are at the front after the descending order as the target speaker terminal.

[0009] Optionally, the transmitting of the conference data of the target speaker terminal to other conference terminals of the video conference so that the other conference terminals display the conference data includes: if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, sending a highlight notification message to the other conference terminal so that the other conference terminal determines, based on the highlight notification message, whether the video data in the conference data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal; if the video data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal, highlighting the terminal identification information in the conference data of the target speaker terminal screened out this time.

[0010] Optionally, the transmitting of the conference data of the target speaker terminal to other conference terminals of the video conference so that the other conference terminals display the conference data includes: if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, sending speaker switching notification information to the conference server, so that the conference server sets a new target speaker terminal according to the speaker switching notification information, and sends speaker stream switching notification information to the other conference terminals; the other conference terminals are used to determine whether the currently displayed target speaker terminal is the new target speaker terminal according to the speaker stream switching notification information, and if the currently displayed target speaker terminal is not the new target speaker terminal, displaying the video data in the conference data of the new target speaker terminal.

[0011] Optionally, the displaying of the video data in the conference data of the new target speaker terminal includes: the other conference terminals are also used to clear the video data on all split screens currently displayed, and display the video data of the new target speaker terminal on all split screens in sequence.

[0012] Optionally, the step of calculating the volume information includes: obtaining amplitude values ​​of two adjacent bytes in the audio field of the audio data packet; and calculating an average value of the amplitude values ​​of the audio field as an average decibel value of the volume information.

[0013] An embodiment of the present invention also discloses a video conference processing device, which is applied to a conference terminal, and the device includes: an audio data packet acquisition module, which is used to acquire audio data packets of multiple speaker terminals in the video conference; an audio data packet parsing module, which is used to parse out the volume information of each of the audio data packets; a target terminal screening module, which is used to screen out a target speaker terminal from the multiple speaker terminals according to the volume information, and send speaker evaluation notification information to a conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal; a conference data transmission module, which is used to transmit the conference data of the target speaker terminal to other conference terminals of the video conference if the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, so that the other conference terminals display the conference data.

[0014] Optionally, the audio data packet parsing module is used to parse the volume information from a packet header field of each audio data packet; wherein the volume information is calculated by the speaker terminal of each audio data packet.

[0015] Optionally, the target terminal screening module includes: a volume storage and update module, used to store each volume information in a volume data table and update the volume data table; a volume sorting module, used to arrange the volume information in the volume data table in descending order; and a target terminal determination module, used to determine the speaker terminals corresponding to a preset number of the volume information in front after descending order as the target speaker terminals.

[0016] Optionally, the conference data transmission module includes: a highlight notification module, which is used to send highlight notification information to the other conference terminals if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, so that the other conference terminals can judge whether the video data in the conference data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal according to the highlight notification information; if the video data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal, the terminal identification information in the conference data of the target speaker terminal screened out this time is highlighted.

[0017] Optionally, the conference data transmission module includes: a speaker switching notification module, which is used to send speaker switching notification information to the conference server if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, so that the conference server sets a new target speaker terminal according to the speaker switching notification information, and sends speaker flow switching notification information to the other conference terminals; the other conference terminals are used to determine whether the currently displayed target speaker terminal is the new target speaker terminal according to the speaker flow switching notification information, and if the currently displayed target speaker terminal is not the new target speaker terminal, then display the video data in the conference data of the new target speaker terminal.

[0018] Optionally, the other conference terminals are further used to clear the video data on all the split screens currently being displayed, and to display the video data of the new target speaking terminal on all the split screens in sequence.

[0019] Optionally, the speaker terminal is used to calculate the volume information, and the speaker terminal includes: an amplitude value acquisition module, used to obtain the amplitude value of two adjacent bytes in the audio field of the audio data packet; a decibel value calculation module, used to calculate the average value of the amplitude value of the audio field as the average decibel value of the volume information.

[0020] An embodiment of the present invention also discloses an electronic device, comprising: one or more processors; and one or more machine-readable media storing instructions thereon, which, when executed by the one or more processors, enables the electronic device to execute the video conference processing method as described above.

[0021] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the video conference processing method as described above.

[0022] The embodiments of the present invention include the following advantages:

[0023] The video conference processing scheme provided by the embodiment of the present invention is applied to a conference terminal. The conference terminal obtains audio data packets of multiple speaker terminals in the video conference. Then, the volume information of each audio data packet is parsed. Next, the target speaker terminal is screened out from the multiple speaker terminals according to the volume information, and speaker evaluation notification information is sent to the conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal. Furthermore, when the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, the conference data of the target speaker terminal is transmitted to other conference terminals of the video conference, so that the other conference terminals display the conference data.

[0024] The embodiment of the present invention can automatically obtain and analyze the volume information of each audio data packet by parsing the audio data packets of multiple speaker terminals, thereby automatically screening out the target speaker terminal. Not only the accuracy of speaker identification is improved, but also the dependence on manual intervention is reduced, and the communication efficiency of the conference is greatly improved. By introducing the speaker evaluation process, the conference server can further verify and evaluate the status of the target speaker terminal after the conference terminal screens out the target speaker terminal, and finally determine the target speaker terminal. This evaluation process can take into account multiple factors, such as the network status of the target speaker terminal, the working status of the audio and video equipment, and the current background noise level. When the status evaluation result shows that the target speaker terminal meets the conference speech conditions, the conference terminal will transmit its conference data to other conference terminals. This mechanism can significantly reduce the audio and video quality problems caused by the poor status of the target speaker terminal, thereby improving the fluency of the entire conference and the reliability of the speech. By automatically screening and transmitting the conference data of the target speaker terminal, it is ensured that other conference terminals quickly receive the most relevant speech content, thereby reducing the information transmission delay and improving the real-time and accuracy of conference communication. The process of screening speakers through volume information can effectively avoid the problem of misidentification caused by background noise interference. Compared with traditional systems that may mistakenly identify noise sources as speakers, more sophisticated volume information processing can more accurately identify actual speakers, thereby improving the quality of meetings. Through automated volume information parsing and screening, the meeting management process is simplified, reducing the need for meeting hosts or technical support personnel, making the meeting process smoother and more efficient. By automatically identifying and highlighting active speakers, a more intuitive communication experience is provided for meeting participants. Users no longer need to operate manually or worry about missing key speeches, thereby improving the overall user experience and enhancing the interactivity and participation of video conferences. For scenarios where multiple members speak at the same time, it can effectively respond to and process multiple audio sources, and by transmitting the audio data of the target speaker, other participants can focus on the main speech content, thereby avoiding information overload. This is especially important for large meetings or multi-party discussions.

[0025] In summary, the embodiments of the present invention can significantly improve the communication efficiency and quality of meetings, reduce the risks of misidentification and noise interference, and provide users with a more intelligent and user-friendly meeting experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for processing a video conference according to an embodiment of the present invention;

[0027] Figure 2 It is a structural diagram of a conference based on video networking according to an embodiment of the present invention;

[0028] Figure 3 It is a flowchart of a conference speech management solution based on a visual network according to an embodiment of the present invention;

[0029] Figure 4 It is a structural block diagram of a video conference processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] The video conference processing solution proposed in the embodiment of the present invention automatically identifies active speakers by acquiring and parsing audio data packets of multiple speaker terminals. Specifically, the volume information of each audio data packet is extracted, and then the target speaker terminal is screened out according to the volume, and the conference server is notified to evaluate the target speaker terminal to determine the target speaker terminal. When the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, its conference data is transmitted to other conference terminals. Through this automated process, the solution effectively improves the accuracy of speaker identification and the real-time nature of communication, reduces the interference of background noise, and simplifies conference management, ultimately providing users with a more efficient and intelligent conference experience.

[0032] Reference Figure 1 , shows a flowchart of a method for processing a video conference according to an embodiment of the present invention. The method for processing a video conference can be applied to a conference terminal (the conference terminal can be a conference management terminal such as a host terminal), and the method for processing a video conference can specifically include the following steps:

[0033] Step 101: Acquire audio data packets from multiple speaking terminals in a video conference.

[0034] In the embodiment of the present invention, the audio data packet contains the voice information of each speaker, which serves as the processing basis for the subsequent steps. The process of obtaining the audio data packet involves capturing and recording the audio stream transmitted in real time by each speaker terminal to ensure that the audio input from all participants can be continuously received.

[0035] To better understand this process, consider an actual conference scenario. In a remote conference with ten participants, each member accesses the conference through their own terminal device. Whenever a member starts speaking, the terminal device converts the member's voice into a digital signal, packages it into a data packet, and transmits it to the conference server or conference terminal through the network.

[0036] The process of obtaining audio data packets requires the conference terminal to have efficient network transmission capabilities and a good data receiving mechanism. Especially when the network conditions are poor or there are delays, the conference terminal needs to have a certain fault tolerance to ensure the integrity and continuity of the audio data. For example, when the network is briefly interrupted, the conference terminal should be able to make up for the lost data packets through a cache mechanism or retransmission strategy to avoid adverse effects on the conference effect.

[0037] Step 102: parse the volume information of each audio data packet.

[0038] In the embodiment of the present invention, after obtaining the audio data packets, the conference terminal needs to further parse these data packets to extract the key information therein, namely, volume information. The volume information may include but is not limited to the amplitude of the audio signal, which reflects the voice intensity of the speaker and is an important indicator for judging the speaker's activity.

[0039] The process of parsing volume information usually involves signal processing of audio data. In audio signal processing, volume is usually directly related to the amplitude of the signal. The conference terminal needs to convert the digital signal in the audio data packet into corresponding amplitude data, and then calculate the average or peak value of the amplitude to obtain the corresponding volume information.

[0040] Suppose in a video conference, a speaker has a loud voice, and the amplitude value of his audio data packet will be significantly higher than that of other speakers. By analyzing these amplitude values, the conference terminal can quickly identify the speaker's volume information and give priority to his speech data in subsequent steps.

[0041] In practical applications, the analysis of volume information does not only rely on simple amplitude calculations, but may also involve more complex processing. For example, when there is background noise, a noise reduction algorithm is needed to filter out non-speaking sound interference, so as to extract more accurate speaker volume information.

[0042] Another factor to consider is the situation where multiple speakers are speaking at the same time. In this case, it is necessary to be able to parse multiple audio packets at the same time and separate the independent volume information of each speaker.

[0043] Step 103, select a target speaker terminal from multiple speaker terminals according to volume information, and send speaker evaluation notification information to the conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal.

[0044] In an embodiment of the present invention, after completing the volume information analysis, it is necessary to screen out a target speaker terminal from multiple speaker terminals based on the volume information. The target speaker terminal usually refers to speakers with a louder volume and higher activity, such as the main speaker, whose speech content should be transmitted and displayed first.

[0045] The core of the screening process is the comparison and sorting of volume information. The volume information of all speaker terminals is compared and sorted according to the volume. Generally, speakers with louder volumes are more likely to be the most active speakers at the moment, so these terminals with louder volumes are preferred as target speaker terminals. However, relying solely on volume may lead to some misjudgments, such as a sudden increase in background noise. Therefore, other factors, such as speech duration and speech frequency, may be combined to further optimize the screening process.

[0046] Assume that in a video conference, audio packets are obtained from ten speaker terminals and the volume information of each audio packet is parsed. By comparison, it is found that the volume of three terminals is significantly higher than that of the others. At this time, the volume information of these three terminals will be further evaluated to determine whether there is background noise interference. If the volume of one terminal increases suddenly and lasts for a short time, it may be identified as a noise source rather than an actual speaker. In this case, the other two terminals with stable and louder volumes will be selected as the target speaker terminals.

[0047] The conference terminal encapsulates the identification information of the target speaker terminal and the corresponding volume information into a speaker evaluation notification message and sends it to the conference server. The speaker evaluation notification message not only contains the basic information of the currently screened target speaker terminal, but also may contain additional conference context information, such as the topic of the current conference, the video content that has been displayed, or other real-time data. The sending of this speaker evaluation notification message enables the conference server to obtain the latest speaker information in a timely manner and perform further processing.

[0048] After receiving the speaker evaluation notification message, the conference server can perform a series of processing operations. For example, the conference server can perform a status evaluation on the target speaker terminal to determine whether there are problems such as network delay and increased packet loss rate, which may affect the audio or video transmission quality of the target speaker terminal. Based on the status evaluation results, the conference server can finally determine the target speaker terminal. In addition, the conference server can further adjust the audio / video transmission parameters of the target speaker terminal, such as dynamically adjusting the bit rate, switching to a backup transmission channel, or allocating more bandwidth to the target speaker terminal to ensure the smoothness and quality of audio and video transmission.

[0049] Step 104: if the status evaluation result indicates that the target speaker terminal meets the conference speaking condition, the conference data of the target speaker terminal is transmitted to other conference terminals of the video conference, so that the other conference terminals display the conference data.

[0050] In an embodiment of the present invention, the conference speaking conditions may include but are not limited to the stability of the network connection and the working status of the audio and video equipment. Among them, the stability of the network connection is used to evaluate whether the network connection of the target speaker terminal is stable enough, such as whether the delay, packet loss rate, etc. are within an acceptable range. If the network condition is not good, the audio and video quality of the target speaker terminal may be affected, resulting in other participants being unable to clearly hear or see the speech content. Therefore, the stability of the network connection is an important condition to ensure smooth speaking. The working status of the audio and video equipment is used to evaluate whether the microphone, camera and other equipment of the target speaker terminal are working properly. If these devices are faulty or not configured properly (such as too low volume, blurred video, etc.), even if the speaker tries to speak, other participants may not be able to receive clear audio and video signals. Therefore, the normal working status of the audio and video equipment is also a key condition.

[0051] The transmission process involves encoding and packaging of data. After the target speaker terminal is screened, the audio data, video data (if any) and other relevant conference information of the terminal need to be encoded. The encoding process not only includes the compression and formatting of audio and video signals, but may also include encryption of data to ensure data security during transmission. After the data is packaged, it is transmitted to other conference terminals through the network.

[0052] During the data transmission process, it is also necessary to deal with possible network fluctuations and data loss. In order to ensure the reliability of data transmission, redundant transmission and error correction mechanisms are usually adopted. For example, if a data packet is lost during transmission, the lost part can be recovered through a retransmission mechanism or from redundant data to ensure that the data that finally reaches each conference terminal is complete and accurate.

[0053] When other conference terminals receive the conference data from the target speaker terminal, they will immediately decode the data and display it to all participants. This process requires other conference terminals to have fast data decoding capabilities and efficient media rendering mechanisms to ensure that audio and video content can be displayed synchronously.

[0054] The video conference processing scheme provided by the embodiment of the present invention is applied to a conference terminal. The conference terminal obtains audio data packets of multiple speaker terminals in the video conference. Then, the volume information of each audio data packet is parsed. Next, the target speaker terminal is screened out from the multiple speaker terminals according to the volume information, and speaker evaluation notification information is sent to the conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal. Furthermore, when the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, the conference data of the target speaker terminal is transmitted to other conference terminals of the video conference, so that the other conference terminals display the conference data.

[0055] The embodiment of the present invention can automatically obtain and analyze the volume information of each audio data packet by parsing the audio data packets of multiple speaker terminals, thereby automatically screening out the target speaker terminal. Not only the accuracy of speaker identification is improved, but also the dependence on manual intervention is reduced, and the communication efficiency of the conference is greatly improved. By introducing the speaker evaluation process, the conference server can further verify and evaluate the status of the target speaker terminal after the conference terminal screens out the target speaker terminal, and finally determine the target speaker terminal. This evaluation process can take into account multiple factors, such as the network status of the target speaker terminal, the working status of the audio and video equipment, and the current background noise level. When the status evaluation result shows that the target speaker terminal meets the conference speech conditions, the conference terminal will transmit its conference data to other conference terminals. This mechanism can significantly reduce the audio and video quality problems caused by the poor status of the target speaker terminal, thereby improving the fluency of the entire conference and the reliability of the speech. By automatically screening and transmitting the conference data of the target speaker terminal, it is ensured that other conference terminals quickly receive the most relevant speech content, thereby reducing the information transmission delay and improving the real-time and accuracy of conference communication. The process of screening speakers through volume information can effectively avoid the problem of misidentification caused by background noise interference. Compared with traditional systems that may mistakenly identify noise sources as speakers, more sophisticated volume information processing can more accurately identify actual speakers, thereby improving the quality of meetings. Through automated volume information parsing and screening, the meeting management process is simplified, reducing the need for meeting hosts or technical support personnel, making the meeting process smoother and more efficient. By automatically identifying and highlighting active speakers, a more intuitive communication experience is provided for meeting participants. Users no longer need to operate manually or worry about missing key speeches, thereby improving the overall user experience and enhancing the interactivity and participation of video conferences. For scenarios where multiple members speak at the same time, it can effectively respond to and process multiple audio sources, and by transmitting the audio data of the target speaker, other participants can focus on the main speech content, thereby avoiding information overload. This is especially important for large meetings or multi-party discussions.

[0056] In summary, the embodiments of the present invention can significantly improve the communication efficiency and quality of meetings, reduce the risks of misidentification and noise interference, and provide users with a more intelligent and user-friendly meeting experience.

[0057] In an exemplary embodiment of the present invention, an implementation method for parsing the volume information of each audio data packet is to parse the volume information from the header field of each audio data packet; wherein the volume information is calculated by the respective speaker terminal of each audio data packet. In a video conferencing system, parsing the volume information of an audio data packet is one of the key steps to ensure that the conference system can accurately identify the speaker and effectively transmit information. In one implementation, a conference terminal (such as a host terminal) obtains the volume information by parsing the header field of each audio data packet. The volume information is calculated by each speaker terminal before sending the audio data packet and embedded in the header field of the data packet. After receiving the audio data packet, the conference terminal can directly extract the volume information from the header without performing additional complex calculations, which greatly improves processing efficiency.

[0058] Specifically, the header field of the audio data packet usually contains various metadata related to the data packet, such as timestamp, data packet sequence number, and terminal identification of the speaker. In this implementation, when each speaker terminal generates an audio data packet, it processes the audio signal, calculates the volume information of the signal, and embeds this information into the header field of the data packet. This process is usually completed at the hardware layer or software layer of the terminal, and can be performed in real time to ensure the accuracy and timeliness of the volume information.

[0059] Assume that in a video conference, a speaker uses his terminal device to participate in the conference. When the speaker starts speaking, the terminal device processes his voice signal and calculates the volume information of the voice signal. After that, the terminal device embeds the volume information as an independent field into the packet header of the audio data packet and transmits the packet to the conference server through the network. The conference server forwards the audio data packet to the conference terminal, which can immediately extract the volume information from the packet header without further processing the audio data itself.

[0060] The volume information in the embodiment of the present invention is pre-calculated at the speaker's terminal and embedded in the data packet. After receiving the audio data packet, the conference terminal can obtain the volume information without complex signal processing. The computing burden of the conference terminal or the server is reduced, and the audio data can be processed and filtered more quickly, especially in the scene where multiple speakers speak at the same time. This advantage is particularly obvious. Since the volume information is directly calculated by the speaker's terminal, this means that the volume information can more accurately reflect the actual volume status of the speaker. For example, the terminal device can dynamically adjust the calculation method of the volume information according to factors such as the speaker's voice characteristics and the ambient noise level to ensure that the volume information finally embedded in the data packet is the most accurate and most suitable for the current environment. By delegating the calculation task of the volume information to each terminal device, it can adapt to different conference scenarios and user needs more flexibly. Each terminal can select the most suitable volume information calculation method according to its own hardware performance and network conditions.

[0061] In an exemplary embodiment of the present invention, an implementation method of filtering out a target speaker terminal from multiple speaker terminals according to volume information is as follows: storing each volume information in a volume data table and updating the volume data table; arranging the volume information in the volume data table in descending order; and determining the speaker terminals corresponding to a preset number of volume information at the front after descending order as the target speaker terminals.

[0062] In actual application, after receiving the volume information, the conference terminal (such as the host terminal) stores the volume information in the volume data table. The volume data table can be regarded as a real-time updated database table, which is specifically used to record and manage the volume information of all current speaker terminals. Whenever the conference terminal receives a new audio data packet and parses its volume information, it will store this volume information together with other associated information of the audio data packet (such as terminal identification, timestamp, etc.) in the volume data table. If the speaker terminal already has a corresponding record in the volume data table, the record will be updated to reflect the latest volume status of the terminal.

[0063] After the volume data table is updated, the conference endpoint will sort the volume information in the volume data table in descending order. The purpose of this step is to quickly determine which speaker endpoints are the most active at the current moment. By sorting the volume information from high to low, it is possible to intuitively see which speaker endpoint has the highest volume, which usually means that the speaker of that endpoint is actively speaking. This sorting operation can be implemented with the help of a quick sorting algorithm to ensure that it remains efficient when processing large amounts of data.

[0064] After the sorting is completed, the conference terminal will filter out the speaker terminals with the highest volume information from the volume data table according to the preset number. For example, the top four speaker terminals with the highest volume may be preset as the target speaker terminals. These target speaker terminals will be marked as the main speakers of the current conference and will be processed first. The conference terminal will then send the audio data packets of these target speaker terminals to other conference terminals to ensure that all participants can receive the voice of the main speaker in a timely manner.

[0065] Suppose that in a meeting, there are ten participants located in different regions. At this time, the host terminal receives the volume information from these ten speaker terminals and stores this volume information in the volume data table. Then, the host terminal sorts these volume information in descending order and finds that the volume of four terminals is significantly higher than that of other terminals. This means that the speakers of these four terminals are the most active at the current moment. The host terminal marks these four terminals as target speaker terminals and preferentially transmits their audio data to all participants, ensuring that the voices of these main speakers can be clearly and real-timely conveyed to other members.

[0066] The embodiment of the present invention can quickly and accurately identify the most active speakers in complex multi-speaker scenarios by storing volume information in a volume data table and sorting it. It is particularly suitable for large-scale meetings or remote collaboration, and can effectively improve the communication efficiency of the meeting. The dynamic update mechanism of the volume data table ensures that the volume changes of each speaker terminal are reflected in real time, thereby ensuring that the selection of the target speaker terminal is based on the latest volume information. This reduces misjudgments caused by delays or outdated data, and further improves response speed and accuracy. In addition, by presetting the number of screenings, it can flexibly adapt to the needs of meetings of different sizes and types. In smaller meetings, the number of target speaker terminals can be reduced to ensure the focus of the discussion. In large meetings, this number can be increased to cover more speakers and promote more extensive communication.

[0067] In an exemplary embodiment of the present invention, a method of transmitting conference data of a target speaker terminal to other conference terminals of a video conference so that other conference terminals display conference data is as follows: if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, a highlight notification message is sent to other conference terminals so that other conference terminals can judge whether the video data in the conference data of the target speaker terminal screened out this time is displayed on the screen of other conference terminals based on the highlight notification message. If the video data of the target speaker terminal screened out this time is displayed on the screen of other conference terminals, the terminal identification information in the conference data of the target speaker terminal screened out this time is highlighted.

[0068] In actual application, when a new target speaker terminal is screened, it will be compared with the target speaker terminal screened last time. If the target speaker terminal screened this time is different from the last time, a highlight notification message will be sent to other conference terminals. The purpose of this notification message is to remind each terminal that a new main speaker has appeared and needs to be visually emphasized.

[0069] After receiving the highlight notification information, other conference terminals will determine whether the video data of the currently filtered target speaker terminal has been displayed on the screen based on this information. If it has been displayed on the screen, these conference terminals will highlight the terminal identification information in the conference data of the target speaker terminal. This highlighting can take many forms, such as adding a highlight border to the current speaker's screen on the screen, displaying his name or terminal identification information in a more conspicuous color, or displaying the words "current speaker" on the speaker's screen. These visual markers can help other participants quickly identify the main speaker, thereby improving the focus and communication efficiency of the meeting.

[0070] Suppose that in a meeting. At the beginning of the meeting, the host terminal selects a speaker A as the target speaker terminal and displays his video image on the screens of all participants. A few minutes later, another speaker B starts speaking. The host terminal detects that B's volume information exceeds A, so B is set as the new target speaker terminal. Since the target speaker terminal selected this time is different from the last time, the host terminal immediately sends a highlight notification message to all other conference terminals. After receiving the notification, these terminals will determine whether B's image has been displayed on the screen. If it has been displayed, a highlight border is added around B's image so that all participants can recognize at a glance that B is the current main speaker.

[0071] The embodiment of the present invention can quickly switch between multiple speakers and display the current main speaker to all participants in real time through an automated notification and highlighting mechanism. This dynamic identification method ensures the fluency of the meeting and the focus of communication. Even in a scene where multiple speakers frequently switch, participants can always know who the current speaker is. Visual highlighting allows the main speaker to be presented more clearly to the participants, reducing communication barriers that may be caused by unclear speakers. Especially in large meetings or situations involving many participants, this visual prompt helps participants stay focused on the content of the meeting and reduce distractions. By dynamically screening and highlighting the target speaker terminal, it can adapt to different meeting scenarios and needs, thereby improving the flexibility and adaptability of the system. This automated processing method reduces the need for human intervention, reduces operational complexity, and enables the conference system to run more efficiently.

[0072] In an exemplary embodiment of the present invention, a method of transmitting conference data of a target speaker terminal to other conference terminals of a video conference so that other conference terminals display conference data is as follows: if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, a speaker switching notification message is sent to a conference server so that the conference server sets a new target speaker terminal according to the speaker switching notification message, and sends a speaker stream switching notification message to other conference terminals. Other conference terminals are used to determine whether the currently displayed target speaker terminal is a new target speaker terminal according to the speaker stream switching notification message. If the currently displayed target speaker terminal is not a new target speaker terminal, the video data in the conference data of the new target speaker terminal is displayed.

[0073] In a video conferencing system, in order to ensure that conference participants can obtain and pay attention to the content of the current main speaker in real time, it is necessary to effectively manage the switching between multiple speaker terminals. When it is detected that the current target speaker terminal is different from the last target speaker terminal, a series of operations will be triggered to display the conference data of the new main speaker in other conference terminals. In an implementation method, when the host terminal selects a new target speaker terminal through volume analysis or other determination methods, if the terminal is different from the target speaker terminal selected last time, the host terminal will send a speaker switching notification message to the conference server. This notification message includes the identification information of the new target speaker terminal currently detected, as well as the relevant conference data. This enables the conference server to update its records about the main speaker in a timely manner.

[0074] After receiving the speaker switching notification information, the conference server will set the new target speaker terminal as the current main speaker. Then, the conference server will send the speaker switching notification information to all other conference terminals. The purpose of this notification information is to inform each terminal that the current main speaker has changed, and provide the new target speaker terminal's identifier and related conference data.

[0075] Next, after receiving the speaker's stream switching notification information, each conference terminal will immediately determine whether the currently displayed target speaker terminal has been updated to a new speaker. If the terminal detects that the currently displayed target speaker terminal is not the latest speaker, it will automatically switch to the new target speaker terminal and display the new speaker's conference data on the screen, including video data, audio data and other related information. This process is usually real-time, ensuring that all participants in the meeting can simultaneously focus on the content of the main speaker.

[0076] Suppose that in a meeting. Halfway through the meeting, the host terminal detects that the volume of a technical person in charge C exceeds that of the current main speaker B. Since the target speaker terminal detected this time is different from the previous one, the host terminal immediately sends a speaker switching notification message to the conference server. After receiving the message, the conference server sets C as the new main speaker and immediately sends a speaker switching notification message to all other conference terminals. At this time, all terminals participating in the press conference will quickly switch to C's screen to ensure that all participants can clearly see and hear C's speech.

[0077] The embodiments of the present invention can effectively avoid information transmission delays caused by frequent switching of multiple speakers by dynamically updating the target speaker terminal and promptly synchronizing the information of the new speaker to all conference terminals. Participants can obtain the latest speaker content at the first time to ensure the timeliness and accuracy of communication. By utilizing the speaker switching notification information and the speaker flow switching notification information, speakers can be automatically managed in complex conference scenarios without manual intervention. This not only reduces the complexity of operations, but also reduces the risk of human error operations, and improves the overall efficiency of the meeting. By updating and switching to the new target speaker terminal in real time, participants can better understand the key points of the meeting and reduce confusion or information omissions caused by changes in speakers.

[0078] In an exemplary embodiment of the present invention, an implementation method for displaying the video data in the conference data of the new target speaker terminal is that other conference terminals are also used to clear the video data on all the split screens currently displayed, and display the video data of the new target speaker terminal on all the split screens in sequence.

[0079] In this implementation, when the conference server selects a new target speaker terminal, it needs to synchronize this change to all participants' terminal devices. After receiving the notification information, each terminal immediately performs the operation of clearing the currently displayed content. Specifically, each terminal will clear all video data currently displayed on the split screen to make room for the new speaker data. This operation ensures that no old or irrelevant speaker content will remain on the screen, avoiding possible confusion or misunderstanding.

[0080] Next, the participant terminal will display the video data of the new target speaker terminal on all split screens in turn. This means that no matter how many split screens are set on the conference terminal, each split screen will synchronously display the video image of the new main speaker. For example, on a participant terminal with four split screens, the participant terminal will display the same image of the latest speaker on all four split screens, or the first split screen will display the image of the first speaker terminal, the second split screen will display the image of the second speaker terminal, the third split screen will display the image of the third speaker terminal, and the fourth split screen will display the image of the fourth speaker terminal. This full coverage display method ensures that participants will not miss any visual information of the main speaker, especially in important occasions where the current speaker's content needs to be emphasized.

[0081] Suppose that in a meeting, the host terminal detects that a senior manager D has become the new target speaker terminal, and the conference server will send a speaker switching notification message to all conference terminals. After receiving the notification message, all participants' terminals will immediately clear the video data on the current split screen and display D's video image on all split screens. Regardless of whether the participants use a single-screen display or a multi-screen display, D's image will occupy all available screen space to ensure that D's speech can get the attention of the whole audience. This display method is particularly suitable for important meeting scenarios, such as when the focus of the meeting is dominated by a speaker, or when participants need to pay full attention to the speech of a key speaker.

[0082] In an embodiment of the present invention, when switching to a new speaker, the old content will be completely cleared, avoiding confusion that may be caused by the superposition or parallel display of different speakers' screens. This method is particularly useful for complex meetings, ensuring that the key points of each speech can be fully displayed. By synchronously displaying the screen of the latest speaker on all split screens, it is ensured that each participant can focus on the current main speaker. This full-screen display method can effectively improve the focus of the meeting, making the speaker's expression more influential and the participants feel more involved. By clearing the old content and automatically displaying the new speaker, it can adapt to different meeting scenarios, reduce the need for human intervention, and reduce the complexity of operation. This not only improves the fluency of the meeting, but also enhances the stability and reliability of the system.

[0083] In an exemplary embodiment of the present invention, the step of calculating the volume information includes: obtaining the amplitude values ​​of two adjacent bytes in the audio field of the audio data packet; calculating the average value of the amplitude value of the audio field as the average decibel value of the volume information. In a video conferencing system, in order to effectively identify and process the audio information of multiple speakers, the step of calculating the volume information is particularly critical. By accurately calculating the volume information of each speaker, the current main speaker can be screened out and his audio and video data can be displayed to other participants in a priority manner. In this process, the calculation of the volume information is based on the analysis and processing of the amplitude value in the audio data packet.

[0084] Specifically, it is necessary to extract the audio field from the audio data packet. The audio field is usually composed of multiple bytes, and each two adjacent bytes together represent the amplitude value of a sampling point. The amplitude value reflects the strength or volume of the sound waveform. Therefore, by analyzing these amplitude values, the speaker's volume information can be inferred. In order to calculate the volume information, the amplitude values ​​of two adjacent bytes are obtained from the audio data packet, and then the average of these amplitude values ​​is calculated, and finally this average value is used as the average decibel value of the volume information.

[0085] Suppose that an audio data packet of a speaker A is being processed. A's audio data packet contains multiple sampling points, each of which is represented by the amplitude value of two adjacent bytes. First, these amplitude values ​​are extracted, and then the average of the amplitude values ​​of all sampling points is calculated. For example, if the extracted amplitude values ​​are 10, 12, 11, 9, 13, then the average of these values ​​is calculated, that is, (10+12+11+9+13) / 5=11. This 11 is the average decibel value of speaker A, and this value is used as A's volume information for subsequent speaker identification and processing.

[0086] In practical applications, this method of calculating the average decibel value based on the amplitude value helps to accurately identify the actual volume of each speaker. By calculating the average decibel value, a more stable and accurate volume information can be obtained. In addition, it can also effectively deal with the problem of background noise in meetings. In a meeting scene, there may be a variety of background noises, such as keyboard tapping, air conditioning, etc., which may interfere with the accurate judgment of the speaker's volume. However, by calculating the average decibel value of the amplitude value, these short-term noise interferences can be filtered out, and only the long-lasting speaking sounds can be focused on. This method can significantly improve the accuracy of volume information, allowing the system to more reliably identify the real speaker without being misled by noise.

[0087] Based on the above description of the embodiment of a video conference processing method, a conference speech management solution based on a video network is introduced below. Figure 2, shows a structural diagram of a conference based on a visual network according to an embodiment of the present invention. The conference based on a visual network may involve: a conference server, a main control device, a sub-control device 1, ..., a sub-control device n, a terminal device 1, ..., a terminal device n. A main control device may be cascaded with multiple sub-control devices, and a sub-control device may be cascaded with multiple terminal devices. The main control device, the sub-control device, and the terminal device are all connected to the visual network.

[0088] In this conference based on the visual network, when the participating terminal pushes an audio data packet to the conference members, it first collects the local audio signal, calculates the average decibel value of the audio packet, and puts it into the header field of the audio data packet, and then sends the audio data packet to all participating members. After the host terminal receives multiple audio data packets, it obtains the decibel value in each audio data packet in real time and records or updates it locally, and detects the records in the volume data table in real time, sorting them from large to small according to the decibel value. The host terminal notifies all participating terminals to highlight the names of the four participating terminals with the highest decibel values. The host terminal notifies the conference server to set the participating terminal with the highest decibel value as speaker one. The participating terminal highlights the name of the participating terminal of the speaker. When speaker one changes, the display screen is rebound.

[0089] Reference Figure 3 , shows a flow chart of a conference speech management solution based on a visual network according to an embodiment of the present invention. A conference based on a visual network may involve a conference server, a core server and a terminal device. Among them, the core server includes a main control device and a sub-control device.

[0090] On the terminal device side:

[0091] The audio sending terminal pushes audio data.

[0092] The audio sending terminal here can be understood as the speaker terminal. The pushed audio data is the local audio data packet collected by the speaker terminal.

[0093] Calculate the decibel value of the audio data collected by the terminal.

[0094] For example, the data volume of an audio data packet is specified as size = 1024*2 bytes, 2 bytes represent an amplitude, and the total absolute value sum is calculated (i.e. the total accumulated value of 1024*2 bytes, for example, the first and second bytes are 100, the third and fourth bytes are 50... then sum = 100+50+.....)

[0095] The average decibel value is calculated by the total absolute value sum, that is, db = sum / (size / 2)

[0096] After the decibel value is calculated, 2 bytes are added to the header of the audio data packet to be sent to store the average decibel value of the audio data packet.

[0097] Send audio data packets to sub-control devices.

[0098] The sub-control device is responsible for data forwarding and forwards the audio data packets to the audio receiving terminals, that is, all terminal devices.

[0099] On the terminal device side:

[0100] Determine whether the terminal device is a host terminal.

[0101] If the terminal device is not the host terminal, the audio is output normally.

[0102] If the terminal device is a host terminal, the received multi-channel audio package information is saved.

[0103] Sort by decibel value in descending order.

[0104] For example, after the host terminal receives multiple audio packets (there may be multiple participating terminals speaking at the same time), it obtains the decibel value in each audio packet in real time and records or updates it locally. The recording method is:

[0105] Channel Number Channel Address Decibel Value Terminal number 1 0xa 55 1001 2 0xb 60 1002 3 0xc 77 1003

[0106] The host terminal detects the data in the list every 500 milliseconds and sorts them from large to small according to the decibel value.

[0107] Determine whether the first 4 terminals are consistent with the previous terminals.

[0108] If the first 4 terminals are the same as the last terminal, the audio is output normally without any other processing.

[0109] If the first four endpoints are not the same as the last endpoint (due to the scenario where there are too many speakers, only the four speakers with the highest decibel values ​​are obtained from the list), then:

[0110] The host terminal sends a voice message notification to all participating terminals. The notification information mainly includes: operation code (voice message), sender (host terminal), receiver terminal (all participating terminals), voice message list, etc.

[0111] Voice information list: such as [(terminal 1 name, terminal 1 device number), (terminal 2 name, terminal 2 device number), (terminal 3 name, terminal 3 device number), (terminal 4 name, terminal 4 device number)]

[0112] The host terminal sends the voice information notification to the corresponding sub-control device, which forwards the data to the sub-control device corresponding to the receiving terminal. Finally, the sub-control device corresponding to the receiving terminal forwards the voice information notification data to the receiving terminal device.

[0113] After receiving the voice message notification, the participating terminal parses the voice message list and determines whether the terminal in the voice message list is on the current terminal split screen (the current terminal display mode can be set to multiple split screens, such as two split screens, four split screens, eight split screens, etc., and each split screen can display a conference member terminal screen).

[0114] If it does not exist, it is not processed.

[0115] If it exists, the corresponding terminal name is highlighted.

[0116] At the same time, the host terminal obtains the terminal number with the largest decibel value at present, and determines whether the terminal with the largest decibel value is the same as the last time.

[0117] If they are consistent, the audio is output normally without any other processing.

[0118] If they are inconsistent, then:

[0119] The host terminal sends a notification to the conference server to switch to the speaker. The notification mainly includes: operation code (switch to speaker 1), receiving end (conference server), sender (host terminal number), information (terminal number with the highest current voice decibel value), etc.

[0120] When the speaker is switched, the sub-control device corresponding to the host terminal forwards the data to the main control device, and then the main control device forwards the data to the conference server.

[0121] The conference server obtains the terminal with the largest decibel value and sets it as the new speaker one. After the setting is successful, a switch notification for speaker one is sent to all terminals. The switch notification is forwarded to the sub-control device through the data of the main control device, and then forwarded to all terminal devices by the sub-control device data. The switch notification contains the following main information: operation code (switching notification for speaker one), sender (conference server), receiving terminal number (all participating terminals), and conference speaker information (mainly including the host device number, speaker one device number, speaker two device number, etc. in the current conference).

[0122] After receiving the speaker switching notification from the conference server, all terminals parse out the new speaker number and determine whether the terminal displayed on the first split screen on the current split screen is consistent with the new speaker.

[0123] If they are consistent, no action is taken.

[0124] If they are inconsistent, all screen displays on the current terminal split screen will be cleared.

[0125] The terminal device parses the information in each video stream received. The video stream information mainly includes: channel number, channel address, terminal number, type (h264 / h265, etc.), video stream, etc.

[0126] If the terminal number corresponding to the received video stream is the same as that of speaker 1, the video stream of speaker 1 is pushed to the first split-screen display, and the video streams of other channels are bound to other split-screen displays in turn.

[0127] In addition, the host terminal selects the participating terminal to be locked through the remote control. After the locking is successful, the participating terminal automatically switches to the first speaker and is displayed on the first split screen of all participating terminals. In the subsequent speaker polling service, the locked terminal device always remains unchanged on the first split screen.

[0128] The host device sends a notification to lock the speaker to the conference server. The communication information mainly includes: operation code (lock the speaker), sender terminal device number (host device), receiver (conference server), locked terminal number, etc.

[0129] After receiving the notification of locking the speaker, the conference server parses the locked terminal number and sets it as the new speaker one. The subsequent process is the same as setting the new speaker above, which will not be repeated here.

[0130] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0131] Reference Figure 4 , shows a structural block diagram of a video conference processing device according to an embodiment of the present invention. The video conference processing device is applied to a conference terminal (the conference terminal may be a conference management terminal such as a host terminal), and the video conference processing device may specifically include the following modules.

[0132] The audio data packet acquisition module 41 is used to acquire audio data packets of multiple speaking party terminals in the video conference;

[0133] The audio data packet parsing module 42 is used to parse out the volume information of each audio data packet;

[0134] A target terminal screening module 43 is used to screen a target speaker terminal from the plurality of speaker terminals according to the volume information, and send speaker evaluation notification information to the conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a state evaluation result, so as to determine the target speaker terminal;

[0135] The conference data transmission module 44 is configured to transmit the conference data of the target speaker terminal to other conference terminals of the video conference if the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, so that the other conference terminals display the conference data.

[0136] In an exemplary embodiment of the present invention, the audio data packet parsing module 42 is used to parse the volume information from the packet header field of each audio data packet;

[0137] The volume information is calculated by the speaker terminal of each audio data packet.

[0138] In an exemplary embodiment of the present invention, the target terminal screening module 43 includes:

[0139] A volume storage and updating module, used for storing each volume information in a volume data table and updating the volume data table;

[0140] A volume sorting module, used for sorting the volume information in the volume data table in descending order;

[0141] The target terminal determination module is used to determine the speaker terminals corresponding to the preset number of volume information arranged in descending order as the target speaker terminals.

[0142] In an exemplary embodiment of the present invention, the conference data transmission module 44 includes:

[0143] A highlight notification module is used to send highlight notification information to the other conference terminals if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, so that the other conference terminals can judge whether the video data in the conference data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal based on the highlight notification information; if the video data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal, the terminal identification information in the conference data of the target speaker terminal screened out this time is highlighted.

[0144] In an exemplary embodiment of the present invention, the conference data transmission module 44 includes:

[0145] A speaker switching notification module is used to send speaker switching notification information to the conference server if the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, so that the conference server sets a new target speaker terminal according to the speaker switching notification information, and sends speaker flow switching notification information to the other conference terminals;

[0146] The other conference terminals are used to determine whether the currently displayed target speaker terminal is the new target speaker terminal according to the speaker switching notification information, and if the currently displayed target speaker terminal is not the new target speaker terminal, then display the video data in the conference data of the new target speaker terminal.

[0147] In an exemplary embodiment of the present invention, the other conference terminals are further used to clear the video data on all the split screens currently being displayed, and to display the video data of the new target speaking terminal on all the split screens in sequence.

[0148] In an exemplary embodiment of the present invention, the speaker terminal is used to calculate the volume information, and the speaker terminal includes:

[0149] An amplitude value acquisition module, used to acquire the amplitude values ​​of two adjacent bytes in the audio field of the audio data packet;

[0150] The decibel value calculation module is used to calculate the average value of the amplitude value of the audio field as the average decibel value of the volume information.

[0151] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0153] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0154] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0158] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0159] The above is a detailed introduction to a video conference processing method and a video conference processing system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing a video conference, characterized in that: Applied to a conference terminal, the method includes: Obtain audio data packets from multiple speaking terminals in a video conference; Parsing the volume information of each audio data packet; Filtering a target speaker terminal from the plurality of speaker terminals according to the volume information, and sending speaker evaluation notification information to a conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a state evaluation result, thereby determining the target speaker terminal; If the status evaluation result indicates that the target speaker terminal meets the conference speaking condition, the conference data of the target speaker terminal is transmitted to other conference terminals of the video conference, so that the other conference terminals display the conference data.

2. The method according to claim 1, characterized in that The step of parsing the volume information of each audio data packet includes: Parsing the volume information from the packet header field of each audio data packet; The volume information is calculated by the speaker terminal of each audio data packet.

3. The method according to claim 1, characterized in that The step of selecting a target speaker terminal from the plurality of speaker terminals according to the volume information includes: storing each volume information in a volume data table, and updating the volume data table; Arrange the volume information in the volume data table in descending order; The speaker terminals corresponding to the preset number of volume information arranged in descending order are determined as the target speaker terminals.

4. The method according to claim 1, characterized in that: The transmitting the conference data of the target speaker terminal to other conference terminals of the video conference so that the other conference terminals display the conference data includes: If the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, a highlight notification message is sent to the other conference terminals, so that the other conference terminals can judge whether the video data in the conference data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal according to the highlight notification message; if the video data of the target speaker terminal screened out this time is displayed on the screen of the other conference terminal, the terminal identification information in the conference data of the target speaker terminal screened out this time is highlighted.

5. The method according to claim 1, characterized in that The transmitting the conference data of the target speaker terminal to other conference terminals of the video conference so that the other conference terminals display the conference data includes: If the target speaker terminal screened out this time is different from the target speaker terminal screened out last time, sending speaker switching notification information to the conference server, so that the conference server sets a new target speaker terminal according to the speaker switching notification information, and sends speaker flow switching notification information to the other conference terminals; The other conference terminals are used to determine whether the currently displayed target speaker terminal is the new target speaker terminal according to the speaker switching notification information, and if the currently displayed target speaker terminal is not the new target speaker terminal, then display the video data in the conference data of the new target speaker terminal.

6. The method according to claim 5, characterized in that The displaying of the video data in the conference data of the new target speaker terminal includes: The other conference terminals are also used to clear the video data on all the split screens currently being displayed, and to display the video data of the new target speaking terminal on all the split screens in sequence.

7. The method according to claim 2, characterized in that The step of calculating the volume information includes: Acquire the amplitude values ​​of two adjacent bytes in the audio field of the audio data packet; An average value of the amplitude values ​​of the audio field is calculated as an average decibel value of the volume information.

8. A video conference processing device, characterized in that: Applied to a conference terminal, the device comprises: An audio data packet acquisition module is used to acquire audio data packets from multiple speaking party terminals in a video conference; An audio data packet parsing module, used to parse out the volume information of each audio data packet; a target terminal screening module, configured to screen a target speaker terminal from the plurality of speaker terminals according to the volume information, and send speaker evaluation notification information to a conference server, so that the conference server evaluates the target speaker terminal according to the speaker evaluation notification information to obtain a status evaluation result, so as to determine the target speaker terminal; The conference data transmission module is used to transmit the conference data of the target speaker terminal to other conference terminals of the video conference if the status evaluation result indicates that the target speaker terminal meets the conference speaking conditions, so that the other conference terminals display the conference data.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the video conference processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the video conference processing method as described in any one of claims 1 to 7.

Images