Video conference processing method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video conference processing method and system. The method comprises the following steps: regularly obtaining split screen display information of conference participating terminals and training rotation terminal information; according to the split-screen display information and the rotation training terminal information, judging whether a target terminal currently displayed by the conference participating terminal contains a to-be-displayed rotation training terminal or not; and if the currently displayed target terminal comprises the to-be-displayed rotation training terminal, determining a first split-screen area according to the split-screen display information, and copying a video stream of the currently displayed target terminal to the first split-screen area. According to the embodiment of the invention, a dynamic split-screen display and video stream copying mechanism is introduced, so that a blank screen phenomenon caused when the same spokesman is displayed by the rotation training terminal and the target terminal is avoided, the resource allocation efficiency and stability of the system are improved, and the fluency of a conference and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to a method for processing a video conferencing, a system for processing a video conferencing, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the widespread application of video conferencing technology, the concurrent display of multiple terminals and multiple video sources has become one of the core functions of modern conference systems. A typical video conferencing system consists of multiple terminal devices, including the speaker terminal, the participant terminal, and the round-robin terminal. The speaker terminal is usually responsible for displaying the video screen of the participant who is currently speaking, while the round-robin terminal is used to display the video screens of multiple participants in turn, so that conference participants can view the video screens of more people.

[0003] However, in actual applications, when the training terminal and the speaker terminal need to display the same video source (i.e. the screen of the same speaker) at the same time, video display anomalies may occur, especially when there is no clear priority processing mechanism for the requests of the two terminals for video stream resources. In this case, the display requests from different terminals may not be processed correctly, resulting in conflicts in video sources or contention for video stream resources.

[0004] Especially in a system with a high priority for the rotation terminal, when the speaker displayed by the rotation terminal is the same as the speaker terminal, the rotation terminal may obtain the video stream resource first, while the speaker terminal cannot obtain the video stream normally, which eventually leads to a black screen on the speaker terminal. This black screen phenomenon not only affects the smoothness of the meeting, but also may have a negative impact on the experience of the participants, especially when the video is interrupted during an important speech. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a method for processing a video conference, a processing system for a video conference, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In order to solve the above problems, an embodiment of the present invention discloses a method for processing a video conference, which includes: regularly obtaining split-screen display information and rotation terminal information of the participating terminals; judging whether the target terminal currently displayed by the participating terminals includes the rotation terminal to be displayed according to the split-screen display information and the rotation terminal information; if the currently displayed target terminal includes the rotation terminal to be displayed, determining a first split-screen area according to the split-screen display information, and copying the video stream of the currently displayed target terminal to the first split-screen area.

[0007] Optionally, judging whether the target terminal currently displayed by the participating terminal includes the training terminal to be displayed according to the split-screen display information and the training terminal information includes: determining the first terminal identification information of the currently displayed target terminal corresponding to each split-screen area of ​​the participating terminal from the split-screen display information, and reading the second terminal identification information of the training terminal to be displayed from the training terminal information; judging whether the target terminal currently displayed includes the training terminal to be displayed according to the first terminal identification information and the second terminal identification information.

[0008] Optionally, if the currently displayed target terminal includes the training terminal to be displayed, determining the first split-screen area according to the split-screen display information, including: if the currently displayed target terminal includes the training terminal to be displayed, determining the split-screen area outside the split-screen area corresponding to the training terminal to be displayed as a candidate split-screen area; and selecting the first split-screen area from the candidate split-screen area.

[0009] Optionally, selecting the first split-screen area from the candidate split-screen areas includes: when the split-screen display information indicates that the split-screen mode of the participating terminal is a two-split-screen mode, using the candidate split-screen area as the first split-screen area; when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode, selecting the first split-screen area according to the display content of the candidate split-screen area.

[0010] Optionally, when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode, the first split-screen area is selected according to the display content of the candidate split-screen area, including: when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode, the first split-screen area whose display content is an abnormal picture is selected from the candidate split-screen areas.

[0011] Optionally, after determining whether the target terminal currently displayed by the participating terminal includes the training terminal to be displayed based on the split-screen display information and the training terminal information, the method further includes: if the currently displayed target terminal changes from including the training terminal to be displayed to not including the training terminal to be displayed, restoring the video stream of the first split-screen area.

[0012] Optionally, if the currently displayed target terminal changes from including the training terminal to be displayed to not including the training terminal to be displayed, the method also includes: pushing the video stream of the training terminal to be displayed to a second split-screen area; wherein the second split-screen area is the split-screen area corresponding to the previous training terminal to be displayed.

[0013] An embodiment of the present invention also discloses a video conference processing system, which includes: an information periodic acquisition module, used to periodically acquire split-screen display information and rotation terminal information of participating terminals; a rotation terminal judgment module, used to judge whether the target terminal currently displayed by the participating terminal includes the rotation terminal to be displayed based on the split-screen display information and the rotation terminal information; a video stream copying module, used to determine a first split-screen area based on the split-screen display information if the currently displayed target terminal includes the rotation terminal to be displayed, and copy the video stream of the currently displayed target terminal to the first split-screen area.

[0014] Optionally, the training terminal judgment module includes: a terminal identification information determination module, used to determine the first terminal identification information of the currently displayed target terminal corresponding to each split-screen area of ​​the participating terminal from the split-screen display information, and read the second terminal identification information of the training terminal to be displayed from the training terminal information; a terminal identification information judgment module, used to judge whether the currently displayed target terminal includes the training terminal to be displayed based on the first terminal identification information and the second terminal identification information.

[0015] Optionally, the video stream copying module includes: a candidate split-screen area determination module, which is used to determine the split-screen area outside the split-screen area corresponding to the training terminal to be displayed as a candidate split-screen area if the currently displayed target terminal includes the training terminal to be displayed; and a first split-screen area selection module, which is used to select the first split-screen area from the candidate split-screen area.

[0016] Optionally, the first split-screen area selection module is used to use the candidate split-screen area as the first split-screen area when the split-screen display information indicates that the split-screen mode of the participating terminal is a two-split-screen mode; when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode, select the first split-screen area according to the display content of the candidate split-screen area.

[0017] Optionally, the first split-screen area selection module is used to select the first split-screen area whose display content is an abnormal picture from the candidate split-screen areas when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode.

[0018] Optionally, the system also includes: a split-screen area recovery display module, which is used to restore the video stream of the first split-screen area after the training terminal judgment module determines whether the target terminal currently displayed by the participating terminal includes the training terminal to be displayed based on the split-screen display information and the training terminal information. If the currently displayed target terminal changes from including the training terminal to be displayed to not including the training terminal to be displayed,

[0019] Optionally, the system also includes: a video stream pushing module, which is used to push the video stream of the training terminal to be displayed to the second split-screen area if the currently displayed target terminal changes from including the training terminal to be displayed to not including the training terminal to be displayed; wherein the second split-screen area is the split-screen area corresponding to the previous training terminal to be displayed.

[0020] An embodiment of the present invention also discloses an electronic device, comprising: one or more processors; and one or more machine-readable media storing instructions thereon, which, when executed by the one or more processors, enables the electronic device to execute the video conference processing method as described above.

[0021] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the video conference processing method as described above.

[0022] The embodiments of the present invention include the following advantages:

[0023] The video conference processing scheme provided by the embodiment of the present invention periodically obtains the split-screen display information and the rotation terminal information of the participating terminal. Then, it is determined whether the target terminal currently displayed by the participating terminal includes the rotation terminal to be displayed based on the split-screen display information and the rotation terminal information. If the currently displayed target terminal includes the rotation terminal to be displayed, the first split-screen area is determined based on the split-screen display information, and the video stream of the currently displayed target terminal is copied to the first split-screen area.

[0024] The embodiment of the present invention ensures that the video source can be properly processed when the rotating terminal displays a speaker that is consistent with the target terminal by regularly acquiring split-screen display information and rotating terminal information. By copying the video stream of the target terminal to the first split-screen area, video source conflicts are avoided, thereby ensuring the stability and continuity of the video picture and reducing the occurrence of black screen phenomena. Based on the split-screen display information and the rotating terminal information, it is dynamically determined whether the target terminal currently displayed by the participating terminal includes the rotating terminal, so that video stream resources are allocated according to the real-time judgment result. Compared with the resource contention or improper allocation problems that may occur in the background technology, this solution optimizes the utilization efficiency of system resources through pre-judgment and reasonable allocation mechanisms, ensures that each terminal can reasonably obtain video stream resources when multiple sources are displayed concurrently, and avoids video display problems caused by insufficient or incorrect resource allocation. By displaying a copy of the video stream in the first split-screen area, the black screen problem caused by the target terminal and the rotating terminal when displaying the same speaker is avoided. Compared with the black screen phenomenon in the background technology, it effectively improves the user's visual experience in video conferencing, especially when multiple speakers take turns to speak, ensuring the continuity of the video so that participants can watch the speech content smoothly. It avoids the negative impact on participants due to video interruption or black screen problems, and improves the quality and experience of the meeting. Since the rotation terminal and split-screen display information are regularly obtained, and video conflicts are actively processed when detected, it can effectively prevent abnormal situations that may occur in the video conferencing system and reduce video interruptions caused by the system's inability to correctly handle video stream conflicts. By pre-identifying and handling conflicts, the conference system is provided with higher reliability and fault tolerance, and the overall operating efficiency of video conferencing is improved.

[0025] In summary, the embodiment of the present invention avoids the black screen phenomenon caused by the training terminal and the target terminal displaying the same speaker by introducing a dynamic split-screen display and video stream copy mechanism, improves the resource allocation efficiency and stability of the system, and improves the fluency of the meeting and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for processing a video conference according to an embodiment of the present invention;

[0027] Figure 2 It is a structural block diagram of a video conference processing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The video conference processing solution proposed in the embodiment of the present invention periodically obtains the split-screen display information and the training terminal information of the participating terminals, and intelligently determines whether the current target terminal includes the training terminal to be displayed. When the target terminal and the training terminal display the same video source, the first split-screen area is determined according to the split-screen display information, and the video stream of the target terminal is copied to the first split-screen area, avoiding the black screen phenomenon caused by the video source conflict. Through this flexible split-screen processing mechanism, the stability of the video stream display of multiple terminals in the meeting can be ensured, while improving the efficiency of resource allocation and optimizing the user's conference experience.

[0030] Reference Figure 1 , shows a flowchart of a method for processing a video conference according to an embodiment of the present invention. The method for processing a video conference may specifically include the following steps:

[0031] Step 101, regularly obtain the split-screen display information and rotation terminal information of the participating terminals.

[0032] In an embodiment of the present invention, split-screen display information refers to how the conference terminal divides the screen to display the video streams of multiple participants. For example, in a common four-split screen layout, the video streams of participants A, B, C, and D will be displayed in four areas of the screen respectively. This information needs to be acquired and updated in real time to ensure that the screen allocation meets the needs of the conference process and participants. On the other hand, the rotation terminal information refers to which participants' screens are to be displayed in turn.

[0033] Periodic acquisition is usually achieved through a communication protocol between a server and a terminal device, and may adopt a request response mechanism based on Hypertext Transfer Protocol (HTTP) or Web Socket to ensure that the latest display information is acquired each time.

[0034] For example, suppose there are 10 participants in a video conference, and their split-screen display information and round-robin terminal information are obtained from all participating terminals every 5 seconds. On participant terminal 1, the current split-screen display information may indicate that the screen is divided into four areas, displaying participants 2, 3, 4, and 5 respectively, while the round-robin terminal information indicates that the videos of participants 6, 7, 8, and 9 are displayed in turn.

[0035] Step 102: determine whether the target terminal currently displayed by the participating terminal includes the terminal to be displayed in rotation according to the split-screen display information and the terminal information in rotation.

[0036] In an embodiment of the present invention, in a video conferencing system, the target terminal usually has a higher priority and is used to display the participant who is currently speaking. The round-robin terminal is used to display the video streams of multiple participants in turn. In order to avoid two terminals displaying the same speaker at the same time, accurate judgment is required. If the speaker displayed by the round-robin terminal and the target terminal is the same, it may cause reuse or conflict of video stream resources, and even cause a black screen problem on a certain terminal. Therefore, judging whether the two contain the same speaker is the key to ensuring the normal operation of the conference display.

[0037] In actual application, the current screen layout on the target terminal is parsed according to the split-screen display information and compared with the information of the rotation terminal. For example, the split-screen display information may show that the target terminal is displaying the video streams of participants A, B, and C, while the rotation terminal is currently displaying the video stream of participant B. In this case, it is determined that the display content of the target terminal includes participant B being displayed by the rotation terminal.

[0038] For example, in a video conference, assuming that the target terminal currently displays the videos of four speakers on the split screen, and the training terminal is training one of the speakers. After analyzing the display information, it is determined that the speaker C to be displayed by the training terminal is displayed in a split screen area of ​​the target terminal.

[0039] Step 103: if the currently displayed target terminal includes the polling terminal to be displayed, determine the first split screen area according to the split screen display information, and copy the video stream of the currently displayed target terminal to the first split screen area.

[0040] In an embodiment of the present invention, the target terminal is determined to contain the portion that the polling terminal needs to display, and a suitable split-screen area is identified through split-screen display information. On this basis, the video stream of the speaker is not requested again, but the video stream is directly copied from the target terminal. Copying in the computer field refers to duplicating existing data, rather than reacquiring or generating new data. In this scenario, copying the video stream can avoid repeated requests for video resources, reduce network load, and improve resource utilization efficiency.

[0041] For example, in a four-screen video conference, the target terminal is showing participants A, B, C, and D, and the rotation terminal also plans to display the video of participant C. At this time, it is determined that a certain split-screen area of ​​the target terminal is already displaying participant C, so the first split-screen area is selected from the split-screen areas corresponding to participants A, B, and D, and the video stream of participant C is copied to the first split-screen area.

[0042] The video conference processing scheme provided by the embodiment of the present invention periodically obtains the split-screen display information and the rotation terminal information of the participating terminal. Then, it is determined whether the target terminal currently displayed by the participating terminal includes the rotation terminal to be displayed based on the split-screen display information and the rotation terminal information. If the currently displayed target terminal includes the rotation terminal to be displayed, the first split-screen area is determined based on the split-screen display information, and the video stream of the currently displayed target terminal is copied to the first split-screen area.

[0043] The embodiment of the present invention ensures that the video source can be properly processed when the rotating terminal displays a speaker that is consistent with the target terminal by regularly acquiring split-screen display information and rotating terminal information. By copying the video stream of the target terminal to the first split-screen area, video source conflicts are avoided, thereby ensuring the stability and continuity of the video picture and reducing the occurrence of black screen phenomena. Based on the split-screen display information and the rotating terminal information, it is dynamically determined whether the target terminal currently displayed by the participating terminal includes the rotating terminal, so that video stream resources are allocated according to the real-time judgment result. Compared with the resource contention or improper allocation problems that may occur in the background technology, this solution optimizes the utilization efficiency of system resources through pre-judgment and reasonable allocation mechanisms, ensures that each terminal can reasonably obtain video stream resources when multiple sources are displayed concurrently, and avoids video display problems caused by insufficient or incorrect resource allocation. By displaying a copy of the video stream in the first split-screen area, the black screen problem caused by the target terminal and the rotating terminal when displaying the same speaker is avoided. Compared with the black screen phenomenon in the background technology, it effectively improves the user's visual experience in video conferencing, especially when multiple speakers take turns to speak, ensuring the continuity of the video so that participants can watch the speech content smoothly. It avoids the negative impact on participants due to video interruption or black screen problems, and improves the quality and experience of the meeting. Since the rotation terminal and split-screen display information are regularly obtained, and video conflicts are actively processed when detected, it can effectively prevent abnormal situations that may occur in the video conferencing system and reduce video interruptions caused by the system's inability to correctly handle video stream conflicts. By pre-identifying and handling conflicts, the conference system is provided with higher reliability and fault tolerance, and the overall operating efficiency of video conferencing is improved.

[0044] In summary, the embodiment of the present invention avoids the black screen phenomenon caused by the training terminal and the target terminal displaying the same speaker by introducing a dynamic split-screen display and video stream copy mechanism, improves the resource allocation efficiency and stability of the system, and improves the fluency of the meeting and user experience.

[0045] In an exemplary embodiment of the present invention, an implementation method for judging whether the target terminal currently displayed by the participating terminal includes the training terminal to be displayed based on the split-screen display information and the training terminal information is to determine the first terminal identification information of the currently displayed target terminal corresponding to each split-screen area of ​​the participating terminal from the split-screen display information, and read the second terminal identification information of the training terminal to be displayed from the training terminal information; and judge whether the currently displayed target terminal includes the training terminal to be displayed based on the first terminal identification information and the second terminal identification information. Specifically, a mechanism of identification information matching is adopted to compare the terminal identification information of the target terminal and the training terminal to determine whether the two involve the same video source, thereby effectively avoiding video conflicts.

[0046] In actual application, the current display content of each split-screen area is extracted from the split-screen display information of the participating terminal. The split-screen display information usually includes terminal identifications for multiple video streams. For example, the screen of a participating terminal may be divided into four areas, each showing the video images of four participants. Each video screen will correspond to a unique terminal identification information, namely the first terminal identification information, which is used to distinguish different target terminals. For example, if target terminal 1 is displaying the screen of participant A, the first terminal identification information may be "A1-01", while target terminal 2 may display participant B, whose first terminal identification information is "B1-02". By parsing these first terminal identification information, the specific situation of the target terminal displayed in each split-screen area can be identified.

[0047] Next, the second terminal identification information of the polling terminal to be displayed is read from the polling terminal information. The polling terminal is used to display the video streams of multiple participants in turn so that the conference participants can view the status of more participants. For example, the polling terminal displays the screen of participant C, and the second terminal identification information read from the polling terminal information may be "C1-03".

[0048] After obtaining the first terminal identification information in the split-screen display information and the second terminal identification information of the rotation terminal, the two will be compared. This step is the key to determining whether the current target terminal overlaps with the rotation terminal. For example, assuming that the first split-screen area of ​​the target terminal is displaying participant C, and the rotation terminal displays the same participant C, after comparison, it is found that the first terminal identification "C1-03" and the second terminal identification "C1-03" are the same. At this point, it is determined that the target terminal contains the rotation terminal to be displayed. This means that without additional processing, the two terminals will display the same video stream at the same time, which may lead to reuse or conflict of video resources.

[0049] For example, in a video conference with ten participants, the participant terminal is currently displaying the screens of participants A, B, C, and D, while the round-robin terminal is displaying the screens of participants C and E. When the round-robin terminal is the participant terminal corresponding to participant C, the first terminal identifier "C1-03" of participant C being displayed by the participant terminal is obtained from the split-screen display information, and the second terminal identifier "C1-03" of participant C to be displayed is read from the round-robin terminal information. After comparison, it is determined that the two terminals will display participant C at the same time. In this case, measures will be taken to avoid video source conflicts, such as copying the video stream to different display areas or adjusting the display priority.

[0050] The embodiment of the present invention can effectively identify potential conflicts between the target terminal and the training terminal by comparing the terminal identification information. This avoids the black screen phenomenon or resource contention problem caused by the training terminal and the target terminal displaying the same video stream. Especially in a complex conference environment with multiple terminals and multiple video streams, this processing mechanism can improve the stability of the system and the consistency of video display, ensuring that participants can watch the conference video screen seamlessly.

[0051] In an exemplary embodiment of the present invention, if the currently displayed target terminal includes a training terminal to be displayed, then an implementation method of determining the first split-screen area according to the split-screen display information is as follows: if the currently displayed target terminal includes a training terminal to be displayed, then the split-screen area outside the split-screen area corresponding to the training terminal to be displayed is determined as a candidate split-screen area; and the first split-screen area is selected from the candidate split-screen area. In this implementation, when it is determined that the currently displayed target terminal includes a training terminal to be displayed, it is necessary to further determine the appropriate split-screen area based on the split-screen display information to avoid conflicts and repeated displays of video resources. A specific solution is to use other split-screen areas outside the split-screen area corresponding to the training terminal to be displayed as candidate split-screen areas, and select a suitable split-screen area from these candidate split-screen areas, called the first split-screen area, for displaying the video picture.

[0052] In actual application, by parsing the split-screen display information, it is confirmed that the target terminal is currently displaying the same video stream as the rotation terminal. Assume that in a meeting, the screen of the target terminal is divided into multiple split-screen areas, and each split-screen area displays the screen of different participants. Detect the allocation of these split-screen areas, especially which split-screen areas have been used to display the video of the current speaker. Next, identify which split-screen areas are not occupied or do not display duplicate video streams. These split-screen areas that do not involve the rotation terminal become candidate split-screen areas. For example, if the participating terminal is divided into four split-screen areas, one of the evaluation areas displays the video of the current speaker A, and the rotation terminal also plans to display the video of speaker A, then the remaining three split-screen areas are marked as candidate split-screen areas. Candidate split-screen areas may contain videos of other participants or blank areas.

[0053] From these candidate split screen areas, one or more will be selected as the first split screen area for displaying the video content of the polling terminal. Usually, the first split screen area is selected according to a specific strategy, which may include giving priority to blank split screen areas or split screen areas that do not display important content. This process ensures that the allocation of video sources will not conflict.

[0054] For example, in a video conference with 10 participants, the split-screen area currently displayed by the participant terminal shows the screens of participants A, B, and C, while the rotation terminal shows the videos of participants C and D. If it is found that a split-screen area of ​​the participant terminal shows the video of participant C, the split-screen area that does not show participant C is used as a candidate split-screen area (for example, the area showing participants A and B), and then a split-screen area is selected from these candidate split-screen areas as the first split-screen area, and the video stream of participant C is copied to the first split-screen area for display.

[0055] The embodiment of the present invention optimizes the allocation of video resources by flexibly adjusting the display order of video streams between split-screen areas, and is particularly suitable for complex conference scenarios with multiple terminals and multiple video streams. In addition, the split-screen area can be dynamically adjusted based on real-time judgment, avoiding the occurrence of problems such as black screen or video stream conflict, thereby improving the stability of the conference system and user experience.

[0056] In an exemplary embodiment of the present invention, an implementation method of selecting the first split-screen area from the candidate split-screen areas is: when the split-screen display information indicates that the split-screen mode of the participating terminal is a two-split-screen mode, the candidate split-screen area is used as the first split-screen area; when the split-screen display information indicates that the split-screen mode of the participating terminal is not a two-split-screen mode, the first split-screen area is selected according to the display content of the candidate split-screen area.

[0057] In actual application, when it is detected that the participating terminal is in two-split screen mode, the processing logic is relatively simple. In two-split screen mode, the display screen of the participating terminal is divided into two split screen areas, each used to display a different video stream. In this case, the candidate split screen area is automatically used as the first split screen area, and there is no need to further screen the candidate split screen area. For example, in a video conference, the video source displayed by the current speaker A and the training terminal is the same, and the other idle split screen area (i.e., another candidate area) is used to display the content of the terminal to be trained.

[0058] When the split-screen mode is not a two-split-screen mode (such as a three-split-screen, a four-split-screen, etc.), the situation becomes more complicated. The split-screen display information may indicate that the participating terminal uses a three-split-screen, a four-split-screen, or even more screen split modes. At this point, it is necessary to further analyze the display content in the candidate split-screen area to select the most suitable first split-screen area to display the video stream of the terminal to be trained. This process usually involves making priority judgments based on the existing content of the split-screen area. For example, if the content currently displayed in a candidate split-screen area is relatively minor, or the candidate split-screen area is displaying a blank screen or a black screen, the candidate split-screen area will be preferentially selected as the first split-screen area. This can avoid conflicts and ensure that the important speaker's screen is displayed in a prominent area.

[0059] For example, if the screen of a participating terminal is divided into four split-screen areas, two of which display the participants who are speaking, another displays auxiliary conference information (such as file sharing screen), and the fourth is a blank split-screen area. In this case, the blank split-screen area will be selected as the first split-screen area to display the content of the terminal to be trained. If all candidate split-screen areas display different participant video streams, the split-screen area with less important content will be selected as the first split-screen area based on the actual needs of the meeting, such as the importance of the participant, the priority of the speech, and other factors.

[0060] The embodiments of the present invention enhance the flexibility of the video conferencing system, can automatically adjust the display area according to different split-screen modes, and reduce the complexity of system management and operation. It ensures the efficient use of video stream resources and avoids problems such as black screen and freeze caused by resource contention or improper allocation. Especially in complex conference scenarios with multiple participants and multiple video streams, this mechanism can ensure that the screen of each participant is reasonably displayed, improving the overall quality of the meeting and user experience. The first split-screen area selection method is highly scalable and can be customized according to the actual needs of the meeting to meet the split-screen display requirements in different conference scenarios.

[0061] In an exemplary embodiment of the present invention, when the split-screen display information indicates that the split-screen mode of the participating terminal is not a two-split-screen mode, an implementation method of selecting a first split-screen area based on the display content of the candidate split-screen area is that when the split-screen display information indicates that the split-screen mode of the participating terminal is not a two-split-screen mode, a first split-screen area whose display content is an abnormal picture is selected from the candidate split-screen area.

[0062] In the actual scenario of video conferencing, the split-screen display mode in multi-screen mode is very common, especially when the participating terminals support the concurrent display of multiple video streams, this kind of problem is particularly prominent. For example, when the split-screen mode is three-split or four-split, different screen areas may display different content, including the speaker's video, file sharing or conference information. However, due to the limitations of network conditions or terminal device performance, sometimes some split-screen areas will display abnormal images, which may appear as black screens, static images or image delays.

[0063] In actual application, when the split-screen mode of the participating terminal is not a two-split-screen mode, it is necessary to select the most suitable split-screen area from multiple candidate split-screen areas to display the terminal video stream to be trained. At this time, a reasonable selection strategy is to give priority to those split-screen areas that display abnormal images as the first split-screen area. This is because the split-screen area that displays abnormal images often has lost its due display effect, and if it is not replaced in time, it may affect the quality of the entire meeting. For example, in a four-split-screen conference terminal, three of the split-screen areas display different speakers or important meeting information, while the fourth split-screen area has a black screen. In this case, the black screen area can be automatically identified as an abnormal screen area, and the split-screen area can be used as the first split-screen area to display the terminal content to be trained. This replacement can not only maximize the use of screen space, but also avoid repeated display or occupying important display areas.

[0064] The embodiments of the present invention can improve the intelligence and automation level of the video conferencing system. By automatically detecting abnormal conditions in the split-screen area, the display content of the split-screen area can be dynamically adjusted without manual intervention to ensure that each split-screen area is displaying valid information. By giving priority to the use of split-screen areas that display abnormal content, interference with normal video streams is avoided, thereby maintaining the overall coordination and integrity of the conference screen. In addition, when an abnormality occurs in a split-screen area, it is ensured that invalid images will not be displayed for a long time during the meeting, especially during the rotation of important speakers, to ensure seamless switching of video streams.

[0065] In an exemplary embodiment of the present invention, after determining whether the target terminal currently displayed by the participating terminal includes the training terminal to be displayed based on the split-screen display information and the training terminal information, an implementation method is to restore the video stream of the first split-screen area if the currently displayed target terminal is changed from including the training terminal to be displayed to not including the training terminal to be displayed.

[0066] In the actual application scenarios of video conferencing, the switching between the target terminal and the rotation terminal is a very common phenomenon. When the video sources of the target and rotation terminals are the same, it is necessary to properly handle the priority of resources to avoid duplicate display or resource conflicts. However, when the target terminal changes from including the rotation terminal to be displayed to not including it, it means that the speaker has changed. At this time, it is necessary to quickly restore the original video stream of the first split screen area to ensure the timely update of the screen content.

[0067] In actual application, suppose that in a conference, the target terminal currently displays the same video stream as the rotation terminal - for example, the video of speaker A. When the conference enters the next stage, the target terminal needs to switch to speaker B, and the rotation terminal continues to display the videos of other participants in rotation, and the target terminal no longer contains the rotation terminal to be displayed. At this time, the split-screen area originally used to display speaker A's video should be restored to ensure that speaker B's video can be smoothly displayed in the first split-screen area. Through this mechanism, the problem of video stream interruption during speaker switching is avoided, so that the conference video screen maintains continuity.

[0068] As another example, assume that the split-screen display mode of the conference system is a three-part screen mode, in which one area is used to display the current speaker, and the other two areas are used to display the video of the rotation terminal and other conference information. When the content displayed in area 1 and rotation terminal A is the same, the video stream of rotation terminal A is copied to area 2 for display. When switching from rotation terminal A to rotation terminal B, the display content of area 1 is restored, for example, the video stream of speaker 1 before the video stream of rotation terminal A is restored.

[0069] In an embodiment of the present invention, when the target terminal no longer includes the training terminal, the original split-screen display can be restored immediately to ensure the continuity and consistency of the video image. This is particularly important for large-scale meetings or meetings with multiple participants, because the video stream may be delayed or interrupted during the speaker switching process. When the target terminal and the training terminal no longer share the same video source, the split-screen area of ​​the video stream of the training terminal originally used to display the copy can be released to avoid repeated occupation of resources, thereby improving the overall performance of the system.

[0070] In an exemplary embodiment of the present invention, if the currently displayed target terminal changes from including a training terminal to be displayed to not including a training terminal to be displayed, the video stream of the training terminal to be displayed is pushed to the second split-screen area; wherein the second split-screen area is the split-screen area corresponding to the previous training terminal to be displayed.

[0071] In a video conferencing system, it is often necessary to manage multiple video streams and split-screen displays to ensure that the conference is conducted efficiently. For example, a video conference may use a four-split-screen mode, where one split-screen area is dedicated to displaying the current speaker, and the other three areas are used to display the video streams of the rotation terminal. When the target terminal in the conference is the same as the rotation terminal, it is necessary to ensure the consistency and correctness of this display.

[0072] In actual application, when the current target terminal displays a video stream of a training terminal, and the progress of the conference causes the training terminal to change, the target terminal no longer contains the training terminal, and the video stream of the new training terminal needs to be pushed to a new split-screen area. This new split-screen area is the second split-screen area, which corresponds to the split-screen area where the last training terminal to be displayed is located. The purpose of this is to ensure that the transition of the video stream is smooth when the training terminal changes, and does not cause blank space or content loss in the video display.

[0073] Assume that in a video conference, the split-screen mode of the video conference is a four-split-screen mode, in which each split-screen area is used to display different content. Area A displays the video of the current speaker, and areas B, C, and D display the videos of the rotation terminals. When the speaker is participant A, area A displays the screen of participant A, while area B displays the screen of rotation terminal 1, area C displays the screen of rotation terminal 2, and area D displays the screen of rotation terminal 3. As the meeting progresses, the speaker changes from participant A to participant B, and the video stream of speaker B is pushed to area A. At this time, area B originally displayed the screen of rotation terminal 1, and it is necessary to push the video stream of rotation terminal 4 to area B, push the video stream of rotation terminal 5 to area C, and push the video stream of rotation terminal 6 to area C, so as to keep the screen of the rotation terminal continuously displayed in the split-screen area.

[0074] The embodiment of the present invention ensures the continuous display of video content when the target terminal and the rotation terminal change by pushing the video stream to the second split-screen area, avoiding interruption or blank of the video stream. This processing method can improve the fluency and professionalism of the meeting. By reasonably allocating the split-screen area, the display resources of the conference terminal are effectively utilized. It ensures that each video stream is displayed in its designated area, avoiding waste or conflict of resources. Participants can continue to see the content of the rotation terminal in the split-screen area, regardless of how the display content of the target terminal changes. This approach improves the user experience and makes the transition of conference content more natural and smooth. This processing mechanism enhances the flexibility of the system and can quickly adapt to various changes in the meeting, including speaker changes and adjustments of rotation terminals. It improves the adaptability of the system in processing multiple video streams and ensures the stability of video display.

[0075] Based on the above description of an embodiment of a method for processing a video conference, a copying scheme for rotating video streams in a video network conference is introduced below.

[0076] 1. Start the meeting: Pamir (the manager or scheduler of the conference system) first selects the participants and starts the meeting. During this process, Pamir needs to set up Speaker 1 and Speaker 2 to ensure that there are clear roles of main speaker and auxiliary speaker in the meeting. The setting of Speaker 1 and Speaker 2 is to ensure that the meeting has a clear speaking structure and visual arrangement.

[0077] 2. Set the terminal split-screen mode: After the meeting starts, Pamir selects participants in the participant list and sets each participant's terminal to two-split-screen mode. Specifically, the left split-screen displays the screen of speaker one, and the right split-screen displays the screen of speaker two. This split-screen setting ensures that participants can see the video streams of both speakers at the same time. Pamir transparently transmits (i.e. directly sends) the split-screen mode command to all participant terminals to ensure that each terminal displays the corresponding video content according to the settings.

[0078] 3. Enable rotation training: Pamir selects the participants who need rotation training and sets the rotation training interval (such as 3 seconds, 5 seconds, 10 seconds, or a custom time). The rotation training function enables all participants to see the screen specified by the conference host. If the participant push function is not enabled, only the host can see the screen specified by him. When each rotation training interval arrives, Pamir sends a stream switching instruction to the scheduling server according to the rotation training sequence. After receiving the stream switching instruction, the scheduling server pushes the video stream of the current rotation training terminal to the terminals of each participant to achieve dynamic switching of the screen.

[0079] 4. Set up terminal video stream copy: During the training process, if the screen of the training terminal is the same as that of the second speaker, Pamir will start the video stream copy function to avoid the black screen problem of the first speaker. The specific operation is to copy the video stream of the second speaker to the display area of ​​the first speaker. In this way, even if the second speaker is the same as the training terminal, the screen on the left side can still be displayed continuously to ensure the continuity and integrity of the meeting.

[0080] 5. Turn off terminal video stream copying: When the screen of the training terminal is no longer consistent with the second speaker, Pamir will turn off the video stream copying function. This means that if the training terminal switches to other screens and no longer affects the screen of the first speaker, the video stream copying operation will be automatically stopped to avoid unnecessary resource occupation and system burden.

[0081] 6. Stop rotation training: When the meeting ends or the rotation training function is no longer needed, Pamir will stop the rotation training operation and stop sending the stream switching command. At this time, all video stream switching operations will stop, and the meeting will be restored to the initial setting state, ensuring that system resources are managed reasonably and ending the rotation training task of the meeting.

[0082] By setting up a video stream copy, the embodiment of the present invention can avoid the black screen phenomenon of the speaker one when the speaker two is the same as the training terminal, thereby ensuring the continuity and stability of the conference video. The combination of the training function and the video stream copy mechanism enables the conference to dynamically adapt to various screen switching requirements and ensures that the video images of different participants can be displayed in a timely manner. The clear speaking split-screen mode and the effective management of the training function enhance the experience of the participants and make the display of the conference content more comprehensive and accurate. By reasonably turning on and off the video stream copy function, the use of resources can be optimized and unnecessary resource consumption and system burden can be avoided.

[0083] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0084] Reference Figure 2 , shows a structural block diagram of a video conference processing system according to an embodiment of the present invention. The video conference processing system may specifically include the following modules.

[0085] The information periodic acquisition module 21 is used to periodically acquire the split-screen display information of the participating terminals and the rotation terminal information;

[0086] The round-robin terminal determination module 22 is used to determine whether the target terminal currently displayed by the participating terminal includes the round-robin terminal to be displayed according to the split-screen display information and the round-robin terminal information;

[0087] The video stream copying module 23 is used to determine a first split screen area according to the split screen display information if the currently displayed target terminal includes the polling terminal to be displayed, and copy the video stream of the currently displayed target terminal to the first split screen area.

[0088] In an exemplary embodiment of the present invention, the polling terminal determination module 22 includes:

[0089] A terminal identification information determination module is used to determine the first terminal identification information of the currently displayed target terminal corresponding to each split-screen area of ​​the participating terminal from the split-screen display information, and read the second terminal identification information of the training terminal to be displayed from the training terminal information;

[0090] The terminal identification information judgment module is used to judge whether the currently displayed target terminal includes the training terminal to be displayed according to the first terminal identification information and the second terminal identification information.

[0091] In an exemplary embodiment of the present invention, the video stream copy module 23 includes:

[0092] A candidate split screen area determination module, configured to determine the split screen area outside the split screen area corresponding to the to-be-displayed polling terminal as a candidate split screen area if the currently displayed target terminal includes the to-be-displayed polling terminal;

[0093] The first split screen area selection module is used to select the first split screen area from the candidate split screen areas.

[0094] In an exemplary embodiment of the present invention, the first split-screen area selection module is used to use the candidate split-screen area as the first split-screen area when the split-screen display information indicates that the split-screen mode of the participating terminal is a two-split-screen mode; when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode, select the first split-screen area according to the display content of the candidate split-screen area.

[0095] In an exemplary embodiment of the present invention, the first split-screen area selection module is used to select the first split-screen area whose display content is an abnormal picture from the candidate split-screen areas when the split-screen display information indicates that the split-screen mode of the participating terminal is not the two-split-screen mode.

[0096] In an exemplary embodiment of the present invention, the system further comprises:

[0097] The split-screen area recovery display module is used to restore the video stream of the first split-screen area after the rotation terminal judgment module 22 determines whether the target terminal currently displayed by the participating terminal includes the rotation terminal to be displayed based on the split-screen display information and the rotation terminal information. If the currently displayed target terminal changes from including the rotation terminal to be displayed to not including the rotation terminal to be displayed,

[0098] In an exemplary embodiment of the present invention, the system further comprises:

[0099] A video stream pushing module, configured to push the video stream of the terminal to be displayed in rotation to the second split-screen area if the currently displayed target terminal is changed from one that includes the terminal to be displayed in rotation to one that does not include the terminal to be displayed in rotation;

[0100] Among them, the second split-screen area is the split-screen area corresponding to the last training terminal to be displayed.

[0101] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0102] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0103] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0104] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0108] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0109] The above is a detailed introduction to a video conference processing method and a video conference processing system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing a video conference, characterized in that: The method comprises: Regularly obtain the split-screen display information of participating terminals and the training terminal information; Determine whether the target terminal currently displayed by the participating terminal includes the terminal to be displayed in rotation according to the split-screen display information and the terminal information in rotation; If the currently displayed target terminal includes the polling terminal to be displayed, a first split-screen area is determined according to the split-screen display information, and the video stream of the currently displayed target terminal is copied to the first split-screen area.

2. The method according to claim 1, characterized in that: The determining, according to the split-screen display information and the round-robin terminal information, whether the target terminal currently displayed by the conference participant terminal includes the round-robin terminal to be displayed includes: Determine the first terminal identification information of the currently displayed target terminal corresponding to each split-screen area of ​​the participating terminal from the split-screen display information, and read the second terminal identification information of the training terminal to be displayed from the training terminal information; It is determined whether the currently displayed target terminal includes the polling terminal to be displayed according to the first terminal identification information and the second terminal identification information.

3. The method according to claim 2, characterized in that If the currently displayed target terminal includes the polling terminal to be displayed, determining the first split-screen area according to the split-screen display information includes: If the currently displayed target terminal includes the polling terminal to be displayed, determining the split screen area outside the split screen area corresponding to the polling terminal to be displayed as a candidate split screen area; The first split screen area is selected from the candidate split screen areas.

4. The method according to claim 3, characterized in that The selecting the first split screen area from the candidate split screen areas includes: When the split-screen display information indicates that the split-screen mode of the conference participant terminal is a two-split-screen mode, using the candidate split-screen area as the first split-screen area; When the split-screen display information indicates that the split-screen mode of the conference participant terminal is not the two-split-screen mode, the first split-screen area is selected according to the display content of the candidate split-screen area.

5. The method according to claim 4, characterized in that When the split-screen display information indicates that the split-screen mode of the conference participant terminal is not the two-split-screen mode, selecting the first split-screen area according to display content of the candidate split-screen area includes: When the split-screen display information indicates that the split-screen mode of the conference participant terminal is not the two-split-screen mode, the first split-screen area whose display content is an abnormal picture is selected from the candidate split-screen areas.

6. The method according to claim 1, characterized in that After determining whether the target terminal currently displayed by the participating terminal includes the terminal to be displayed in rotation according to the split-screen display information and the terminal information in rotation, the method further includes: If the currently displayed target terminal is changed from including the polling terminal to be displayed to not including the polling terminal to be displayed, the video stream of the first split-screen area is restored.

7. The method according to claim 6, characterized in that If the currently displayed target terminal is changed from including the polling terminal to be displayed to not including the polling terminal to be displayed, the method further includes: Pushing the video stream of the polling terminal to be displayed to the second split-screen area; Among them, the second split-screen area is the split-screen area corresponding to the last training terminal to be displayed.

8. A video conference processing system, characterized in that: The system comprises: The information periodic acquisition module is used to periodically acquire the split-screen display information of the participating terminals and the rotation terminal information; A rotation terminal judgment module, used for judging whether the target terminal currently displayed by the participating terminal includes the rotation terminal to be displayed according to the split-screen display information and the rotation terminal information; The video stream copying module is used to determine the first split screen area according to the split screen display information if the currently displayed target terminal includes the polling terminal to be displayed, and copy the video stream of the currently displayed target terminal to the first split screen area.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the video conference processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the video conference processing method as described in any one of claims 1 to 7.

Images