Video conference control method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video conference control method and system, and the method comprises the steps: collecting conference operation data of a conference terminal in real time; detecting an abnormal state of the conference terminal according to the conference operation data to obtain an abnormal detection result; and executing corresponding automatic control measures according to the anomaly detection result so as to maintain a video conference function. According to the embodiment of the invention, by collecting the conference operation data in real time and executing the automatic control measures, the tedious steps of manually configuring the MCU and controlling the terminal are eliminated, the conference demand (such as terminal addition or parameter adjustment) can be quickly responded, the operation time and the labor cost are remarkably reduced, and the problem of low efficiency of the traditional method is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to a control method for a video conference, a control system for a video conference, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of video conferencing technology, its applications in various industries are becoming increasingly widespread, providing convenience for remote communication. However, existing video conference systems have deficiencies in the cascading of Multipoint Control Units (MCUs) and terminal control.

[0003] Traditional methods usually rely on manual configuration of MCUs and control terminals, with low efficiency and low resource utilization. When network anomalies (such as latency, packet loss) or equipment failures (such as camera, microphone failures) occur, there is a lack of automatic response capabilities and manual intervention is required, resulting in a decline or interruption in the conference quality. Therefore, there is an urgent need for an automated and intelligent control method to improve efficiency, optimize resources, and enhance system reliability. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a control method for a video conference, a control system for a video conference, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] To solve the above problems, embodiments of the present invention disclose a control method for a video conference, the method comprising:

[0006] Real-time collecting the conference operation data of the conference terminal;

[0007] Detecting the abnormal state of the conference terminal according to the conference operation data to obtain an abnormality detection result;

[0008] Performing corresponding automated control measures according to the abnormality detection result to maintain the video conference function.

[0009] Optionally, the detecting the abnormal state of the conference terminal according to the conference operation data to obtain an abnormality detection result includes:

[0010] Extracting the feature data of at least one of network data, device data, and audio-video stream data from the conference operation data;

[0011] Analyzing the feature data based on preset rules and / or a preset model to obtain the abnormality detection result.

[0012] Optionally, performing corresponding automated control measures according to the anomaly detection result includes:

[0013] Classifying the anomaly detection result to obtain an anomaly type;

[0014] Identifying a corresponding anomaly cause according to the anomaly type;

[0015] Performing the automated control measures corresponding to the anomaly cause according to an automated processing mechanism.

[0016] Optionally, performing the automated control measures corresponding to the anomaly cause according to an automated processing mechanism includes:

[0017] Performing the automated control measures according to at least one of a rule engine mechanism, a dynamic adjustment mechanism, and a disaster recovery switching mechanism.

[0018] Optionally, performing the automated control measures according to the rule engine mechanism includes:

[0019] Automatically triggering an automated control operation corresponding to the anomaly cause according to a preset processing rule.

[0020] Optionally, performing the automated control measures according to the dynamic adjustment mechanism includes:

[0021] When the anomaly cause is related to the network state, dynamically adjusting audio and video parameters according to the network state;

[0022] And / or, when the anomaly cause is related to the system load, dynamically allocating system resources according to the system load.

[0023] Optionally, performing the automated control measures according to the disaster recovery switching mechanism includes:

[0024] When the anomaly cause is a network reason, automatically switching the video conference transmission path to a backup path;

[0025] And / or, when the anomaly cause is a device reason, automatically replacing the faulty device with a backup device.

[0026] An embodiment of the present invention also discloses a control system for a video conference, and the system includes:

[0027] A data acquisition module, configured to collect meeting operation data of a meeting terminal in real time;

[0028] An anomaly detection module, configured to detect an abnormal state of the meeting terminal according to the meeting operation data to obtain an anomaly detection result;

[0029] An automatic control module, configured to execute corresponding automatic control measures according to the anomaly detection result to maintain the video conferencing function.

[0030] Optionally, the anomaly detection module includes:

[0031] A feature data extraction module, configured to extract feature data of at least one of network data, device data, and audio-video stream data from the conference operation data;

[0032] A feature data analysis module, configured to analyze the feature data based on a preset rule and / or a preset model to obtain the anomaly detection result.

[0033] Optionally, the automatic control module includes:

[0034] A detection result classification module, configured to classify the anomaly detection result to obtain an anomaly type;

[0035] An anomaly cause identification module, configured to identify a corresponding anomaly cause according to the anomaly type;

[0036] A control measure execution module, configured to execute the automatic control measure corresponding to the anomaly cause according to an automatic processing mechanism.

[0037] Optionally, the control measure execution module is configured to execute the automatic control measure according to at least one of a rule engine mechanism, a dynamic adjustment mechanism, and a disaster recovery switching mechanism.

[0038] Optionally, the control measure execution module includes:

[0039] A rule engine module, configured to automatically trigger an automatic control operation corresponding to the anomaly cause according to a preset processing rule.

[0040] Optionally, the control measure execution module includes:

[0041] A dynamic adjustment module, configured to dynamically adjust audio-video parameters according to the network status when the anomaly cause is related to the network status; and / or, dynamically allocate system resources according to the system load when the anomaly cause is related to the system load.

[0042] Optionally, the control measure execution module includes:

[0043] A disaster recovery switching module, configured to automatically switch the video conferencing transmission path to a backup path when the anomaly cause is a network reason; and / or, automatically replace a faulty device with a backup device when the anomaly cause is a device reason.

[0044] An embodiment of the present invention also discloses an electronic device, including: one or more processors; and one or more machine-readable media storing instructions thereon, which, when executed by the one or more processors, cause the electronic device to execute the control method of the video conference as described above.

[0045] An embodiment of the present invention also discloses a computer-readable storage medium, and the computer program stored therein causes a processor to execute the control method of the video conference as described above.

[0046] The embodiments of the present invention have the following advantages:

[0047] The control solution of the video conference provided by the embodiments of the present invention collects the conference operation data of the conference terminal in real time; detects the abnormal state of the conference terminal according to the conference operation data to obtain an abnormal detection result; and executes corresponding automatic control measures according to the abnormal detection result to maintain the video conference function.

[0048] Compared with the background technology, the embodiments of the present invention have the following beneficial effects:

[0049] By collecting the conference operation data in real time and executing automatic control measures, the cumbersome steps of manual configuration of the MCU and terminal control are eliminated, and the conference requirements (such as terminal joining or parameter adjustment) can be quickly responded to, significantly reducing the operation time and labor cost, and solving the problem of low efficiency of the traditional method. Detecting abnormalities according to the conference operation data and executing corresponding automatic control measures (such as reducing the bit rate to reduce bandwidth occupancy) realizes the dynamic allocation of resources. Compared with the resource waste caused by traditional manual control, it can optimize the network bandwidth, computing resources and device utilization rate according to real-time requirements, and improve the overall resource utilization efficiency. Automatically detecting network abnormalities (such as latency, packet loss) or device failures (such as camera failure), and quickly executing automatic control measures (such as switching to a standby device), it can quickly respond to abnormalities without manual intervention, reducing the risk of conference interruption or quality degradation, and overcoming the defect of the traditional system lacking automatic response ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of the steps of a control method for a video conference according to an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of the steps of a video conference MCU cascading and terminal intelligent control solution according to an embodiment of the present invention;

[0052] Figure 3 is a block diagram of the structure of a control system for a video conference according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] An embodiment of the present invention provides a control solution for a video conference, which dynamically monitors the network status and device operation by collecting the conference operation data of the conference terminal in real time. Based on the conference operation data, the abnormal status of the conference terminal is detected, and network anomalies or device failures are quickly identified. And corresponding automated control measures are executed according to the abnormal detection results, such as adjusting audio-visual parameters or switching to standby devices, to maintain the stable operation of the video conference function. The embodiment of the present invention overcomes the defects of the traditional system relying on manual operation, low efficiency, and insufficient abnormal response, and significantly improves the conference efficiency, resource utilization rate, and system reliability.

[0055] Refer to Figure 1 , which shows the step flowchart of a control method for a video conference according to an embodiment of the present invention. This control method for a video conference can be applied to a video conference system or a video conference control system, simply referred to as a system. The control method for this video conference can specifically include the following steps:

[0056] Step 101, collect the conference operation data of the conference terminal in real time.

[0057] The conference operation data refers to various types of information related to the operation status and performance of the video conference terminal, specifically including network data, device data, audio-visual stream data, and so on. Network data covers the performance parameters of network transmission, such as latency (reflecting the time for data packet transmission), packet loss rate (indicating the proportion of lost data packets), and bandwidth utilization rate (measuring the degree of network resource usage). Device data involves the operation status of the conference terminal hardware, such as whether the camera is online and capturing images normally, whether the microphone is effectively collecting sound, and whether the speaker is outputting audio normally. Audio-visual stream data includes the quality indicators of the transmitted audio and video, such as bit rate (data transmission rate), frame rate (number of video frames per second), resolution (image clarity), and audio-visual synchronization status. These conference operation data are continuously obtained through the data acquisition module in the system, usually using real-time monitoring technologies, such as periodically sampling the network status through network protocols, or detecting the device operation through hardware interfaces. The acquisition process needs to ensure high frequency and low latency to reflect the instantaneous status of the conference terminal, such as collecting the network latency and packet loss rate once per second, or checking the working status of the camera every few milliseconds. In addition, the collected conference operation data is usually stored in the system cache for subsequent analysis and processing, and at the same time supports parallel acquisition of multiple terminals to meet the requirements of large-scale video conference scenarios.

[0058] Step 102, detect the abnormal status of the conference terminal according to the conference operation data to obtain an abnormal detection result.

[0059] An abnormal state refers to the situation where the conference terminal deviates from normal operation during the running process, mainly including two categories: network anomalies and equipment failures. In the detection process, key feature data is extracted from the conference operation data. For example, the average latency, packet loss rate, and jitter value are extracted from the network data, the online status of the camera or the signal strength of the microphone is extracted from the equipment data, and indicators such as bitrate fluctuations or frame rate drops are extracted from the audio-video stream data. Subsequently, these feature data are analyzed based on preset rules or preset models. The preset rules can be threshold-based judgments. For example, when the network latency exceeds 100 milliseconds or the packet loss rate is higher than 5%, it is determined as a network anomaly. When the camera status shows "offline" or the signal-to-noise ratio of the microphone is lower than 30 decibels, it is determined as an equipment failure. The preset model may adopt machine learning methods, such as classification models trained by historical data (such as decision trees or neural networks), to identify complex abnormal patterns, such as network congestion or intermittent equipment failures. For example, in a certain conference, if the system detects that the network latency of the conference terminal continuously exceeds 200 milliseconds and the packet loss rate reaches 10% at the same time, the model may comprehensively determine it as a network anomaly and generate corresponding anomaly detection results. After the detection is completed, the anomaly detection results are output in a structured form, which may include the anomaly type (such as "network anomaly" or "equipment failure"), the degree of anomaly (such as "slight" or "severe"), and specific parameters (such as "latency value: 250 milliseconds").

[0060] Step 103, execute corresponding automated control measures according to the anomaly detection results to maintain the video conferencing function.

[0061] The automated control measures are processing means dynamically selected by the system according to the anomaly type and cause, mainly including a rule engine mechanism, a dynamic adjustment mechanism, and a disaster recovery switching mechanism. Specifically, the system first classifies the anomaly detection results to determine the anomaly type (such as network anomaly or equipment failure) and identifies the cause of the anomaly (such as high latency or camera failure). Subsequently, the system performs operations according to the automated processing mechanism. For example, if the anomaly detection result shows a network anomaly (such as the packet loss rate exceeding 5%), the system may reduce the audio-video parameters through the dynamic adjustment mechanism (such as reducing the bitrate from 1 Mbps to 512 kbps, or the resolution from 1080p to 720p) to reduce bandwidth occupancy and restore smoothness; if an equipment failure (such as microphone failure) is detected, the system automatically switches to the backup microphone or enables the audio-only mode through the disaster recovery switching mechanism.

[0062] In practical applications, assume that the camera of a certain conference terminal suddenly goes offline. The system will immediately detect this anomaly, generate a "device failure" result, and automatically call the backup camera to replace the faulty device, all without manual intervention. The execution of control measures depends on preset rules in the rule engine (such as "reduce the bit rate when the latency > 100ms") or dynamic adjustment logic (such as allocate resources according to the load) to ensure a quick response.

[0063] In addition, the system also supports disaster recovery switching. For example, when the network path becomes unavailable due to congestion, it automatically switches to the backup transmission path (such as switching from the public network to a dedicated line). The implementation of these measures not only maintains the audio and video transmission and communication connection of the video conference, but also optimizes resource utilization and ensures the stability of multi-terminal collaborative operation.

[0064] The control solution for the video conference provided by the embodiments of the present invention collects the conference operation data of the conference terminal in real time; detects the abnormal state of the conference terminal according to the conference operation data to obtain an anomaly detection result; and executes corresponding automated control measures according to the anomaly detection result to maintain the video conference function.

[0065] Compared with the background technology, the embodiments of the present invention have the following beneficial effects:

[0066] By collecting the conference operation data in real time and executing automated control measures, the cumbersome steps of manual configuration of the MCU and terminal control are eliminated, and the conference requirements (such as terminal joining or parameter adjustment) can be quickly responded to, significantly reducing the operation time and labor costs, and solving the problem of low efficiency of traditional methods. Detecting anomalies according to the conference operation data and executing corresponding automated control measures (such as reducing the bit rate to reduce bandwidth occupancy) realizes the dynamic allocation of resources. Compared with the resource waste caused by traditional manual control, it can optimize the network bandwidth, computing resources, and device usage rate according to real-time requirements, improving the overall resource utilization efficiency. Automatically detecting network anomalies (such as latency, packet loss) or device failures (such as camera failure), and quickly executing automated control measures (such as switching to a backup device) can quickly respond to anomalies without manual intervention, reducing the risk of conference interruption or quality degradation, and overcoming the defect of the lack of automatic response ability in traditional systems.

[0067] In an exemplary embodiment of the present invention, one implementation manner of detecting the abnormal state of the conference terminal according to the conference operation data to obtain an anomaly detection result is: extracting the feature data of at least one of the network data, device data, and audio and video stream data from the conference operation data; analyzing the feature data based on preset rules and / or preset models to obtain an anomaly detection result.

[0068] In this embodiment, feature data is extracted from the conference operation data, which includes three categories: network data, device data, and audio-video stream data. Network data refers to the parameters related to network transmission performance, such as latency (time delay in network transmission), packet loss rate (abbreviated as PLR), and bandwidth utilization, which reflect the quality of communication of the conference terminals. Device data involves the operating status of the conference terminal hardware, such as whether the camera is working properly, whether the microphone (abbreviated as MIC) captures a valid sound signal, and the output status of the speaker (abbreviated as SPK). Audio-video stream data includes the quality metrics of audio-video transmission, such as bit rate, frame rate, and resolution. The extracted feature data is the key part of these data. For example, the "average latency value" is extracted from network data, the "camera online status" is extracted from device data, or the "degree of frame rate drop" is extracted from audio-video stream data. Subsequently, these feature data are analyzed based on preset rules and / or preset models. The preset rules can be simple threshold judgments. For example, when the latency exceeds 100 milliseconds or the packet loss rate is higher than 5%, it is determined that there is a network anomaly; when the microphone signal strength is lower than a certain threshold, it is determined that there is a device failure. The preset model may adopt machine learning techniques, such as a support vector machine (abbreviated as SVM) or a neural network (abbreviated as NN) trained by historical data, to analyze complex anomaly patterns, such as identifying network jitter or intermittent device malfunctions. For example, in a certain conference, if the system extracts that the packet loss rate of a certain conference terminal reaches 10%, it can be determined as a network anomaly through the preset rules, and an anomaly detection result of "network anomaly, packet loss rate exceeds the standard" is generated. After the analysis is completed, the anomaly detection result is output in a structured form, which may include information such as the anomaly type (such as "network anomaly" or "device failure") and anomaly parameters (such as "packet loss rate: 10%").

[0069] This embodiment effectively solves the low-efficiency problem of anomaly detection relying on manual monitoring in traditional video conferencing systems through the dual steps of feature extraction and analysis. Among them, feature extraction enables the system to focus on key data, avoid the interference of irrelevant information, and improve the detection efficiency. The combination of preset rules and / or models provides diverse analysis means, which can not only quickly handle common anomalies but also deal with complex scenarios, enhancing the accuracy and adaptability of detection.

[0070] In an exemplary embodiment of the present invention, one implementation of performing corresponding automated control measures according to the anomaly detection result is as follows: Classify the anomaly detection result to obtain the anomaly type; Identify the corresponding anomaly cause according to the anomaly type; Execute the automated control measure corresponding to the anomaly cause according to the automated processing mechanism.

[0071] This implementation classifies the anomaly detection result to determine the anomaly type. The anomaly detection result contains the anomaly status information of the conference terminal, such as "network anomaly" or "device failure". The classification process is to analyze the specific parameters or features in the anomaly detection result and classify it into a clear anomaly type. For example, classifying "delay exceeding 100 milliseconds and packet loss rate higher than 5%" as "network anomaly", or classifying "camera offline" as "device failure". Next, identify the corresponding anomaly cause according to the anomaly type. This process aims to deeply explore the root cause of the anomaly. For example, if the anomaly type is "network anomaly", the anomaly cause may be "network congestion" or "insufficient bandwidth"; if the anomaly type is "device failure", the cause may be "camera hardware damage" or "microphone signal interruption". Finally, the system executes the automated control measure corresponding to the anomaly cause according to the automated processing mechanism. The automated processing mechanism may include a rule engine mechanism, a dynamic adjustment mechanism, and a disaster recovery switching mechanism. For example, if the anomaly cause is "network congestion", the system may reduce the bit rate of the audio and video stream from 1 Mbps to 512 kbps or adjust the resolution from 1080p to 720p through the dynamic adjustment mechanism to reduce bandwidth occupancy; if the anomaly cause is "camera hardware damage", the backup camera will be automatically enabled to replace the faulty device through the disaster recovery switching mechanism.

[0072] In an actual scenario, assume that the anomaly detection result of a certain conference terminal shows "packet loss rate (reaching 10%)", which is classified as "network anomaly", and the identified cause is "insufficient bandwidth". The system will automatically reduce the frame rate to 15 frames per second to ensure the smoothness of the conference.

[0073] This implementation significantly improves the intelligence level of anomaly handling through a structured process of classification, cause identification, and automated execution. Among them, classification processing and cause identification ensure the precise matching of control measures with anomalies, avoiding the inefficiency of blind adjustment. The implementation of the automated processing mechanism eliminates the delay of manual intervention, improving the response speed and system efficiency.

[0074] In an exemplary embodiment of the present invention, one implementation of executing the automated control measure corresponding to the anomaly cause according to the automated processing mechanism is as follows: Execute the automated control measure according to at least one of the rule engine mechanism, the dynamic adjustment mechanism, and the disaster recovery switching mechanism.

[0075] In this embodiment, the execution of the automated control measures depends on at least one of three mechanisms: the rule engine mechanism, the dynamic adjustment mechanism, and the disaster recovery switching mechanism. The rule engine mechanism is a way to automatically trigger control operations based on preset rules. For example, when the cause of the anomaly is "network latency exceeds 100 milliseconds", the rule engine may trigger the preset operation of "reducing the bit rate to 512 kbps" to quickly relieve network pressure. The dynamic adjustment mechanism dynamically optimizes resource allocation according to the real-time state of the conference terminal. For example, if the cause of the anomaly is "insufficient bandwidth", the system may automatically adjust the resolution of the audio-video stream from 1080p to 720p, or reduce the frame rate from 30 frames per second to 15 frames per second to reduce bandwidth occupancy and ensure smoothness. The disaster recovery switching mechanism enables the use of backup resources in case of severe anomalies. For example, when the cause of the anomaly is "camera hardware failure", the system will automatically switch to a backup camera, or when the current transmission path becomes unavailable due to "network congestion", it will switch to a backup path (such as switching from the public network to a dedicated line).

[0076] In practical applications, assume that the cause of the anomaly of a certain conference terminal is "packet loss rate as high as 10%". The system may first reduce the bit rate through the rule engine mechanism. If the problem is not alleviated, it will switch to a backup path with low latency through the disaster recovery switching mechanism. The selection of such mechanisms can be used alone (such as only adjusting parameters) or executed in combination (such as reducing the frame rate and switching devices simultaneously), depending on the nature and severity of the cause of the anomaly.

[0077] This embodiment forms a comprehensive anomaly handling system through the diverse combination of the rule engine mechanism, the dynamic adjustment mechanism, and the disaster recovery switching mechanism. Among them, the cooperation of multiple mechanisms improves the flexibility and coverage of handling, and can take optimal measures for different causes of anomalies (such as network or device problems). Automated execution eliminates the delay and errors of manual operations, and improves the response speed and system efficiency.

[0078] In an exemplary embodiment of the present invention, an implementation manner of executing the automated control measures according to the rule engine mechanism is: automatically triggering an automated control operation corresponding to the cause of the anomaly according to the preset processing rules.

[0079] In this embodiment, the rule engine mechanism is a processing method that automatically triggers control measures based on predefined logic. Its key component is the preset processing rules. These processing rules are pre-set condition-action pairs in the system, aiming to perform corresponding automated control operations for specific abnormal reasons. The abnormal reasons come from the aforementioned steps (such as "identifying the corresponding abnormal reason according to the abnormal type"), and may include "excessive network latency", "exceeding packet loss rate", "camera failure", or "microphone signal interruption", etc. The preset processing rules are defined in the "if-then" logic form. For example, "if the network latency exceeds 100 milliseconds, then reduce the bit rate to 512 kbps" or "if the microphone signal strength is lower than 30 decibels, then switch to the backup microphone". During the execution process, the system matches the abnormal reason with the conditions in the rule library. Once the match is successful, the corresponding control operation is automatically triggered.

[0080] For example, in a certain video conference, if it is detected that the packet loss rate of a conference terminal reaches 10%, the system automatically adjusts the frame rate from 30 frames per second to 15 frames per second according to the preset rule "reduce the frame rate when the packet loss rate > 5%", so as to reduce the network load and restore the smoothness of the conference. Similarly, if the abnormal reason is "camera offline", the rule may be "enable the backup camera when the camera is offline", and the system will immediately switch the device to ensure that the video transmission is not interrupted. The preset processing rules can be flexibly adjusted or extended according to actual needs. For example, new rules can be added to handle abnormal situations such as "insufficient bandwidth" or "no sound from the speaker", so as to improve the adaptability of the system.

[0081] This embodiment forms an efficient closed-loop for exception handling through automated control operations driven by preset processing rules. Among them, the clear logic of the preset rules ensures the accurate correspondence between the abnormal reason and the control operation, improving the processing efficiency and accuracy. The automatic trigger mechanism eliminates the delay of manual operation and significantly shortens the response time.

[0082] In an exemplary embodiment of the present invention, an implementation manner of performing automated control measures according to the dynamic adjustment mechanism is as follows: when the abnormal reason is related to the network state, the audio and video parameters are dynamically adjusted according to the network state; and / or, when the abnormal reason is related to the system load, the system resources are dynamically allocated according to the system load.

[0083] In this embodiment, the dynamic adjustment mechanism is a processing method that flexibly adjusts system operation parameters or resources based on real-time status, and is divided into two main scenarios. The first scenario is when the cause of the anomaly is related to the network status. The system dynamically adjusts the audio and video parameters according to the network status. The network status includes indicators such as latency, packet loss rate, and bandwidth utilization. If these indicators show anomalies (such as latency exceeding 100 milliseconds or packet loss rate higher than 5%), the system will automatically adjust the audio and video parameters. For example, it will reduce the bitrate from 1 Mbps to 512 kbps, or reduce the resolution from 1080p to 720p, or even adjust the frame rate from 30 frames per second to 15 frames per second to reduce the network load. For example, in a certain meeting, if the network status of a meeting terminal shows that the packet loss rate reaches 10%, the system may dynamically reduce the bitrate and frame rate to ensure the stability of the audio and video stream transmission. The second scenario is when the cause of the anomaly is related to the system load. The system dynamically allocates system resources according to the system load. The system load usually refers to the computing resource occupancy rate of the multipoint control unit or server. If the load is too high (such as CPU usage exceeding 80%), the system will migrate the tasks of some meeting terminals to the MCU with a lower load, or reallocate bandwidth resources to balance the operation. For example, if a certain MCU has a too high load due to processing too many terminals at the same time, the system may transfer the audio and video stream processing tasks of some terminals to the standby MCU.

[0084] This embodiment implements precise optimization for the abnormal causes of network status and system load through the dynamic adjustment mechanism, forming a flexible and efficient control system. Among them, adjusting the audio and video parameters according to the network status and allocating resources according to the system load ensure the efficient use of resources and the smoothness of the meeting. The dynamically adaptive processing method significantly improves the system's response speed and adaptability, and avoids the inefficiency of manual adjustment.

[0085] In an exemplary embodiment of the present invention, an implementation manner of executing an automatic control measure according to the disaster recovery switching mechanism is as follows: when the cause of the anomaly is a network reason, automatically switch the video conference transmission path to the standby path; and / or, when the cause of the anomaly is a device reason, automatically replace the faulty device with a standby device.

[0086] In this embodiment, the disaster recovery switching mechanism is a processing method for quickly restoring system functions in the event of a serious anomaly, and it is divided into two main scenarios. The first scenario is when the anomaly is due to network reasons, the system automatically switches the video conferencing transmission path to a backup path. Network reasons may include network congestion, excessive latency, or an excessive packet loss rate. For example, if the packet loss rate of the current path reaches 10% and continuously affects the audio and video stream transmission, the system will detect this anomaly and automatically switch to the backup path (such as switching from the public network to a dedicated line, or selecting a network node with lower latency) to restore the communication quality. For example, in a cross-regional meeting, if the main transmission path causes the video to freeze due to network congestion, the system can quickly switch to a pre-configured backup dedicated line to ensure that the meeting is not interrupted. The second scenario is when the anomaly is due to equipment reasons, the system automatically replaces the faulty equipment with a backup device. Equipment reasons may involve hardware failures, such as a camera being unable to capture images, a microphone being silent, or a speaker having no output. If the camera of a meeting terminal suddenly goes offline, the system will identify this anomaly and automatically activate the backup camera to replace the faulty device to maintain the continuous transmission of the video stream.

[0087] This embodiment provides a quick recovery solution for network and equipment anomalies through the disaster recovery switching mechanism, forming a strong fault response ability. Among them, automatically switching to the backup path or device significantly reduces the impact of anomalies on the continuity of the meeting. The quick execution without manual intervention improves the response efficiency and shortens the fault recovery time.

[0088] Based on the above related description of an embodiment of a control method for a video conference, a video conference MCU cascading and terminal intelligent control solution is introduced below. This video conference MCU cascading and terminal intelligent control solution is applied to a video conference system (hereinafter referred to as the system), aiming to achieve the automation and intelligence of MCU cascading and terminal control, improve system efficiency, optimize resource utilization, and enhance disaster recovery capabilities.

[0089] The system uses an XMCU (a specific type of multi-point control unit, which is used as a docking gateway role and is responsible for coordinating and managing the communication between multiple video conference terminals) as the docking gateway to achieve the cascading docking of the conference management system and the cloud communication MCU. The XMCU schedules multiple video conference system interfaces through middleware, including the sending and feedback of instructions such as meeting status query, group meeting, adding a terminal to the meeting, setting a speaker, removing a terminal, and ending a meeting. Before the meeting starts, the system initializes the connections of the chair terminal and multiple controlled terminals. When the states of both the chair terminal and the controlled terminals are idle, the communication connection is established. The XMCU calls all the interfaces of the cloud communication platform to support the meeting operation function, audio and video scheduling function, and information query function to ensure the integrity of the system docking.

[0090] Refer to Figure 2, showing a schematic flow chart of the steps of a video conferencing MCU cascading and terminal intelligent control solution according to an embodiment of the present invention.

[0091] Step 201, collect meeting operation data in real time.

[0092] During the meeting, the XMCU collects the meeting operation data of all meeting terminals in real time. The meeting operation data includes network data (such as latency, packet loss rate, bandwidth utilization), device data (such as the status of cameras, microphones, speakers), and audio-video stream data (such as bit rate, frame rate, resolution, audio-video synchronization status). For example, the network data of a certain meeting terminal shows an average latency of 50 milliseconds, a packet loss rate of 2%, and a bandwidth utilization of 70%; the device data indicates that the camera is online and the resolution is 1080p, and the signal-to-noise ratio of the microphone is 40 decibels; the audio-video stream data records a bit rate of 1 Mbps and a frame rate of 30 frames per second. These data are collected once per second through the data collection module and stored in the system cache, providing a basis for subsequent anomaly detection.

[0093] The XMCU, as the core monitoring unit of the system, is responsible for continuously tracking the operating status of all participating terminals. The monitoring content includes two aspects: network conditions and device status. To achieve this goal, each participating terminal is configured with a data collection and reporting function, specifically calculating key network parameters by sending and receiving data packets and regularly reporting the results to the XMCU. Specifically, the meeting terminal can package the monitoring content into a status report at fixed intervals (such as 1 second) and report it to the XMCU. The reported meeting operation data includes, but is not limited to, terminal identification, timestamp, and network parameter values, etc. In addition, the status of the meeting device is detected by the meeting terminal through the hardware interface, including whether the camera is online, whether the microphone is collecting sound normally, and whether the speaker is outputting effectively. These status information are also included in the reported data. For example, a certain meeting terminal may report "latency 50ms, jitter 5ms, packet loss rate 2%, camera online, microphone normal". Step 202, anomaly detection and result generation.

[0094] The XMCU detects the abnormal status of the conference terminal based on the collected or reported conference operation data and generates an abnormal detection result. In specific implementation, the system extracts feature data from the conference operation data: network features (such as average latency, packet loss rate, jitter), device features (such as camera online status, microphone signal strength), and audio-video features (such as bitrate fluctuation, frame rate drop). Analysis is performed based on preset rules and machine learning models. The preset rules include: determining network abnormality when the latency > 100ms or the packet loss rate > 5%, and determining device failure when the camera is offline or the microphone signal-to-noise ratio < 30dB. The machine learning model (such as random forest or neural network) is trained with historical data to identify complex abnormal patterns. For example, when the latency of a certain terminal rises to 250 milliseconds and the packet loss rate reaches 10%, the system determines it as "network abnormality" based on the rules and generates the result "network abnormality, latency 250ms, packet loss rate 10%"; when the camera resolution of another terminal drops below 720p, it is determined as "device failure", and the result is "device failure, camera abnormality".

[0095] If the XMCU does not receive the conference operation data from the conference terminal at regular intervals, it can be determined that the conference terminal is in an abnormal state, which can specifically be abnormal states such as network abnormality or device failure. For example, the conference terminal should report the conference operation data at a fixed period (such as every second). If the XMCU does not receive the report from a certain conference terminal within the specified time (such as 3 seconds), it may be that the conference terminal is disconnected, there is a power failure, or the network is completely interrupted. The XMCU will initiate an active detection. If there is still no response, it is initially determined as a device failure.

[0096] Step 203, execute automated control measures.

[0097] Based on the abnormal detection result, the XMCU executes corresponding automated control measures, which are divided into three stages: classification, cause identification, and measure execution. First, classify the abnormal detection result to determine the abnormal type (such as "network abnormality" or "device failure"). Second, identify the cause of the abnormality according to the abnormal type. For example, the cause of "network abnormality" may be "network congestion", and the cause of "device failure" may be "camera failure". Finally, execute measures according to the automated processing mechanism, including the rule engine mechanism, dynamic adjustment mechanism, and disaster tolerance switching mechanism.

[0098] Rule engine mechanism: Automatically trigger operations according to preset processing rules. For example, when the cause of the abnormality is "network latency > 100ms", the rule triggers to reduce the bitrate to 512kbps; when "camera failure" occurs, the rule enables the backup camera.

[0099] Dynamic adjustment mechanism: In a video conferencing system, Quality of Service (QoS) refers to a comprehensive metric of the consistency, reliability, and user experience provided by the network and system during audio and video transmission. QoS is typically used to measure network performance (such as latency, jitter, packet loss rate) and audio and video quality (such as clarity, smoothness, synchronization), and directly affects the communication effect of the conference terminal. The QoS audio and video quality table is a pre-set reference table that correlates network status, audio and video parameters with quality levels, and is used to guide parameter adjustment. The QoS audio and video quality table may include but is not limited to: input parameters, including network condition metrics such as latency, jitter, packet loss rate, etc.; output parameters, including audio and video adjustment targets such as bit rate, resolution, frame rate, etc.; quality levels, including QoS levels in different scenarios such as high level, medium level, low level, etc.

[0100] The XMCU extracts the current network parameters from the conference operation data of the conference terminal. For example, the latency is 150ms and the packet loss rate is 8%. The XMCU dynamically adjusts the audio and video parameters according to the QoS (Quality of Service) audio and video quality table, such as reducing the resolution from 1080p to 720p and the frame rate from 30 frames per second to 15 frames per second. When the exception reason is "system overload" (such as the MCU load exceeding 80%), resources are dynamically allocated and some terminals are migrated to an MCU with a lower load.

[0101] Disaster recovery switching mechanism: When the exception reason is "network congestion", the system automatically switches the transmission path from the public network to a dedicated line or a candidate network; when the "microphone fails", it automatically switches to a backup microphone or enables a separate audio mode.

[0102] In the system, the XMCU obtains the audio and video switching instruction (such as "set speaker") sent by the chair terminal, and sends an instruction to the central control system through the central control interface according to the scheduling result. The central control system controls the video matrix and the audio matrix through the video matrix interface and the audio matrix interface, and outputs the audio and video code streams of the specified terminal to the target screen. For example, the video code stream of the chair terminal is input into the video matrix through the video matrix interface, and the audio code stream is input into the audio matrix through the audio matrix interface, and finally output to the large screen.

[0103] It should be noted that for the method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0104] Reference Figure 3 , a structural block diagram of a control system for a video conference according to an embodiment of the present invention is shown. The control system for the video conference may specifically include the following modules.

[0105] A data acquisition module 31, configured to collect the meeting operation data of the meeting terminal in real time;

[0106] An anomaly detection module 32, configured to detect the abnormal state of the meeting terminal according to the meeting operation data to obtain an anomaly detection result;

[0107] An automatic control module 33, configured to execute corresponding automatic control measures according to the anomaly detection result to maintain the video conference function.

[0108] In an exemplary embodiment of the present invention, the anomaly detection module 32 includes:

[0109] A feature data extraction module, configured to extract feature data of at least one of network data, device data, and audio-video stream data from the meeting operation data;

[0110] A feature data analysis module, configured to analyze the feature data based on a preset rule and / or a preset model to obtain the anomaly detection result.

[0111] In an exemplary embodiment of the present invention, the automatic control module 33 includes:

[0112] A detection result classification module, configured to perform classification processing on the anomaly detection result to obtain an anomaly type;

[0113] An anomaly cause identification module, configured to identify a corresponding anomaly cause according to the anomaly type;

[0114] A control measure execution module, configured to execute the automatic control measure corresponding to the anomaly cause according to an automatic processing mechanism.

[0115] In an exemplary embodiment of the present invention, the control measure execution module is configured to execute the automatic control measure according to at least one of a rule engine mechanism, a dynamic adjustment mechanism, and a disaster tolerance switching mechanism.

[0116] In an exemplary embodiment of the present invention, the control measure execution module includes:

[0117] A rule engine module, configured to automatically trigger an automatic control operation corresponding to the anomaly cause according to a preset processing rule.

[0118] In an exemplary embodiment of the present invention, the control measure execution module includes:

[0119] A dynamic adjustment module, configured to dynamically adjust audio and video parameters according to the network status when the abnormal cause is related to the network status; and / or, when the abnormal cause is related to the system load, dynamically allocate system resources according to the system load.

[0120] In an exemplary embodiment of the present invention, the control measure execution module includes:

[0121] A disaster recovery switching module, configured to automatically switch the video conference transmission path to a standby path when the abnormal cause is a network reason; and / or, when the abnormal cause is a device reason, automatically replace the faulty device with a standby device.

[0122] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, please refer to the partial description of the method embodiment.

[0123] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts among the embodiments can be referred to each other.

[0124] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0125] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0126] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one process Figure 1 or more processes and / or blocks Figure 1 or more blocks specified in one block or more blocks.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operational steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 or more processes and / or blocks Figure 1 or more blocks specified in one block or more blocks.

[0128] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0129] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0130] The above has introduced in detail a method for controlling a video conference and a control system for a video conference provided by the present invention. Specific examples are used in this text to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A control method for a video conference, characterized in that, The method includes: Collecting the meeting operation data of the meeting terminal in real time; Detecting the abnormal state of the meeting terminal according to the meeting operation data to obtain an abnormal detection result; Performing corresponding automatic control measures according to the abnormal detection result to maintain the video conferencing function.

2. The method according to claim 1, characterized in that, The detecting the abnormal state of the meeting terminal according to the meeting operation data to obtain an abnormal detection result includes: Extracting the feature data of at least one of network data, device data, and audio-video stream data from the meeting operation data; Analyzing the feature data based on preset rules and / or a preset model to obtain the abnormal detection result.

3. The method according to claim 1, wherein The performing corresponding automatic control measures according to the abnormal detection result includes: Performing classification processing on the abnormal detection result to obtain an abnormal type; Identifying the corresponding abnormal cause according to the abnormal type; Executing the corresponding automatic control measure according to the abnormal cause according to the automatic processing mechanism.

4. The method according to claim 3, characterized in that The executing the corresponding automatic control measure according to the abnormal cause according to the automatic processing mechanism includes: Executing the automatic control measure according to at least one of a rule engine mechanism, a dynamic adjustment mechanism, and a disaster recovery switching mechanism.

5. The method according to claim 4, characterized in that The executing the automatic control measure according to the rule engine mechanism includes: Automatically triggering an automatic control operation corresponding to the abnormal cause according to a preset processing rule.

6. The method according to claim 4, wherein The executing the automatic control measure according to the dynamic adjustment mechanism includes: When the abnormal cause is related to the network state, dynamically adjusting the audio-video parameters according to the network state; And / or, when the abnormal cause is related to the system load, dynamically allocating system resources according to the system load.

7. The method according to any one of claims 4 to 6, characterized in that The executing the automatic control measure according to the disaster recovery switching mechanism includes: When the abnormal cause is a network reason, automatically switching the video conferencing transmission path to a backup path; And / or, when the abnormal cause is a device reason, automatically replacing the faulty device with a backup device.

8. A control system for a video conference, characterized in that, The system includes: A data collection module for collecting the meeting operation data of the meeting terminal in real time; An abnormal detection module for detecting the abnormal state of the meeting terminal according to the meeting operation data to obtain an abnormal detection result; An automatic control module for performing corresponding automatic control measures according to the abnormal detection result to maintain the video conferencing function.

9. An electronic device, characterized in that, Includes: One or more processors; And One or more machine-readable media storing instructions, which when executed by the one or more processors, cause the electronic device to execute the control method of the video conference according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program stored therein causes the processor to execute the control method of the video conference according to any one of claims 1 to 7.

Images