Video conferencing quality monitoring method, device, server, electronic equipment, and storage medium based on virtual terminal polling

By creating virtual terminal instances with signaling connection, bitstream reception, and decoding capabilities, the problem of monitoring audio and video quality after decoding in video conferencing is solved, enabling timely detection and location of quality anomalies without adding hardware, modifying the network, or relying on terminal modifications.

CN123123556APending Publication Date: 2026-11-17SHANGHAI CHUANGSHITONG SOFTWARE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611189131.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-06
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively monitor the audio and video quality of each participant in a video conference without adding hardware, modifying the existing network, or relying on changes to the participants' terminals. In particular, they are unable to detect and locate application-layer quality anomalies such as black screens, distorted screens, frozen images, mute, or intermittent audio.

Method used

Create virtual terminal instances with video conferencing signaling connectivity, audio and video stream reception, and decoding capabilities. Control these instances to join the target conference as participants. Through the media stream control mechanism of the conference server, sequentially receive and decode the audio and video streams of each participant, perform quality checks, and generate quality monitoring results.

Benefits of technology

It enables effective monitoring of the audio and video quality of each participant in a video conference after decoding, and can promptly detect and locate quality anomalies, reducing reliance on manual troubleshooting, and eliminating the need for hardware probes, network modifications, and terminal modifications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123556A_ABST
    Figure CN123123556A_ABST
Patent Text Reader

Abstract

This application provides a video conferencing quality monitoring method, apparatus, server, electronic device, and storage medium based on virtual terminal polling, relating to the field of video conferencing technology. The method includes: creating a virtual terminal instance dedicated to quality monitoring; controlling the virtual terminal instance to join a target conference organized by a conference server as a participant; obtaining a list of participants in the target conference; invoking the media stream control mechanism of the conference server; iterating through the participant list according to a preset polling strategy; sequentially designating each participant as the receiving target and receiving their audio and video streams; decoding and quality testing the streams to obtain quality indicators for each participant; and generating quality monitoring results. This application can monitor the decoded audio and video quality of each participant without adding hardware acquisition equipment or modifying the conference network and participant terminals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of video conferencing technology, and more specifically, to a method, apparatus, server, electronic device, and storage medium for monitoring video conferencing quality based on virtual terminal polling. Background Technology

[0002] Video conferencing systems typically connect multiple participants through a conferencing server and forward, mix, or perform other media processing on the audio and video data. As the scale of the meeting increases, the network environment, acquisition equipment, encoding equipment, and access methods of each participant may differ, potentially leading to quality anomalies such as black screens, distorted images, frozen screens, muted audio, noise, or intermittent playback. Some of these anomalies are related to network transmission status, others to the acquisition or encoding status at the conferencing site, and still others are only detected when the audio and video are actually displayed.

[0003] One method for monitoring video conferencing quality involves querying meeting statistics through the management interface of the meeting server. This includes information such as participant online status, bandwidth, packet loss rate, jitter, and connection status. The meeting's operational status is then determined based on these statistics. This method does not require additional hardware probes or port mirroring or optical splitting modifications to the meeting network, resulting in relatively low implementation costs.

[0004] However, the conference server management interface typically returns connection status and transmission layer statistics. While these data can reflect whether participants are online and whether there is packet loss or jitter in the link, they are difficult to directly reflect the actual presentation status of the video and audio during the meeting. When a participant experiences camera obstruction, malfunction of the acquisition device, abnormal encoding output, prolonged video freeze, microphone malfunction, or intermittent audio interruption, the online status, bandwidth, or packet loss rate statistics on the conference server side may not show obvious abnormalities, making it difficult to promptly ascertain the actual audio and video quality status of that participant.

[0005] Another quality monitoring method relies on each participant's terminal to collect and report quality data. This requires installing collection components, opening local interfaces, or performing software adaptation on the actual participating terminals. When the meeting is large-scale, the types of participating terminals are complex, the terminal management permissions are not uniform, or there are temporary accesses from external terminals, it is difficult to implement the method of deploying collection components on each participating terminal individually. Moreover, the collected parameters mainly reflect the terminal device status and network conditions, and cannot directly reflect the quality of the decoded audio and video presentation.

[0006] Therefore, without adding hardware, modifying the existing network architecture, or relying on changes to the participating terminals, it is difficult to effectively monitor the audio and video quality of each participant in a video conference. When application-layer quality anomalies such as black screens, distorted screens, frozen images, mute, or intermittent audio occur during a meeting, the discovery and location of these anomalies still rely heavily on manual investigation. Anomaly discovery is delayed, historical records are lacking, and post-event investigations lack data support.

[0007] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The purpose of this application is to provide a video conferencing quality monitoring method, device, server, electronic device, and computer-readable storage medium based on virtual terminal polling, so as to monitor the audio and video quality status of each participant in the video conference without adding hardware devices for conference quality acquisition, without modifying the existing conference network, and without relying on the modification of the participating terminals.

[0009] According to a first aspect of the embodiments of this application, a video conferencing quality monitoring method based on virtual terminal polling is provided, the method comprising: Create a virtual terminal instance, which is dedicated to quality monitoring and has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability; Control the virtual terminal instance to join the target meeting organized by the meeting server as a participant; Obtain the list of participants in the target meeting, and call the media stream control mechanism of the meeting server according to the list of participants. According to the preset polling strategy, traverse the list of participants in a loop, and sequentially designate the receiving target of the virtual terminal instance as each participant in the list of participants, and sequentially receive the audio and video streams corresponding to the currently designated participants. The audio and video streams are decoded and their quality is tested to obtain the quality indicators corresponding to each participant. Based on the quality indicators corresponding to each participant, the quality monitoring results of the target meeting are generated.

[0010] In one exemplary embodiment of this application, receiving the audio and video stream corresponding to the currently designated participant includes: when the receiving target of the virtual terminal instance is designated as the currently designated participant, receiving the uplink audio and video stream sent by the currently designated participant to the conference server by the conference server based on the designation result of the media stream control mechanism, as the audio and video stream corresponding to the currently designated participant.

[0011] In one exemplary embodiment of this application, the audio / video stream includes at least one of an audio stream or a video stream.

[0012] In one exemplary embodiment of this application, obtaining the list of participants for the target meeting includes: after the meeting is established, calling the meeting query interface of the meeting server, and / or receiving the list of participants sent by the meeting server to the virtual terminal instance, and obtaining the complete list of participants for the target meeting as the basis list for subsequent polling.

[0013] In one exemplary embodiment of this application, the method further includes: determining the number of virtual terminal instances required to complete the polling monitoring of the target meeting based on the number of participants in the target meeting, the monitoring duration required for a single participant, and the maximum allowed time configured by the user to complete a round of full participant monitoring; if the determined number of virtual terminal instances is greater than 1, creating multiple virtual terminal instances and controlling the multiple virtual terminal instances to join the target meeting as participants; dividing the participants in the participant list into multiple participant groups; and controlling the multiple virtual terminal instances to perform receiving target specification, audio and video stream reception, decoding, and quality detection for the corresponding participant groups.

[0014] In one exemplary embodiment of this application, determining the number of virtual terminal instances required to complete the polling monitoring of the target meeting includes: according to the formula Determine the number of virtual terminal instances, where N is the number of participants, t is the monitoring duration required for a single participant, and T is the maximum allowed time configured by the user to complete one round of monitoring of all participants. This indicates rounding up to the nearest integer.

[0015] In one exemplary embodiment of this application, controlling the plurality of virtual terminal instances to perform receiving target specification, audio and video stream reception, decoding, and quality detection on the corresponding participant groups includes at least one of the following: setting different polling start times for each virtual terminal instance to achieve time staggering; performing load balancing allocation based on the historical quality data of each participant; and prioritizing the allocation of VIP participants to virtual terminal instances with remaining computing resources higher than a preset threshold.

[0016] In one exemplary embodiment of this application, the method further includes: formulating a dynamic polling strategy based on the participant list, according to at least one of the following factors: participant priority, historical quality results, and meeting status; sequentially specifying receiving targets according to the dynamic polling strategy, receiving and decoding the bitstream through the media stream control mechanism, performing quality detection on the decoded data and recording indicators; and dynamically adjusting subsequent polling strategies based on the detection results.

[0017] In one exemplary embodiment of this application, the dynamic polling strategy includes at least one of the following: a priority-based polling strategy, which determines the priority based on the participant's VIP identifier, meeting role, and device type, and allocates the monitoring frequency according to the priority; a historical quality-based polling strategy, which dynamically adjusts the monitoring frequency based on historical quality scores; and an event-triggered polling strategy, which triggers a continuous monitoring mode when a network event, quality event, or meeting event is detected.

[0018] In one exemplary embodiment of this application, the decoding and quality detection of the audio and video stream includes: decoding the audio and video stream to obtain original video frames and / or Pulse Code Modulation (PCM) audio data; and performing quality detection on the original video frames and / or PCM audio data to obtain quality indicators.

[0019] In one exemplary embodiment of this application, the quality detection includes at least one of the following: traditional audio and video anomaly detection, including black screen detection, screen distortion detection, freeze detection, silence detection, noise detection, and audio and video synchronization detection; quality evaluation based on traditional signal processing, including objective quality evaluation based on pixel differences or structural similarity; and perceptual quality evaluation based on artificial intelligence and deep learning, including non-referenced or referenced perceptual quality evaluation based on neural networks.

[0020] In one exemplary embodiment of this application, the media stream control mechanism includes at least one of the following: a media stream control interface, a subscription mechanism, a selection mechanism, a forwarding mechanism, a routing mechanism, and a media switching mechanism.

[0021] In one exemplary embodiment of this application, after completing the quality detection of the currently designated participant, the signaling connection state of the virtual terminal instance in the target conference is maintained, and the receiving target is designated as the next participant through the media flow control mechanism.

[0022] In one exemplary embodiment of this application, the method further includes: when the target conference uses encrypted transmission, completing identity authentication and encryption parameter negotiation with the conference server through a pre-configured certificate, a pre-shared key, or a standard key negotiation mechanism; decrypting the encrypted audio and video streams received by the virtual terminal instance, and performing the decoding and quality detection on the decrypted audio and video streams.

[0023] In one exemplary embodiment of this application, the step of invoking the media stream control mechanism of the conference server includes: invoking the media stream control mechanism of the conference server through an abstract adaptation layer, wherein the abstract adaptation layer defines a unified adaptation interface specification and includes multiple adapters corresponding to different types of conference servers; loading the corresponding adapter according to the type of conference server corresponding to the target conference, and converting the call based on the unified adaptation interface specification into an interface call supported by the conference server through the loaded adapter.

[0024] According to a second aspect of the embodiments of this application, a video conferencing quality monitoring device based on virtual terminal polling is provided. The device is a collection of logical functional modules deployed on a monitoring server, including: The virtual terminal module is used to instantiate and maintain the lifecycle of virtual terminal instances. The virtual terminal instances are dedicated to quality monitoring and have video conferencing signaling connection capabilities, audio and video stream receiving capabilities, and audio and video stream decoding capabilities. The management module is used to control virtual terminal instances to join the target meeting established by the meeting server and to obtain the list of participants in the target meeting; The polling control module is used to traverse the participant list in a loop according to a preset polling strategy, and sequentially designate each participant in the participant list as the receiving target of the virtual terminal instance. The media control module is used to invoke the media stream control mechanism of the conference server, so that the conference server sends the audio and video stream of the currently designated participant to the virtual terminal instance through the media channel, and the virtual terminal instance receives the audio and video stream corresponding to the currently designated participant. The quality inspection module is used to decode and inspect the distributed audio and video streams to obtain the corresponding quality indicators for each participant. The results aggregation module is used to generate quality monitoring results for the target meeting based on the quality indicators corresponding to each participant.

[0025] In one exemplary embodiment of this application, the apparatus further includes: a virtual terminal scheduling module, configured to determine the required number of virtual terminal instances, the determination including: dynamically calculating based on the number of participants in the target meeting, the monitoring duration required for a single participant, and the maximum allowed time configured by the user to complete a full round of monitoring of all participants, or using a preset number; when the number of virtual terminal instances is greater than 1, triggering the virtual terminal module to dynamically create multiple virtual terminal instances, and dividing the participants in the participant list into multiple participant groups, so as to allocate them to the corresponding virtual terminal instances for parallel monitoring.

[0026] According to a third aspect of the embodiments of this application, a video conferencing quality monitoring server is provided. The server is a dedicated monitoring server independent of the conferencing server and is connected to an external conferencing server via a standard network protocol. The server includes: one or more processors; a memory coupled to the processors; and a network interface. The memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of the first aspects.

[0027] In one exemplary embodiment of this application, the server is deployed with a virtual container runtime environment that supports the batch creation and destruction of virtual terminal instances.

[0028] In an exemplary embodiment of this application, when the processor executes the computer program, it further implements: dynamically calculating, or using a preset number, the required number of virtual terminal instances based on the number of participants in the target meeting, the monitoring time required for a single participant, and the maximum allowed time for completing a full round of participant monitoring configured by the user; when the number of virtual terminal instances is greater than 1, dynamically creating multiple virtual terminal instances, and dividing the participants in the participant list into multiple participant groups to be assigned to the corresponding virtual terminal instances for parallel monitoring.

[0029] According to a fourth aspect of the embodiments of this application, an electronic device is provided, comprising: a processor; and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the video conferencing quality monitoring method based on virtual terminal polling as described in any one of the first aspects.

[0030] According to a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the video conferencing quality monitoring method based on virtual terminal polling as described in any one of the first aspects.

[0031] The technical solutions provided in this application embodiment may include the following beneficial effects: The video conferencing quality monitoring method based on virtual terminal polling in the example embodiments of this application creates a virtual terminal instance dedicated to quality monitoring, possessing video conferencing signaling connection capabilities, audio and video stream reception capabilities, and audio and video stream decoding capabilities. This virtual terminal instance is then controlled to join the target conference organized by the conference server as a participant. This allows the quality monitoring process to utilize the target conference's existing conference access mechanism to enter the conference processing flow, without requiring hardware probes to collect conference quality data, bypassing packet capture or port mirroring modifications to the existing conference network, or adding collection components or opening collection interfaces on the actual participant terminals. Furthermore, it obtains the target conference... The system retrieves a list of participants for the meeting and invokes the media stream control mechanism of the meeting server according to the list. It iterates through the list of participants and sequentially assigns the receiving target of each virtual terminal instance to each participant. This allows the virtual terminal instance to receive the uplink audio and video streams sent to the meeting server by each participant in a time-sharing manner, thereby obtaining the actual audio and video content corresponding to each participant, which is different from the statistical data on the meeting server side. The system then decodes and performs quality testing on the received audio and video streams to obtain the quality indicators corresponding to each participant. Based on these indicators, it generates the quality monitoring results of the target meeting, ensuring that the quality monitoring results reflect the actual quality status of the corresponding picture and / or sound of each participant after decoding. Therefore, it is possible to effectively monitor the audio and video quality status of each participant in a video conference without adding hardware equipment for conference quality acquisition, without modifying the existing conference network, and without relying on the modification of the participant's terminal. Furthermore, when application layer quality anomalies such as black screen, distorted screen, frozen screen, mute, or intermittent audio occur during the conference, the anomaly can be attributed to the specific participant based on the corresponding quality indicators of each participant, reducing the reliance on manual feedback and post-event investigation.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0034] Figure 1 The schematic diagram illustrates a system architecture diagram according to some embodiments of this application.

[0035] Figure 2 The flowchart illustrates a video conferencing quality monitoring method based on virtual terminal polling according to some embodiments of this application.

[0036] Figure 3 A schematic diagram of a virtual terminal polling flowchart according to some embodiments of this application is shown.

[0037] Figure 4 The illustration shows a schematic diagram of a multi-virtual terminal collaboration mechanism according to some embodiments of this application.

[0038] Figure 5 A schematic diagram of a multipoint control unit adapter layer architecture according to some embodiments of this application is shown.

[0039] Figure 6 A schematic diagram illustrating an encrypted conferencing support architecture according to some embodiments of this application is provided.

[0040] Figure 7 A flowchart illustrating the stream switching stability determination process according to some embodiments of this application is shown.

[0041] Figure 8 The diagram illustrates a structural block diagram of a video conferencing quality monitoring device based on virtual terminal polling according to some embodiments of this application.

[0042] Figure 9 The schematic diagram illustrates the structural schematic of a computer system of an electronic device according to some embodiments of this application.

[0043] Figure 10 A schematic diagram of a computer-readable storage medium according to some embodiments of this application is shown.

[0044] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0046] Furthermore, the accompanying drawings are for illustrative purposes only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0047] In this example embodiment, a video conferencing quality monitoring method based on virtual terminal polling is first provided. This method can be executed by an electronic device, which can deploy a video conferencing quality monitoring system. Specifically, the electronic device can be a workstation, server, private cloud node, cluster computing platform, or other computing device with capabilities for video conferencing access, virtual terminal operation, media stream control, audio and video decoding, and quality detection. The electronic device can be configured with a processor, memory, and program instructions for performing video conferencing quality monitoring tasks. It can be further configured with a virtual terminal management environment, a conference access environment, a polling control environment, a stream receiving and decoding environment, a quality detection environment, and a result generation environment. These environments are used to create virtual terminal instances, control the virtual terminal instances to join the target conference organized by the conference server as participants, obtain the list of participants in the target conference, and invoke the media stream control mechanism of the conference server according to the participant list. Following a preset polling strategy, the participant list is iterated through, and each participant in the list is designated as the receiving target of the virtual terminal instance. The audio and video streams corresponding to the currently designated participants are received sequentially, decoded, and quality-detected to obtain the quality indicators corresponding to each participant. Finally, the quality monitoring results of the target conference are generated based on the quality indicators corresponding to each participant.

[0048] The virtual terminal management environment can be used to create virtual terminal instances with video conferencing signaling connectivity, audio and video stream reception, and audio and video stream decoding capabilities. The conference access environment can be used to control virtual terminal instances to join the target conference as participants. The polling control environment can be used to obtain the list of participants in the target conference and, according to the participant list, invoke the media stream control mechanism of the conference server, iterating through the participant list according to a preset polling strategy, and sequentially designating each participant in the list as the receiving target for the virtual terminal instance. The stream reception and decoding environment can be used to receive the audio and video streams corresponding to the currently designated participant and decode them. The quality detection environment can be used to perform quality detection on the decoded video frames and / or audio data to obtain quality indicators for each participant. The result generation environment can be used to generate quality monitoring results for the target conference based on the quality indicators for each participant.

[0049] Figure 1 The schematic illustration shows the architecture of a video conferencing quality monitoring system according to some embodiments of this application. References Figure 1As shown, the video conferencing quality monitoring system may include multiple participant terminals 110-1, 110-2, ..., 110-N, a conference server 120, and a monitoring server 130, where N is a positive integer. The conference server 120 may be a multipoint control unit (MCU) or a selective forwarding unit (SFU) used to establish the target conference and perform conference control and media stream processing on each participant terminal accessing the target conference.

[0050] Each participating terminal 110-1, 110-2, ..., 110-N communicates with the conference server 120. Each participating terminal can collect audio and / or video data from its corresponding participant, encode the collected audio and / or video data, and send the encoded uplink audio and video streams to the conference server 120. The conference server 120 receives the uplink audio and video streams sent by each participating terminal and forwards, selects, or performs other media processing on the uplink audio and video streams according to the media processing method of the target conference.

[0051] The monitoring server 130 can be set up independently of the conference server 120, but can communicate with the conference server 120. The monitoring server 130 can run a virtual terminal instance 131 and is equipped with a quality detection module 132. The virtual terminal instance 131 has video conferencing signaling connection capabilities, conference query capabilities, media stream control capabilities, audio and video stream reception capabilities, and audio and video stream decoding capabilities. The quality detection module 132 is used to perform quality detection on the audio and video streams received by the virtual terminal instance 131.

[0052] In some embodiments, the monitoring server 130 controls the virtual terminal instance 131 to join the target meeting organized by the conference server 120 as a participant. After the virtual terminal instance 131 establishes a signaling connection with the conference server 120, it can call the conference query mechanism of the conference server 120 to obtain the list of participants in the target meeting, and determine the participants that need to be monitored based on the list of participants.

[0053] Virtual terminal instance 131 iterates through the participant list according to a preset polling strategy and sends a media stream control command to the conference server 120 to designate the receiving target of virtual terminal instance 131 as the participant currently to be monitored. Based on the media stream control command, the conference server 120 determines the audio and video stream corresponding to the currently designated participant from the uplink audio and video streams sent by each participant's terminal, and then sends the audio and video stream corresponding to the currently designated participant to virtual terminal instance 131.

[0054] After receiving the audio and video stream corresponding to the currently designated participant, virtual terminal instance 131 transmits the audio and video stream to quality detection module 132. Quality detection module 132 decodes the audio and video stream and performs quality detection on the decoded original video frames and / or audio data to obtain the quality index corresponding to the currently designated participant. The quality index may include at least one of the following: black screen detection result, distorted screen detection result, frozen screen detection result, silent screen detection result, noise detection result, audio and video synchronization detection result, and perceived quality score.

[0055] After completing the quality check of the currently designated participant, the virtual terminal instance 131 maintains the signaling connection with the conference server 120 and switches the receiving target to the next participant in the participant list. The conference server 120 then sends the audio and video streams corresponding to the next participant to the virtual terminal instance 131. By repeatedly executing the receiving target designation, audio and video stream reception, and quality check, the quality indicators corresponding to each participant can be obtained sequentially, and the quality monitoring results of the target conference can be generated accordingly. Therefore, without deploying quality acquisition components on the terminal side of each participant, the decoded audio and video quality status of each participant can be monitored based on the audio and video streams actually sent to the conference server 120.

[0056] Figure 2 The illustration schematically depicts a flowchart of a video conferencing quality monitoring method based on virtual terminal polling according to some embodiments of this application. (Reference) Figure 2 As shown, the video conferencing quality monitoring method based on virtual terminal polling may include the following steps: Step S210: Create a virtual terminal instance. The virtual terminal instance has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability. Step S220: Control the virtual terminal instance to join the target meeting established by the meeting server as a participant; Step S230: Obtain the list of participants in the target meeting, and call the media stream control mechanism of the meeting server according to the list of participants. According to the preset polling strategy, traverse the list of participants in a loop, and designate the receiving target of the virtual terminal instance as each participant in the list of participants in turn, and receive the audio and video streams corresponding to the currently designated participants in turn. Step S240: Decode and perform quality testing on the audio and video streams to obtain the quality indicators corresponding to each participant, and generate the quality monitoring results of the target meeting based on the quality indicators corresponding to each participant.

[0057] According to the video conferencing quality monitoring method based on virtual terminal polling in this example embodiment, a virtual terminal instance with video conferencing signaling connection capability, audio and video stream reception capability, and audio and video stream decoding capability is created. This virtual terminal instance is then controlled to join the target conference organized by the conference server as a participant. This allows the quality monitoring process to utilize the existing conference access mechanism of the target conference to enter the conference processing flow, without needing to collect conference quality data through hardware probes, without needing to modify the existing conference network for bypass packet capture, and without needing to add collection components or open collection interfaces on the actual participant terminal side. The method continues to obtain the participant list of the target conference and calls the media stream control mechanism of the conference server according to the participant list. Following a preset polling strategy, the participant list is traversed in a loop, and the virtual terminal instance is sequentially designated as the receiving target. For each participant, virtual terminal instances can receive the audio and video streams corresponding to each participant. The received audio and video streams are then decoded and their quality checked to obtain quality indicators for each participant. Based on these indicators, quality monitoring results for the target meeting are generated, reflecting the actual quality of the video and / or audio for each participant. This allows for effective monitoring of the audio and video quality of each participant in a video conference without adding hardware for meeting quality acquisition, modifying the existing conference network, or altering the participant terminals. Furthermore, when application-layer quality anomalies such as black screens, distorted screens, frozen images, mute, or intermittent audio occur during the meeting, the participants involved can be identified based on their respective quality indicators, reducing reliance on manual feedback and post-event investigation.

[0058] In some embodiments, the virtual terminal instance joining the target conference, invoking the media stream control mechanism of the conference server to specify the receiving target, and decoding and quality checking the audio and video streams can constitute a continuous quality monitoring chain. After the virtual terminal instance joins the target conference as a participant, it can access the target conference and receive media data without modifying the real participant terminal. The conference server, based on the result of the media stream control mechanism call, sends the audio and video streams corresponding to the currently specified participant to the virtual terminal instance, enabling the virtual terminal instance to obtain the audio and video content corresponding to each participant in a time-sharing manner according to the participant list. The electronic device further decodes and performs quality checks on the received audio and video streams, and associates the obtained quality indicators with the corresponding participants. Through the above processing, a continuous processing flow from conference access and stream acquisition to quality detection and result generation can be formed, thereby monitoring the audio and video quality status of each participant in the target conference.

[0059] The video conferencing quality monitoring method based on virtual terminal polling in this example embodiment will be further explained below.

[0060] Step S210: Create a virtual terminal instance. The virtual terminal instance has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability.

[0061] Virtual terminal polling refers to a processing method that sequentially receives the target data through virtual terminal instances to obtain the audio and video streams corresponding to different participants in the target conference in a time-sharing manner. Video conference quality monitoring refers to the process of detecting the audio and video data corresponding to the participants in the video conference and judging the audio and video quality status of the conference based on the detection results.

[0062] A virtual terminal instance refers to a terminal instance created by an electronic device for accessing video conferencing and performing quality monitoring tasks. This virtual terminal instance may not correspond to a physical conference terminal in an actual meeting room, but rather serve as an execution object for accessing the target conference and receiving audio and video streams. The electronic device refers to a computing device capable of executing the methods of this embodiment, such as a server, workstation, private cloud node, or cluster computing platform. Creating a virtual terminal instance means that the electronic device generates the virtual terminal instance and configures the virtual terminal instance with the necessary operating conditions for subsequent access to the conference, receiving streams, and decoding streams.

[0063] Video conferencing signaling connectivity refers to the ability of a virtual terminal instance to establish a signaling connection with the conference server for conference access, hold, and control, in accordance with the signaling protocol adopted by the video conferencing system. Audio and video stream reception capability refers to the ability of a virtual terminal instance to receive audio and / or video streams sent to it by the conference server. Audio and video stream decoding capability refers to the ability of a virtual terminal instance to decode the received audio and / or video streams to obtain video frames and / or audio data that can be used for quality inspection.

[0064] In some embodiments, the electronic device can create virtual terminal instances on a monitoring server. A monitoring server is a server used to deploy video conferencing quality monitoring programs and run virtual terminal instances. The electronic device can configure conference access parameters, signaling connection parameters, media reception parameters, and media decoding parameters for the virtual terminal instances. Conference access parameters indicate the information required for the virtual terminal instance to access the target conference; signaling connection parameters establish a signaling connection between the virtual terminal instance and the conference server; media reception parameters configure the method by which the virtual terminal instance receives audio and video streams; and media decoding parameters configure the method by which the virtual terminal instance decodes the audio and video streams.

[0065] In some embodiments, the audio and video stream decoding capabilities of a virtual terminal instance may include video decoding capabilities and / or audio decoding capabilities. Video decoding capabilities may include decoding H.264 and / or H.265 video streams to output raw video frames; audio decoding capabilities may include decoding AAC, G.711, and / or G.722 audio streams to output Pulse Code Modulation (PCM) audio data. The raw video frames and / or PCM audio data can be used as the processing objects for subsequent quality inspection. The specific supported encoding formats can be configured according to the stream format distributed by the conference server in the target conference, the encoding capabilities of the conference terminal, and the requirements of the quality inspection task.

[0066] In some embodiments, the audio and video stream decoding capability of the virtual terminal instance can be implemented using a media decoding engine. As an optional implementation, the media decoding engine can employ an open-source media processing framework, such as FFmpeg or GStreamer; alternatively, it can use a self-developed decoding component. After decoding the received audio and video streams into raw video frames and / or PCM audio data, the media decoding engine can send the decoding results to the quality inspection stage, rather than using them for display or playback. In other words, the decoded audio and video data is not rendered and output in the virtual terminal instance; it is directly used as the processing object for quality inspection, thereby further reducing the virtual terminal instance's resource consumption for display and playback.

[0067] In some embodiments, the video conferencing signaling connectivity of a virtual terminal instance can be implemented by integrating a standard signaling protocol stack. Taking the SIP protocol stack as an example, the virtual terminal instance can send a call invitation (INVITE) request carrying a Session Description Protocol (SDP) session description to the conference server; after the conference server returns an SDP response, the virtual terminal instance completes media format negotiation accordingly; and a media channel is established after the signaling handshake is completed. Taking the H.323 protocol stack as an example, the virtual terminal instance can complete call establishment and media channel negotiation through H.225 call signaling and H.245 capability negotiation. The aforementioned signaling connectivity capabilities support conference access, conference hold, and conference control for the virtual terminal instance.

[0068] In some embodiments, the processing time involved in the quality monitoring process performed by the virtual terminal instance can be configured to a range that meets the real-time monitoring requirements. As an optional implementation, the signaling connection establishment time of the virtual terminal instance can be one to three seconds, the bitstream reception latency can be less than one hundred milliseconds, and the single-frame decoding time for a 1080-line progressive scan (1080p) video frame can be less than fifty milliseconds. These values ​​are typical values ​​based on general video conferencing platforms. In other embodiments, these values ​​can be adjusted according to the monitoring server performance, network conditions, and bitstream resolution. For example, when processing higher resolution video frames, the single-frame decoding time can be appropriately increased.

[0069] In some embodiments, the video conferencing signaling protocol may include the H.323 protocol or the Session Initiation Protocol (SIP). Both the H.323 protocol and SIP can be used for call setup, conference access, and signaling interaction in video conferencing. The specific protocol used can be determined based on the support of the target conference and the conference server.

[0070] In some embodiments, when creating a virtual terminal instance, the media sending module of the virtual terminal instance can be disabled or configured to be inactive, so that the virtual terminal instance functions as a participant that only receives media and does not send actual media data. Specifically, the virtual terminal instance can declare its media sending capability as empty or not supporting the sending direction during the Session Description Protocol (SDP) or H.245 capability negotiation process, so that the conference server can recognize that the virtual terminal instance will not send audio and video media streams used for mixing to the target conference. Furthermore, the virtual terminal instance may not create a media sending port or send Real-time Transport Protocol (RTP) media packets. When it is necessary to meet the conference protocol keep-alive or compatibility requirements, the virtual terminal instance may also send silent frames, black video frames, or other placeholder media data that does not occupy actual uplink media bandwidth. In this way, the virtual terminal instance can meet the access requirements of the target conference, while reducing the impact of the virtual terminal instance on uplink media bandwidth and conference mixing processing.

[0071] In some embodiments, the electronic device can also create and manage one or more virtual terminal instances to form a virtual terminal pool. A virtual terminal pool refers to a collection of instances consisting of multiple virtual terminal instances pre-created or created on demand by the electronic device. The electronic device can allocate idle virtual terminal instances from the virtual terminal pool to join the target meeting based on the monitoring task of the target meeting; after the monitoring task of the target meeting ends, the electronic device can release the corresponding virtual terminal instance or restore it to an idle state for reuse in subsequent meeting quality monitoring tasks. Virtual terminal pool management can reduce the time required to repeatedly create virtual terminal instances and facilitate instance scheduling in multi-meeting or large-scale meeting scenarios.

[0072] In existing quality monitoring methods, relying on real participating terminals to collect and report quality data typically requires installing collection components, opening local interfaces, or performing software adaptation on the real participating terminals. This is difficult to apply to meeting scenarios with complex terminal types, inconsistent terminal management permissions, or temporary access from external terminals. The virtual terminal instance creation method described above allows virtual terminal instances to be created and run by electronic devices. Their video conferencing signaling connectivity, audio / video stream reception, and audio / video stream decoding capabilities can be uniformly configured by the monitoring side, without requiring modifications to individual real participating terminals. Furthermore, by configuring virtual terminal instances not to send actual audio / video media data, the impact of virtual terminal instances on uplink media bandwidth and meeting mixing processing can be reduced. Through virtual terminal pool management, virtual terminal instances can be reused in multiple or large-scale meeting monitoring tasks, reducing the time overhead of repeatedly creating instances.

[0073] Understandably, by creating virtual terminal instances with video conferencing signaling connectivity, audio and video stream reception capabilities, and audio and video stream decoding capabilities, a processing foundation can be provided for subsequently joining the target conference as a participant, receiving audio and video streams corresponding to different participants, and decoding and quality checking the audio and video streams.

[0074] Step S220: Control the virtual terminal instance to join the target meeting organized by the meeting server as a participant.

[0075] In this context, "participant identity" refers to the identity of a participant in the target meeting. After a virtual terminal instance joins the target meeting as a participant, it can be identified and managed by the meeting server as a participant within the target meeting. The meeting server refers to the meeting control equipment or control system used to establish, connect to, and manage multi-party video conferences, such as a Multipoint Control Unit (MCU) or Selective Forwarding Unit (SFU), which can perform meeting control and media processing for multiple participants joining the meeting. The target meeting refers to the video conference for which quality monitoring is required.

[0076] In some embodiments, an electronic device can control a virtual terminal instance to initiate a meeting access request to the meeting server based on the meeting information of the target meeting. Meeting information refers to information indicating the access location and conditions of the target meeting, and may include a meeting address, meeting number, meeting identifier, or access authentication information. A meeting access request is a signaling request sent by the virtual terminal instance to the meeting server, requesting to join the target meeting.

[0077] In some embodiments, when a virtual terminal instance joins a target conference, it can declare a receive-only mode through signaling negotiation. Taking SIP signaling as an example, the virtual terminal instance can declare the `a=recvonly` attribute in the SDP session description of the INVITE request, causing the conference server to establish only a downlink media channel from the conference server to the virtual terminal instance, without establishing an uplink media channel from the virtual terminal instance to the conference server. The virtual terminal instance can retain the necessary Real-time Transport Control Protocol (RTCP) feedback capability to report the reception status to the conference server. By declaring receive-only mode, the virtual terminal instance can meet the access requirements of the target conference while avoiding sending uplink audio and video streams used for mixing to the conference server.

[0078] In some embodiments, after receiving a meeting access request, the meeting server can process the virtual terminal instance according to the target meeting's access procedure, enabling the virtual terminal instance to join the target meeting. After the virtual terminal instance joins the target meeting, the meeting server can assign session information or participant identifiers to the virtual terminal instance. Session information refers to information used to maintain the connection status of the virtual terminal instance in the target meeting; participant identifiers refer to the identification information used by the meeting server to distinguish different participants in the target meeting.

[0079] In some embodiments, after a virtual terminal instance joins the target conference as a participant, the conference server can verify the virtual terminal instance's permissions according to the permission model for ordinary participants. When the virtual terminal instance invokes the media stream control mechanism, it can carry the participant identifier, session information, or authentication information assigned to it by the conference server. The conference server determines whether the virtual terminal instance has the permission to perform the specified operation of receiving the target based on the participant's identity. Therefore, the virtual terminal instance can complete the specified receiving target and audio / video stream reception based on its ordinary participant identity without using administrator privileges or changing the conference server's security configuration.

[0080] In some embodiments, after a virtual terminal instance joins the target conference, it can occupy one participant seat and the corresponding signaling port and media receiving port. Since the virtual terminal instance can be configured not to send actual audio and video media data, its uplink media bandwidth usage can be lower than that of a regular real participant terminal; the downlink bandwidth of the virtual terminal instance can be related to the audio and video streams it currently receives from the participants. For the conference server, the virtual terminal instance can act as a regular participant for signaling maintenance and media forwarding, without requiring additional conference mixing and output processing. This configuration reduces the impact on the conference server's mixing and computing resources when the virtual terminal instance joins the target conference and performs quality monitoring.

[0081] In existing methods, if quality monitoring requires administrator privileges, special meeting control permissions, or changes to the security configuration of the meeting server, it increases the complexity of the meeting system configuration and is not conducive to deployment in existing meeting systems. By configuring the permissions and resource usage of ordinary participants as described above, after a virtual terminal instance joins the target meeting as a participant, it can perform subsequent target-specified operations based on the participant identifier and session information assigned by the meeting server, without needing additional administrator privileges or changes to the meeting server's security configuration. Furthermore, the virtual terminal instance does not need to send actual audio or video media data, nor does it participate in the meeting server's mixed-stream output. Therefore, it mainly occupies signaling maintenance and media reception resources at the ordinary participant level, which helps reduce the impact of the quality monitoring process on the meeting server's operational load.

[0082] Understandably, by controlling virtual terminal instances to join the target meeting as participants, virtual terminal instances can enter the target meeting's processing flow without requiring real participating terminals to add acquisition components or open acquisition interfaces, thus providing an access foundation for subsequently obtaining audio and video streams corresponding to different participants through the meeting server.

[0083] Step S230: Obtain the list of participants in the target meeting, and call the media stream control mechanism of the meeting server according to the list of participants. According to the preset polling strategy, traverse the list of participants in turn, and designate each participant in the list as the receiving target of the virtual terminal instance in turn, and receive the audio and video streams corresponding to the currently designated participants in turn.

[0084] The participant list refers to the collection of information about participants in the target meeting. A participant is any entity that connects to the target meeting and can include meeting room terminals, personal terminals, or other endpoints capable of connecting. The participant list may include at least one of the following: participant identifier, meeting room identifier, terminal identifier, participant name, connection status, or participant index used for media flow control. The participant list is used to determine the participants in the target meeting that need to be monitored sequentially.

[0085] The media stream control mechanism refers to the mechanism provided by the conference server to control the receiving targets of virtual terminal instances. It can be implemented as at least one of the following: a media stream control interface, a subscription mechanism, a selection mechanism, a forwarding mechanism, a routing mechanism, or a media switching mechanism. Taking the media stream control interface as an example, by calling the media stream control interface, the conference server can change the participants corresponding to the audio and video streams received by the virtual terminal instance. The receiving target refers to the participant currently configured to receive its corresponding audio and video streams on the virtual terminal instance. Specifying the receiving target sequentially as each participant in the participant list means switching the receiving object of the virtual terminal instance to different participants one by one according to the order of the participants in the participant list.

[0086] The currently designated participant refers to the participant corresponding to the current receiving target of the virtual terminal instance during a single polling process. The audio and video streams refer to the audio and / or video streams corresponding to the participant. An audio stream is a data stream formed by encoding sound data, and a video stream is a data stream formed by encoding video data. The audio and video streams corresponding to the currently designated participant refer to the audio and / or video streams sent by the conference server to the virtual terminal instance after the virtual terminal instance's receiving target has been designated as that participant.

[0087] In some embodiments, after a virtual terminal instance joins a target conference, the electronic device can obtain a list of participants for that conference. Obtaining the participant list can include: actively querying via a conference query interface provided by the conference server, and / or receiving a participant list sent by the conference server to the virtual terminal instance. The conference query interface refers to the interface provided by the conference server for querying participant information in a target conference; this interface can be a management interface of the conference server (e.g., a REST API provided by a conference management platform), or a query interface accessible to participants (e.g., a subscription query interface based on SIP conference event packets). If authentication is required for the conference query interface, the electronic device can use the participant identity of the virtual terminal instance or pre-configured authentication information to complete the call. Furthermore, in some conference systems, the conference server actively sends a participant list to each participant, which the virtual terminal instance, as a participant, can receive. The electronic device can determine the participants requiring quality monitoring based on the participant list and determine the target receiving order for each participant during the polling process.

[0088] In some embodiments, the electronic device can control the virtual terminal instance to invoke the media stream control mechanism (e.g., media stream control interface) of the conference server according to the participant list, specifying a participant in the participant list as the receiving target of the virtual terminal instance. Based on the specified receiving target, the conference server sends the audio and video stream corresponding to the currently specified participant to the virtual terminal instance. After receiving the audio and video stream corresponding to the currently specified participant, the electronic device can continue to control the virtual terminal instance to specify the next participant in the participant list as the receiving target and receive the audio and video stream corresponding to the next participant.

[0089] In some embodiments, the electronic device can sequentially traverse each participant in the participant list, allowing the virtual terminal instance to receive the audio and video streams corresponding to each participant at different time periods. Thus, the virtual terminal instance can poll and monitor multiple participants in a target meeting by sequentially changing the receiving target, without needing to configure a separate terminal-side acquisition program for each real participant.

[0090] Figure 3 This schematically illustrates the virtual terminal polling process. (Reference) Figure 3As shown, after a virtual terminal instance joins the target conference, it obtains the list of participants in the target conference, initializes index i to 1, and designates the i-th participant as the receiving target. Subsequently, it receives the audio and video streams corresponding to the currently designated participant, decodes and performs quality checks on the audio and video streams, and records the corresponding quality indicators. After completing the quality check for the current participant, it determines whether all participants have been traversed. If not, it sets i = i + 1, designates the next participant as the receiving target, and repeats the above process; if so, it completes this round of polling and waits for the next polling cycle to arrive before re-obtaining the participant list and starting the next round of polling.

[0091] In some embodiments, after a virtual terminal instance designates a participant as its receiving target, it can receive the audio and video streams corresponding to that participant and continuously receive them for a preset duration. The preset reception duration can be configured based on the sample length required for quality detection. After completing the reception, decoding, and quality detection of the audio and video streams corresponding to that participant, the electronic device can continue to designate the virtual terminal instance as the next participant in the participant list and repeat the process of designating the receiving target, receiving the audio and video streams, decoding, and quality detection. After traversing all participants in the participant list, the electronic device can wait for the next polling cycle to arrive before restarting the next polling cycle. The next polling cycle can be configured based on the meeting size, real-time requirements, and the processing capacity of the monitoring server.

[0092] Existing methods of querying statistical data based on the conference server management interface typically only obtain statistical information such as online status, bandwidth, packet loss rate, or jitter, which is insufficient to directly reflect the actual video and audio status of different participants during the meeting. Through the polling reception process described above, the virtual terminal instance does not merely obtain statistical data from the conference server side, but rather receives the audio and video streams corresponding to each participant sequentially according to the participant list. This allows subsequent decoding and quality inspection to process the actual audio and video content of specific participants. By setting preset reception durations and the next polling cycle, continuous monitoring can be established among different participants, ensuring that each participant in the target meeting can be periodically monitored.

[0093] Understandably, obtaining the list of participants for the target meeting and invoking the media stream control mechanism of the meeting server according to the participant list, then iterating through the participant list according to a preset polling strategy, and sequentially assigning the receiving target of each virtual terminal instance to each participant, allows the virtual terminal instances to receive the audio and video streams corresponding to different participants. Compared to simply querying the meeting server's statistical data, this process provides a data foundation for subsequent quality inspection based on the actual audio and video content.

[0094] Step S240: Decode and perform quality testing on the audio and video streams to obtain the quality indicators corresponding to each participant, and generate the quality monitoring results of the target meeting based on the quality indicators corresponding to each participant.

[0095] Decoding refers to the process of parsing and restoring the encoded audio and video streams. Through decoding, video streams can be restored to video frames, and audio streams can be restored to audio data. A video frame is the picture frame data obtained after decoding the video stream. Audio data is the sound data obtained after decoding the audio stream. Video frames and / or audio data can be used as the processing objects for quality inspection.

[0096] Quality inspection refers to the process of judging the quality status of decoded video frames and / or audio data. Quality inspection can be used to determine the video and / or audio quality status of each participant. Quality indicators refer to the results obtained from quality inspection, which can be used to characterize the audio and video quality status of each participant. Quality monitoring results refer to the meeting monitoring results generated based on the quality indicators corresponding to each participant, which can be used to characterize the audio and video quality of each participant in the target meeting.

[0097] In some embodiments, after the virtual terminal instance receives the audio and video stream corresponding to the currently designated participant, the electronic device can decode the audio and video stream to obtain video frames and / or audio data corresponding to the currently designated participant. For the video stream, decoding yields video frames representing the screen content; for the audio stream, decoding yields audio data representing the sound content.

[0098] In some embodiments, the electronic device can perform quality checks on the decoded video frames and / or audio data to obtain a quality index corresponding to the currently designated participant. The video quality index can be used to characterize the degree of image abnormality, and the audio quality index can be used to characterize the degree of sound abnormality. After completing the quality check for the currently designated participant, the electronic device can associate the obtained quality index with the currently designated participant. After sequentially completing the quality check for each participant in the participant list, the electronic device can obtain the quality index corresponding to each participant.

[0099] In some embodiments, quality detection of decoded video frames can employ machine learning models or deep neural network algorithms, including at least one of black screen detection, screen distortion detection, freeze detection, jitter detection, color cast detection, sharpness detection, and camera occlusion detection. Black screen detection can be based on the brightness and variance of the video frame, outputting a black screen determination result or a black screen quantification score. Screen distortion detection can be based on edge gradients or abnormal block identification results in the video frame, outputting the proportion of abnormal blocks. Freeze detection can be based on the inter-frame difference between adjacent video frames, outputting the freeze duration. Jitter detection can be based on the temporal changes in motion vectors, outputting a jitter amplitude quantification result. Color cast detection can be based on chroma component statistics, outputting a color cast degree score. Sharpness detection can be quantified based on spatial frequency and edge sharpness, outputting a sharpness score. Camera occlusion detection can be based on sudden scene changes or changes in image content, outputting an occlusion determination result.

[0100] In some embodiments, the threshold parameters used for the video quality detection described above can be configurable. For meetings primarily focused on presentation content, the brightness and variance thresholds used for black screen detection can be adjusted; for high frame rate meetings, the consecutive static frame count threshold used for freeze detection can be adjusted; and for video streams with different resolutions or encoding formats, different clarity evaluation benchmarks can be used. The above detection algorithm can serve as a specific implementation for quality analysis of audio and video streams obtained through virtual terminal instances.

[0101] In some embodiments, quality detection of decoded audio and video data can employ various types of quality detection algorithms, including at least one of the following. The first type is traditional audio and video anomaly detection, such as the aforementioned black screen detection, distorted screen detection, freeze detection, silence detection, noise detection, and audio-video synchronization detection, used to identify explicit anomalies in the image and sound. The second type is quality evaluation based on traditional signal processing, such as peak signal-to-noise ratio (PSNR) evaluation based on pixel differences, objective quality evaluation based on structural similarity (SSIM), or referenceless quality evaluation based on natural scene statistical features, including the Blind / Referenceless Image Spatial Quality Evaluator (BRISQUE) and the Natural Image Quality Evaluator (NIQE), used to objectively quantify image quality. The third category is perceptual quality assessment based on artificial intelligence and deep learning. Examples include no-reference video quality assessment based on neural networks, such as content-aware and semantic feature aggregation-based methods; and deep feature assessment based on perceptual similarity, including Learned Perceptual Image Patch Similarity (LPIPS) and Deep Image Structure and Texture Similarity (DISTS). These three types of quality detection algorithms can be used individually or in combination. They are applied to the decoded raw video frames and / or PCM audio data, rather than network layer statistics or encoded bitstream parameters.

[0102] The audio-video synchronization detection can be based on the deviation between the timestamp of the decoded video frame and the timestamp of the audio data, and output the duration of audio-video asynchrony. When the duration of audio-video asynchrony exceeds a preset threshold, it can be determined that there is an audio-video asynchrony anomaly in the audio-video bitstream corresponding to the currently designated participant.

[0103] In some embodiments, quality detection of the decoded audio data can employ machine learning models or deep neural network algorithms, and may include at least one of silence detection, feedback detection, noise detection, volume anomaly detection, and intermittent detection. Silence detection can be based on the relationship between the audio signal energy and a preset energy threshold, and output the silence duration. Feedback detection can be based on narrowband high-frequency peaks in the audio spectrum, and output the feedback frequency and intensity. Noise detection can be based on the signal-to-noise ratio (SNR), and output a noise level score. Volume anomaly detection can be based on the relationship between the root mean square (RMS) value and a preset normal range, and output the degree of volume deviation. Intermittent detection can be based on intermittent interruptions in the audio stream, and output the intermittent frequency.

[0104] In some embodiments, the threshold parameters used for the audio quality detection described above can be configurable parameters. For example, the energy threshold and duration threshold for silence detection, the spectral characteristic threshold for feedback detection, the signal-to-noise ratio threshold for noise detection, the normal range threshold for abnormal volume detection, and the intermittent frequency threshold for discontinuous detection can be adjusted according to the audio device type, the noise level of the conference environment, or the audio encoding format.

[0105] In some embodiments, the electronic device can generate quality monitoring results for the target meeting based on the quality indicators corresponding to each participant. The quality monitoring results for the target meeting may include the quality indicators of each participant in the target meeting, or the quality status determined based on the quality indicators of each participant. The quality status refers to the audio and video quality of the participant as determined by the quality indicators, such as a normal state, an abnormal state, or states corresponding to different degrees of abnormality. Therefore, the quality monitoring results can reflect the audio and video quality of each participant in the target meeting.

[0106] In some embodiments, the electronic device can also call the network management interface of the conference server through a virtual terminal instance or a monitoring server to query the network quality data corresponding to the currently designated participant. The network quality data may include at least one of the following: transmit-direction packet loss rate, receive-direction packet loss rate, number of transmit-direction packet losses, number of receive-direction packet losses, number of consecutive packet losses, and network jitter. The electronic device can fuse and correlate the quality indicators obtained based on decoded video frames and / or audio data with the network quality data obtained through the network management interface to generate a comprehensive quality assessment result for the target conference. Through this processing, based on the actual audio and video content and the application layer quality status, the analysis can be further aided by combining network transmission status to help analyze the causes of quality anomalies.

[0107] Existing methods based solely on application-layer audio and video detection results, while capable of identifying video or audio anomalies, struggle to determine whether these anomalies are related to network transmission status. Conversely, methods relying solely on network statistics fail to reflect the actual video and audio quality after decoding. By fusing and correlating network quality data with audio and video quality indicators, decoded video frames and / or audio data can reflect the actual video and audio quality of the participants, while network quality data can reflect packet loss, jitter, or continuous packet loss in the transmission direction for the corresponding participants. Fusing and correlating these two metrics allows for a more comprehensive basis for determining the cause of quality anomalies such as black screens, distorted images, frozen screens, mute, or intermittent audio, by simultaneously considering both network transmission status and the decoded content.

[0108] In some embodiments, the electronic device can store the quality indicators, network quality data, and / or comprehensive quality assessment results corresponding to each participant in a database. The database can associate and store monitoring data according to the target meeting identifier, participant identifier, monitoring time, and quality indicator type. The electronic device can also perform alarm judgments based on preset alarm rules for the quality assessment results. Preset alarm rules may include quality indicators falling below a preset quality threshold, anomalies occurring in multiple consecutive polling cycles, network quality data exceeding a preset anomaly range, or at least one of multiple quality anomalies occurring in the same participant. When a serious quality problem is detected in a participant, the electronic device can generate an alarm message and send it to the operation and maintenance terminal or platform to prompt operation and maintenance personnel to handle the corresponding participant.

[0109] Existing methods for meeting quality maintenance that rely on manual observation or feedback are prone to problems such as delayed detection of quality anomalies, lack of historical records of anomalies, and insufficient data for post-event investigations. By using the aforementioned database storage and alarm judgment, quality indicators, network quality data, and comprehensive quality assessment results can be stored in association with the target meeting identifier, participant identifier, and monitoring time, providing a data foundation for subsequent queries, backtracking, and trend analysis. Furthermore, alarm judgments can be made based on preset alarm rules to generate timely alarm information when participants experience serious quality problems or continuous anomalies, thereby reducing reliance on manual feedback and post-event investigations for quality anomaly detection.

[0110] Understandably, by decoding and quality testing the audio and video streams received by the virtual terminal instance, the actual audiovisual quality status of the participants in the target meeting can be determined based on the decoded video frames and / or audio data. By generating the quality monitoring results of the target meeting according to the quality indicators corresponding to each participant, a correspondence can be established between quality anomalies and specific participants, thereby facilitating the discovery and location of meeting quality anomalies.

[0111] In one specific embodiment, the above method can be applied to quality monitoring of small and medium-sized video conferences. The monitoring server can be a general-purpose server, configured with computing and network resources for running virtual terminal instances, receiving audio and video streams, performing decoding, and quality detection. The conference server can be a conference control device that supports specifying the receiving target or specifying viewing capabilities. The electronic device can create a virtual terminal instance on the monitoring server and control the virtual terminal instance to join the target conference; the virtual terminal instance can sequentially call the media stream control mechanism of the conference server to specify the receiving target as each participant in the target conference, and receive the audio and video streams corresponding to each participant. The electronic device can decode and perform quality detection on the received audio and video streams, and associate the obtained quality indicators with the corresponding participants.

[0112] In some embodiments, step S230 above, receiving the audio and video stream corresponding to the currently designated participant, may include: when the receiving target of the virtual terminal instance is designated as the currently designated participant, the conference server, based on the designation result of the media stream control mechanism, sends the uplink audio and video stream sent by the currently designated participant to the conference server to the virtual terminal instance; the virtual terminal instance receives the uplink audio and video stream as the audio and video stream corresponding to the currently designated participant.

[0113] Here, "currently designated participant" refers to the participant designated as the receiving target of the virtual terminal instance during this polling process. The receiving target refers to the participant for whom the virtual terminal instance is currently configured to receive its corresponding audio and video streams. The designation result of the media stream control mechanism refers to the media delivery object relationship determined by the conference server based on the participant identifier, meeting room identifier, or receiving target parameters carried when the virtual terminal instance invokes the media stream control mechanism. This designation result instructs the conference server which participant's audio and video stream to use as the delivery content when sending audio and video streams to the virtual terminal instance.

[0114] Uplink audio and video streams refer to the audio and / or video streams sent by the currently designated participant to the conference server. Here, "uplink" is relative to the transmission direction from the currently designated participant to the conference server, distinguishing it from the downlink streams distributed by the conference server to participants and the mixed streams formed by the conference server after mixing multiple participants. Sending the uplink audio and video streams to the virtual terminal instance means that after receiving the uplink audio and video streams sent by the currently designated participant, the conference server, based on the specified result of the receiving target, sends the uplink audio and video streams to the virtual terminal instance.

[0115] In some embodiments, the virtual terminal instance can designate the receiving target as the currently designated participant according to the order in the participant list. Upon receiving this designation, the conference server can establish a media delivery relationship between the virtual terminal instance and the corresponding bitstream of the currently designated participant. After this media delivery relationship is established, the conference server can deliver the uplink audio and video bitstreams sent by the currently designated participant to the virtual terminal instance according to the designation result. Upon receiving the uplink audio and video bitstreams, the virtual terminal instance can use them as the audio and video bitstreams corresponding to the currently designated participant for subsequent decoding and quality testing.

[0116] In some embodiments, the uplink audio and video streams corresponding to the currently designated participant may include the video stream uploaded by the currently designated participant, the audio stream uploaded by the currently designated participant, or both the video and audio streams uploaded by the currently designated participant. If the currently designated participant only enables video or only enables audio, the virtual terminal instance can receive the stream corresponding to the actual media type uploaded by the currently designated participant; if the currently designated participant uploads both audio and video, the virtual terminal instance can receive the video and audio streams corresponding to the currently designated participant and subsequently perform decoding and quality checks on them respectively.

[0117] In some embodiments, the uplink audio and video streams sent by the conference server to the virtual terminal instance can maintain a correspondence with the currently designated participant. Upon receiving the uplink audio and video stream, the electronic device can associate it with the participant identifier of the currently designated participant. The quality metrics obtained from subsequent decoding and quality testing of the uplink audio and video stream can also be associated with the participant identifier of the currently designated participant. This avoids confusion of quality metrics among different participants and provides a participant-level data foundation for generating quality monitoring results for the target conference.

[0118] Understandably, by designating the receiving target of the virtual terminal instance as the currently designated participant, the conference server distributes the uplink audio and video streams sent by the currently designated participant to the conference server back to the virtual terminal instance. This allows the virtual terminal instance to obtain an independent stream corresponding to the currently designated participant, rather than simply receiving statistical information such as connection status, bandwidth, and packet loss rate from the conference server side. Furthermore, since this uplink audio and video stream originates from the audio and video content sent to the conference server by the currently designated participant, subsequent decoding and quality testing of this uplink audio and video stream can map the obtained quality indicators to the currently designated participant, thereby facilitating the assessment of the specific participant's audio and video quality status.

[0119] Existing methods of analyzing the overall conference mixing or downlink distribution results from the conference server struggle to distinguish whether quality anomalies originate from a specific participant or from the conference server's mixing, downlink distribution, or other conference processing stages. By using the aforementioned uplink audio and video stream reception method, the virtual terminal instance receives the uplink audio and video stream sent to the conference server by the currently designated participant. Therefore, the quality metrics subsequently obtained based on this stream can more directly correspond to the acquisition status, encoding output status, and transmission status between the currently designated participant and the conference server. Consequently, when quality anomalies are detected, a clearer correlation can be established between the quality metrics and the currently designated participant.

[0120] In this context, "receiving target switching" refers to the process of changing the media source currently being received by a virtual terminal instance from one participant to another. In this embodiment, the receiving target switching occurs after the virtual terminal instance invokes the media stream control mechanism of the conference server. Based on the specified result of the media stream control mechanism, the conference server updates the media delivery relationship corresponding to the virtual terminal instance, causing the virtual terminal instance to change from receiving the audio and video stream from the previous participant to receiving the audio and video stream from the currently designated participant.

[0121] In some embodiments, after joining a target conference, a virtual terminal instance establishes and maintains a continuous media transmission channel (such as an RTP / RTCP channel) with the conference server. During polling monitoring, when the virtual terminal instance invokes the media flow control mechanism to switch the receiving target from the first participant to the second participant, the media transmission channel remains connected. This switching of the receiving target essentially triggers the media routing matrix within the conference server to change the source of the incoming stream from the uplink stream corresponding to the first participant to the uplink stream corresponding to the second participant. Since there is no need to re-perform the signaling handshake and connection establishment of the media channel, this embodiment achieves low-latency, seamless stream switching, thereby ensuring the high efficiency of polling monitoring.

[0122] Figure 7 A flowchart illustrating the bitstream switching stability determination process according to some embodiments of this application is shown schematically. (Reference) Figure 7As shown, in some embodiments, after the receiving target of the virtual terminal instance is switched from the previous participant to the currently designated participant, the electronic device can perform a stream switching stability determination to determine that the received audio and video streams after the switch have reached a stable state that can be used for quality detection. Specifically, after the receiving target is switched, the electronic device starts receiving the audio and video stream corresponding to the currently designated participant and starts the stability determination timer; the electronic device detects the arrival status of key frames in the received video stream. If no key frames are received, it can request the encoding end to send key frames by sending an FIR message or a PLI message; after receiving key frames, the electronic device detects the continuously decoded video frames. When a preset number of consecutive video frames are decoded normally, the frame rate is within a preset range, and no decoding errors occur, it determines that the audio and video streams have reached a stable state and starts performing quality detection sampling for a preset reception duration; if a stable state is not reached within the preset timeout period, the electronic device records this switching anomaly, re-designates the receiving target as the currently designated participant, or skips the currently designated participant and marks it in the quality monitoring results. By determining the stability of the bitstream switching, we can avoid misjudging transitional states such as incomplete frames or non-critical frame decoding abnormalities during the switching process as quality abnormalities of the participants, thereby improving the accuracy of quality indicators.

[0123] The meeting query interface refers to the interface provided by the meeting server for querying information about participants in a target meeting. When a virtual terminal instance calls the meeting query interface, it means that after joining the target meeting, the virtual terminal instance sends a query request to the meeting server to obtain participant information and receives the query results returned by the meeting server. The current participant list refers to the list of participants obtained through the meeting query interface before the start of this round of polling. The current participant list may include at least one of the following: participant identifier currently in the target meeting, meeting room identifier, terminal identifier, participant name, connection status, or participant index used for media stream control.

[0124] In some embodiments, the above method may further include: determining the number of virtual terminal instances required to complete the polling monitoring of the target meeting based on the number of participants in the target meeting, the monitoring duration required for a single participant, and the target polling cycle (i.e., the maximum allowed time configured by the user to complete a round of full participant monitoring); if the determined number of virtual terminal instances is greater than 1, creating multiple virtual terminal instances and controlling the multiple virtual terminal instances to join the target meeting as participants; dividing the participants in the participant list into multiple participant groups; and controlling the multiple virtual terminal instances to perform receiving target specification, audio and video stream reception, decoding, and quality detection for the corresponding participant groups.

[0125] Multiple virtual terminal instances refer to two or more virtual terminal instances used to perform polling monitoring tasks. Each virtual terminal instance has video conferencing signaling connectivity, audio and video stream reception, and audio and video stream decoding capabilities, and can join the target conference as a participant. A participant group refers to a subset of participants from the participant list. A participant group can be assigned to a virtual terminal instance, which is responsible for polling and monitoring the participants in that group. Target designation refers to the virtual terminal instance calling the media stream control interface of the conference server to designate a specific participant in the corresponding participant group as the media source it is currently receiving. Audio and video stream reception refers to the virtual terminal instance receiving the audio and / or video streams corresponding to the designated participant after the target has been specified.

[0126] In some embodiments, when the number of participants in the target meeting is small, a single virtual terminal instance can sequentially perform target designation, audio / video stream reception, decoding, and quality detection for each participant in the participant list. When the number of participants in the target meeting is large, if a single virtual terminal instance still polls all participants one by one according to the participant list, the time required to complete one round of polling monitoring will increase with the number of participants, causing the quality indicators of some participants to fail to update within the target polling period. Therefore, the electronic device can determine how many virtual terminal instances are needed to jointly complete the polling monitoring of the target meeting based on the number of participants in the target meeting, the monitoring time required for each participant, and the target polling period.

[0127] In some embodiments, when the number of determined virtual terminal instances is greater than one, the electronic device can create multiple virtual terminal instances on the monitoring server and control each virtual terminal instance to join the target conference as a participant. After joining the target conference, each virtual terminal instance can be identified and managed by the conference server as an independent participant. Since each virtual terminal instance has an independent receiving target, each virtual terminal instance can invoke the media stream control mechanism of the conference server to specify its receiving target as a participant in the corresponding participant group and receive the audio and video streams corresponding to that participant.

[0128] In some embodiments, the electronic device can divide the participants in the participant list into multiple participant groups based on the determined number of virtual terminal instances. The number of participant groups can be the same as the determined number of virtual terminal instances, or it can be adjusted according to the actual monitoring resources. When dividing the participant groups, participants can be assigned to different participant groups sequentially according to the order of the participant list; alternatively, they can be divided in a way that the number of participants in each participant group is similar, so that the monitoring workload undertaken by multiple virtual terminal instances is relatively balanced. For example, if there are sixty participants in the target meeting that need to be monitored and it is determined that three virtual terminal instances are needed, the sixty participants can be divided into three participant groups, each containing twenty participants, and each group can be assigned to three virtual terminal instances.

[0129] In some embodiments, when an electronic device controls multiple virtual terminal instances to perform polling monitoring on their respective participant groups, each virtual terminal instance can sequentially receive targets only within the participant group assigned to it. For example, a first virtual terminal instance can sequentially specify the participants in a first participant group as the receiving targets, a second virtual terminal instance can sequentially specify the participants in a second participant group as the receiving targets, and a third virtual terminal instance can sequentially specify the participants in a third participant group as the receiving targets. Multiple virtual terminal instances can execute the above processing in parallel within the same polling cycle, thereby jointly completing the quality monitoring of multiple participants in the target meeting.

[0130] Figure 4 This illustration schematically depicts a multi-virtual terminal collaboration mechanism according to some embodiments of this application. (Reference) Figure 4 As shown, the electronic device can divide the list of participants in the target meeting into a first participant group, a second participant group, and a third participant group, and assign them to a first virtual terminal instance, a second virtual terminal instance, and a third virtual terminal instance, respectively. Each virtual terminal instance performs target designation, audio and video stream reception, decoding, and quality detection in parallel for its respective participant group within the same polling cycle, and sends the obtained quality indicators to the monitoring server for aggregation to form the quality monitoring results of the target meeting.

[0131] In some embodiments, when each virtual terminal instance performs polling monitoring within its corresponding participant group, it can be processed in the same way as a single virtual terminal instance. That is, each virtual terminal instance can obtain or receive participant group information assigned to it, sequentially call the media stream control mechanism of the conference server according to the order of participants in that group, specify the receiving target as the corresponding participant, receive the audio and video stream corresponding to the currently specified participant, and decode and perform quality checks on the audio and video stream to obtain the quality indicators of the corresponding participant. Therefore, collaborative polling by multiple virtual terminal instances does not change the quality detection method for a single participant, but rather shortens the time required to complete the monitoring of all participants through grouping and parallel execution.

[0132] In some embodiments, each virtual terminal instance can send its obtained quality metrics to the monitoring server separately. The monitoring server can associate the quality metrics obtained by different virtual terminal instances with the corresponding participants based on the allocation relationship between the participants and the virtual terminal instances. Through the above processing, even if the participants in the target meeting are monitored separately by multiple virtual terminal instances, a unified quality monitoring result for the target meeting can be formed on the monitoring server, avoiding the problem of scattered quality metric sources or confused participant correspondence caused by multiple instances executing in parallel.

[0133] Understandably, by determining the number of virtual terminal instances based on the number of participants in the target meeting, the monitoring duration required for each participant, and the target polling cycle, and by creating multiple virtual terminal instances and adding them to the target meeting when the determined number of virtual terminal instances is greater than one, multiple virtual terminal instances can share the polling monitoring task of the target meeting. By dividing the participant list into multiple participant groups and controlling multiple virtual terminal instances to perform target designation, audio and video stream reception, decoding, and quality detection for their respective participant groups, the problem of excessively long monitoring cycles caused by a single virtual terminal instance polling a large number of participants one by one can be reduced, ensuring that the participant quality indicators in large-scale video conferences can be updated within the target polling cycle.

[0134] Existing single-virtual-terminal polling methods are easily limited by the number of participants in large-scale meetings. The more participants there are, the longer the time required for a full round of monitoring, resulting in some participants' quality status not being updated in a timely manner within the target polling period. By using multi-virtual-terminal collaborative polling, the number of virtual terminal instances is determined based on the number of participants, the monitoring time required for each participant, and the target polling period. Participants are divided into multiple participant groups, each polled in parallel by multiple virtual terminal instances. This decouples meeting size from the polling period, thereby improving the timeliness of full participant quality monitoring in large-scale meetings.

[0135] In some embodiments, determining the number of virtual terminal instances required to complete the polling monitoring of the target meeting based on the number of participants in the target meeting, the monitoring time required for each participant, and the target polling cycle may include: determining the total monitoring time required to complete one round of full participant monitoring based on the number of participants in the target meeting and the monitoring time required for each participant; determining the number of virtual terminal instances based on the relationship between the total monitoring time and the target polling cycle, so that the time for each virtual terminal instance to complete one round of full participant monitoring after parallel polling does not exceed the target polling cycle; allocating participants in the participant list to the corresponding virtual terminal instances according to the determined number of virtual terminal instances, and aggregating the quality indicators obtained from each virtual terminal instance on the monitoring server.

[0136] In this context, "full-participant monitoring in one round" refers to the process of completing a quality monitoring cycle for all participants in the target meeting. "Total monitoring time" refers to the estimated time required to complete one round of full-participant monitoring when a single virtual terminal instance sequentially monitors all participants. "Parallel polling" refers to a processing method where multiple virtual terminal instances perform polling monitoring on different participant groups within the same polling cycle. "Assigning participants to corresponding virtual terminal instances" means determining which virtual terminal instance is responsible for monitoring each participant. "Monitoring server" refers to the server used to run video conferencing quality monitoring tasks and aggregate monitoring results. "Aggregating quality metrics from various virtual terminal instances" means that the monitoring server receives and integrates the quality metrics generated by multiple virtual terminal instances to form the quality monitoring results for the target meeting.

[0137] In some embodiments, the electronic device can determine the total monitoring time required to complete a full round of monitoring of all participants based on the number of participants in the target meeting and the monitoring time required for each participant. For example, if the number of participants to be monitored in the target meeting is N, and the monitoring time required for each participant is t, then the total monitoring time can be determined based on the product of N and t. Here, N can represent the number of participants in the participant list, and t can represent the average time required for a virtual terminal instance to complete one effective monitoring of a participant. This total monitoring time is used to characterize the time required for a single virtual terminal instance to sequentially complete the monitoring of all participants without parallel polling by multiple virtual terminals.

[0138] In some embodiments, the electronic device can determine the number of virtual terminal instances based on the relationship between the total monitoring duration and the target polling period. If the total monitoring duration does not exceed the target polling period, one virtual terminal instance can complete a round of full participant monitoring; if the total monitoring duration exceeds the target polling period, the number of virtual terminal instances can be increased, allowing multiple virtual terminal instances to each undertake a portion of the participant polling monitoring task. In some embodiments, the number of virtual terminal instances can be determined based on the proportional relationship between the number of participants, the monitoring duration required by a single participant, and the target polling period. By using the above method of determining the number, the time required for each virtual terminal instance to complete a round of full participant monitoring after parallel polling can be kept within the target polling period.

[0139] In some embodiments, to reduce unnecessary virtual terminal instance usage, the number of virtual terminal instances can be determined to be the minimum number required to meet the polling cycle requirements. By determining the number of virtual terminal instances to be the minimum number required to meet the polling cycle requirements, the resource consumption caused by an excessive number of virtual terminal instances can be reduced while meeting the monitoring time requirements for a round of full participation.

[0140] In some embodiments, the determined number of virtual terminal instances can be an integer value that satisfies the target polling cycle requirement. For example, when the number of virtual terminal instances calculated based on the proportional relationship is not an integer, an integer not less than the result can be used as the number of virtual terminal instances. This determined number of virtual terminal instances allows for parallel polling by multiple virtual terminal instances, reducing the number of participants each virtual terminal instance handles, thus ensuring that the time to complete one round of full participant monitoring meets the target polling cycle requirement. The above method of determining the number is only one example; electronic devices can also adjust the number of virtual terminal instances based on monitoring server resources, the target meeting size, or the available number of virtual terminal instances, provided that the target polling cycle requirement is met.

[0141] In some embodiments, allocating participants from the participant list to corresponding virtual terminal instances according to the determined number of virtual terminal instances may include: dividing the participant list into a corresponding number of participant groups based on the number of virtual terminal instances, and allocating one virtual terminal instance to each participant group. Electronic devices can perform balanced allocation according to the order of the participant list, for example, assigning several adjacent participants to the same virtual terminal instance; or they can perform allocation in a round-robin manner, for example, assigning the first participant in the participant list to the first virtual terminal instance, the second participant to the second virtual terminal instance, and so on in a round-robin fashion. All of the above allocation methods are used to ensure that participants in the target meeting are assigned to corresponding virtual terminal instances for monitoring.

[0142] In some embodiments, after participant allocation is completed, each virtual terminal instance can perform polling monitoring according to the participants assigned to it. For participants assigned to a particular virtual terminal instance, that virtual terminal instance can sequentially specify these participants as the receiving targets, receive the corresponding audio and video streams, and decode and perform quality checks on the received audio and video streams. After multiple virtual terminal instances execute the above process respectively, a full round of monitoring of all participants can be completed within the target polling period.

[0143] In some embodiments, the quality metrics obtained by each virtual terminal instance can be sent to a monitoring server. The monitoring server can aggregate the quality metrics obtained by each virtual terminal instance according to the participant identifier, the virtual terminal instance identifier, and the monitoring time. The participant identifier is used to distinguish different participants, the virtual terminal instance identifier is used to distinguish the virtual terminal instance that generated the quality metrics, and the monitoring time is used to determine the monitoring time corresponding to the quality metrics. Through the above aggregation process, the monitoring server can integrate the quality metrics obtained by multiple virtual terminal instances into the quality monitoring results of the target meeting.

[0144] In some embodiments, when aggregating quality metrics, the monitoring server can organize the quality metrics corresponding to the same participant in different polling cycles according to the monitoring time, or it can organize the quality metrics corresponding to different participants in the same polling cycle according to the participant identifier. If multiple virtual terminal instances output quality metrics in parallel over time, the monitoring server can classify these quality metrics into the same round of full participant monitoring results based on the monitoring time or polling cycle identifier. This maintains the uniformity and traceability of quality monitoring results even when multiple virtual terminal instances are polling in parallel.

[0145] Understandably, by determining the total monitoring time based on the number of participants and the monitoring time required by each participant, and by determining the number of virtual terminal instances based on the relationship between the total monitoring time and the target polling cycle, the number of virtual terminal instances can be matched with the target meeting scale and monitoring cycle requirements. By allocating participants according to the determined number of virtual terminal instances and aggregating the quality indicators obtained from each virtual terminal instance on the monitoring server, a unified quality monitoring result can be formed under the collaborative polling of multiple virtual terminal instances, thereby improving the scalability and timeliness of quality monitoring for large-scale video conferencing.

[0146] In some embodiments, the target polling period can be configured by the user as the maximum allowed time to complete one round of full participant monitoring. In this case, the electronic device can determine the number of virtual terminal instances required to complete the polling monitoring of the target meeting based on the number of participants in the target meeting, the monitoring time required for each participant, and the maximum allowed time configured by the user to complete one round of full participant monitoring. As an optional implementation, the number of virtual terminal instances can be determined according to the formula... Where N is the number of participants, t is the monitoring duration required for a single participant, and T is the maximum allowed time configured by the user to complete a full round of monitoring of all participants. This indicates rounding up. For example, when the number of participants N to be monitored in the target meeting is sixty, the required monitoring time t for a single participant is five seconds, and the maximum allowed time T configured by the user is one hundred seconds, the result calculated according to the above formula is... This means that three virtual terminal instances are needed to poll in parallel to complete a full round of monitoring of all participants within 100 seconds. In other embodiments, the above calculation result can be adjusted based on the available resources of the monitoring server, provided that the maximum allowable time requirement is met. For example, if the calculation result is three but the monitoring server only has two idle virtual terminal instance slots, the target polling period can be appropriately extended or virtual terminal instances can be created in batches.

[0147] In some embodiments, when an electronic device controls multiple virtual terminal instances to perform monitoring on their respective participant groups, the multiple virtual terminal instances can be coordinated and scheduled. As an optional implementation, the electronic device can set different polling start times for each virtual terminal instance, staggering the target switching actions of each virtual terminal instance to achieve peak-shaving and prevent multiple virtual terminal instances from simultaneously initiating media stream control calls to the conference server, thus avoiding a sudden increase in the conference server's load. As another optional implementation, the electronic device can perform load balancing based on the historical quality data of each participant. For example, participants with historically large fluctuations in quality indicators and requiring longer monitoring times can be distributed to different participant groups, making the actual monitoring load borne by each virtual terminal instance relatively balanced. As yet another optional implementation, the electronic device can prioritize assigning participants identified as VIPs to virtual terminal instances with remaining computing resources exceeding a preset threshold, so that the audio and video streams corresponding to important participants can be received and processed more stably. The above three methods can be implemented individually or in combination.

[0148] Through the above-mentioned collaborative scheduling, the media flow control calls between virtual terminal instances are staggered in time, which alleviates the instantaneous processing pressure on the conference server; load balancing distribution makes the time for each virtual terminal instance to complete the monitoring of its respective participant group more similar, avoiding individual virtual terminal instances becoming the time bottleneck of the entire round of monitoring; priority allocation for VIP participants makes the quality monitoring of important participants more reliable.

[0149] In some embodiments, when a virtual terminal instance invokes the media stream control mechanism of the conference server to specify a receiving target, this media stream control mechanism can have several specific implementation forms. As one optional implementation, the media stream control mechanism can be a media stream control interface exposed by the conference server. The virtual terminal instance calls this interface, carrying the target participant's identifier, to request the conference server to send the corresponding participant's uplink audio and video streams. As another optional implementation, the media stream control mechanism can be a subscription mechanism, where the virtual terminal instance subscribes to the media stream of a specified participant from the conference server, and the conference server actively pushes the uplink stream to the virtual terminal instance upon receiving it. As yet another optional implementation, the media stream control mechanism can be a selection mechanism, where the virtual terminal instance submits a selection request for a participant to be received to the conference server, and the conference server selects the corresponding stream from multiple uplink streams for delivery. In addition, media flow control mechanisms can also be forwarding mechanisms, routing mechanisms, or media switching mechanisms. These mechanisms can change the receiving target of the virtual terminal instance to a specified participant through the configuration of the stream forwarding rules on the conference server, the update of the internal media routing matrix, or the media source switching operation. One or more of these media flow control mechanisms can be selected and used in combination, depending on the platform type and capabilities of the conference server.

[0150] Different media stream control mechanisms have different specific carrying methods on the conference server side, but they can all achieve the effect of specifying the receiving target of the virtual terminal instance as a specified participant and triggering the corresponding uplink bitstream to be sent. Therefore, the embodiments of this application can be compatible with the media stream control capabilities of various types of conference servers.

[0151] In some embodiments, the virtual terminal instance can specify the receiving target by invoking the media stream control mechanism of the conference server, which can be implemented in at least one of the following ways. As a first optional implementation, the virtual terminal instance can request the conference server to update its receiving target based on the SIP re-INVITE signaling, triggering the conference server to switch the media source sent to the virtual terminal instance to the stream corresponding to the target participant. As a second optional implementation, for conference servers using a selective forwarding unit architecture, the virtual terminal instance can send a media stream switching request instruction containing the target participant's user ID to the conference server through the HyperText Transfer Protocol (HTTP) Representational State Transfer (REST) ​​application programming interface. As a third optional implementation, the virtual terminal instance can subscribe to the media stream of a specified participant from the conference server based on the SIP SUBSCRIBE / NOTIFY mechanism or a Webhook subscription mechanism. The conference server will then actively push the uplink stream from that participant to the virtual terminal instance upon receiving it. The above methods can be used individually or in combination, depending on the type and capabilities of the conference server. Furthermore, after the target switching is complete, the virtual terminal instance can also, based on the feedback messages of the RTCP protocol, request the encoding end of the currently designated participant's bitstream to send keyframes by sending Full Intra Request (FIR) messages or Picture Loss Indication (PLI) messages. This accelerates the recovery of the image after the switch and shortens the waiting time for obtaining a valid bitstream that can be used for quality inspection.

[0152] As a typical application scenario, in a multipoint control unit (MCU) architecture, the conference server aggregates and forwards or mixes the uplink audio and video streams from various participants. Virtual terminal instances can call the designated viewing (selective viewing) control interface provided by the MCU, requesting the MCU to switch and send the stream corresponding to the currently designated participant to the virtual terminal instance. Upon receiving the selection request, the MCU switches the media source sent to the virtual terminal instance to the uplink audio and video stream corresponding to the currently designated participant. Therefore, even when the MCU performs mixed-stream output of the conference video, the virtual terminal instance can still obtain the audio and video stream corresponding to a single participant, thus ensuring that quality indicators can pinpoint the specific participant.

[0153] Existing methods, if supporting only a single stream acquisition method, are difficult to adapt to conference server providers from different vendors and with different architectures. Through the specific implementation of the aforementioned multiple media stream control mechanisms, virtual terminal instances can specify the target participant and acquire the corresponding uplink stream whenever the conference server supports different capabilities such as re-INVITE, REST interface, RTCP feedback, or subscription push. This improves the adaptability of this application's solution to various conference server platforms.

[0154] In some embodiments, after the virtual terminal instance completes the quality check on the currently designated participant, the electronic device can maintain the signaling connection state of the virtual terminal instance in the target conference without causing the virtual terminal instance to exit the target conference. Then, through the media flow control mechanism, the receiving target of the virtual terminal instance is designated as the next participant. That is, the virtual terminal instance only accesses the conference once during a round of polling. Subsequent switching between participants is completed by updating the media delivery relationship within the conference server while maintaining the signaling connection, without requiring a new signaling handshake and media channel establishment for conference access.

[0155] In existing methods, if each participant needs to rejoin and rejoin the meeting, it incurs repeated signaling handshake overhead and may affect the meeting server's participant management due to frequent meeting entry and exit. By using the aforementioned method of maintaining a signaling connection while switching the receiving target, the virtual terminal instance maintains a single meeting access state throughout the entire polling round. Switching the receiving target only involves updating the media distribution relationship within the meeting server, resulting in lower bitstream switching latency and reduced disruption to the meeting server's participant management. This improves the continuity and efficiency of polling monitoring.

[0156] In some embodiments, the polling strategy adopted by the virtual terminal instance after obtaining the participant list can be a dynamic polling strategy. The electronic device can formulate a dynamic polling strategy based on the obtained participant list, according to at least one of the following factors: participant priority, historical quality results, and meeting status. Following this dynamic polling strategy, receiving targets are sequentially specified, and the bitstream is received and decoded through a media stream control mechanism. The decoded data undergoes quality testing and metrics are recorded. Subsequent polling strategies are then dynamically adjusted based on the test results. Compared to polling participants in a fixed order with equal probability, a dynamic polling strategy can allocate monitoring resources to participants who require more attention.

[0157] As an optional implementation, dynamic polling can be a priority-based strategy. Electronic devices can determine the priority of each participant based on their VIP status, meeting role, and device type, and allocate monitoring frequencies accordingly. As an optional hierarchical approach, participants can be divided into multiple priority levels with separate monitoring frequencies: VIP participants, moderators, or speakers are assigned the highest priority, included in every round of polling; participants with a history of quality issues are assigned the second priority, included every two rounds of polling; ordinary participants are assigned the third priority, included every three rounds of polling; and participants using mobile devices, older devices, or other potentially problematic devices are assigned the fourth priority, included every five rounds of polling, with the lowest monitoring frequency. The above priority levels and corresponding monitoring frequencies are merely examples and can be adjusted based on meeting size and monitoring resources. This allows key participants to be monitored more frequently.

[0158] As an alternative implementation, a dynamic polling strategy can be based on historical quality. Electronic devices can dynamically adjust their monitoring frequency according to the historical quality scores of each participant. As an optional tiering method, historical quality scores can be divided into multiple intervals, each with its own monitoring frequency: for participants with historical quality scores of 90 or higher, the monitoring frequency is reduced, for example, once every four rounds of polling; for participants with historical quality scores between 80 and 89, the standard monitoring frequency is maintained, for example, once every two rounds of polling; for participants with historical quality scores between 70 and 79, the monitoring frequency is increased, for example, included in every round of polling; and for participants with historical quality scores below 70, continuous monitoring is performed, for example, included in three consecutive rounds of polling. The above scoring intervals and corresponding monitoring frequencies are merely examples and can be adjusted according to the actual quality distribution. Thus, monitoring resources can be concentrated on participants more likely to have quality issues.

[0159] As another optional implementation, dynamic polling can be an event-triggered polling strategy. Electronic devices can trigger continuous monitoring of relevant participants when they detect network events, quality events, or meeting events. For example, when a participant experiences network events such as network jitter or a sudden increase in packet loss, or quality events such as black screen, screen distortion, or mute, continuous monitoring of that participant can be started immediately; when the meeting enters important agenda items such as voting or presentations, the monitoring frequency for all participants or relevant participants can be increased.

[0160] The aforementioned polling strategies based on priority, historical quality, and event triggering can be implemented individually or in combination. Electronic devices can also dynamically adjust subsequent polling strategies based on the quality detection results of each round. For example, when a quality issue is detected in a participant, their priority can be immediately increased and continuous monitoring initiated; the monitoring frequency can be temporarily increased when a VIP participant speaks; and the monitoring frequency for ordinary participants can be temporarily decreased when the conference server load is too high. Through dynamic polling strategies and their real-time adjustments, key participants and abnormal participants can receive more timely quality monitoring even with limited monitoring resources, balancing the breadth and focus of monitoring.

[0161] In some embodiments, the electronic device can listen for participant change events through an event subscription mechanism during the polling process, and update the base list and execute corresponding processing logic when a change in the participant list is detected. When a new participant is detected joining, the electronic device can add the new participant to the base list and assign a monitoring priority to the new participant. When a participant is detected leaving, the electronic device can remove the participant from the base list and release the monitoring resources corresponding to the participant. When a participant's status change is detected, the electronic device can re-evaluate the participant's priority and dynamically adjust the monitoring strategy for that participant. When a change in meeting size is detected, the electronic device can recalculate the required number of virtual terminal instances and dynamically expand or shrink the virtual terminal instances as needed. By reusing the base list in a stable state and performing incremental updates only when participant changes are detected, the complete participant list can be avoided in each round of polling, balancing monitoring efficiency and real-time performance.

[0162] The aforementioned dynamic polling strategy is also applicable to scenarios involving parallel monitoring of multiple virtual terminals. In such scenarios, each virtual terminal instance can independently execute the dynamic polling strategy of its respective participant group, and the coordination mechanism can comprehensively manage the polling of each virtual terminal instance. For example, if a participant group experiences concentrated quality anomalies, the coordination mechanism can temporarily allocate some monitoring capabilities of other virtual terminal instances to that participant group.

[0163] In some embodiments, virtual terminal instances can be deployed on a local monitoring server, and the local monitoring server can be located in the same network domain as the conference server. The virtual terminal instance can establish a signaling connection with the conference server and receive audio and video streams via a local area network (LAN). By deploying the virtual terminal instance in the same domain as the conference server, the interference of additional network latency, jitter, or packet loss caused by cross-public network transmission on the quality detection results can be reduced, making the quality indicators more accurately reflect the quality status of the audio and video streams of the participants in the target conference.

[0164] If existing monitoring nodes transmit data to the conference server via the public internet or complex cross-domain networks, the latency, jitter, or packet loss of the monitoring link itself may be superimposed on the quality detection results, causing the quality indicators to fail to accurately reflect the status of the audio and video streams of the participants in the target conference. By deploying the virtual terminal instances on a local monitoring server within the same network domain as the conference server, as described above, the impact of the monitoring link itself on the quality detection results can be reduced, improving the reliability of the quality assessment results.

[0165] Figure 6 A schematic diagram illustrating an encrypted conferencing support architecture according to some embodiments of this application is provided. (Reference) Figure 6 As shown, before joining a target conference using encrypted transmission, the virtual terminal instance pre-configures authentication information such as certificates and pre-shared keys on the monitoring server. The virtual terminal instance negotiates encryption parameters with the conference server through a standard key negotiation mechanism, and the conference server authenticates the virtual terminal instance. After successful authentication, the conference server sends encrypted audio and video streams to the virtual terminal instance. The virtual terminal instance uses the negotiated decryption key to decrypt the encrypted audio and video streams and then decodes and performs quality checks on the decrypted audio and video streams.

[0166] In some embodiments, the target conference may employ encrypted transmission. Before joining a target conference using encrypted transmission, a virtual terminal instance can pre-configure certificates, pre-shared keys, or other authentication information for identity verification on the monitoring server. When initiating a conference access request, the virtual terminal instance can negotiate encryption parameters with the conference server through a standard key negotiation mechanism. The conference server can authenticate the virtual terminal instance and allow it to join the target conference after successful authentication. After receiving the encrypted audio and video streams, the virtual terminal instance can decrypt the encrypted audio and video streams using the negotiated decryption key, and then decode and perform quality checks on the decrypted audio and video streams. Through this process, target conferences employing media encryption and / or signaling encryption can also be subject to the aforementioned video conferencing quality monitoring method.

[0167] In some embodiments, the virtual terminal instance can support the media encryption protocol and / or signaling encryption protocol used by the target conference. The virtual terminal instance can determine the corresponding authentication method, key negotiation method, and stream decryption method based on the platform type, encryption configuration, and protocol version of the conference server. For different encryption protocol versions used by different conference server platforms, the virtual terminal instance can achieve compatibility by configuring corresponding encryption parameters or loading corresponding adaptation components. Therefore, decoding and quality checks can still be performed on the decrypted audio and video streams even when the target conference uses different encryption configurations.

[0168] Existing quality monitoring methods may not be able to directly obtain decryptable audio and video streams when the target meeting uses media encryption and / or signaling encryption, or may require additional compromise of the meeting's security mechanisms for analysis. With the encrypted meeting support methods described above, virtual terminal instances, after completing authentication and encryption parameter negotiation through pre-configured certificates, pre-shared keys, or standard key negotiation mechanisms, can join the encrypted meeting as legitimate participants and decrypt the received encrypted audio and video streams. The decrypted audio and video streams then enter the decoding and quality inspection process, enabling the aforementioned quality monitoring methods to be applied even to target meetings using encrypted transmission.

[0169] Figure 5 A schematic diagram illustrating a multipoint control unit adapter layer architecture according to some embodiments of this application is provided. (Reference) Figure 5 As shown, the abstract adaptation layer is set between the virtual terminal instance and various types of conference servers. The unified adaptation interface specification defined by the abstract adaptation layer can include a participant list query interface, a receiving target setting interface, a network statistics query interface, and an authentication parameter configuration interface. For different types of conference servers such as multipoint control units (MCUs) and selective forwarding units (SFUs), the abstract adaptation layer sets up corresponding adapters. Each adapter converts the calls based on the unified adaptation interface specification into actual interface calls supported by the corresponding conference server.

[0170] In some embodiments, virtual terminal instances can be compatible with multiple conference server platforms through an abstract adaptation layer. The abstract adaptation layer is an interface encapsulation layer set between the virtual terminal instance and different conference server platforms, used to shield the interface differences between different manufacturers or models of conference servers. The abstract adaptation layer can define a unified adaptation interface specification, which may include at least one of the following: a participant list query interface, a receiving target setting interface, and a network statistics query interface. For different types of conference servers, corresponding adapters can be set, and each adapter encapsulates the actual interface call method of the corresponding conference server according to the unified adaptation interface specification. When the virtual terminal instance performs a quality monitoring task, it can load the corresponding adapter according to the type of conference server corresponding to the target conference, and complete the participant list query, receiving target specification, audio and video stream reception, and network statistics query through the unified interface. Therefore, the same monitoring and control logic can be reused under different conference server platforms.

[0171] In some embodiments, the abstract adaptation layer can provide a unified Application Programming Interface (API), which may include at least one of a conference query API, a media stream control API, a network statistics query API, and an authentication parameter configuration API. For conference servers from different vendors or with different models, the adapter can convert the unified API into the actual interface call format supported by the corresponding conference server platform. At runtime, the virtual terminal instance can automatically load the corresponding adapter based on the conference server type, interface version, or platform identifier corresponding to the target conference to complete cross-platform quality monitoring tasks.

[0172] In existing conferencing systems, different vendors or models of conferencing servers may have differences in calling formats, parameter names, and return results for interfaces such as participant list query, receiving target setting, network statistics query, and authentication parameter configuration. Developing separate monitoring logic for each conferencing server would increase cross-platform deployment and maintenance costs. The aforementioned abstract adaptation layer encapsulates these differences through a unified interface specification, with corresponding adapters handling specific platform interface conversions. This allows virtual terminal instances to execute quality monitoring tasks based on a unified monitoring and control logic, reducing cross-platform deployment and maintenance costs.

[0173] In some embodiments, a single monitoring server can run multiple virtual terminal instance groups simultaneously. Different virtual terminal instance groups can correspond to different target conferences and join their respective target conferences as participants. Each virtual terminal instance group can independently obtain the participant list for its corresponding target conference, independently perform target specification, audio and video stream reception, decoding, and quality detection, and generate quality monitoring results for the corresponding target conference. Different virtual terminal instance groups can run independently of each other, sharing the monitoring server's computing, storage, and network resources. This process enables the same monitoring server to support parallel quality monitoring of multiple video conferences.

[0174] Existing single-conference sequential monitoring methods can easily lead to insufficient utilization of monitoring server resources when multiple conferences are running simultaneously, or require the deployment of independent monitoring environments for different conferences. The multi-conference parallel monitoring method described above allows the monitoring server to run multiple virtual terminal instance groups simultaneously, with each virtual terminal instance group corresponding to a different target conference. This enables the parallel execution of processes such as participant list acquisition, target specification, audio and video stream reception, decoding, and quality testing across multiple conferences, all in isolation. The independent operation of different virtual terminal instance groups avoids interference between monitoring tasks from different conferences, while simultaneously improving the utilization of the monitoring server's computing, storage, and network resources.

[0175] In some embodiments, since the virtual terminal instance is dedicated to quality monitoring and has its audio / video acquisition, encoding, display, and playback functions removed, its resource consumption can be significantly lower than that of a full soft terminal instance with all of the above functions enabled. As an example, in the same test environment, a comparison is made between a full soft terminal instance and the virtual terminal instance of this embodiment: the CPU utilization of the full soft terminal instance is approximately 45%, the GPU utilization is approximately 60%, and the memory usage is approximately 256 megabytes; the CPU utilization of the virtual terminal instance of this embodiment is approximately 9%, the GPU utilization is approximately 15%, and the memory usage is approximately 85 megabytes. That is, the CPU utilization, GPU utilization, and memory usage of the virtual terminal instance are reduced by approximately 80%, 75%, and 67%, respectively, compared to the full soft terminal instance. The above values ​​are examples based on specific test conditions and may vary depending on the test environment.

[0176] The example conditions for the above tests are as follows: the tests are based on an open-source video conferencing platform and a self-built selective forwarding unit server, simulating three meeting sizes: 50 participants, 100 participants, and 500 participants; the monitoring duration for a single virtual terminal instance is five seconds, the video resolution is 1080p, the encoding format is H.264, and the bitrate is 2 megabits per second; the test server is configured with a multi-core CPU, a dedicated graphics processor, and 32 gigabytes of memory; the baseline network conditions for the test environment are a bandwidth of no less than 100 megabits per second, a network latency of less than 10 milliseconds, and a packet loss rate of less than 0.1%; each configuration is tested repeatedly several times, and the average value is taken. In the comparison benchmark, the complete soft terminal instance has all functional modules such as audio and video acquisition, encoding, decoding, display, and playback enabled, while the virtual terminal instance in this application embodiment does not have the above-mentioned acquisition, encoding, display, and playback functions.

[0177] In some embodiments, due to the low resource consumption of virtual terminal instances, a single monitoring server can support a higher number of virtual terminal instances than a single full-fledged soft terminal instance. As an example, a full-fledged soft terminal instance needs to simultaneously receive and decode multiple streams forwarded by the conference server; its single-instance resource consumption increases with the conference size, while the number of instances supported by a single server decreases. In contrast, the virtual terminal instances in this embodiment only receive the audio and video streams corresponding to the currently designated participant at any given time; their single-instance resource consumption is essentially independent of the conference size, and a single server can support approximately 350 instances. Therefore, the same monitoring server can support more virtual terminal instances, thus supporting parallel monitoring of large-scale conferences with fewer monitoring servers.

[0178] In some embodiments, since the present application uses pure software virtual terminal instances for monitoring, there is no need to deploy hardware probe devices or modify the existing conference network for optical splitting or port mirroring. Therefore, it has advantages over hardware probe-based solutions in terms of deployment and operation and maintenance costs. As an example, a hardware probe-based solution requires approximately RMB 50,000 in hardware costs per monitoring node and approximately RMB 10,000 in deployment and maintenance costs per year, with a monitoring cost of approximately RMB 500 per conference. In contrast, the solution in the present application requires no dedicated hardware investment, with deployment and maintenance costs of approximately RMB 2,000 per year and a monitoring cost of approximately RMB 50 per conference. The above cost figures are illustrative and may vary depending on the deployment scale and operation method. The above example data regarding resource consumption, carrying capacity, and cost are used to illustrate the resource and cost characteristics of the pure software monitoring solution in the present application embodiments and do not constitute a limitation on the scope of protection of this application.

[0179] In some embodiments, the functions implemented by the aforementioned virtual terminal management environment, conference access environment, polling control environment, stream receiving and decoding environment, quality detection environment, and result generation environment can be respectively provided by... Figure 8 The virtual terminal module 810, management module 820, polling control module 830, media control module 840, quality detection module 850 and result aggregation module 860 in the illustrated embodiment are implemented.

[0180] The virtual terminal module 810 can be used to create and manage one or more virtual terminal instances on the monitoring server. Each virtual terminal instance can be implemented in software and has video conferencing signaling connectivity, audio and video stream reception, and audio and video stream decoding capabilities. The video conferencing signaling connectivity capability may include the ability to perform call establishment, capability negotiation, conference joining, and conference control signaling interaction based on the H.323 protocol or the Session Initiation Protocol (SIP); the audio and video stream decoding capability may include decoding H.264 video streams and / or H.265 video streams to output raw video frames, and decoding AAC audio streams and / or G.711 audio streams to output Pulse Code Modulation (PCM) audio data.

[0181] The management module 820 can connect to the virtual terminal module 810 to control the virtual terminal instances created or allocated by the virtual terminal module 810. These instances join the target conference organized by the conference server as participants via H.323 or SIP protocols, and call the conference query interface of the conference server and / or receive the participant list from the conference server through the virtual terminal instances to obtain the participant list of the target conference. After joining the target conference, the virtual terminal instance can be identified by the conference server as an independent participant. The conference server can assign session information, participant identifiers, or authentication information to the virtual terminal instance, enabling it to participate in signaling interaction and media stream control processes in the target conference as a regular participant.

[0182] The polling control module 830 can be connected to the management module 820. It iterates through the participant list according to a preset polling strategy, sequentially designating each participant in the participant list as the receiving target for the virtual terminal instance. The media control module 840 can be connected to the polling control module 830. It generates and sends media stream control instructions to the conference server. These instructions instruct the conference server to distribute the audio and video streams of the currently designated participant to the virtual terminal instance via the media channel, allowing the virtual terminal instance to receive the audio and video streams corresponding to the currently designated participant. In some embodiments, the audio and video streams corresponding to the currently designated participant can be the uplink audio and video streams sent by that participant to the conference server. The polling control module 830 can associate and cache the received audio and video streams with the corresponding participant identifiers for later use by the quality detection module 850.

[0183] The quality detection module 850 can be connected to the polling control module 830 to decode and perform quality detection on the received audio and video streams, obtaining the quality indicators corresponding to each participant. The quality detection module 850 can perform at least one of the following on the decoded video frames: black screen detection, screen tearing detection, freeze detection, jitter detection, color cast detection, sharpness detection, and camera occlusion detection. It can also perform at least one of the following on the decoded audio data: silence detection, feedback detection, noise detection, abnormal volume detection, and intermittent playback detection. The result aggregation module 860 can be connected to the quality detection module 850 to generate the quality monitoring results of the target meeting based on the quality indicators corresponding to each participant, and can associate the quality indicators corresponding to each participant with the target meeting identifier, participant identifier, and monitoring time.

[0184] In some embodiments, the virtual terminal module 810 can also disable the media transmission module of the virtual terminal instance or configure the media transmission module of the virtual terminal instance to be inactive when creating the virtual terminal instance, so that the virtual terminal instance is functionally equivalent to a participant that only receives but does not send actual media data. The virtual terminal instance can declare its media transmission capability to be empty or that it does not support transmission direction during the Session Description Protocol (SDP) or H.245 capability negotiation process, and can also choose not to create a media transmission port or send Real-time Transport Protocol (RTP) media packets. When it is necessary to meet the conference protocol keep-alive or compatibility requirements, the virtual terminal instance can send silent frames, black video frames, or other placeholder media data that does not occupy actual uplink media bandwidth. This configuration allows the conference server to treat the virtual terminal instance as a normal participant, while reducing the impact of the virtual terminal instance on uplink media bandwidth and mixing processing.

[0185] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0186] Furthermore, in this example embodiment, a video conferencing quality monitoring device based on virtual terminal polling is also provided, referring to... Figure 8 As shown, the video conferencing quality monitoring device 800 based on virtual terminal polling includes: a virtual terminal module 810, a management module 820, a polling control module 830, a media control module 840, a quality detection module 850, and a result aggregation module 860. Wherein: The virtual terminal module 810 is used to instantiate and maintain the lifecycle of the virtual terminal instance. The virtual terminal instance is dedicated to quality monitoring and has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability. The management module 820 is used to control virtual terminal instances to join the target meeting established by the meeting server and to obtain the list of participants in the target meeting; The polling control module 830 is used to traverse the participant list in a loop according to a preset polling strategy, and sequentially designate the receiving target of the virtual terminal instance as each participant in the participant list; The media control module 840 is used to call the media flow control mechanism of the conference server (e.g., to generate and send media flow control instructions to the conference server), so that the conference server can send the audio and video streams of the currently designated participant to the virtual terminal instance through the media channel, and the virtual terminal instance can receive the audio and video streams corresponding to the currently designated participant. The quality inspection module 850 is used to decode and inspect the distributed audio and video streams to obtain the corresponding quality indicators for each participant. The results aggregation module 860 is used to generate the quality monitoring results of the target meeting based on the quality indicators corresponding to each participant.

[0187] In some embodiments, the polling control module 830 and the media control module 840 can also be used to: when the receiving target of the virtual terminal instance is specified as the currently specified participant, receive the uplink audio and video stream sent by the currently specified participant to the conference server by the conference server based on the specified result of the media flow control mechanism, and use the uplink audio and video stream as the audio and video stream corresponding to the currently specified participant.

[0188] In some embodiments, the video conferencing quality monitoring device may further include a virtual terminal scheduling module. The virtual terminal scheduling module can be used to determine the number of virtual terminal instances required to complete the polling monitoring of the target conference based on the number of participants in the target conference, the monitoring duration required for each participant, and the target polling cycle; if the determined number of virtual terminal instances is greater than one, create multiple virtual terminal instances and control the multiple virtual terminal instances to join the target conference as participants; divide the participants in the participant list into multiple participant groups; and control the multiple virtual terminal instances to perform target assignment, audio and video stream reception, decoding, and quality detection for their respective participant groups.

[0189] In some example embodiments of this application, based on the foregoing scheme, the polling control module 830 and the media control module 840 are configured as follows: When the receiving target of the virtual terminal instance is specified as the currently designated participant, the conference server will send the uplink audio and video streams sent by the currently designated participant to the conference server to the virtual terminal instance based on the specified result of the media flow control mechanism. The virtual terminal instance receives the uplink audio and video streams, which are then used as the audio and video streams corresponding to the currently designated participant.

[0190] In some example embodiments of this application, based on the foregoing solution, the above-mentioned device further includes a virtual terminal scheduling module, which is configured to: Based on the number of participants in the target meeting, the monitoring duration required for each participant, and the target polling cycle, determine the number of virtual terminal instances required to complete the polling monitoring of the target meeting. If the number of determined virtual terminal instances is greater than 1, create multiple virtual terminal instances and control each virtual terminal instance to join the target meeting as a participant. Divide the participants in the participant list into multiple participant groups; Control multiple virtual terminal instances to perform target specification, audio and video stream reception, decoding, and quality detection for their respective participant groups.

[0191] In some example embodiments of this application, based on the foregoing scheme, the virtual terminal scheduling module is configured as follows: Based on the number of participants in the target meeting and the monitoring time required for each participant, determine the total monitoring time required to complete one round of full participant monitoring; Based on the relationship between the total monitoring duration and the target polling cycle, the number of virtual terminal instances is determined so that the time for each virtual terminal instance to complete a full round of monitoring of all participants after parallel polling does not exceed the target polling cycle. Based on the determined number of virtual terminal instances, the participants in the participant list are assigned to the corresponding virtual terminal instances, and the quality indicators obtained from each virtual terminal instance are aggregated on the monitoring server.

[0192] In some embodiments, the virtual terminal module 810 can also be used to disable or configure the media sending module of the virtual terminal instance to an inactive state, so that the virtual terminal instance does not send actual audio and video media data for mixing to the target conference. The virtual terminal module 810 can also be used to create and manage a virtual terminal pool, allocate idle virtual terminal instances from the virtual terminal pool according to the monitoring task of the target conference, and release virtual terminal instances or restore virtual terminal instances to an idle state after the monitoring task is completed.

[0193] In some embodiments, the management module 820 can also be used to complete identity authentication and encryption parameter negotiation through pre-configured certificates, pre-shared keys or key negotiation mechanisms when the virtual terminal instance joins a target conference using encrypted transmission, and to decrypt the encrypted audio and video stream after the virtual terminal instance receives it, so that the decrypted audio and video stream can enter the decoding and quality inspection process.

[0194] In some embodiments, the quality detection module 850 can also be used to perform at least one of the following on the decoded video frames: black screen detection, screen tearing detection, freeze detection, jitter detection, color cast detection, sharpness detection, and camera occlusion detection, and generate video quality indicators. The quality detection module 850 can also be used to perform at least one of the following on the decoded audio data: silence detection, howling detection, noise detection, abnormal volume detection, and intermittent detection, and generate audio quality indicators.

[0195] In some embodiments, the result aggregation module 860 can also be used to call the network management interface of the conference server, query the network quality data corresponding to the currently designated participant, and fuse and correlate the quality indicators obtained based on the decoded video frames and / or audio data with the network quality data to generate a comprehensive quality assessment result for the target conference. The result aggregation module 860 can also be used to store the quality indicators, network quality data and / or comprehensive quality assessment results corresponding to each participant in the database, and generate alarm information according to preset alarm rules.

[0196] In some embodiments, the video conferencing quality monitoring device may further include an adapter layer module. The adapter layer module can be used to encapsulate the interface differences between different conference server platforms through a unified adapter interface specification, and load the corresponding adapter according to the conference server type corresponding to the target conference, so as to complete participant list query, target specification, and network statistics query through a unified interface.

[0197] In some embodiments, the video conferencing quality monitoring device may further include a multi-conference parallel monitoring module. The multi-conference parallel monitoring module can be used to run multiple virtual terminal instance groups on a monitoring server, so that different virtual terminal instance groups correspond to different target conferences, and each group performs tasks such as obtaining the participant list, specifying the receiving target, receiving audio and video streams, decoding, and quality detection for its respective target conference, thereby generating quality monitoring results for different target conferences.

[0198] The specific details of each module of the video conferencing quality monitoring device based on virtual terminal polling have been described in detail in the corresponding video conferencing quality monitoring method based on virtual terminal polling, so they will not be repeated here.

[0199] It should be noted that although several modules or units of the video conferencing quality monitoring device based on virtual terminal polling have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0200] Furthermore, in an exemplary embodiment of this application, a video conferencing quality monitoring server is also provided. This server can be a dedicated monitoring server independent of the conferencing server, communicating with an external conferencing server via a standard network protocol. In other words, the monitoring server and the conferencing server are physically and deployment-wise separated, interacting via a standard network protocol. The monitoring server does not alter the internal structure and security configuration of the conferencing server, thus eliminating the need for intrusive modifications to the existing conferencing system.

[0201] In some embodiments, the monitoring server is deployed with a virtual container runtime environment, supporting the batch creation and destruction of virtual terminal instances. The virtual container runtime environment can be a container engine, a lightweight virtual machine, or other runtime environment capable of isolating and managing the lifecycle of virtual terminal instances. Through the virtual container runtime environment, the monitoring server can create multiple virtual terminal instances in batches according to the size of the target meeting and the needs of the monitoring task, and destroy the virtual terminal instances in batches after the monitoring task is completed, thereby releasing computing resources. As an optional implementation, each virtual terminal instance can run in an independent container instance, isolating them from each other in terms of resource consumption and fault impact; as another optional implementation, multiple virtual terminal instances can also run in the same container instance to reduce the overhead of container creation and scheduling.

[0202] The server may include one or more processors, memory coupled to the processors, and a network interface. The memory stores a computer program. When the processor executes the computer program, it implements the video conferencing quality monitoring method based on virtual terminal polling as described in any of the preceding embodiments. This method involves creating virtual terminal instances, controlling the virtual terminal instances to join the target conference organized by the conference server as participants, obtaining a list of participants and invoking the conference server's media stream control mechanism to poll and receive the audio and video streams corresponding to each participant, decoding and quality checking the audio and video streams, and generating quality monitoring results. The network interface is used for communication between the server and an external conference server.

[0203] In some embodiments, when the server's processor executes a computer program, it can also determine the number of virtual terminal instances and schedule parallel monitoring. Specifically, the processor can dynamically calculate the required number of virtual terminal instances based on the number of participants in the target meeting, the monitoring duration required for a single participant, and the maximum allowed time configured by the user to complete a full round of monitoring of all participants; or it can use a preset number as the number of virtual terminal instances. If the determined number of virtual terminal instances is greater than one, the processor can dynamically create multiple virtual terminal instances using a virtual container runtime environment and divide the participants in the participant list into multiple participant groups to allocate to the corresponding virtual terminal instances for parallel monitoring.

[0204] By integrating the dynamic calculation, batch creation, and participant grouping scheduling of virtual terminal instances into a dedicated monitoring server independent of the conference server, the number of virtual terminal instances can be flexibly expanded or contracted according to the conference size without modifying the conference server. This allows the dedicated monitoring server to adapt to different monitoring needs from small and medium-sized conferences to large-scale conferences.

[0205] Furthermore, in an exemplary embodiment of this application, an electronic device capable of implementing the above-described video conferencing quality monitoring method based on virtual terminal polling is also provided.

[0206] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented as: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0207] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present application. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0208] like Figure 9 As shown, the electronic device 900 is presented in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including storage unit 920 and processing unit 910), and a display unit 940.

[0209] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this application. For example, the processing unit 910 can perform actions such as... Figure 2Step S210 shows the creation of a virtual terminal instance, which has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability; Step S220 shows the control of the virtual terminal instance to join the target conference established by the conference server as a participant; Step S230 shows the acquisition of the participant list of the target conference, and the calling of the media stream control mechanism of the conference server according to the participant list, traversing the participant list in a loop according to the preset polling strategy, and sequentially designating each participant in the participant list as the receiving target of the virtual terminal instance, and sequentially receiving the audio and video stream corresponding to the currently designated participant; Step S240 shows the decoding and quality detection of the audio and video streams, obtaining the quality indicators corresponding to each participant, and generating the quality monitoring results of the target conference based on the quality indicators corresponding to each participant.

[0210] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0211] Storage unit 920 may also include a program / utility 924 having a set (at least one) program module 925, such program module 925 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0212] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0213] Electronic device 900 can also communicate with one or more external devices 970 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0214] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this application.

[0215] In exemplary embodiments of this application, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this application may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.

[0216] refer to Figure 10 As shown, a program product 1000 for implementing the above-described video conferencing quality monitoring method based on virtual terminal polling, according to an embodiment of this application, is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0217] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0218] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0219] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0220] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0221] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0222] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0223] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0224] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for video conference quality monitoring based on virtual terminal polling, characterized in that, The method includes: Create a virtual terminal instance, which is dedicated to quality monitoring and has video conferencing signaling connection capability, audio and video stream receiving capability, and audio and video stream decoding capability; Control the virtual terminal instance to join the target meeting organized by the meeting server as a participant; Obtain the list of participants in the target meeting, and call the media stream control mechanism of the meeting server according to the list of participants. According to the preset polling strategy, traverse the list of participants in a loop, and sequentially designate the receiving target of the virtual terminal instance as each participant in the list of participants, and sequentially receive the audio and video streams corresponding to the currently designated participants. The audio and video streams are decoded and their quality is tested to obtain the quality indicators corresponding to each participant. Based on the quality indicators corresponding to each participant, the quality monitoring results of the target meeting are generated.

2. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, Receiving the audio and video stream corresponding to the currently designated participant includes: When the receiving target of the virtual terminal instance is specified as the currently specified participant, the uplink audio and video stream sent by the currently specified participant to the conference server, which is issued by the conference server based on the specified result of the media stream control mechanism, is received as the audio and video stream corresponding to the currently specified participant.

3. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The audio and video streams include at least one of audio streams or video streams.

4. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The process of obtaining the list of participants for the target meeting includes: After the meeting is established, the meeting query interface of the meeting server is invoked, and / or the list of participants sent by the meeting server to the virtual terminal instance is received to obtain the complete list of participants of the target meeting, which serves as the basis list for subsequent polling.

5. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The method further includes: Based on the number of participants in the target meeting, the monitoring time required for each participant, and the maximum allowed time configured by the user to complete a round of full participant monitoring, determine the number of virtual terminal instances required to complete the polling monitoring of the target meeting. If the number of determined virtual terminal instances is greater than 1, multiple virtual terminal instances are created, and the multiple virtual terminal instances are controlled to join the target meeting as participants. The participants in the participant list are divided into multiple participant groups; The multiple virtual terminal instances are controlled to perform target specification, audio and video stream reception, decoding, and quality detection for their respective participant groups.

6. The virtual terminal polling based video conference quality monitoring method of claim 5, wherein, The determination of the number of virtual terminal instances required to complete the polling monitoring of the target meeting, based on the number of participants in the target meeting, the monitoring time required for each participant, and the maximum allowed time configured by the user to complete a round of full participant monitoring, includes: According to the formula determining the number of virtual terminal instances, wherein N is the number of participants, t is the monitoring time required for a single participant, T is the maximum allowed time configured by the user to complete a round of full-quantity participant monitoring, denotes rounding up.

7. The virtual terminal polling based video conference quality monitoring method of claim 5, wherein, The control of the multiple virtual terminal instances to perform receiving target specification, audio and video stream reception, decoding, and quality detection on the corresponding participant groups includes at least one of the following: Set different polling start times for each virtual terminal instance to achieve staggered polling times; Load balancing is performed based on the historical quality data of each participant. VIP attendees will be prioritized for allocation to virtual terminal instances with remaining computing resources exceeding a preset threshold.

8. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The method further includes: Based on the list of participants, a dynamic polling strategy is formulated according to at least one of the following factors: participant priority, historical quality results, and meeting status. The receiving targets are sequentially specified according to the dynamic polling strategy, the bitstream is received and decoded through the media stream control mechanism, and the quality of the decoded data is checked and the indicators are recorded. The subsequent polling strategy is dynamically adjusted based on the test results.

9. The virtual terminal polling based video conference quality monitoring method of claim 8, wherein, The dynamic polling strategy includes at least one of the following: Priority-based polling strategy: Priority is determined based on the participant's VIP ID, meeting role, and device type, and monitoring frequency is allocated according to priority; Historical quality-based polling strategy: Dynamically adjust the monitoring frequency based on historical quality scores; Event-triggered polling strategy: When a network event, quality event, or conference event is detected, a continuous monitoring mode is triggered.

10. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The decoding and quality detection of the audio and video streams include: The audio and video streams are decoded to obtain the original video frames and / or Pulse Code Modulation (PCM) audio data; The original video frames and / or pulse code modulation (PCM) audio data are subjected to quality detection to obtain quality indicators.

11. The virtual terminal polling based video conference quality monitoring method of claim 10, wherein, The quality inspection includes at least one of the following: Traditional audio and video anomaly detection: black screen detection, screen flickering detection, freeze detection, mute detection, noise detection, audio and video synchronization detection; Quality assessment based on traditional signal processing: objective quality assessment based on pixel differences or structural similarity; Perceptual quality assessment based on artificial intelligence and deep learning: Neural network-based no-reference or reference-based perceptual quality assessment.

12. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The media stream control mechanism includes at least one of the following: media stream control interface, subscription mechanism, selection mechanism, forwarding mechanism, routing mechanism, and media switching mechanism.

13. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, After completing the quality check of the currently designated participant, the signaling connection status of the virtual terminal instance in the target conference is maintained, and the receiving target is designated as the next participant through the media flow control mechanism.

14. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The method further includes: When the target conference uses encrypted transmission, identity authentication and encryption parameter negotiation are completed with the conference server through pre-configured certificates, pre-shared keys, or standard key negotiation mechanisms. The encrypted audio and video streams received by the virtual terminal instance are decrypted, and the decoding and quality detection are performed on the decrypted audio and video streams.

15. The virtual terminal polling based video conference quality monitoring method of claim 1, wherein, The media stream control mechanism invoked by the conference server includes: The media stream control mechanism of the conference server is invoked through the abstract adaptation layer. The abstract adaptation layer defines a unified adaptation interface specification and includes multiple adapters corresponding to different types of conference servers. Based on the type of the meeting server corresponding to the target meeting, load the corresponding adapter, and use the loaded adapter to convert the call based on the unified adaptation interface specification into an interface call supported by the meeting server.

16. A virtual terminal polling based video conference quality monitoring apparatus, characterized by, The device is a collection of logical functional modules deployed on a monitoring server, including: The virtual terminal module is used to instantiate and maintain the lifecycle of virtual terminal instances. The virtual terminal instances are dedicated to quality monitoring and have video conferencing signaling connection capabilities, audio and video stream receiving capabilities, and audio and video stream decoding capabilities. The management module is used to control virtual terminal instances to join the target meeting established by the meeting server and to obtain the list of participants in the target meeting; The polling control module is used to traverse the participant list in a loop according to a preset polling strategy, and sequentially designate each participant in the participant list as the receiving target of the virtual terminal instance. The media control module is used to invoke the media stream control mechanism of the conference server, so that the conference server sends the audio and video stream of the currently designated participant to the virtual terminal instance through the media channel, and the virtual terminal instance receives the audio and video stream corresponding to the currently designated participant. The quality inspection module is used to decode and inspect the distributed audio and video streams to obtain the corresponding quality indicators for each participant. The results aggregation module is used to generate quality monitoring results for the target meeting based on the quality indicators corresponding to each participant.

17. The virtual terminal polling based video conference quality monitoring apparatus of claim 16, wherein, The device further includes: a virtual terminal scheduling module, used to determine the required number of virtual terminal instances, the determination including: The calculation is performed dynamically based on the number of participants in the target meeting, the monitoring time required for each participant, and the maximum allowed time to complete a full round of monitoring of all participants as configured by the user, or a preset number can be used. When the number of virtual terminal instances is greater than 1, the virtual terminal module is triggered to dynamically create multiple virtual terminal instances and divide the participants in the participant list into multiple participant groups, which are then assigned to the corresponding virtual terminal instances for parallel monitoring.

18. An electronic device, comprising: It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-15.

20. A video conference quality monitoring server, characterized by The server is a dedicated monitoring server independent of the conference server, and communicates with the external conference server via a standard network protocol. The server includes: one or more processors; a memory coupled to the processors; and a network interface. The memory stores a computer program, and when the processor executes the computer program, it implements the method as described in any one of claims 1-15.

21. The video conference quality monitoring server of claim 20, wherein, The server is equipped with a virtual container runtime environment, which supports the batch creation and destruction of virtual terminal instances.

22. The video conference quality monitoring server of claim 20, wherein, When the processor executes the computer program, it also performs the following: dynamically calculates the required number of virtual terminal instances based on the number of participants in the target meeting, the monitoring time required for a single participant, and the maximum allowed time for completing a full round of participant monitoring configured by the user, or uses a preset number; when the number of virtual terminal instances is greater than 1, it dynamically creates multiple virtual terminal instances and divides the participants in the participant list into multiple participant groups to be assigned to the corresponding virtual terminal instances for parallel monitoring.