Video conference automatic roll call method and system based on audio and video intelligent analysis

Abstract

The invention provides a video conference automatic roll call method and system based on audio and video intelligent analysis. The method comprises the following steps: setting roll call parameters; collecting at least one kind of data of a conference terminal to be detected, wherein the data comprises audio data and video data; determining the communication state of the conference terminal based on the audio analysis result and / or the video analysis result; and generating a roll call result. According to the invention, by introducing an intelligent audio and video analysis technology, a traditional manual roll call mode is converted into automatic processing, the roll call efficiency is remarkably improved, and the waiting time is saved; a flexible parameter configuration mechanism is adopted, and application requirements of different scenes can be met; the video conference terminal supports portrait recognition and background image comparison functions, effectively prevents improper use of the conference terminal, and improves the standardization of the video conference.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to an automatic roll call method and system for video conferencing based on intelligent audio and video analysis. Background Art

[0002] In video conferencing scenarios, in order to ensure the quality of the meeting and the attendance of participants, the host usually needs to conduct a video roll call before the meeting starts. In the traditional video conference roll call process, the host needs to manually call the names of each terminal one by one to verify the communication status, including the normal operation of audio and video. This manual roll call method has many technical problems: when there are a large number of conference terminals, the host needs to invest a lot of time and energy in one-to-one polling; when checking the audio and video effects, it is often necessary to repeatedly confirm and debug, resulting in a long roll call process; other participants need to wait for the roll call to be completed before starting the meeting, resulting in a waste of time.

[0003] Similar problems also exist in the field of video surveillance. Managers need to regularly check the operating status of monitoring terminals to ensure that the audio and video transmission of the equipment is normal. Currently, they mainly rely on manual polling to access each monitoring terminal and check the picture quality and audio status one by one. This manual inspection method is not only labor-intensive, but also prone to omissions and errors, and it is difficult to meet the operation and maintenance needs of large-scale video surveillance systems. With the increase in the number of conference terminals and monitoring terminals, the traditional manual roll call and inspection method can no longer meet the actual application needs.

[0004] For some important occasions, in addition to confirming the communication status of the equipment, it is also necessary to verify the identity of the participants and the standardization of the meeting venue. The existing technology lacks solutions for differentiated detection for different terminals and different security levels, making it difficult to meet personalized needs such as identity authentication and scene verification. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for automatic roll call in video conferences based on intelligent analysis of audio and video, so as to solve the problem of cumbersome and inefficient roll call process in video conferences in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a method for automatic roll call in a video conference based on intelligent analysis of audio and video, comprising:

[0007] Setting roll call parameters, which include preset voice samples, preset video samples and device personalized configuration information;

[0008] Collecting at least one kind of data of the conference terminal to be detected, wherein the data includes audio data and video data;

[0009] Performing speech recognition processing on the audio data to obtain an audio analysis result and / or performing image recognition processing on the video data to obtain a video analysis result;

[0010] Determining the communication status of the conference terminal based on the audio analysis result and / or the video analysis result; and

[0011] Generate a roll call result and feed the roll call result back to the host terminal.

[0012] Optionally, the preset voice sample includes:

[0013] Standard phonetic keywords; and

[0014] The multilingual audio data corresponds to the standard voice keyword.

[0015] Optionally, the preset video sample includes:

[0016] Standard portrait data; and / or

[0017] Standard background image data.

[0018] Optionally, the device personalized configuration information includes:

[0019] Specifies the detection mode of the conference terminal, which includes audio-only detection, video-only detection, or audio-video dual detection.

[0020] Optionally, collecting audio data and video data of the conference terminal to be detected includes:

[0021] Automatically trigger the acquisition; or

[0022] Respond to manual trigger instructions to collect.

[0023] Optionally, the step of performing speech recognition processing on the audio data to obtain an audio analysis result includes:

[0024] Comparing the audio data with the preset voice samples in sequence;

[0025] When it is detected that the preset voice sample is included in the audio data, it is determined that the audio verification has passed, and an analysis result that the audio is normal is generated;

[0026] When the audio data fails to pass the comparison with the current preset voice sample and there are still unmatched preset voice samples, obtaining the next preset voice sample for comparison;

[0027] When all preset voice samples are compared and all fail, an analysis result of audio anomaly is generated.

[0028] Optionally, the step of performing image recognition processing on the video data to obtain a video analysis result includes:

[0029] Performing picture quality detection on the video data to determine whether there is an abnormal state;

[0030] When the image quality detection passes and the device personalized configuration information includes preset portrait data, performing portrait comparison;

[0031] When the picture quality detection passes and the device personalized configuration information includes preset background image data, performing background image comparison;

[0032] Generate video analysis results.

[0033] Optionally, the process of generating a roll call result and feeding the roll call result back to the host terminal includes:

[0034] Displaying the roll call result on the host terminal;

[0035] When an abnormal communication status is detected, a re-roll call option is displayed on the roll call result interface;

[0036] In response to the triggering operation of the re-roll option, the audio and video data collection is re-executed.

[0037] The present invention also provides a video conference automatic roll call system based on audio and video intelligent analysis, comprising:

[0038] A parameter configuration module, used to configure roll call parameters, wherein the roll call parameters include preset voice samples, preset video samples and device personalized configuration information;

[0039] The data collection module is used to respond to the roll call trigger command and collect the audio data and video data of the specified conference terminal;

[0040] A data transmission module, used for transmitting the audio data and video data to a data analysis module;

[0041] A data analysis module, configured to perform speech recognition processing on the audio data to obtain an audio analysis result and / or perform image recognition processing on the video data to obtain a video analysis result;

[0042] A status determination module, used for comprehensively determining the communication status of the conference terminal based on the audio analysis result and the video analysis result;

[0043] A result feedback module, used for generating roll call results and feeding back the roll call results to the host terminal;

[0044] A display module is used to display the roll call result interface on the host terminal;

[0045] The exception handling module is used to trigger the re-roll process when an abnormal communication status is detected.

[0046] Optionally, the data analysis module includes:

[0047] A speech recognition unit, configured to perform speech recognition processing;

[0048] The image analysis unit is used to perform picture normality detection, portrait comparison and background image comparison.

[0049] Compared with the prior art, the present invention has at least the following technical effects:

[0050] The present invention introduces preset voice samples and preset video samples, combined with intelligent analysis technology, to achieve automatic detection and determination of the communication status of conference terminals, without the need for manual polling by the host, greatly improving the efficiency of video conference roll call. At the same time, through a unified data transmission and analysis module, the stability and reliability of the roll call process are guaranteed.

[0051] The present invention also supports flexible detection mode configuration and personalized settings, and can selectively perform audio detection, video detection or dual detection according to the security level and usage scenarios of different terminals. Through the comparison function of standard portrait data and background image data, it effectively solves the identity authentication and scene verification requirements in important occasions, and further improves the security and standardization of the video conferencing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A block diagram of a method for automatic roll call in a video conference based on intelligent analysis of audio and video in an embodiment of the present invention;

[0053] Figure 2 The module diagram of a video conference automatic roll call system based on intelligent analysis of audio and video in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following is a description of a video conference automatic roll call method and system based on audio and video intelligent analysis of the present invention in conjunction with a schematic diagram, which shows a preferred embodiment of the present invention. It should be understood that those skilled in the art can modify the present invention described herein and still achieve the beneficial effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art and not as a limitation of the present invention.

[0055] The present invention is described in more detail in the following paragraphs by way of example with reference to the accompanying drawings. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and are not in exact proportions, and are only used to facilitate and clearly assist in illustrating the purpose of the embodiments of the present invention.

[0056] Embodiment 1

[0057] Please refer to Figure 1 This embodiment provides a method for automatic roll call in a video conference based on intelligent analysis of audio and video, and its main steps are as follows:

[0058] S1. Setting roll call parameters, wherein the roll call parameters include preset voice samples, preset video samples and device personalized configuration information;

[0059] S2. Collect at least one data of the conference terminal to be detected, wherein the data includes audio data and video data;

[0060] S3, performing speech recognition processing on the audio data to obtain an audio analysis result and / or performing image recognition processing on the video data to obtain a video analysis result;

[0061] S4, determining the communication status of the conference terminal based on the audio analysis result and / or the video analysis result; and

[0062] S5. Generate a roll call result and feed the roll call result back to the host terminal.

[0063] In this embodiment, compared to the method that requires the host to manually call the roll, the present invention adopts automated audio and video analysis technology to complete the roll call without human intervention, greatly reducing the workload of the host. By synchronously collecting and analyzing audio and / or video data, the communication status of the conference terminal can be selectively detected, while also avoiding the cumbersome process of separately checking audio and video in traditional technology, significantly improving the efficiency of roll call. The present invention adopts an automated analysis and judgment method to replace manual subjective judgment, which not only improves the accuracy and consistency of the detection, but also reduces the misjudgment caused by human factors, making the roll call result more reliable.

[0064] Specifically, in step S1, the preset voice sample includes a standard voice keyword and multilingual audio data corresponding to the standard voice keyword.

[0065] The multilingual audio data may include speech samples in different languages ​​or dialects.

[0066] In a specific example, the system can pre-record voice keywords in multiple languages ​​such as Mandarin, English, and Cantonese, support recognition of multiple languages ​​and dialects, allow users to expand voice samples according to actual needs, and improve the system's adaptability to different accents.

[0067] It should be noted that during the voice configuration process, a matching test of the voice configuration parameters is required. Specifically, the system will perform a matching test on the configured voice keywords and the standard audio, requiring a 100% matching rate to ensure the accuracy of the subsequent roll call process.

[0068] In a specific example, the system uses a common speech transcription technology to convert speech into text, and then detects whether the transcribed text contains preset keywords, thereby achieving speech recognition and matching. This method has high versatility and reliability.

[0069] The preset video sample includes standard portrait data and / or standard background image data.

[0070] In actual applications, standard portrait data can be collected in advance for conference terminals that require dedicated personnel to be on duty; for terminals that require meetings to be held in fixed locations, standard background image data can be collected in advance.

[0071] The device personalized configuration information includes a detection mode of a specified conference terminal, and the detection mode includes audio detection only, video detection only, or audio and video dual detection. This flexible configuration mechanism can adapt to the actual needs of different terminals and improve the practicality of the system.

[0072] In a specific example, flexible personalized configuration options are provided for different types of terminal devices:

[0073] Audio capture configuration:

[0074] You can configure specific devices not to collect audio data, which is suitable for pure video surveillance devices; you can set the trigger conditions and collection duration for audio collection.

[0075] Portrait data configuration:

[0076] Suitable for important conference terminals that require dedicated personnel on duty; supports entering portrait data from multiple angles to improve recognition accuracy; can set the strictness of portrait comparison.

[0077] Background data configuration:

[0078] Applicable to terminals that require to attend meetings in fixed locations; supports input of standard background image data; can set the allowable range of background changes.

[0079] In specific application scenarios, the combination of these personalized configurations can effectively meet the security management needs of different occasions:

[0080] In important meeting scenarios, you can enable both portrait recognition and background detection to ensure the standardization of participants and venues. In ordinary meeting scenarios, you can only perform basic audio and video quality detection. In monitoring scenarios, you can focus on configuring video picture detection parameters and ignore audio-related configurations.

[0081] Specifically, in step S2, the data collection method includes:

[0082] Automatically trigger collection: the system can automatically start the collection process at a predetermined time; and

[0083] Respond to manual triggering instructions to collect data: The host can manually trigger the collection operation according to actual needs.

[0084] In the actual collection process, the collection of audio and video data is carried out synchronously, and the longest collection time is 10 seconds. This embodiment not only ensures the integrity of the detection, but also avoids excessive waiting time, thereby improving the meeting efficiency.

[0085] The acquisition module adopts an audio and video acquisition method well known to those skilled in the art, which is a common method used by those skilled in the art and will not be described in detail.

[0086] Specifically, in step S3, the step of performing speech recognition processing on the audio data to obtain an audio analysis result specifically includes:

[0087] S311, comparing the audio data with the preset voice samples in sequence;

[0088] S312, when it is detected that a certain preset voice sample is included in the audio data, it is determined that the audio verification passes, and an analysis result that the audio is normal is generated;

[0089] S313, when the audio data fails to pass the comparison with the current audio sample, obtaining the next preset voice sample for comparison;

[0090] S314: When all preset voice samples have been compared and all have failed, an analysis result of audio anomaly is generated.

[0091] Specifically, in step S3, the step of performing image recognition processing on the video data to obtain the video analysis result specifically includes:

[0092] S321, performing picture quality detection on the video data to determine whether there is an abnormal state;

[0093] S322, when the picture quality detection passes and the device personalized configuration information includes preset portrait data, performing portrait comparison;

[0094] S323, when the picture quality detection passes and the device personalized configuration information includes preset background image data, performing background image comparison;

[0095] S324: Generate video analysis results.

[0096] It should be noted that, in steps S322-S323, if the picture quality detection passes, but if the device personalized configuration information containing preset portrait data or preset background image data is not configured, steps S322-S323 are skipped.

[0097] When performing picture quality detection on video data, the system specifically detects the following abnormal conditions:

[0098] Freeze: Detects whether the video frame is still or unchanged.

[0099] Video stream interruption: Detects whether there is video stream data transmission.

[0100] Abnormal brightness: Check whether the overall brightness of the picture is within a reasonable range.

[0101] Color cast: Check if the image color is abnormally shifted.

[0102] Signal loss or omission: Detects the integrity and continuity of the video signal.

[0103] In addition, step S5 specifically includes:

[0104] S51, displaying the roll call result on the host terminal;

[0105] S52, when an abnormal communication status is detected, displaying a re-roll call option on the roll call result interface;

[0106] S53: Responding to the triggering operation of the re-roll option, re-execute the audio and video data collection.

[0107] In this embodiment, audio analysis adopts a step-by-step comparison mechanism, video analysis includes quality detection and feature comparison, and supports re-collection when abnormalities are detected. The detailed analysis mechanism effectively improves the reliability of roll call.

[0108] In summary, through the configuration mechanism of standard voice keywords and multilingual audio data, this embodiment can adapt to the application requirements of different language environments, significantly improving the versatility and practicality of the system. In addition, this embodiment introduces a comparison mechanism for standard portrait data and background image data, which further solves the following problems: ensuring that designated personnel are on duty, preventing conference terminals from being moved to non-specified locations, and ensuring the standardization of video conferences. A flexible detection mode configuration scheme is adopted. This differentiated detection strategy not only ensures the detection effect, but also optimizes the use of system resources. A multi-level comparison strategy is adopted in the analysis and processing process to ensure the accuracy of the detection results.

[0109] Embodiment 2

[0110] This embodiment provides a video conference automatic roll call system based on audio and video intelligent analysis. Figure 2 , including: parameter configuration module, data acquisition module, data transmission module, data analysis module, state determination module and result feedback module.

[0111] The parameter configuration module is used to configure roll call parameters, which include preset voice samples, preset video samples and device personalized configuration information.

[0112] The data acquisition module is connected to the parameter configuration module and is used to respond to the roll call trigger instruction and collect the audio data and video data of the designated conference terminal.

[0113] The data acquisition module includes an audio acquisition unit and a video acquisition unit, which are responsible for the acquisition of audio data and video data respectively.

[0114] The data transmission module is connected to the data acquisition module and is used to transmit the audio data and video data to the data analysis module.

[0115] The data transmission adopts a network transmission method commonly used in the art, and a suitable transmission protocol can be selected according to the actual network environment.

[0116] Furthermore, the data analysis module includes:

[0117] The speech recognition unit is used to perform speech recognition processing.

[0118] The image analysis unit is used to perform picture normality detection, portrait comparison and background image comparison.

[0119] In a specific example, when performing speech recognition processing, the system adopts the following specific steps:

[0120] 1. Use common speech transcription technology to convert the collected audio data into text;

[0121] 2. Perform word segmentation and keyword extraction on the transcribed text content;

[0122] 3. Match the extracted keywords with the preset voice keywords;

[0123] 4. If the match is successful, the audio verification is considered to be successful;

[0124] 5. If the match fails, try matching with samples in other languages ​​or dialects.

[0125] Furthermore, the status determination module is connected to the data analysis module and is used to comprehensively determine the communication status of the conference terminal based on the audio analysis result and the video analysis result.

[0126] The result feedback module is connected to the state determination module, and is used to generate a roll call result and feed the roll call result back to the host terminal.

[0127] In addition, the system includes:

[0128] A display module is used to display the roll call result interface on the host terminal;

[0129] The exception handling module is used to trigger the re-roll process when an abnormal communication status is detected.

[0130] In summary, the present invention provides an automatic roll call method and system for video conferences based on intelligent audio and video analysis, which realizes the automation of conference roll call by introducing audio and video analysis, greatly improves the efficiency of roll call, and saves the waiting time of participants. A flexible parameter configuration mechanism is adopted, which supports multi-language audio samples and personalized detection modes, and can adapt to the application requirements of different scenarios. It supports portrait recognition and background image comparison functions, effectively prevents improper use of conference terminals, and improves the standardization and security of video conferences; in addition, through automated audio and video quality detection, communication anomalies are discovered and fed back in time, thereby improving the reliability of the video conferencing system. The present invention is not only suitable for conference roll call scenarios, but can also be used for equipment inspections in fields such as video surveillance, and has a wide range of application value.

[0131] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A video conference automatic roll call method based on audio and video intelligent analysis, characterized in that: include: Setting roll call parameters, which include preset voice samples, preset video samples and device personalized configuration information; Collecting at least one kind of data of the conference terminal to be detected, wherein the data includes audio data and video data; Performing speech recognition processing on the audio data to obtain an audio analysis result and / or performing image recognition processing on the video data to obtain a video analysis result; Determining the communication status of the conference terminal based on the audio analysis result and / or the video analysis result; as well as Generate a roll call result and feed the roll call result back to the host terminal.

2. The method for automatic roll call in video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The preset voice samples include: Standard phonetic keywords; and The multilingual audio data corresponds to the standard voice keyword.

3. The method for automatic roll call in video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The preset video samples include: Standard portrait data; and / or Standard background image data.

4. The method for automatic roll call in video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The device personalized configuration information includes: Specifies the detection mode of the conference terminal, which includes audio-only detection, video-only detection, or audio-video dual detection.

5. The method for automatic roll call in video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The collecting of audio data and video data of the conference terminal to be detected includes: Automatically trigger the acquisition; or Respond to manual trigger instructions to collect.

6. The method for automatic roll call in a video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The step of performing speech recognition processing on the audio data to obtain an audio analysis result comprises: Comparing the audio data with the preset voice samples in sequence; When it is detected that the preset voice sample is included in the audio data, it is determined that the audio verification has passed, and an analysis result that the audio is normal is generated; When the audio data fails to pass the comparison with the current preset voice sample and there are preset voice samples that have not been compared, obtaining the next preset voice sample for comparison; When all preset voice samples are compared and all fail, an analysis result of audio anomaly is generated.

7. The method for automatic roll call in a video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The step of performing image recognition processing on the video data to obtain a video analysis result comprises: Performing picture quality detection on the video data to determine whether there is an abnormal state; When the image quality detection passes and the device personalized configuration information includes preset portrait data, performing portrait comparison; When the picture quality detection passes and the device personalized configuration information includes preset background image data, performing background image comparison; Generate video analysis results.

8. The method for automatic roll call in a video conference based on intelligent audio and video analysis according to claim 1 is characterized in that: The process of generating the roll call result and feeding back the roll call result to the host terminal includes: Displaying the roll call result on the host terminal; When an abnormal communication status is detected, a re-roll call option is displayed on the roll call result interface; In response to the triggering operation of the re-roll option, the audio and video data collection is re-executed.

9. An automatic roll call system for video conferences based on intelligent audio and video analysis, characterized in that: include: A parameter configuration module, used to configure roll call parameters, wherein the roll call parameters include preset voice samples, preset video samples and device personalized configuration information; The data collection module is used to respond to the roll call trigger command and collect the audio data and video data of the specified conference terminal; A data transmission module, used for transmitting the audio data and video data to a data analysis module; A data analysis module, configured to perform speech recognition processing on the audio data to obtain an audio analysis result and / or perform image recognition processing on the video data to obtain a video analysis result; A status determination module, used for comprehensively determining the communication status of the conference terminal based on the audio analysis result and the video analysis result; A result feedback module, used for generating roll call results and feeding back the roll call results to the host terminal; A display module is used to display the roll call result interface on the host terminal; The exception handling module is used to trigger the re-roll process when an abnormal communication status is detected.

10. The video conference automatic roll call system based on audio and video intelligent analysis according to claim 9 is characterized in that: The data analysis module includes: A speech recognition unit, configured to perform speech recognition processing; The image analysis unit is used to perform picture normality detection, portrait comparison and background image comparison.

Images