Video conference exception identification processing method and device, electronic equipment and storage medium
Through deep learning algorithm models, the problem that cannot be identified and processed in real time in the prior art is solved, and the quality and user experience of video conferences are improved.
Patent Information
- Application Number
- CN202411301780.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-10
AI Technical Summary
The prior art cannot accurately identify and handle abnormalities in video conferences in real time, resulting in equipment failures, network problems or abnormal video source, degradation or interruption of conference quality, affecting work efficiency and communication effect.
Deep learning algorithm model, including convolutional neural network and recurrent neural network, processes video sources and terminal real-time data, recognizes exception information, and processes it according to preset exception processing strategies, including switching video sources, adjusting code rates, Automatic disconnection and reconnection and switch backup devices.
Real-time identification and processing of abnormalities in video conferencing is realized, the quality and user experience of meetings are improved, the possibility of interruptions and quality declines is reduced, and a stable and efficient video conferencing experience is ensured.
Smart Images
Figure CN119110056A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video communication technology, and in particular to a method, device, electronic device and storage medium for identifying and processing video conference anomalies. Background Art
[0002] With the rapid development of information technology, remote conferences such as video conferences and online conferences have become an important part of daily work.
[0003] However, in the existing technology, the monitoring of conference quality mainly relies on manual monitoring, which is unable to identify and handle anomalies in video conferences in real time and accurately. During the meeting, the quality of the meeting is reduced or even interrupted due to equipment failure, network problems or video source anomalies, which seriously affects work efficiency and communication effectiveness. Summary of the invention
[0004] In view of this, the present application provides a video conference anomaly identification and processing method, device, electronic device and storage medium, which are used to solve the technical problem of being unable to accurately identify and process video conference anomalies in real time.
[0005] In the first aspect, the present application provides a method for identifying and processing video conference anomalies, which adopts the following technical solutions: Acquire real-time data of the video conference, the real-time data including video source real-time data and terminal real-time data; Processing the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time abnormal information; Execute a preset exception handling strategy according to the real-time exception information; Generate exception handling result information according to the preset exception handling strategy.
[0006] In an optional implementation, the processing of the video source real-time data and the terminal real-time data according to a preset deep learning algorithm model to obtain real-time abnormal information includes: The real-time data of the video source is trained according to a convolutional neural network model to obtain real-time abnormal information of the video source; The real-time data of the terminal is trained according to a recurrent neural network model to obtain real-time abnormal information of the terminal.
[0007] In an optional implementation, executing a preset exception handling strategy according to the real-time exception information includes: When the real-time abnormal information is the real-time abnormal information of the video source, executing a first preset abnormality handling strategy according to the real-time abnormal information of the video source; When the real-time exception information is the real-time exception information of the terminal, executing a second preset exception handling strategy according to the real-time exception information of the terminal; When the real-time exception information includes the real-time exception information of the video source and the real-time exception information of the terminal, a third preset exception handling strategy is executed according to the real-time exception information of the video source and the real-time exception information of the terminal.
[0008] In an optional implementation, executing a first preset exception handling strategy according to the real-time exception information of the video source includes: Get the preset backup video source; When the operation state of the preset backup video source is normal, switching the video source to the preset backup video source; When the running state of the preset backup video source is abnormal, a preset background image is obtained, and the preset backup video source is switched to the preset background image.
[0009] In an optional implementation, executing a second preset exception handling strategy according to the real-time exception information of the terminal includes: Obtaining the abnormal information type of the real-time abnormal information of the terminal; When the abnormal information type is abnormal audio and video quality, adjusting the audio and video bit rate of the terminal according to the real-time abnormal information of the terminal; When the abnormal information type is a video connection abnormality, automatically disconnecting and reconnecting the terminal; When the abnormal information type is device performance abnormality, the terminal is switched to a standby terminal.
[0010] In an optional implementation, executing a third preset exception handling strategy according to the real-time exception information of the video source and the real-time exception information of the terminal includes: Determine a first exception handling priority of the real-time exception information of the video source and a second exception handling priority of the real-time exception information of the terminal; Comparing the first exception handling priority with the second exception handling priority to obtain a comparison result; Determining an execution order of the first preset exception handling strategy and the second preset exception handling strategy according to the comparison result; Exception processing is performed on the real-time exception information of the video source and the real-time exception information of the terminal according to the execution order.
[0011] In an optional embodiment, the method further comprises: When the real-time abnormal information is not identified, face recognition and identity recognition are started to identify the video source and the participants of the terminal to obtain a recognition result; Acquire feature information of the recognition result, match the feature with a pre-stored feature template according to the feature, and if the match is successful, record the conference information of the video conference.
[0012] In a second aspect, the present application provides a video conference anomaly identification and processing device, the system comprising: A data acquisition module is used to obtain real-time data of the video conference, wherein the real-time data includes real-time data of the video source and real-time data of the terminal; An abnormality identification module, used to process the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time abnormality information; An exception handling module is used to execute a preset exception handling strategy according to the real-time exception information; The information integration module is used to generate exception handling result information according to the preset exception handling strategy.
[0013] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the video conferencing anomaly identification and processing method when executing the computer program.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the video conference anomaly identification and processing method.
[0015] The present application obtains real-time data of a video conference; trains the real-time data according to a preset deep learning algorithm model to obtain specific real-time exception information; and executes a preset exception handling strategy based on the real-time exception information. When a real-time exception occurs at a video source, a first preset exception handling strategy is executed; when a real-time exception occurs at a terminal, a second preset handling strategy is executed; when both are abnormal, a third preset exception handling strategy is executed, thereby resolving abnormal situations occurring in a video conference, and obtaining exception handling result information based on the real-time exception information and the preset exception handling strategy, so as to accurately identify, process and record abnormal information in a video conference in real time, thereby improving the quality of the video conference and the user's video experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A flowchart of a method for identifying and processing video conference anomalies shown in an embodiment of the present application; Figure 2 A functional module diagram of a video conference anomaly identification and processing device shown in an embodiment of the present application; Figure 3 This is a flowchart of the operation of the video conference anomaly identification and processing system shown in an embodiment of the present application; Figure 4 A structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION
[0017] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.
[0018] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0019] See also Figure 1 As shown, the video conference anomaly identification and processing method shown in the embodiment of the present application includes the following steps.
[0020] S11, obtain real-time data of the video conference.
[0021] The real-time data includes real-time data of the video source and real-time data of the terminal, and the video conference includes a source device that initiates the video conference and multiple terminals that participate in the video conference.
[0022] Video source real-time data refers to the real-time data information collected from the source device that provides the video signal, which may include the video image data, audio data, device status data, transmission parameter data, and environmental data of the source device; terminal real-time data refers to the real-time data information collected from the terminals participating in the meeting (such as computers, smart phones, video conferencing equipment, etc.), which may include the terminal's audio data, video data, network communication data, device status data, and user behavior data.
[0023] Through the data acquisition module of the video conferencing system, the real-time data of the video source of the source device and the real-time data of each terminal participating in the video conference can be captured in real time, and the collected real-time data of the video source and the real-time data of the terminal can be sent to the main server, thereby obtaining the real-time data of the video conference.
[0024] In the above embodiment, by acquiring the real-time data of the video source and the real-time data of the terminal of the video conference, abnormal problems existing in the video conference can be discovered in real time. For example, the system can analyze the real-time data of the video source, so as to timely discover abnormal problems existing in the source device, such as blurred images, frozen images, etc. The system can analyze the real-time data of the video source, and can timely discover abnormal problems existing in the terminal, such as network delays, equipment failures, etc.
[0025] S12, processing the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time abnormal information.
[0026] Among them, the preset deep learning algorithm model is located in the anomaly recognition module of the main server, including a convolutional neural network model and a recurrent neural network model; the real-time anomaly information includes real-time anomaly information of the video source and real-time anomaly information of the terminal.
[0027] The convolutional neural network model can effectively extract spatial features when processing image data, which is used to train the real-time data of the video source and obtain abnormal information of the video source; the convolutional neural network model can effectively process sequence data and capture the timing information between the data, which is used to train the real-time data of the terminal and obtain the real-time abnormal information of the terminal.
[0028] It should be noted that the real-time abnormal information of the video source mainly refers to the abnormal display of the screen of the video source device, which may include signal loss, screen deformation or freezing, frame rate loss, etc. The real-time abnormal information of the terminal mainly involves the abnormal audio transmission between the terminal and the source device, which may include sound distortion, audio or video freeze, screen freezing, etc. The real-time abnormal information of the video source and the real-time abnormal information of the terminal will overlap in certain abnormalities. For example, if the screen freezes during a video conference, it may be the screen abnormality of the source device or the screen abnormality of the terminal. It is necessary to train the real-time data according to the preset deep learning algorithm model to determine the specific source and form of expression of the real-time abnormal information.
[0029] In the above embodiment, by training real-time data with a preset deep learning algorithm model, real-time abnormal information of the video source and the terminal in the video conference can be obtained. Based on the real-time abnormal information, the system can execute corresponding strategies to solve the real-time abnormal information, thereby ensuring the quality of the meeting and user experience.
[0030] In an optional implementation, the processing of the video source real-time data and the terminal real-time data according to a preset deep learning algorithm model to obtain real-time abnormal information includes: The real-time data of the video source is trained according to a convolutional neural network model to obtain real-time abnormal information of the video source; The real-time data of the terminal is trained according to a recurrent neural network model to obtain real-time abnormal information of the terminal.
[0031] After obtaining the real-time data of the video source, it is trained through the convolutional neural network model. The convolutional neural network model can learn various features of the picture in the real-time data of the video source, such as edges, textures, colors, etc., and construct spatial features through multi-layer convolution and pooling operations, thereby obtaining real-time abnormal data of the video source such as blurred video pictures and color anomalies.
[0032] After obtaining the real-time data of the video source, it is trained through the recurrent neural network model. The real-time data of the terminal mainly includes the real-time interaction of audio data and video data based on time series between the terminal and the source device, that is, the real-time data of the terminal will change over time. The recurrent neural network model can capture the time dependency in the real-time data of the terminal. Through the training of the recurrent neural network model, the system can identify real-time abnormal information of the terminal such as sound distortion and freeze.
[0033] In the above embodiment, real-time data is trained by a preset deep learning algorithm model, so as to accurately capture the real-time exception information of the video source and the real-time exception information of the terminal, provide accurate real-time exception information support for the system to handle exceptions, and ensure the quality of video conferencing.
[0034] S13: Execute a preset exception handling strategy according to the real-time exception information.
[0035] According to the real-time abnormal information in the real-time data obtained by the abnormal identification module, it can be divided into three abnormal situations. The first is that the real-time abnormal information only includes the real-time abnormal information of the video source, that is, the abnormality of the video conference only occurs in the source device; the second is that the real-time abnormal information only includes the real-time abnormal information of the terminal, that is, the abnormality of the video conference only occurs in the terminal; the third is that the real-time abnormal information includes both the real-time abnormal information of the video source and the real-time abnormal information of the terminal, that is, there are abnormalities in both the source device and the terminal of the video conference.
[0036] Based on the above three abnormal situations, the preset abnormality handling strategies include a first preset abnormality handling strategy, a second preset abnormality handling strategy and a third preset abnormality handling strategy. In the abnormality handling module of the system, the corresponding preset abnormality handling strategy can be automatically called for processing according to the above abnormal situation.
[0037] In the above embodiment, by presetting the exception handling strategy, the system can respond to the acquired abnormal situation in a timely manner and execute the preset processing strategy, thereby minimizing the possibility of video conference interruption or quality degradation, improving the overall quality and efficiency of the video conference, and providing users with a stable and efficient video conferencing experience.
[0038] In an optional implementation, executing a preset exception handling strategy according to the real-time exception information includes: When the real-time abnormal information is the real-time abnormal information of the video source, executing a first preset abnormality handling strategy according to the real-time abnormal information of the video source; When the real-time exception information is the real-time exception information of the terminal, executing a second preset exception handling strategy according to the real-time exception information of the terminal; When the real-time exception information includes the real-time exception information of the video source and the real-time exception information of the terminal, a third preset exception handling strategy is executed according to the real-time exception information of the video source and the real-time exception information of the terminal.
[0039] In the exception handling module, when the acquired real-time exception information only includes the real-time exception information of the video source, it is determined that the real-time video conference exception occurs in the source device, and the corresponding first preset exception handling strategy is executed. When the acquired real-time exception information only includes the real-time exception information of the terminal, it is determined that the real-time video conference exception occurs in the terminal, and the corresponding second preset exception handling strategy is executed. When the acquired real-time exception information includes the real-time exception information of the video source and the real-time exception information of the terminal, it is determined that the real-time video conference exception occurs both in the source device and in the terminal, and the corresponding third preset exception handling strategy is executed.
[0040] In the above embodiment, the real-time abnormal information is targeted at the abnormal source and the corresponding processing strategy is adopted, so that the real-time abnormal situation can be solved more efficiently. For example, when the abnormality only involves the video source, the system will focus on the abnormal problem of the source device and execute the first preset abnormality processing strategy; similarly, when the abnormality only involves the terminal, the second processing strategy is executed for the terminal. When the abnormality involves both the source device and the terminal, the third preset abnormality processing strategy is adopted to maximize the continuity and quality of the meeting.
[0041] In an optional implementation, executing a first preset exception handling strategy according to the real-time exception information of the video source includes: Get the preset backup video source; When the operation state of the preset backup video source is normal, switching the video source to the preset backup video source; When the running state of the preset backup video source is abnormal, a preset background image is obtained, and the preset backup video source is switched to the preset background image.
[0042] Among them, the preset backup video source is a pre-set known video input, which can include a screen sharing device, a live stream, or other video sources. These sources can come from different line channels, such as different hardware devices, network transmission channels, or software input sources. The system can switch or adjust as needed. The preset background screen is a background image pre-set in the system, which can be a meeting theme, company logo, brand element, etc., which is used to switch to the preset background screen for display when the backup video source is abnormal.
[0043] During a video conference, the system will continuously obtain the operating status of the video source and the preset backup video source. When the source device has real-time abnormal information about the video source, the system will immediately switch the video source to the preset backup video source to prevent affecting the video conference screen display of the source device. When the preset backup video source also has abnormal operation, the system will switch the conference screen to the preset background screen to prevent video screen loss or other abnormal situations.
[0044] In the above embodiment, when the video source of the source device is abnormal, the system will immediately switch to the preset backup video source, thereby reducing the possibility of video conference interruption and improving the stability of the system and user experience. At the same time, if the preset backup video source is also abnormal, it can also switch to the preset backup background screen to ensure that the user can still keep the meeting going in multiple abnormal situations.
[0045] In an optional implementation, executing a second preset exception handling strategy according to the real-time exception information of the terminal includes: Obtaining the abnormal information type of the real-time abnormal information of the terminal; When the abnormal information type is abnormal audio and video quality, adjusting the audio and video bit rate of the terminal according to the real-time abnormal information of the terminal; When the abnormal information type is a video connection abnormality, automatically disconnecting and reconnecting the terminal; When the abnormal information type is device performance abnormality, the terminal is switched to a standby terminal.
[0046] The types of abnormal information of the real-time abnormal information of the terminal may include abnormal audio and video quality, abnormal video connection, and abnormal device performance. Among them, abnormal audio and video quality may include blurred or stuck video images, intermittent or noisy audio, increased audio delay, etc.; abnormal video connection may include network disconnection resulting in the inability to transmit video, server failure resulting in connection interruption, abnormal network configuration, etc. Abnormal device performance may include excessive CPU or GPU load, slow operation due to insufficient memory, and failure of camera or microphone devices, etc.
[0047] When the exception handling module detects that the audio and video quality is abnormal, the audio and video bit rate adjustment strategy is executed.
[0048] The audio and video bitrate adjustment strategy is a strategy to dynamically adjust the audio and video transmission bitrate based on the real-time situation of network bandwidth and device performance. In this application, when the network bandwidth is low or the device performance is poor, the system will reduce the transmission bitrate of audio and video to ensure the smoothness and stability of the video conference. When the network conditions improve or the device performance improves, the bitrate can be increased in time to improve the clarity and quality of the audio and video. The specific quantitative values are preset in advance by the relevant conference personnel.
[0049] When the exception handling module detects an abnormal video connection, the disconnection and reconnection strategy is executed. The disconnection and reconnection strategy refers to the strategy for dealing with network connection abnormalities. When a video connection abnormality is detected, such as a network disconnection or server failure, the system automatically attempts to disconnect the current connection and try to reestablish the connection within a preset period of time to prevent the meeting from being interrupted due to network problems and improve the continuity and stability of the meeting.
[0050] When the exception handling module detects abnormal device performance, it executes the backup device switching strategy, which refers to the strategy of automatically switching to the backup terminal when a terminal fails. For example, when the main camera or main microphone of the terminal fails, the system will automatically switch to the backup camera or backup microphone to ensure that the meeting can continue.
[0051] It should be noted that the audio and video bit rate adjustment strategy, disconnection and reconnection strategy, and backup device switching strategy can all be changed according to different conference requirements. In the terminal, multiple exceptions may occur at the same time. The system can execute corresponding processing strategies for different exceptions at the same time, thereby solving the corresponding terminal exceptions and ensuring the stability of the video conference.
[0052] In the above embodiment, based on different types of terminal abnormal information, the system can respond to different abnormal situations of the terminal by automatically adjusting the audio and video bit rate strategy, the automatic reconnection strategy when disconnected, and the backup device switching strategy, ensuring that users can get a stable and smooth video conferencing experience.
[0053] In an optional real-time mode, executing a third preset exception handling strategy according to the real-time exception information of the video source and the real-time exception information of the terminal includes: Determine a first exception handling priority of the real-time exception information of the video source and a second exception handling priority of the real-time exception information of the terminal; Comparing the first exception handling priority with the second exception handling priority to obtain a comparison result; Determining an execution order of the first preset exception handling strategy and the second preset exception handling strategy according to the comparison result; Exception processing is performed on the real-time exception information of the video source and the real-time exception information of the terminal according to the execution order.
[0054] Among them, the exception handling priority of the real-time exception information of the video source and the real-time exception information of the terminal is set according to different real-time scenarios. Based on the importance of the people participating in the video conference, the video source exception and terminal exception handling priority can be dynamically adjusted. For example, when the main person in the video conference uses the terminal, the second exception handling priority of the corresponding terminal can be set higher, and when the main person in the video conference uses the source device, the first exception handling priority of the source device can be set higher.
[0055] When an exception occurs between the video source and the terminal, if the first exception handling priority of the source device is higher, the system will immediately execute the first preset exception handling strategy, and then execute the second preset exception handling strategy; if the second exception handling priority of the terminal is higher, the system will immediately execute the second preset exception handling strategy, and then execute the first preset exception handling strategy.
[0056] In the above embodiment, by processing the real-time abnormal information of the video source and the terminal with priority, the response speed and fault tolerance of the video conferencing system are effectively improved. When facing a complex network and equipment environment, the system can quickly identify and handle key abnormal situations, enhance the stability and reliability of the overall system, and thus improve user satisfaction and the work efficiency of video conferencing.
[0057] S14: Generate exception handling result information according to the preset exception handling strategy.
[0058] When the exception handling module executes the preset exception handling strategy, the information integration module of the system will further analyze whether it is a terminal exception or a video source exception based on the real-time exception information. For terminal exceptions, the system will record the real-time exception information of the terminal, obtain the processing results of the exception handling module, generate a visual processing report based on the real-time exception information and processing results of the terminal, and send it to the corresponding terminal to facilitate maintenance by relevant personnel; similarly, for video source failures, the system will record the real-time exception information of the video source, and generate a visual processing report based on the real-time exception information and processing results of the video source and send it to the corresponding source device. In addition, the system will record and store the corresponding abnormal events or fault information, and users can easily query and generate reports through dedicated interfaces or tools for subsequent analysis and optimization, providing data support for the continuous improvement of meetings.
[0059] In the above embodiment, the system records and integrates the detailed result information of exception processing, and converts it into a visual processing report and sends it to the corresponding terminal or source device, so that relevant personnel can intuitively understand the results and impact of exception processing, and provide a reference basis for relevant personnel to equipment abnormalities, helping them to adjust and repair system problems more quickly and accurately, and realize timely recording, analysis and feedback of exception processing results.
[0060] In an optional embodiment, the method further comprises: When the real-time abnormal information is not identified, face recognition and identity recognition are started to identify the video source and the participants of the terminal to obtain a recognition result; Acquire feature information of the recognition result, match the feature with a pre-stored feature template according to the feature, and if the match is successful, record the conference information of the video conference.
[0061] When real-time abnormal information is obtained, it means that there is no abnormality in the video conference and the conference is carried out normally. The participants are identified by starting face recognition and identity recognition, and the information of the participants, including identity and facial information, is automatically recorded. At the same time, the obtained information is recorded and the feature information is obtained, where the feature information may include facial expression feature information, facial contour feature information, identity level feature information, etc.
[0062] After obtaining the corresponding feature information, the feature information will be matched with the pre-stored feature template to confirm whether the participant is the corresponding correct person, thereby improving the accuracy of conference participant identification. When the match is completed and the participant is confirmed to be the corresponding correct person, the conference information of the video conference will be recorded.
[0063] In the above embodiment, face recognition and identity recognition technology are used to automatically identify and record the participants in the video conference, which greatly improves the efficiency and accuracy of the conference. In addition, by matching with pre-stored feature templates, the accuracy of the identity of the participants can be further ensured to avoid misidentification or identity confusion, which helps to improve the security and management efficiency of the video conference and provides a convenient and reliable means of managing participants for the organizers of the video conference.
[0064] To summarize, the present application obtains real-time data of a video conference; trains the real-time data according to a preset deep learning algorithm model to obtain specific real-time exception information; and executes a preset exception handling strategy based on the real-time exception information. When the video source has a real-time exception, the first preset exception handling strategy is executed; when the terminal has a real-time exception, the second preset handling strategy is executed; when both are abnormal, the third preset exception handling strategy is executed to resolve the corresponding exceptions, and obtains exception handling result information based on the real-time exception information and the preset exception handling strategy, thereby realizing real-time and accurate identification, processing and recording of exception information in the video conference, thereby improving the quality of the video conference and the user's video experience.
[0065] See also Figure 2 The above is a functional module diagram of the video conference anomaly identification and processing device provided in an embodiment of the present application.
[0066] In some embodiments, the video conference abnormality identification processing device 20 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the video conference abnormality identification processing device 20 may be stored in the memory of the video conference abnormality identification processing system and executed by at least one processor to perform (see Figure 1 Description) Function of the video conference exception identification and processing method.
[0067] In this embodiment, the video conference abnormality identification and processing device 20 can be divided into multiple functional modules according to the functions it performs. The functional modules may include: a data collection module 201, an abnormality identification module 202, an abnormality processing module 203, an information integration module 204, and a conference recording module 205.
[0068] The data acquisition module 201 is used to obtain real-time data of the video conference, and the real-time data includes real-time data of the video source and real-time data of the terminal.
[0069] The anomaly identification module 202 is used to process the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time anomaly information.
[0070] The exception handling module 203 is used to execute a preset exception handling strategy according to the real-time exception information.
[0071] The information integration module 204 is used to generate exception handling result information according to the preset exception handling strategy.
[0072] The conference recording module 205 is used to start face recognition and identity recognition to identify the participants of the video source and the terminal when the real-time abnormal information is not identified, and obtain the recognition result; obtain the feature information of the recognition result, match the feature with the pre-stored feature template according to the feature, and if the match is successful, record the conference information of the video conference.
[0073] The present application obtains real-time data of a video conference; trains the real-time data according to a preset deep learning algorithm model to obtain specific real-time exception information; and executes a preset exception handling strategy based on the real-time exception information. When a real-time exception occurs at a video source, a first preset exception handling strategy is executed; when a real-time exception occurs at a terminal, a second preset handling strategy is executed; when both are abnormal, a third preset exception handling strategy is executed to resolve the corresponding exceptions, and exception handling result information is obtained based on the real-time exception information and the preset exception handling strategy, so as to realize real-time and accurate identification, processing and recording of exception information in a video conference, thereby improving the quality of the video conference and the user's video experience.
[0074] See also Figure 3 The figure is an operation flow chart of a video conference exception identification and processing system shown in an embodiment of the present application.
[0075] S301, the meeting begins.
[0076] S302, system initialization. After the system is started, it will perform initialization operations, including loading necessary algorithm models, connecting all participating terminals, setting backup video sources, setting preset background images, and preset switching plans.
[0077] S303, data loading.
[0078] S304, real-time data collection. Execute the technical solution of S11, and collect the real-time data of the video source and the real-time data of each participating terminal in real time through the data collection module.
[0079] S305, real-time data transmission: The collected real-time video source data and the terminal real-time data of each participating terminal are encoded and transmitted to the exception processing module for processing.
[0080] S306, abnormality identification. When an abnormality exists, the technical solution of S12 is executed, and the real-time data of the video source and the real-time data of the terminal are trained through the model of the abnormality identification module to obtain the real-time abnormal information of the video source and the real-time abnormal information of the terminal.
[0081] S307, exception handling. Execute the technical solution of S13, and perform exception handling on the real-time exception information of the video source and the real-time exception information of the terminal through the exception handling module.
[0082] S308, generating fault information. Execute the technical solution of S14 to generate corresponding abnormal fault information according to the real-time abnormal information and the abnormal processing result.
[0083] S309, the meeting proceeds normally. When no real-time abnormal information is identified in S306, the meeting proceeds normally.
[0084] S310, recording meeting information. Start face recognition, identity recognition and behavior recognition technology, record the information of participants, analyze the meeting atmosphere and interaction, and obtain meeting information.
[0085] S311, the conference ends. Obtain abnormal fault information or normal conference information, and save all conference data. Users can query these data and generate reports through specific interfaces or tools to facilitate subsequent analysis and optimization of the video conference.
[0086] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements all or part of the steps of a method for identifying and processing video conference anomalies when executing the computer program.
[0087] See also Figure 4 , which is a functional module diagram of an electronic device according to an embodiment of the present application. In a preferred embodiment of the present application, the electronic device 4 includes a memory 41, at least one processor 42, and at least one communication bus 43.
[0088] Those skilled in the art should understand that Figure 4 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present application, and may be either a bus structure or a star structure. The printer may also include more or less other hardware or software than shown in the figure, or a different arrangement of components.
[0089] The present application embodiment also provides a computer-readable storage medium. The computer-readable storage medium is, for example, a non-volatile memory, such as: a magnetic medium (such as a hard disk, a floppy disk, and a magnetic tape), an optical medium (such as a CDROM disk and a DVD), a magneto-optical medium (such as an optical disk), and a hardware device specially constructed to store and execute computer-executable instructions (such as a read-only memory (ROM), a random access memory (RAM), a flash memory, etc.). Computer-executable instructions are stored on the computer-readable storage medium. The computer-readable storage medium can be executed by one or more processors or processing devices to implement the video conference exception identification processing method in the aforementioned embodiment.
[0090] In addition, it can be understood that the aforementioned embodiments are merely illustrative descriptions of the present application. Under the premise that the technical features do not conflict, the structures do not contradict, and the purpose of the invention of the present application is not violated, the technical solutions of the various embodiments can be arbitrarily combined and used in combination.
[0091] In the several embodiments provided in the present application, it should be understood that the disclosed modules and methods can be implemented in other ways. For example, the system or device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.
[0092] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0093] In addition, each functional unit / module in each embodiment of the present application may be integrated into one processing unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated into one unit / module. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of hardware plus software functional units / modules.
[0094] The above-mentioned integrated unit / module implemented in the form of a software functional unit / module can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for enabling one or more processors of a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for identifying and processing abnormalities in video conferencing. It is characterized in that include: Acquire real-time data of the video conference, the real-time data including video source real-time data and terminal real-time data; Processing the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time abnormal information; Execute a preset exception handling strategy according to the real-time exception information; Generate exception handling result information according to the preset exception handling strategy.
2. The method for identifying and processing video conference anomalies according to claim 1, It is characterized in that The processing of the video source real-time data and the terminal real-time data according to the preset deep learning algorithm model to obtain real-time abnormal information includes: The real-time data of the video source is trained according to a convolutional neural network model to obtain real-time abnormal information of the video source; The real-time data of the terminal is trained according to a recurrent neural network model to obtain real-time abnormal information of the terminal.
3. The method for identifying and processing video conference anomalies according to claim 2, It is characterized in that The executing a preset exception handling strategy according to the real-time exception information includes: When the real-time abnormal information is the real-time abnormal information of the video source, executing a first preset abnormality handling strategy according to the real-time abnormal information of the video source; When the real-time exception information is the real-time exception information of the terminal, executing a second preset exception handling strategy according to the real-time exception information of the terminal; When the real-time exception information includes the real-time exception information of the video source and the real-time exception information of the terminal, a third preset exception handling strategy is executed according to the real-time exception information of the video source and the real-time exception information of the terminal.
4. The method for identifying and processing video conference anomalies according to claim 3, It is characterized in that The executing a first preset exception handling strategy according to the real-time exception information of the video source comprises: Get the preset backup video source; When the operation state of the preset backup video source is normal, switching the video source to the preset backup video source; When the running state of the preset backup video source is abnormal, a preset background image is obtained, and the preset backup video source is switched to the preset background image.
5. The method for identifying and processing video conference anomalies according to claim 3, It is characterized in that The executing a second preset exception handling strategy according to the real-time exception information of the terminal includes: Obtaining the abnormal information type of the real-time abnormal information of the terminal; When the abnormal information type is abnormal audio and video quality, adjusting the audio and video bit rate of the terminal according to the real-time abnormal information of the terminal; When the abnormal information type is a video connection abnormality, automatically disconnecting and reconnecting the terminal; When the abnormal information type is device performance abnormality, the terminal is switched to a standby terminal.
6. The method for identifying and processing video conference anomalies according to claim 3, It is characterized in that The executing a third preset exception handling strategy according to the real-time exception information of the video source and the real-time exception information of the terminal includes: Determine a first exception handling priority of the real-time exception information of the video source and a second exception handling priority of the real-time exception information of the terminal; Comparing the first exception handling priority with the second exception handling priority to obtain a comparison result; Determining an execution order of the first preset exception handling strategy and the second preset exception handling strategy according to the comparison result; Exception processing is performed on the real-time exception information of the video source and the real-time exception information of the terminal according to the execution order.
7. The method for identifying and processing video conference anomalies according to any one of claims 1 to 6, It is characterized in that The method further comprises: When the real-time abnormal information is not identified, face recognition and identity recognition are started to identify the video source and the participants of the terminal to obtain a recognition result; Acquire feature information of the recognition result, match the feature with a pre-stored feature template according to the feature, and if the match is successful, record the conference information of the video conference.
8. A video conference abnormality identification and processing device, It is characterized in that The device comprises: A data acquisition module is used to obtain real-time data of the video conference, wherein the real-time data includes real-time data of the video source and real-time data of the terminal; An abnormality identification module, used to process the real-time data of the video source and the real-time data of the terminal according to a preset deep learning algorithm model to obtain real-time abnormality information; An exception handling module is used to execute a preset exception handling strategy according to the real-time exception information; The information integration module is used to generate exception handling result information according to the preset exception handling strategy.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the video conference anomaly identification and processing method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, It is characterized in that An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the steps of the video conference abnormality identification and processing method as described in any one of claims 1 to 7.