A method and system for identifying abnormal image signals in a video conferencing system

By using multi-scale analysis and adaptive threshold detection, anomalies in the image signal of a video conferencing system are identified in the compressed domain. This solves the problems of high computational resource consumption and false positives and false negatives in existing technologies, and achieves efficient and accurate anomaly detection and management.

CN118741088BActive Publication Date: 2024-11-12INFORMATION & COMMNUNICATION BRANCH STATE GRID JIANGXI ELECTRIC POWER CO
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411223508.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-12
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

Existing methods for identifying image signal anomalies in video conferencing systems require full-frame decoding, which demands high computational resources, leading to increased latency and decreased processing efficiency. Furthermore, fixed threshold judgments are prone to false alarms or missed alarms, making it impossible to comprehensively capture anomalies.

Method used

A multi-scale analysis model is adopted to extract features in the compressed domain, generate anomaly coefficients through discrete cosine transform coefficients and motion vectors, dynamically adjust the threshold by combining network and video parameters, adaptively detect anomalies, and perform analysis at different time and spatial scales.

Benefits of technology

It enables real-time processing of large amounts of video data, improves response speed and resource utilization, enhances the accuracy and comprehensiveness of anomaly detection, reduces false alarms and missed alarms, and improves the management efficiency of video conferencing systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118741088B_ABST
    Figure CN118741088B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for identifying abnormalities in image screen signals of a video conference system, and relates to the technical field of screen abnormality identification. The identification system adopts a multi-scale analysis mode for the current video conference, generates an abnormality coefficient for the current scale analysis mode by analyzing the changes of multiple parameters, dynamically adjusts the abnormality threshold of the current scale analysis mode according to the adjustment index, and compares the abnormality coefficient with the adjusted abnormality threshold to identify whether the current scale analysis mode is abnormal, and generates a corresponding control strategy according to the abnormality identification result. The identification system can not only process a large amount of video data in real time, but also significantly improve the response speed and resource utilization of the video conference system, introduce adaptive threshold detection, and detect video signal abnormalities at different time and space scales, so as to more comprehensively capture various types of abnormalities and effectively improve the accuracy of image screen signal abnormality detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal picture recognition, and in particular to a method and system for recognizing abnormal picture signals of a video conference system. Background Art

[0002] With the widespread use of video conferencing in daily life, such as corporate meetings, distance education, medical collaboration, cross-border cooperation, and social interaction, video conferencing systems usually adopt digital signal transmission technology and use compression algorithms (such as H.264 or H.265) to reduce bandwidth requirements. In video conferencing systems, the identification of image signal anomalies is crucial because these systems rely on high-quality audio and video transmission to ensure the effectiveness of remote communication.

[0003] The prior art has the following deficiencies:

[0004] 1. Existing recognition methods usually require full-frame decoding and processing of the video, which requires high computing resources, especially in real-time video conferencing, which will occupy computing resources, resulting in increased latency and reduced processing efficiency of the video conferencing system;

[0005] 2. The recognition method usually uses a fixed threshold to judge the abnormality of the image signal, which is easy to cause false positives or false negatives in different network environments or video contents. For example, when the network condition is poor, the fixed threshold may cause frequent false positives;

[0006] 3. The recognition method only performs analysis at a single scale (such as a single frame or a single image block), which easily ignores anomalies in a small range or abnormal signals accumulated over a long period of time, resulting in incomplete detection of image signal anomalies. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for identifying anomalies in image screen signals of a video conferencing system. By directly extracting features in the compression domain, the complete decoding of the video is avoided. Not only can a large amount of video data be processed in real time, but also the response speed and resource utilization of the video conferencing system can be significantly improved. Adaptive threshold detection is introduced, and by detecting video signal anomalies at different time and space scales, various types of anomalies can be captured more comprehensively, and the accuracy of image screen signal anomaly detection can be effectively improved to address the shortcomings of the background technology.

[0008] In order to achieve the above object, the present invention provides the following technical solution: a method for identifying abnormal image signals of a video conferencing system, the method comprising the following steps:

[0009] The recognition system adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors, and generates abnormal coefficients for the current scale analysis mode by analyzing the changes of multiple parameters;

[0010] Obtain network parameters and video parameters of the current scale analysis mode, generate an adjustment index based on the network parameters and video parameters, and dynamically adjust the abnormal threshold of the current scale analysis mode according to the adjustment index;

[0011] Compare the abnormal coefficient with the adjusted abnormal threshold, identify whether the current scale analysis mode has abnormalities, and then identify whether other scale analysis modes have abnormalities, and generate corresponding control strategies based on the abnormal identification results;

[0012] When the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value, and whether it is necessary to generate maintenance suggestions for the video conference system is determined based on the analysis results of the maintenance value.

[0013] In a preferred embodiment, the recognition system adopts a multi-scale analysis mode for the current video conference, and the multi-scale analysis mode includes a short-time full-screen analysis mode, a short-time local screen analysis mode, a long-time full-screen analysis mode, and a long-time local screen analysis mode.

[0014] In a preferred embodiment, in the video image encoding process of each scale analysis mode, compressing the video signal into multiple parameters includes the following steps:

[0015] Multiple parameters include discrete cosine transform coefficients and motion vectors;

[0016] In the current scale analysis mode, the video stream is divided into multiple frames of image data, and each frame of image data is divided into a number of image blocks;

[0017] Perform discrete cosine transform on each image block to convert the pixel value in the spatial domain into discrete cosine transform coefficients in the frequency domain. The discrete cosine transform coefficients describe the frequency characteristics of the image block. Image blocks with frequency characteristics less than or equal to the frequency threshold are retained, and image blocks with frequency characteristics greater than the frequency threshold are deleted.

[0018] The block matching algorithm is used to find the area in the previous frame that is most similar to a block in the current frame to generate a motion vector, which is used to describe the movement of the image block between frames.

[0019] In a preferred embodiment, generating anomaly coefficients for the current scale analysis mode by analyzing the changes in multiple parameters includes the following steps:

[0020] Calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude of each scale analysis mode, and comprehensively calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude to obtain the abnormal coefficient of the current scale analysis mode. The expression is:

[0021] , where is the abnormal coefficient, is the discrete cosine transform coefficient fluctuation amplitude, is the motion vector fluctuation amplitude, , are the weight coefficients of the discrete cosine transform coefficient fluctuation amplitude and the motion vector fluctuation amplitude, respectively, and , Both are greater than 0.

[0022] In a preferred embodiment, obtaining network parameters and video parameters of the current scale analysis mode and combining the network parameters and video parameters to generate an adjustment index include the following steps:

[0023] Get the network parameters and video parameters of the current scale analysis mode. The network parameters include network delay and network packet loss rate. Normalize the network delay and network packet loss rate so that their value ranges are mapped to [0,1]. Get the normalized value of network delay and the normalized value of network packet loss rate. Add the normalized value of network delay to the normalized value of network packet loss rate to get the network instability coefficient. The video parameters include scene complexity. The network instability coefficient and scene complexity are calculated comprehensively to get the adjustment index. The expression is:

[0024] , where To adjust the index, is the network instability coefficient, is the scene complexity, , are the proportional coefficients of the network instability coefficient and the scene complexity, and , Both are greater than 0.

[0025] In a preferred embodiment, dynamically adjusting the abnormal threshold of the current scale analysis mode according to the adjustment index includes the following steps:

[0026] The abnormal threshold of the current scale analysis mode is dynamically adjusted through the obtained adjustment index, and the expression is: , where is the adjusted abnormal threshold, To adjust the pre-abnormal threshold, To adjust the index.

[0027] In a preferred embodiment, comparing the abnormal coefficient with the adjusted abnormal threshold to identify whether the current scale analysis mode has an abnormality includes the following steps:

[0028] The anomaly coefficient is compared with the adjusted anomaly threshold. The adjusted anomaly threshold is used to identify whether the current scale analysis mode has an anomaly. If the anomaly coefficient is less than or equal to the adjusted anomaly threshold, it is identified that the current scale analysis mode does not have an anomaly. If the anomaly coefficient is greater than the adjusted anomaly threshold, it is identified that the current scale analysis mode has an anomaly. After repeatedly obtaining the anomaly coefficients of other scale analysis modes, the anomaly coefficients of other scale analysis modes are compared with the adjusted anomaly threshold to identify whether other scale analysis modes have anomalies.

[0029] In a preferred embodiment, generating a corresponding control strategy according to the abnormality identification result includes the following steps:

[0030] When no abnormality is found in all scale analysis modes, no control is made to the video conference. When an abnormality is found in any scale analysis mode, a corresponding control strategy is generated. The control strategy includes the recognition system automatically initiating a frame retransmission request, or restarting the video conference and sending a warning signal prompt.

[0031] In a preferred embodiment, weighted average calculation is performed on the abnormal coefficients of all scale analysis modes to obtain a maintenance value, and judging whether it is necessary to generate maintenance suggestions for the video conferencing system based on the analysis result of the maintenance value, including the following steps:

[0032] The maintenance value is obtained by weighted average calculation of the anomaly coefficients of all scale analysis modes. The expression is: , where To maintain the value, For the The anomaly coefficient of the scale analysis model is For the The weight coefficient of the scale analysis model, and , Indicates the number of scale analysis modes;

[0033] The obtained maintenance value is compared with the preset maintenance threshold. If the maintenance value is less than or equal to the maintenance threshold, it is determined that there is no need to generate maintenance recommendations for the video conferencing system. If the maintenance value is greater than the maintenance threshold, it is determined that there is a need to generate maintenance recommendations for the video conferencing system.

[0034] A video conference system image signal anomaly recognition system, comprising a calculation module, a threshold dynamic adjustment module, an anomaly analysis module, and a maintenance module;

[0035] Calculation module: adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors, and generates abnormal coefficients for the current scale analysis mode by analyzing the changes of multiple parameters;

[0036] Threshold dynamic adjustment module: obtains network parameters and video parameters of the current scale analysis mode, generates an adjustment index based on the network parameters and video parameters, and dynamically adjusts the abnormal threshold of the current scale analysis mode according to the adjustment index;

[0037] Abnormal analysis module: compares the abnormal coefficient with the adjusted abnormal threshold, identifies whether the current scale analysis mode has abnormalities, and then identifies whether other scale analysis modes have abnormalities, and generates corresponding control strategies based on the abnormal identification results;

[0038] Maintenance module: After the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value. Based on the analysis results of the maintenance value, it is determined whether maintenance recommendations need to be generated for the video conference system.

[0039] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0040] 1. The present invention adopts a multi-scale analysis mode for the current video conference through the recognition system, generates an abnormality coefficient for the current scale analysis mode by analyzing the changes of multiple parameters, obtains the network parameters and video parameters of the current scale analysis mode, generates an adjustment index in combination with the network parameters and the video parameters, dynamically adjusts the abnormal threshold of the current scale analysis mode according to the adjustment index, and compares the abnormality coefficient with the adjusted abnormality threshold to identify whether the current scale analysis mode is abnormal, and generates a corresponding control strategy according to the abnormal recognition result. The recognition system avoids complete decoding of the video by directly extracting features in the compression domain, and can not only process a large amount of video data in real time, but also significantly improve the response speed and resource utilization of the video conferencing system, introduces adaptive threshold detection, and detects video signal anomalies at different time and space scales, which can more comprehensively capture various types of anomalies and effectively improve the accuracy of image signal anomaly detection.

[0041] 2. After the video conference is over, the present invention performs weighted average calculation on the abnormal coefficients of all scale analysis modes to obtain a maintenance value, and determines whether it is necessary to generate maintenance suggestions for the video conference system based on the analysis results of the maintenance value, thereby improving the management efficiency of the video conference system;

[0042] 3. In the present invention, the maintenance value is used to reflect the overall abnormal state of the video conference. When the video conference is over, even if all scale analysis modes are detected to have no abnormalities during the video conference (the abnormality coefficient is less than or equal to the adjusted abnormality threshold), when all scale analysis modes are developing in a bad trend (that is, the abnormality coefficient is close to the abnormality threshold), it still indicates that the video conferencing system as a whole may also have abnormalities. By comprehensively analyzing all scale analysis modes, the accuracy of abnormality analysis of the video conferencing system is effectively improved, which facilitates the judgment of whether it is necessary to generate maintenance suggestions for the video conferencing system, thereby improving the maintenance efficiency of the video conferencing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0044] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1: Please refer to Figure 1 As shown, the present embodiment provides a method for identifying abnormal image signals of a video conferencing system, and the identification method includes the following steps:

[0047] The recognition system adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors. By analyzing the changes of multiple parameters, an abnormal coefficient is generated for the current scale analysis mode, and the network parameters and video parameters of the current scale analysis mode are obtained. An adjustment index is generated by combining the network parameters and the video parameters. The abnormal threshold of the current scale analysis mode is dynamically adjusted according to the adjustment index, and the abnormal coefficient is compared with the adjusted abnormal threshold. After identifying whether the current scale analysis mode has an abnormality, it is then identified whether other scale analysis modes have an abnormality. A corresponding control strategy is generated according to the abnormal identification result. When the video conference ends, the abnormal coefficients of all scale analysis modes are weighted averaged to obtain a maintenance value. According to the analysis result of the maintenance value, it is determined whether it is necessary to generate maintenance suggestions for the video conferencing system.

[0048] This application adopts a multi-scale analysis mode for the current video conference through the recognition system, generates an abnormality coefficient for the current scale analysis mode by analyzing the changes of multiple parameters, obtains the network parameters and video parameters of the current scale analysis mode, generates an adjustment index based on the network parameters and video parameters, dynamically adjusts the abnormal threshold of the current scale analysis mode according to the adjustment index, and compares the abnormality coefficient with the adjusted abnormality threshold to identify whether there is an abnormality in the current scale analysis mode, and generates a corresponding control strategy based on the abnormal recognition result. This recognition system avoids complete decoding of the video by directly extracting features in the compression domain. It can not only process a large amount of video data in real time, but also significantly improve the response speed and resource utilization of the video conferencing system, introduces adaptive threshold detection, and detects video signal anomalies at different time and space scales. It can capture various types of anomalies more comprehensively and effectively improve the accuracy of image signal anomaly detection.

[0049] After the video conference is over, this application performs weighted average calculation on the abnormal coefficients of all scale analysis modes to obtain a maintenance value, and determines whether it is necessary to generate maintenance recommendations for the video conferencing system based on the analysis results of the maintenance value, thereby improving the management efficiency of the video conferencing system.

[0050] Embodiment 2: The recognition system adopts a multi-scale analysis mode for the current video conference, and the multi-scale analysis mode includes a short-time full-screen analysis mode, a short-time local-screen analysis mode, a long-time full-screen analysis mode, and a long-time local-screen analysis mode;

[0051] Specifically, in this application, the short time value is 5s and the long time value is 15s.

[0052] In the video image encoding process of each scale analysis mode, the video signal is compressed into multiple parameters, the multiple parameters include discrete cosine transform coefficients and motion vectors, and abnormal coefficients are generated for the current scale analysis mode by analyzing changes in the multiple parameters, including the following steps:

[0053] In the current scale analysis mode, the video stream is divided into multiple frames of image data, and each frame of image data is divided into several image blocks (usually 8x8 or 16x16 pixel blocks) to encode the local area of ​​the image data. Block processing helps to compress the image data more efficiently.

[0054] A discrete cosine transform (DCT) is performed on each image block to convert the pixel values ​​in the spatial domain into discrete cosine transform coefficients in the frequency domain. The discrete cosine transform coefficients describe the frequency characteristics of the image block, usually including low-frequency and high-frequency components:

[0055] The pixel value in the spatial domain is converted into a discrete cosine transform coefficient in the frequency domain. The expression is:

[0056] ,

[0057] In the formula, are discrete cosine transform coefficients, is the pixel value in the spatial domain, corresponding to the position in the image block , is the size of the image block (usually 8x8 or 16x16), : Cosine function, indicating that each discrete cosine transform coefficient corresponds to a cosine wave of a specific frequency, indicating the intensity of the frequency in the image block, and is the normalization coefficient, defined as:

[0058] , and is the normalization coefficient, which is used to correct the transformed energy so that the energy before and after the transformation is consistent. Represents the horizontal frequency component of the image block in the frequency domain, Represents the vertical frequency component of the image block in the frequency domain.

[0059] Low-frequency component: represents the overall brightness and color changes of the image;

[0060] High frequency components: capture image details and edge information.

[0061] Image blocks with frequency characteristics less than or equal to the frequency threshold are retained, and image blocks with frequency characteristics greater than the frequency threshold are deleted, so that low-frequency components are retained and high-frequency components are discarded, thereby achieving image data compression.

[0062] The block matching algorithm is used to find the area in the previous frame that is most similar to a block in the current frame to generate a motion vector, which is used to describe the movement of the image block between frames. Based on the motion vector, the image block of the current frame is predicted, and only the prediction error is encoded, which further reduces inter-frame redundancy and improves compression efficiency.

[0063] The block matching algorithm is used to find the area in the previous frame that is most similar to a block in the current frame to generate a motion vector. The logic for obtaining the motion vector is as follows: within the specified search range, the absolute difference between a block in the current frame and each candidate block in the reference frame is calculated, and the position with the smallest absolute difference is selected as the best matching block. The offset between this position and the original position is the motion vector, which is expressed as: , where is the motion vector, is the position offset that minimizes the sum of the absolute differences;

[0064] The calculation expression of the sum of absolute differences is:

[0065] ,

[0066] In the formula, is the pixel value of the block to be matched in the current frame, and the upper left corner of the block is , is the pixel value of the block to be tested in the reference frame, and the upper left corner of the block is , is the size of the image block (usually 8x8 or 16x16), It is the offset within the search window, indicating the position offset of the test block in the reference frame relative to the current block. The smaller the absolute difference and value, the more similar the block of the current frame is to the block of the reference frame.

[0067] The discrete cosine transform coefficients and motion vectors of each image block are entropy encoded (such as Huffman coding or arithmetic coding) to further compress the data volume. These encoded parameters will be used as a compressed representation of the video signal.

[0068] Calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude of each scale analysis mode, and comprehensively calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude to obtain the abnormal coefficient of the current scale analysis mode. The expression is:

[0069] , where is the abnormal coefficient, is the discrete cosine transform coefficient fluctuation amplitude, is the motion vector fluctuation amplitude, , are the weight coefficients of the discrete cosine transform coefficient fluctuation amplitude and the motion vector fluctuation amplitude, respectively, and , Both are greater than 0.

[0070] The fluctuation amplitude of discrete cosine transform coefficient and the fluctuation amplitude of motion vector are obtained by the general parameter standard deviation calculation formula, and the general parameter standard deviation calculation formula is: , where is the parameter standard deviation, For the The original parameters of the image blocks, is the parameter average value, is the number of image block divisions in the current scale analysis mode;

[0071] The calculation method of the discrete cosine transform coefficient fluctuation amplitude and the motion vector fluctuation amplitude is the same as the calculation method of the general parameter standard deviation, which will not be described in detail here;

[0072] The larger the fluctuation amplitude of the discrete cosine transform coefficient, the greater the difference in the values ​​of the discrete cosine transform coefficients between multiple image blocks, that is, the more likely the video conferencing image is to have abnormalities, such as image blur, artifacts, or mosaics. The larger the fluctuation amplitude of the motion vector, the greater the difference in the values ​​of the motion vector between multiple image blocks, that is, the more likely the video conferencing image is to have abnormalities, such as jitter, freeze, or frame loss.

[0073] Obtaining network parameters and video parameters of the current scale analysis mode, and combining the network parameters and video parameters to generate an adjustment index, includes the following steps:

[0074] The network parameters and video parameters of the current scale analysis mode are obtained. The network parameters include network delay and network packet loss rate. The network delay and network packet loss rate are normalized so that the value range of network delay and network packet loss rate is mapped to [0,1]. The normalized value of network delay and the normalized value of network packet loss rate are obtained. The network instability coefficient is obtained by adding the normalized value of network delay to the normalized value of network packet loss rate. The larger the network instability coefficient is, the worse the network state is. At this time, the recognition system needs to increase the abnormal threshold appropriately to avoid frequent false alarms caused by slight fluctuations in network state. The video parameters include scene complexity. The larger the scene complexity is, the more the recognition system needs to reduce the abnormal threshold appropriately to improve the monitoring of complex scenes. The network instability coefficient and scene complexity are calculated comprehensively to obtain the adjustment index. The expression is:

[0075] , where To adjust the index, is the network instability coefficient, is the scene complexity, , are the proportional coefficients of the network instability coefficient and the scene complexity, and , Both are greater than 0.

[0076] The calculation logic of scene complexity is: use the Sobel operator to calculate the gradient assignment of the current scale analysis mode, the expression is: , where For the image at pixel points The gradient amplitude at Indicates the image at pixel points The horizontal gradient at Indicates the image at pixel points The vertical gradient at

[0077] The scene complexity of the image is calculated based on the gradient amplitude, and the expression is:

[0078] ,

[0079] In the formula, is the scene complexity, is the image size, which means the image has OK Column pixels, is the indicator function, that is, when the gradient amplitude Greater than the amplitude threshold When , the indicator function takes the value of 1, otherwise it takes the value of 0. The greater the complexity of the scene, the more it is necessary to reduce the abnormal threshold and improve the monitoring of complex scenes.

[0080] Dynamically adjusting the abnormal threshold of the current scale analysis mode according to the adjustment index includes the following steps:

[0081] The larger the adjustment index is, the more it is necessary to increase the abnormal threshold of the current scale analysis mode. The abnormal threshold of the current scale analysis mode is dynamically adjusted by the obtained adjustment index. The expression is: , where is the adjusted abnormal threshold, To adjust the pre-abnormal threshold, To adjust the index.

[0082] Compare the anomaly coefficient with the adjusted anomaly threshold to identify whether the current scale analysis mode is abnormal, and then identify whether other scale analysis modes are abnormal, including the following steps:

[0083] The anomaly coefficient is compared with the adjusted anomaly threshold. The adjusted anomaly threshold is used to identify whether the current scale analysis mode has an anomaly. If the anomaly coefficient is less than or equal to the adjusted anomaly threshold, it is identified that the current scale analysis mode does not have an anomaly. If the anomaly coefficient is greater than the adjusted anomaly threshold, it is identified that the current scale analysis mode has an anomaly. After repeatedly obtaining the anomaly coefficients of other scale analysis modes, the anomaly coefficients of other scale analysis modes are compared with the adjusted anomaly threshold to identify whether other scale analysis modes have anomalies.

[0084] Generate a corresponding control strategy based on the abnormality identification results, including the following steps:

[0085] When no abnormality is found in all scale analysis modes, no control is made to the video conference. When an abnormality is found in any scale analysis mode, a corresponding control strategy is generated. The control strategy includes the recognition system automatically initiating a frame retransmission request, or restarting the video conference and sending a warning signal prompt.

[0086] When the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value. Based on the analysis result of the maintenance value, it is determined whether it is necessary to generate maintenance suggestions for the video conference system, including the following steps:

[0087] The maintenance value is obtained by weighted average calculation of the anomaly coefficients of all scale analysis modes. The expression is: , where To maintain the value, For the The anomaly coefficient of the scale analysis model is For the The weight coefficient of the scale analysis model, and , Indicates the number of scale analysis modes;

[0088] The maintenance value is used to reflect the overall abnormal state of the video conference. When the video conference is over, even if all scale analysis modes are detected to be normal during the video conference (the abnormal coefficient is less than or equal to the adjusted abnormal threshold), when all scale analysis modes are developing in a bad trend (that is, the abnormal coefficient is close to the abnormal threshold), it still indicates that the video conference system as a whole may also have abnormalities. By comprehensively analyzing all scale analysis modes, the accuracy of abnormal analysis of the video conference system is effectively improved, which makes it easier to determine whether maintenance recommendations need to be generated for the video conference system, thereby improving the maintenance efficiency of the video conference system.

[0089] The obtained maintenance value is compared with the preset maintenance threshold. If the maintenance value is less than or equal to the maintenance threshold, it is determined that there is no need to generate maintenance recommendations for the video conferencing system. If the maintenance value is greater than the maintenance threshold, it is determined that there is a need to generate maintenance recommendations for the video conferencing system.

[0090] Embodiment 3: A video conferencing system image signal anomaly recognition system described in this embodiment includes a calculation module, a threshold dynamic adjustment module, an anomaly analysis module, and a maintenance module;

[0091] Calculation module: adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors, generates abnormal coefficients for the current scale analysis mode by analyzing the changes of multiple parameters, and sends the abnormal coefficients to the abnormal analysis module and the maintenance module, and the multi-scale analysis mode is sent to the maintenance module;

[0092] Threshold dynamic adjustment module: obtains network parameters and video parameters of the current scale analysis mode, generates an adjustment index based on the network parameters and video parameters, dynamically adjusts the abnormal threshold of the current scale analysis mode according to the adjustment index, and sends the adjusted abnormal threshold to the abnormal analysis module;

[0093] Abnormal analysis module: compares the abnormal coefficient with the adjusted abnormal threshold, identifies whether the current scale analysis mode has abnormalities, then identifies whether other scale analysis modes have abnormalities, and generates corresponding control strategies based on the abnormal identification results;

[0094] Maintenance module: After the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value. Based on the analysis results of the maintenance value, it is determined whether maintenance recommendations need to be generated for the video conference system.

[0095] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0096] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0097] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying abnormal image signals in a video conferencing system, characterized in that: The identification method comprises the following steps: The recognition system adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors, and generates abnormal coefficients for the current scale analysis mode by analyzing the changes of multiple parameters; Obtain network parameters and video parameters of the current scale analysis mode, generate an adjustment index based on the network parameters and video parameters, and dynamically adjust the abnormal threshold of the current scale analysis mode according to the adjustment index; Compare the abnormal coefficient with the adjusted abnormal threshold, identify whether the current scale analysis mode has abnormalities, and then identify whether other scale analysis modes have abnormalities, and generate corresponding control strategies based on the abnormal identification results; When the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value, and whether it is necessary to generate maintenance suggestions for the video conference system is determined based on the analysis results of the maintenance value; The recognition system adopts a multi-scale analysis mode for the current video conference, and the multi-scale analysis mode includes a short-time full-screen analysis mode, a short-time local-screen analysis mode, a long-time full-screen analysis mode, and a long-time local-screen analysis mode.

2. The method for identifying abnormal image signals in a video conferencing system according to claim 1, characterized in that: In the video image encoding process of each scale analysis mode, the video signal is compressed into multiple parameters, including the following steps: Multiple parameters include discrete cosine transform coefficients and motion vectors; In the current scale analysis mode, the video stream is divided into multiple frames of image data, and each frame of image data is divided into a number of image blocks; Perform discrete cosine transform on each image block to convert the pixel value in the spatial domain into discrete cosine transform coefficients in the frequency domain. The discrete cosine transform coefficients describe the frequency characteristics of the image block. Image blocks with frequency characteristics less than or equal to the frequency threshold are retained, and image blocks with frequency characteristics greater than the frequency threshold are deleted. The block matching algorithm is used to find the area in the previous frame that is most similar to a block in the current frame to generate a motion vector, which is used to describe the movement of the image block between frames.

3. A method for identifying abnormal image signals in a video conferencing system according to claim 2, characterized in that: The anomaly coefficients are generated for the current scale analysis mode by analyzing the changes in multiple parameters, including the following steps: Calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude of each scale analysis mode, and comprehensively calculate the discrete cosine transform coefficient fluctuation amplitude and motion vector fluctuation amplitude to obtain the abnormal coefficient of the current scale analysis mode. The expression is: , where is the abnormal coefficient, is the discrete cosine transform coefficient fluctuation amplitude, is the motion vector fluctuation amplitude, , are the weight coefficients of the discrete cosine transform coefficient fluctuation amplitude and the motion vector fluctuation amplitude, respectively, and , Both are greater than 0.

4. The method for identifying abnormal image signals of a video conferencing system according to claim 3, characterized in that: Obtaining network parameters and video parameters of the current scale analysis mode, and combining the network parameters and video parameters to generate an adjustment index, includes the following steps: Get the network parameters and video parameters of the current scale analysis mode. The network parameters include network delay and network packet loss rate. Normalize the network delay and network packet loss rate so that their value ranges are mapped to [0,1]. Get the normalized value of network delay and the normalized value of network packet loss rate. Add the normalized value of network delay to the normalized value of network packet loss rate to get the network instability coefficient. The video parameters include scene complexity. The network instability coefficient and scene complexity are calculated comprehensively to get the adjustment index. The expression is: , where To adjust the index, is the network instability coefficient, is the scene complexity, , are the proportional coefficients of the network instability coefficient and the scene complexity, and , Both are greater than 0.

5. A method for identifying abnormal image signals in a video conferencing system according to claim 4, characterized in that: Dynamically adjusting the abnormal threshold of the current scale analysis mode according to the adjustment index includes the following steps: The abnormal threshold of the current scale analysis mode is dynamically adjusted through the obtained adjustment index, and the expression is: is the adjusted abnormal threshold, To adjust the pre-abnormal threshold, To adjust the index.

6. A method for identifying abnormal image signals in a video conferencing system according to claim 5, characterized in that: Compare the anomaly coefficient with the adjusted anomaly threshold to identify whether the current scale analysis mode is abnormal, including the following steps: The anomaly coefficient is compared with the adjusted anomaly threshold. The adjusted anomaly threshold is used to identify whether the current scale analysis mode has an anomaly. If the anomaly coefficient is less than or equal to the adjusted anomaly threshold, it is identified that the current scale analysis mode does not have an anomaly. If the anomaly coefficient is greater than the adjusted anomaly threshold, it is identified that the current scale analysis mode has an anomaly. After repeatedly obtaining the anomaly coefficients of other scale analysis modes, the anomaly coefficients of other scale analysis modes are compared with the adjusted anomaly threshold to identify whether other scale analysis modes have anomalies.

7. A method for identifying abnormal image signals in a video conferencing system according to claim 6, characterized in that: Generate a corresponding control strategy based on the abnormality identification results, including the following steps: When no abnormality is found in all scale analysis modes, no control is made to the video conference. When an abnormality is found in any scale analysis mode, a corresponding control strategy is generated. The control strategy includes the recognition system automatically initiating a frame retransmission request, or restarting the video conference and sending a warning signal prompt.

8. The method for identifying abnormal image signals of a video conferencing system according to claim 7, characterized in that: The weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value, and whether it is necessary to generate maintenance suggestions for the video conferencing system is determined based on the analysis results of the maintenance value, including the following steps: The maintenance value is obtained by weighted average calculation of the anomaly coefficients of all scale analysis modes. The expression is: , where To maintain the value, For the The anomaly coefficient of the scale analysis model is For the The weight coefficient of the scale analysis model, and , Indicates the number of scale analysis modes; The obtained maintenance value is compared with the preset maintenance threshold. If the maintenance value is less than or equal to the maintenance threshold, it is determined that there is no need to generate maintenance recommendations for the video conferencing system. If the maintenance value is greater than the maintenance threshold, it is determined that there is a need to generate maintenance recommendations for the video conferencing system.

9. A video conference system image signal abnormality recognition system, used to implement the recognition method according to any one of claims 1 to 8, characterized in that: It includes calculation module, threshold dynamic adjustment module, abnormality analysis module and maintenance module; Calculation module: adopts a multi-scale analysis mode for the current video conference, and in the video image encoding process of each scale analysis mode, compresses the video signal into multiple parameters, including discrete cosine transform coefficients and motion vectors, and generates abnormal coefficients for the current scale analysis mode by analyzing the changes of multiple parameters; Threshold dynamic adjustment module: obtains network parameters and video parameters of the current scale analysis mode, generates an adjustment index based on the network parameters and video parameters, and dynamically adjusts the abnormal threshold of the current scale analysis mode according to the adjustment index; Abnormal analysis module: compares the abnormal coefficient with the adjusted abnormal threshold, identifies whether the current scale analysis mode has abnormalities, and then identifies whether other scale analysis modes have abnormalities, and generates corresponding control strategies based on the abnormal identification results; Maintenance module: After the video conference is over, the weighted average calculation of the abnormal coefficients of all scale analysis modes is performed to obtain the maintenance value. Based on the analysis results of the maintenance value, it is determined whether maintenance recommendations need to be generated for the video conference system.