A method and system for audio and video conference quality analysis and evaluation

Abstract

The application provides a method and system for audio and video conference quality analysis and evaluation, relates to the technical fields of audio and video communication diagnosis and image processing diagnosis, and audio and video conference equipment health degree evaluation method, wherein the analysis method comprises: audio and video data collection: operation state monitoring and evaluation of audio and video conference core equipment; collecting key parameters affecting audio and video quality; establishing an evaluation model: converting collected data into quantifiable quality indicators, focusing on three dimensions of video lag, picture blur and audio break; identifying and verifying the voice: establishing a scoring prediction model: integrating equipment operation state data and three main dimension indicators to obtain the overall conference quality score and the reliability of the score, the application takes the severity of picture blur and other factors into account, the error of the static model is reduced, and the voice stability value is divided according to the region, so that the stability value can effectively reflect the overall audio quality in the conference room.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio and video communication diagnosis and image processing diagnosis, in particular to a method and system for audio and video conference quality analysis and evaluation. BACKGROUND

[0002] The core purpose of audio and video conference quality analysis and evaluation is to monitor and optimize technical performance in the conference process in real time, maintain the stability, clarity and fluency of communication, and ensure the efficiency and good meeting experience of remote collaboration in the audio and video conference process. At the same time, based on data analysis, the network configuration and device performance are optimized, which can improve the reliability of the conference, reduce the communication cost of enterprises, and ultimately support the efficient collaboration and decision continuity of cross-regional teams. In short, it is a key technical guarantee for maintaining efficient collaboration in a digital office environment.

[0003] The existing technology for audio and video conference quality analysis and evaluation mainly relies on basic network parameters, ignores terminal factors such as device performance and coding fluctuations, and cannot locate the root cause of lag and blur. Moreover, the final analysis result needs to be scored manually, which is time-consuming and easily affected by individual differences, and is difficult to scale. There are defects in static models, and fixed weights cannot dynamically adapt to changes in the scene. In particular, the severity of factors such as picture blur cannot be considered in the scope of consideration, there is a lack of side verification means, and it is difficult to find information problems such as semantic breaks, thereby reducing the usability of the final audio and video conference quality analysis and evaluation result. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application aims to provide a method and system for audio and video conference quality analysis and evaluation to solve the problems raised in the background art. The severity of factors such as picture blur is considered in the scope of consideration, the blur weight is adjusted in real time by measuring the resolution degradation index, the error of the static model is reduced, multiple dimensional features are integrated, and the device parameters and speech recognition verification improve the final score of the evaluation model, thereby improving the effectiveness of the final score model, realizing effective prediction of the subsequent audio and video conference quality, and dividing the speech stability value according to the region, so that the stability value can effectively reflect the quality of the overall audio conference process in the conference room.

[0005] In order to achieve the above-mentioned purpose, the present application is realized by the following technical scheme: a method for audio and video conference quality analysis and evaluation, comprising the following steps:

[0006] S1. Audio and video data acquisition: Collect key parameters that affect audio and video quality, including network interface card (NIC) performance, synchronously collect frame rate (FPS) and bit rate (bps) fluctuation information, obtain audio and video conferencing equipment health detection data, and generate evaluation results based on the collected audio and video equipment health detection data;

[0007] S2. Establish an evaluation model: Transform the collected data into quantifiable quality indicators, focusing on three dimensions: video stuttering, image blurring, and audio interruption. The three dimensions are then weighted and fused into a comprehensive evaluation model. The actual resolution is verified after image blurring. The weight allocation in the evaluation model is adaptively adjusted by quantifying the degree of resolution degradation.

[0008] S3. Verify speech recognition: Verify the stability of information transmission through speech recognition (ASR), capture the completeness of semantics to improve the comprehensiveness of the evaluation, quantify information fidelity, and finally integrate and output speech stability values. Build a dedicated recognition system, control the speech recognition device to move to different areas in the meeting scene for recognition, and divide speech stability values ​​according to the area.

[0009] S4. Establish a scoring prediction model: Use an LSTM network to capture dynamic changes in the meeting. The regression model is used as the input to the outputs of steps S1, S2, and S3: data collection indicators, evaluation model scores, and voice stability values. The target variable is the overall quality score. Based on the audio and video equipment health evaluation results obtained in S1, a reliability analysis is performed on the overall quality score to generate the validity probability of the meeting quality score and predict the overall meeting quality score, thus achieving forward-looking optimization.

[0010] Furthermore, in step S1, a high-precision timestamp is uniformly added during the data collection process, and instantaneous jitter is filtered out by a weighted moving average algorithm to finally generate a continuous and smooth fluctuation curve, which intuitively reflects network congestion or equipment performance bottlenecks.

[0011] After defining the unified data model, edge computing preprocessing is introduced, and CRC check code is added to avoid the influence of collection errors on the model in step S4; the health of audio and video equipment is tested, the audio equipment test items include total harmonic distortion, noise and frequency response, the video equipment test items include color restoration error and brightness uniformity, the network AV test items include code stream delay, a standard level sinusoidal wave signal is introduced into the audio equipment, and an audio analyzer is used to measure total harmonic distortion plus noise, signal-to-noise ratio, frequency response, channel separation, output level / power; and an LCR table is used to measure the capacitance / susceptance drift of key components, the equivalent series resistance rise value; the power conversion efficiency of the current equipment is recorded by a power analyzer, and compared with the original data at the time of factory shipment, to obtain the decline amplitude of the power conversion efficiency; physical wear test, motor bearing noise spectrum analysis are carried out on mobile storage, cable, fan, plug-in interface; the infrared thermal imager scans the temperature difference anomaly caused by oxidation and dust accumulation of the heat sink.

[0012] Further, in step S2, the video freezing time proportion is the proportion of freezing duration to total meeting time, and freezing is limited to frame rate below threshold <5fps; the picture blur period proportion is used to quantify the time proportion of picture blur; the audio break proportion is the time proportion of audio interruption or distortion;

[0013] Based on the frame rate and code rate fluctuation information collected in step S1, the freezing weight is adjusted, and based on weighted average, the 0-1 standardized score is output, and the output data evaluation model score is integrated.

[0014] Further, the verification of the actual resolution includes the following contents:

[0015] Real-time intercept the video frame rendered by the terminal, exclude the interference of device scaling or interface superposition, pre-process each frame to remove blur, select the text area that appears stably in the picture, identify the character size through OCR engine (Tesseract), and calculate the actual pixel density combined with the standard physical size of the character. If the text is missing, SIFT feature point matching is used: compare the preset size of the known object in the original video stream with the pixel span of the object in the rendered frame, calculate the scaling ratio, and the calculation formula is:

[0016] Actual resolution = encoding resolution x (distance of feature points in rendered frame / distance of feature points in original stream)

[0017] And according to the measured actual resolution, the resolution degradation index (RDI) is obtained, and the weight correction factor (λ) is integrated into the comprehensive evaluation model, and the calculation formula of the comprehensive evaluation model score is:

[0018] Comprehensive score = (freezing proportion x 0.4) + (blur proportion x 0.35) + (break proportion x 0.25)

[0019] New fuzzy weight = 0.35 x lambda;

[0020] The score calculation formula of the comprehensive evaluation model after introducing the resolution correction is:

[0021] .

[0022] Further, in step S3, using an automatic speech recognition (ASR) engine, the conference audio stream is transcribed into text in real time, and the key parameters include: text completeness, break detection, and semantic consistency. The text completeness calculates the similarity between the recognized text and the original or expected content, and the formula is:

[0023] Completeness = 1 - (edit distance / original text length)

[0024] The break detection identifies the missing text caused by audio breaks, and counts the frequency of break events;

[0025] The semantic consistency uses an NLP model to analyze the proportion of sentence breaks in the text context sentence.

[0026] Calculate the final output:

[0027] Stability = 0.5 x Completeness + 0.3 x Inverse of Break Frequency + 0.2 x Consistency.

[0028] And the stability result is fed back to the model in step S2 as a correction factor to adjust the comprehensive evaluation. When the stability is lower than the threshold, the audio break weight is reduced;

[0029] Using the built-in voice recognition device, repeat the voice recognition process in different positions in the conference scene, and the recognition period is not less than 20 minutes. Keep the audio and video content consistent in each period. Each test area collects the corresponding voice stability value, and finally obtains the average stability data result in the entire conference scene.

[0030] Further, in step S4, all features need to be scaled to the [0, 1] interval to eliminate the dimension effect. The input feature vector is input into the scoring prediction model, and the output is the predicted score and the root cause report. Set the total harmonic distortion, noise, frequency response, channel separation, output level / power, power conversion efficiency, and physical wear items tested to be qualified. Compare the average classification of the final audio and video device health and the overall conference quality score collected to verify the reliability of the final conference overall quality score.

[0031] A quality analysis and evaluation system used in the above method, comprising a collection end module, a data processing and storage engine, a quality evaluation engine and a speech recognition verification system, wherein a speech recognition device is built in the conference room in the speech recognition verification system, multiple recognition scenes are switched through the speech recognition device, the stability average data of the audio conference quality in the conference room scene is obtained, and the effectiveness of the audio break analyzer is verified in reverse.

[0032] Further, the quality evaluation engine includes a lag analyzer, a blur detector, an audio break analyzer and a dynamic weight distribution module.

[0033] Further, the blur detector outputs the blur period ratio + actual resolution based on the OpenCV+SSIM algorithm; the audio break analyzer outputs the break event frequency through short-time energy zero-crossing rate double-threshold detection; and the dynamic weight distribution module has a resolution degradation index feedback mechanism established therein, and outputs a fusion formula weight lambda.

[0034] Further, the data processing and storage engine includes a multi-source data access layer, a stream processing layer and a distributed storage layer, the stream processing layer is based on ApacheFlink to build a time window, adopts a dynamic watermark processing clock drift, and adjusts and controls the audio and video equipment performance data and the conference media stream timestamp alignment.

[0035] The beneficial effects of the present application are:

[0036] 1. The method for audio and video conference quality analysis and evaluation covers device performance, encoding parameters and network fluctuations at the same time, combines speech recognition verification content integrity, solves more fuzzy attribution problems, and takes into account the severity of factors such as picture blur, adjusts the blur weight in real time through the measured resolution degradation index, and reduces the error of the static model.

[0037] 2. The present application can comprehensively consider multi-dimensional characteristics, and the device parameters and speech recognition verification can improve the final score of the evaluation model, thereby improving the effectiveness of the final score model and realizing effective prediction of subsequent audio and video conference quality.

[0038] 3. The present application quantifies information fidelity, finally integrates and outputs a voice stability value, builds a special recognition system, controls the speech recognition device to move to different areas in the conference scene for recognition, divides the voice stability value according to the area, so that the stability value can effectively reflect the quality of the overall audio conference process in the conference room. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The flowchart of the method for audio and video conference quality analysis and evaluation of the present application;

[0040] Figure 2 It is a principle block diagram of a quality analysis and evaluation system for audio and video conference of the present application;

[0041] Figure 3 It is a structure diagram of a large-range voice recognition device in a quality analysis and evaluation system for audio and video conference of the present application;

[0042] Figure 4 It is a structure diagram of a first switching unit in the large-range voice recognition device of the present application;

[0043] Figure 5 It is an installation schematic diagram of an independent recognition unit of the present application;

[0044] Figure 6 It is a split diagram of the independent recognition unit of the present application;

[0045] Figure 7 It is a structure diagram of a front end of the recognition and collection framework of the present application;

[0046] Figure 8 It is a structure schematic diagram of a second switching unit part of the present application;

[0047] In the figure: 1, recognition and collection framework; 2, independent recognition unit; 3, first switching unit; 4, second switching unit; 5, support plate; 6, fixed plate; 7, pneumatic assembly; 8, extension plate; 9, connecting rod; 10, first supporting plate; 11, second supporting plate; 12, motor; 13, screw rod; 14, gear; 15, threaded sleeve; 16, lifting plate; 17, sound pickup; 18, loudspeaker; 19, lifting sleeve; 20, plug-in rod; 21, protruding block; 22, receiving hole; 23, spring rod; 24, moving groove; 25, end plate; 26, fixed support wheel; 27, slide rail; 28, rotating frame; 29, pressure receiving plate; 30, driving wheel; 31, inclined plate; 32, toothed belt; 33, rotating sleeve; 34, return spring. DETAILED DESCRIPTION

[0048] In order to make the technical means, creative features, purposes and effects achieved by the present application easy to understand, the present application is further described below in combination with specific embodiments.

[0049] Please refer to Figures 1 to 8 The present application provides the following technical solutions: a method for quality analysis and evaluation of audio and video conference, comprising the following steps:

[0050] S1, audio and video data collection: collect key parameters that affect audio and video quality, including network interface card (NIC) performance, synchronous acquisition frame rate (FPS) and bit rate (bps) fluctuation information, use client SDK to capture data stream key parameters at fixed time intervals, frame rate is dynamically calculated by counting the number of frames decoded and rendered per second, bit rate is real-time converted by accumulating the size of data packets transmitted per unit time, integrate output data collection indicators;

[0051] S2, establish evaluation model: convert collected data into quantifiable quality indicators, focus on three dimensions of video lag, picture blur and audio breakage, and weight and integrate them into a comprehensive evaluation model based on the three dimensions, wherein the actual resolution is verified after picture blur, the resolution degradation degree is quantified, and the weight distribution in the evaluation model is adaptively corrected;

[0052] S3, speech recognition verification: verify information transmission stability through speech recognition (ASR), capture semantic integrity, improve overall evaluation, and quantify information fidelity, finally integrate output voice stability value, build a dedicated recognition system, control the voice recognition device to move to different areas in the conference scene for recognition, and divide the voice stability value according to the area;

[0053] S4, establish a scoring prediction model: use LSTM network to capture conference dynamic changes, use regression model input as the output of steps S1, S2, S3: data collection indicators, evaluation model scores, and voice stability values, target variable is comprehensive quality score, and predict the overall quality score of the conference to realize forward-looking optimization.

[0054] In step S1 of the embodiment, a high-precision timestamp is uniformly stamped during data collection, transient jitter is filtered through a weighted moving average algorithm, and finally a continuous and smooth fluctuation curve is generated, which intuitively reflects network congestion or device performance bottleneck;

[0055] After defining a unified data model, edge computing preprocessing is introduced, and CRC check code is added to avoid the impact of collection errors on the model in step S4.

[0056] In step S2 of the embodiment, the video lag time ratio is the proportion of lag duration to total conference time, and the lag is limited to a frame rate below the threshold <5fps, and the calculation formula is:

[0057] Lag time ratio = (lag duration / total duration) x 100%

[0058] Picture blur period ratio is used to quantify the time proportion of picture blur, and the calculation formula is:

[0059] Blur ratio = (blur period / total period) x 100%

[0060] Audio disruption ratio is the proportion of audio interruption or distortion in the audio, and the calculation formula is:

[0061] Disruption ratio = (disruption time / total time) x 100%

[0062] Based on the frame rate and code rate fluctuation information collected in step S1, the stall weight is adjusted, and based on the weighted average, a 0-1 normalized score is output, and the output data evaluation model score is integrated.

[0063] In this embodiment, the verification of the actual resolution includes the following contents:

[0064] Real-time intercept the video frame rendered by the terminal, exclude the interference of device scaling or interface superposition, deblur preprocessing is performed on each frame, select the text area that appears stably in the picture, recognize the character size through the OCR engine (Tesseract), and calculate the actual pixel density combined with the standard physical size of the character. If the text is missing, SIFT feature point matching is used: compare the preset size of the known object in the original video stream with the pixel span of the object in the rendered frame, calculate the scaling ratio, and the calculation formula is:

[0065] Actual resolution = encoding resolution x (distance of feature points in rendered frame / distance of feature points in original stream)

[0066] And according to the measured actual resolution, the resolution degradation index (RDI) is obtained, and the weight correction factor (λ) is integrated into the comprehensive evaluation model, wherein the resolution degradation index is:

[0067] RDI =

(nominal resolution - actual resolution) / nominal resolution

[0068] The weight correction factor is set as:

[0069] When RDI < 0.02, λ = 1.0;

[0070] When 0.02 ≤ RDI < 0.05, λ = 1.2;

[0071] When RDI ≥ 0.05, λ = 1.5

[0072] The calculation formula of the comprehensive evaluation model score is:

[0073] Comprehensive score = (stall ratio x 0.4) + (blur ratio x 0.35) + (disruption ratio x 0.25)

[0074] New blur weight = 0.35 x λ;

[0075] The calculation formula of the comprehensive evaluation model score after introducing the resolution correction is:

[0076] .

[0077] In step S3, the conference audio stream is transcribed into text in real time using an automatic speech recognition (ASR) engine, and the key parameters include: text completeness, break detection, and semantic consistency. The text completeness calculates the similarity between the recognized text and the original or expected content, and the formula is:

[0078] Completeness = 1 - (edit distance / original text length)

[0079] The break detection identifies the missing text caused by audio breaks, and counts the frequency of break events.

[0080] The semantic consistency uses an NLP model to analyze the context sentence in the text and the proportion of breaks.

[0081] The final output is calculated as:

[0082] Stability = 0.5 × Completeness + 0.3 × Inverse of Break Frequency + 0.2 × Consistency.

[0083] The stability result is fed back to the model in step S2 as a correction factor to adjust the comprehensive evaluation. When the stability is below the threshold, the audio break weight is reduced.

[0084] The voice recognition device is moved to different positions in the conference scene to repeat the voice recognition process, and the recognition period is not less than 20 minutes. The audio and video content in each period is kept completely consistent, and the corresponding voice stability value is collected in each test area. The average stability data result in the entire conference scene is finally obtained.

[0085] In step S4, all features are scaled to the [0, 1] interval to eliminate the dimension effect, and the input feature vector is input into the scoring prediction model. The output is the predicted score and the root cause report.

[0086] The embodiment also provides a quality analysis and evaluation system used in the above method, which includes a collection end module, a data processing and storage engine, a quality evaluation engine, and a voice recognition verification system. The collection end module collects device performance data through a terminal performance probe. The device performance data includes: the data processing and storage engine is used to clean heterogeneous data and align multi-source data by timestamp. The voice recognition verification system builds a voice recognition device in the conference room, switches multiple recognition scenes through the voice recognition device, obtains the stability mean value data of the audio conference quality in the conference room scene, and verifies the effectiveness of the audio break analyzer in reverse.

[0087] In this embodiment, the quality evaluation engine includes a stutter analyzer, a blur detector, an audio break analyzer, and a dynamic weight distribution module. The quality evaluation engine fuses device health data in real time to construct a quantitative correlation model between hardware performance degradation and quality risk.

[0088] Audio quality level: When the total harmonic distortion is greater than 3% and the signal-to-noise ratio is less than 70 dB, the probability of speech intelligibility score decrease increases. When the frequency response and channel separation are greater than 50 dB, it is determined to be abnormal, which is caused by the aging of the positioning microphone array or the capacitance drift of the circuit board, resulting in the increase of the bottom noise;

[0089] Video quality level: When the color restoration error ΔE is greater than 5 or the brightness uniformity is less than 85%, the picture blur perception rate increases. When the display driver chip temperature rise is greater than 15°C and the power conversion efficiency decreases by more than 12%, color distortion caused by abnormal heat dissipation or aging of components is identified, and the video clarity score decrease probability increases.

[0090] Network transmission level: When the cable impedance deviation is greater than 10% of the initial value and the interface plug-in life exceeds 5000 times, the code stream delay fluctuation is associated. When the LCR detects that the filter inductance value deviates by ±15%, the device packet error rate will increase. At this time, the fan bearing noise spectrum predicts that the heat dissipation failure caused by the sudden packet loss will cause the conference score to decrease.

[0091] In this embodiment, the frame delay threshold model outputs the frame freezing time proportion, the blur detector outputs the blur period proportion and the actual resolution based on OpenCV+SSIM algorithm, the audio break analyzer outputs the break event frequency through the short-time energy zero-crossing rate double-threshold detection, and the dynamic weight distribution module has a resolution degradation index feedback mechanism to output the fusion formula weight λ.

[0092] In this embodiment, the data processing and storage engine includes a multi-source data access layer, a streaming processing layer, and a distributed storage layer. The streaming processing layer is based on Apache Flink to build a time window and uses dynamic watermark processing clock drift to align the performance data of audio and video devices with the conference media stream timestamp.

[0093] The embodiment also provides an audio recognition device used in the audio and video conference quality analysis and evaluation system, which comprises an identification and collection frame 1, independent identification units 2 and a switching unit, wherein the switching unit comprises a first switching unit 3 and a second switching unit 4, a moving groove 24 is formed in the inside of the identification and collection frame 1, the independent identification units 2 are embedded in the inside of the moving groove 24, slide rails 27 are welded on the sides of the identification and collection frame 1, one end of each independent identification unit 2 is embedded in the inside of the slide rail 27, support plates 5 are welded on the two sides of the bottom of the identification and collection frame 1, fixed support wheels 26 are inserted into the bottom of each support plate 5, a lead screw 13 is inserted into the inside of the identification and collection frame 1, the second switching units 4 are installed at the two ends of the lead screw 13, each second switching unit 4 is outwardly penetrated from the same side of the identification and collection frame 1, and the first switching unit 3 is penetrated through the surface of the support plate 5 on one side of the top of the identification and collection frame 1.

[0094] The audio recognition device first performs identification detection on the end area in the large conference scene which needs to be analyzed through the two independent identification units 2 carried on the identification and collection frame 1, the two groups of independent identification units 2 are driven and controlled through the motor 12 at one end of the identification and collection frame 1, so as to drive the two independent identification units 2 to be respectively located at the middle and the two ends of the identification and collection frame 1 to complete the subsequent identification process of the audio device, and realize the conversion purpose in step S3 after identification. After the identification of the two ends in the conference scene area is completed, the driving force part at the end of the second switching unit 4 is pressed on the ground through the operation of the first switching unit 3, so as to drive the whole audio recognition device to move, change the layout range of the whole recognition device in the conference scene, repeat the above process, and perform the audio recognition conversion process on the two end points in the range again, until the automatic audio recognition conversion process is realized in any area of the whole conference scene.

[0095] In the embodiment, the text recognition process of the audio is completed through the two groups of independent identification units 2, the independent identification unit 2 used in the process comprises a lifting plate 16, a pickup 17 and a loudspeaker 18, the bottom of the lifting plate 16 is welded with a lug 21, the bottom of the lug 21 is welded with a plug-in rod 20, one end of the bottom of the lifting plate 16 is welded with a spring rod 23, the bottom of the spring rod 23 is sleeved with a lifting sleeve 19, the bottom of the lifting sleeve 19 is embedded in the inside of the guide rail through the protruding ring, a motor 12 is screwed at one end of the identification and collection frame 1, a lead screw 13 is inserted into the output end of the motor 12, a threaded sleeve 15 is sleeved on the surface of the lead screw 13, a receiving hole 22 is formed in the top of the threaded sleeve 15, and the plug-in rod 20 is used to be embedded in the inside of the receiving hole 22.

[0096] Specifically, the two independent recognition units 2 are respectively located in different areas inside the recognition and collection frame 1. When the first switching unit 3 is not started, both independent recognition units 2 are embedded into the inside of the receiving hole 22 through the plug rod 20, and connected with the threaded sleeve 15. At this time, the motor 12 is started, and the threaded sleeve 15 is moved along the lead screw 13 by the rotation of the lead screw 13. At this time, the two recognition and collection frames 1 are synchronously controlled to move along the lead screw 13, so that the two independent recognition units 2 are respectively moved to the two end positions of the recognition and collection frame 1. By this effect, when the recognition device is in a region in the conference scene, the audio recognition conversion process of the two end positions of the region can be quickly realized. Each time the recognition is performed, the pickup 17 on the independent recognition unit 2 at the end of the recognition and collection frame 1 is used as the collection device, and the pickup 17 on the independent recognition unit 2 in the middle region of the recognition and collection frame 1 is used as the detection group, and the loudspeaker 18 can also output the audio to each other for mutual verification after each recognition is completed.

[0097] In the embodiment, the first switching unit 3 is also provided to adjust the distance between the two independent recognition units 2 in any conference scene coverage range, and change the position of the recognition device in the current conference scene. The first switching unit 3 includes a fixed plate 6, a pneumatic assembly 7, a connecting rod 9, a first supporting plate 10 and a second supporting plate 11. The pneumatic assembly 7 is installed at the end of the fixed plate 6, the top end of the pneumatic assembly 7 is screwed with an extension plate 8, one side of the extension plate 8 is integrally formed with the connecting rod 9, the middle of the connecting rod 9 is integrally formed with the first supporting plate 10, and both ends of the first supporting plate 10 are integrally formed with the second supporting plate 11. The first supporting plate 10 is used to support the independent recognition unit 2 at the middle position, and the second supporting plate 11 is used to support the independent recognition unit 2 at the two end positions. The pneumatic assembly 7 is connected with the air pump built-in the recognition device through the air supply pipeline, and the pneumatic assembly 7 is a two-section operation structure.

[0098] Specifically, when the distance between the two independent recognition units 2 needs to be changed, the pneumatic assembly 7 is controlled to operate to the first stage, at this time the first supporting plate 10 and the second supporting plate 11 are lifted by a small distance, since the first supporting plate 10 is higher than the second supporting plate 11, the first supporting plate 10 can lift the lifting plate 16 in the independent recognition unit 2 in the middle position upwards, the spring rod 23 in the independent recognition unit 2 rises inside the lifting sleeve 19, drives the plug-in rod 20 to be pulled out from the inside of the receiving hole 22, and the end independent recognition unit 2 is not in the range of the first supporting plate 10, and the second supporting plate 11 is lower, so that the lifting effect of the pneumatic assembly 7 cannot lift the end independent recognition unit 2 by the second supporting plate 11, therefore, after the motor 12 is started at this time, the end independent recognition unit 2 can still be moved along the lead screw 13, and finally the distance between the two independent recognition units 2 is controlled.

[0099] The embodiment also realizes the effect of automatically switching the recognition and collection range of the whole recognition device through the second switching unit 4, wherein the second switching unit 4 comprises a rotating frame 28, a driving wheel 30, a gear 14 and a pressure receiving plate 29, the top end of the transmission frame is integrally formed with a rotating sleeve 33, the end of the transmission frame is sleeved with the driving wheel 30, the middle of the driving wheel 30 and the surface of the lead screw 13 are both provided with the gear 14, the two gears 14 are sleeved with a toothed belt 32, the top of the transmission frame is welded with the inclined plate 31, the top end of the inclined plate 31 is integrally formed with the pressure receiving plate 29, the bottom of the pneumatic assembly 7 is aligned with the surface of the pressure receiving plate 29 through a rod, and the surface of the transmission frame is connected with the surface of the recognition and collection frame 1 through a return spring 34.

[0100] When the whole recognition device needs to be moved, after the pneumatic assembly 7 is controlled to extend the two ends of the pneumatic rod by a small distance, the pneumatic assembly 7 is further controlled to increase the internal air pressure. Further increase the extension length of the extended pneumatic rod, so as to further regularly press the first supporting plate 10 and the second supporting plate 11 by means of the pneumatic rod, until the plug-in rod 20 in the two independent recognition units 2 is separated from the receiving hole 22, at the same time, the pneumatic rod at the bottom of the pneumatic assembly 7 is pressed on the pressure receiving plate 29, so as to rotate the whole transmission frame downwards, and press the driving wheel 30 at the end on the ground, at this time, after the motor 12 is started, the gear 14 and the rack are rotated by the lead screw 13, finally the driving wheel 30 is driven by being pressed on the ground, so as to realize the movement purpose of the whole recognition device, until it is moved to another area in the meeting scene, and the above-mentioned recognition and conversion process of the audio quality can be repeated.

[0101] The foregoing merely illustrates the principles of the application and various embodiments are now described with reference to the drawings. This application is not limited to the embodiments described but is intended to encompass any and all changes and modifications within the spirit and scope of the underlying principles thereof. Thus, to those skilled in the art, it will be apparent from this disclosure that various modifications and changes can be made in the implementation without departing from the true spirit and scope of the application.

[0102] In addition, it should be understood that although the description herein is made on the basis of the embodiments, not every embodiment contains only one independent technical solution, and the description herein is made only for the sake of clarity, and those skilled in the art should understand the description as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. A method for audio and video conference quality analysis and evaluation, characterized in that, Comprise the following steps: S1, audio and video data collection: collect key parameters affecting audio and video quality, including network interface card performance, synchronous acquisition frame rate and code rate fluctuation information, obtain audio and video conference equipment health degree detection data, and generate evaluation results according to the collected audio and video equipment health degree detection data; S2, establish evaluation model: convert the collected data into quantifiable quality indicators, focus on three dimensions of video lag, picture blur and audio break, and weightedly fuse the three dimensions into a comprehensive evaluation model, wherein the actual resolution is verified after picture blur, the weight distribution in the evaluation model is corrected by quantifying the resolution degradation degree, the health degree of audio and video equipment is tested, the audio equipment test items include total harmonic distortion, noise and frequency response, the video equipment test items include color restoration error and brightness uniformity; S3, speech recognition verification: verify information transmission stability through speech recognition, capture semantic integrity, improve overall evaluation, and quantify information fidelity, finally integrate output speech stability value, build a dedicated recognition system, control the speech recognition device to move to different areas in the conference scene for recognition, and divide the speech stability value according to the area; S4, establish scoring prediction model: use LSTM network to capture conference dynamic changes, use regression model input as the output of steps S1, S2, S3: data collection indicators, evaluation model score, speech stability value, target variable is comprehensive quality score, according to the audio and video equipment health degree evaluation results obtained in S1, analyze the reliability of the comprehensive quality score, generate the effectiveness probability of the quality score, and the equipment state weight is dynamically adjusted according to the importance of the conference, and predict the overall quality score of the conference, realize forward-looking optimization, and establish a closed loop mechanism of equipment degradation, quality attenuation, strategy response and equipment repair; The method uses an audio recognition device, which comprises a recognition collection frame, independent recognition units and a switching unit, wherein the switching unit comprises a first switching unit and a second switching unit, a moving groove is opened in the inside of the recognition collection frame, the independent recognition units are embedded in the inside of the moving groove, slide rails are welded on the sides of the recognition collection frame, one end of each independent recognition unit is embedded in the inside of the slide rail, support plates are welded on the two sides of the bottom of the recognition collection frame, fixed support wheels are inserted into the bottom of each support plate, a lead screw is inserted into the inside of the recognition collection frame, second switching units are installed at the two ends of the lead screw, each second switching unit penetrates out of the same side of the recognition collection frame, a support plate is welded on one side of the top of the recognition collection frame, and the first switching units are inserted into the surface of the support plate. Through the two independent recognition units carried on the recognition collection frame, the end area in the large conference scene that needs to be analyzed is first recognized and detected, the two groups of independent recognition units are driven and controlled by the motor at one end of the recognition collection frame, so that the two independent recognition units are respectively located in the middle and both ends of the recognition collection frame to complete the subsequent recognition process of the audio device. The first switching unit comprises a fixed plate, a pneumatic assembly, a connecting rod, a first supporting plate and a second supporting plate, the pneumatic assembly is installed at the end of the fixed plate, an extension plate is screwed at the top end of the pneumatic assembly, the connecting rod is integrally formed on one side of the extension plate, the first supporting plate is integrally formed in the middle of the connecting rod, and the second supporting plate is integrally formed at the two ends of the first supporting plate. The second switching unit comprises a rotating frame, a driving wheel, a gear and a pressure plate, a rotating sleeve is integrally formed at the top end of the rotating frame, the driving wheel is sleeved at the end of the rotating frame, the gears are installed on the surface of the driving wheel and the lead screw, the toothed belt is sleeved between the two gears, the inclined plate is welded on the top of the rotating frame, the pressure plate is integrally formed at the top end of the inclined plate, the bottom of the pneumatic assembly is aligned with the surface of the pressure plate through the rod, and the surface of the rotating frame is connected with the surface of the recognition collection frame through the return spring.

2. The method for audio and video conference quality analysis and evaluation according to claim 1, characterized in that: In step S1, a high-precision timestamp is uniformly stamped during data collection, and a weighted moving average algorithm is used to filter transient jitter, finally generating a continuous and smooth fluctuation curve, which directly reflects network congestion or device performance bottleneck; After defining the unified data model, edge computing preprocessing is introduced, and CRC check code is added to avoid the influence of collection errors on the model in step S4; the network AV test project includes code stream delay, the audio device is input with a standard level sinusoidal wave signal, and an audio analyzer is used to measure total harmonic distortion plus noise, signal-to-noise ratio, frequency response, channel separation, output level / power; and an LCR table is used to measure the capacitance / inductance drift of key components and the equivalent series resistance rise value; the power conversion efficiency of the current device is recorded by a power analyzer, and compared with the original data at the time of factory shipment to obtain the decline amplitude of the power conversion efficiency; physical wear test, motor bearing noise spectrum analysis are performed on mobile storage, cable, fan and plug-in interface.

3. The method for audio and video conference quality analysis and evaluation according to claim 1, characterized in that: In step S2, the video freezing time proportion is the proportion of freezing duration to the total conference time, and freezing is defined as a frame rate lower than a threshold value <5fps; the picture blur period proportion is used to quantify the time proportion of picture blur; The audio break proportion is the time proportion of audio interruption or distortion; Based on the frame rate and code rate fluctuation information collected in step S1, the freezing weight is adjusted, and based on the weighted average, a 0-1 standardized score is output, and the output data evaluation model score is integrated.

4. The method for audio and video conference quality analysis and evaluation according to claim 3, characterized in that, The verification of the actual resolution includes the following contents: Real-time intercept the video frame rendered by the terminal, exclude the interference of device scaling or interface superposition, pre-process each frame for deblurring, select the text area that appears stably in the picture, and identify the character size through the OCR engine; Combine the standard physical size of characters to calculate the actual pixel density: if the text is missing, use SIFT feature point matching: compare the preset size of the known object in the original video stream with the pixel span of the object in the rendered frame, calculate the scaling ratio, and the calculation formula is: Actual resolution = encoding resolution × (distance of feature points in rendered frame / distance of feature points in original stream) And according to the measured actual resolution, the resolution degradation index RDI is obtained, and the weight correction factor λ is integrated into the comprehensive evaluation model, and the comprehensive evaluation model score calculation formula is: Comprehensive score = (freezing proportion × 0.4) + (blur proportion × 0.35) + (break proportion × 0.25) New blur weight = 0.35 × λ; The comprehensive evaluation model score calculation formula after introducing the resolution correction is:

5. The method of claim 1, wherein: In step S3, an automatic speech recognition engine is used to convert the conference audio stream into text in real time, and the key parameters include: text integrity, break detection, and semantic consistency. The text integrity calculates the similarity between the recognized text and the original or expected content, and the formula is: Integrity = 1-(edit distance / original text length) The break detection identifies the missing text caused by audio break, and counts the break event frequency; The semantic consistency uses an NLP model to analyze the context sentence in the text; Calculate the final output: Stability = 0.5 × integrity + 0.3 × inverse of break frequency + 0.2 × consistency; And the stability result is fed back to the model in step S2 as a correction factor to adjust the comprehensive evaluation. When the stability is lower than the threshold value, the audio break weight is reduced; Use the built-in speech recognition device to move to different positions in the conference scene and repeat the speech recognition process, with an identification period of no less than 20 minutes, and keep the audio and video content in each period completely consistent. Each test area collects the corresponding voice stability value, and finally obtains the average stability data result in the entire conference scene.

6. The method for audio and video conference quality analysis and evaluation according to claim 1, characterized in that: The step S4 is used for scaling all features to the interval [0, 1], eliminating the dimension influence, inputting the feature vector into the scoring prediction model, and outputting the predicted score and generating the root cause report; the total harmonic distortion, noise, frequency response, channel separation, output level / power, power conversion efficiency, and physical wear items of the test are set to be qualified grades, the average grade of the finally obtained audio / video equipment health degree is compared with the collected conference overall quality score for verification, so as to improve the reliability of the finally obtained conference overall quality score, and the conference overall quality score and the reliability of the score are obtained by comprehensively considering the equipment running state data and the three main dimension indexes; if there is a large difference between the evaluation result generated by the collected audio / video equipment health degree detection data and the conference overall quality score, the reliability rating of the conference overall quality score result is reduced; if the difference between the evaluation result generated by the collected audio / video equipment health degree detection data and the conference overall quality score is small, the reliability rating of the conference overall quality score result is improved.

7. A quality analysis evaluation system used in the method as claimed in claim 1, characterized by: The system comprises a collection end module, a data processing and storage engine, a quality evaluation engine, and a speech recognition verification system. The speech recognition verification system is built in a conference room, and a speech recognition device is used to switch multiple recognition scenes, so as to obtain the stability average data of the audio conference quality in the conference room scene, and the effectiveness of the audio break analyzer is verified in reverse.

8. The mass analysis evaluation system of claim 7, wherein: The quality evaluation engine comprises a stuttering analyzer, a blur detector, an audio break analyzer, and a dynamic weight distribution module. The quality evaluation engine fuses the equipment health degree data in real time to construct a quantitative correlation model of hardware performance degradation and quality risk. At the audio quality level, when the total harmonic distortion is greater than 3% and the signal-to-noise ratio is less than 70 dB, the probability of voice clarity score decrease is increased, and when the frequency response and the channel separation are greater than 50 dB, it is determined that the bottom noise is enhanced due to the aging of the microphone array or the capacitor drift of the circuit board. At the video quality level, when the color restoration error ΔE is greater than 5 or the brightness uniformity is less than 85%, the picture blur perception rate increases, and when the display driver chip temperature rise is greater than 15℃ and the power conversion efficiency decreases by more than 12%, the color distortion caused by abnormal heat dissipation or aging of components is identified, and the probability of video clarity score decrease is increased. At the network transmission level, when the cable impedance deviation is greater than 10% of the initial value and the interface plugging life exceeds 5000 times, the code stream delay fluctuation is associated, and when the LCR detects that the filter inductance value deviates by ±15%, the device packet error rate will increase. At this time, the sudden packet loss caused by the fan bearing noise spectrum prediction of heat dissipation failure will cause the conference score to decrease.

9. The mass analysis evaluation system of claim 8, wherein: The blur detector outputs the blur period ratio and the actual resolution based on the OpenCV+SSIM algorithm. The audio break analyzer outputs the break event frequency through double-threshold detection of short-time energy zero-crossing rate. The dynamic weight distribution module has a resolution degradation index feedback mechanism, and outputs the fusion formula weight λ.

10. The mass analysis evaluation system of claim 8, wherein: The data processing and storage engine comprises a multi-source data access layer, a streaming processing layer and a distributed storage layer, the streaming processing layer is based on Apache Flink to build a time window, adopts dynamic watermark processing clock drift, and adjusts and controls the performance data of audio and video equipment and the time stamp alignment of conference media stream.

Images