A multimedia multi-audio test system

Abstract

The invention discloses a multimedia multi-audio test system, and relates to the technical field of audio test. The invention comprises an audio acquisition module, an audio quality evaluation module, an adaptive adjustment module and a user interaction module; the audio acquisition module is used for automatically extracting audio data in multimedia content and performing preprocessing; and determining the multimedia file format; the invention provides a comprehensive audio quality evaluation method by integrating multiple audio evaluation parameters, such as signal-to-noise ratio, distortion, frequency response, etc. The system not only performs automatic objective evaluation on audio signals, but also combines the subjective listening evaluation of users to ensure the accuracy and practicality of the evaluation results. The comprehensive evaluation mechanism makes the judgment of audio quality more accurate, helps to timely discover and solve audio quality problems, and thus significantly improves the overall sound quality of multimedia content.

Description

Technical Field

[0001] The invention relates to the technical field of audio testing, in particular to a multimedia multi-audio testing system. Background Art

[0002] With the rapid development of multimedia technology, audio, as an important part of multimedia content, its quality directly affects the user experience.

[0003] During multimedia production, transmission and playback, audio signals may be affected by various noises, distortions and synchronization issues, resulting in degraded sound quality.

[0004] Therefore, it is particularly important to accurately and comprehensively test and evaluate audio quality. Existing audio testing methods often rely on a single evaluation indicator or lack an automated testing process, and are unable to comprehensively evaluate audio quality, especially in complex multimedia environments with multiple audio streams.

[0005] Traditional audio test systems usually only focus on certain specific parameters of audio, such as signal-to-noise ratio or distortion, while ignoring the overall performance of the audio signal. In addition, these systems often require manual operation and lack the ability to automatically process multiple audio streams, resulting in inefficiency and difficulty in adapting to rapidly changing multimedia content. In the case of multiple audio streams, such as stereo and surround sound systems, existing test methods have difficulty accurately evaluating sound image positioning and separation, which is critical to providing an immersive audio experience. Therefore, a test system that can automatically process and evaluate multiple audio streams is needed to meet the needs of modern multimedia applications.

[0006] In order to solve the above defects, a technical solution is now provided. Summary of the invention

[0007] The purpose of the present invention is to solve the problems of incomplete evaluation, low operation efficiency and lack of automatic adjustment capability in the existing audio test system when processing multi-audio stream multimedia content, and to propose a multimedia multi-audio test system.

[0008] The purpose of the present invention can be achieved through the following technical solutions:

[0009] A multimedia multi-audio test system, comprising:

[0010] Audio acquisition module, used to automatically extract audio data from multimedia content and perform preprocessing;

[0011] The audio quality evaluation module is used to evaluate the quality of the extracted audio stream. The specific process is as follows:

[0012] Set evaluation criteria and parameters, and configure the degree and focus of evaluation according to user needs or presets. Evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation;

[0013] Visualize the evaluation parameters;

[0014] After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. Specifically, the evaluation results of each evaluation parameter are converted into quantitative values, and then weights are assigned according to the importance of each parameter. After normalization, the quantitative value of each parameter is multiplied by its weight, and then all products are summed to obtain a comprehensive score, which is recorded as the evaluation score.

[0015] Then, a subjective listening evaluation is performed, and the evaluation score obtained is corrected by the subjective parameters of the subjective evaluation to obtain a final score, and the final score obtained is compared with a preset final evaluation threshold score. When the final score is lower than the preset final evaluation threshold score, it is determined that the audio quality does not meet the standard, and an adjustment signaling is sent to the adaptive adjustment module to improve the overall audio quality;

[0016] Adaptive adjustment module, used to automatically adjust the parameters of the audio stream according to the evaluation results to optimize the audio quality;

[0017] The user interaction module provides a user interface that allows users to monitor the synchronization and evaluation process.

[0018] Furthermore, the process of the audio quality evaluation module evaluating different single parameters is as follows:

[0019] Audio signal analysis: Perform signal analysis in the time domain and frequency domain on the pre-processed audio data, and calculate the basic parameters of the signal, including amplitude, frequency and phase;

[0020] Signal-to-noise ratio: measures the ratio of useful signal to background noise in the audio signal, calculates the signal-to-noise ratio, and compares it with the preset threshold to evaluate the clarity of the audio;

[0021] Distortion: Measures the distortion of audio signals during amplification or processing, including harmonic distortion and intermodulation distortion, and evaluates the impact on sound quality;

[0022] Frequency response: Analyzes the response of audio signals at different frequencies to ensure that the audio device or content provides balanced sound across the entire frequency range;

[0023] Dynamic range: measures the difference between the maximum and minimum audible levels of an audio signal, assessing the dynamic expressiveness of audio content;

[0024] Transient response: evaluates the audio signal's ability to respond to rapid changes;

[0025] Stereo and Surround Sound Evaluation: For stereo or surround sound audio, evaluate the positioning and separation of sound images to ensure that the distribution of sound in the space is as expected.

[0026] Furthermore, the specific operation steps of the audio quality evaluation module to visualize the evaluation parameters are as follows:

[0027] The specific visualization method is as follows:

[0028] Audio waveform graph: Displays the time domain waveform of the audio signal, allowing you to intuitively see the amplitude changes of the audio signal. You can also observe the continuity, breakpoints, and noise of the audio signal through the waveform graph.

[0029] Spectrogram: Displays the frequency domain distribution of an audio signal, with frequency as the horizontal axis and amplitude or energy as the vertical axis. The spectrogram is used to identify the frequency components and possible frequency distortion in the audio.

[0030] Signal-to-noise ratio chart: Displays the measurement results of the signal-to-noise ratio through a bar chart or line chart, compares it with the preset threshold, and displays the changes in the signal-to-noise ratio in different time periods or different audio segments;

[0031] Distortion curve: Harmonic distortion and intermodulation distortion are displayed through bar graphs or curves, showing the degree of distortion at different harmonics or frequencies, and the comparison with the standard or threshold;

[0032] Frequency response curve: Draw a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve graph is used to show the uniformity of frequency response and whether there are peaks and valleys;

[0033] Dynamic range bar graph: Use a bar graph to display the maximum and minimum audible levels of the audio signal, which is used to intuitively see the dynamic range of the audio content;

[0034] Transient response graph: Displays the response of the audio signal to transient changes through a time-amplitude graph, which is used to observe the rise time and decay characteristics of the signal;

[0035] Stereo and Surround Positioning Diagrams: Use polar coordinates or three-dimensional sound field diagrams to show the location of various sound sources in stereo or surround audio, allowing you to see the distribution and direction of sound sources in space.

[0036] Furthermore, the specific operation steps of the audio quality evaluation module converting the evaluation result of each evaluation parameter into a quantized value are as follows:

[0037] Audio signal analysis: Quantitative indicators: amplitude, frequency, phase; Conversion method: Use Fourier transform frequency domain analysis method to extract the frequency components of the signal and calculate the amplitude, frequency and phase;

[0038] Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio; Conversion method: Calculate the ratio of signal power to noise power, and then convert it to decibels using the formula: ,in is the signal power, is the noise power;

[0039] Distortion: Quantitative indicators: total harmonic distortion plus noise, intermodulation distortion; Conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave; is the total harmonic distortion, for noise;

[0040] Frequency response: Quantitative indicator: frequency response curve; Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve;

[0041] Dynamic range: Quantitative indicator: dynamic range; Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level;

[0042] Transient response: Quantitative indicators: rise time, overshoot, steady-state error; Conversion method: analyze the behavior of the audio signal during transient changes and measure the time required for the signal to go from 10% to 90% of the maximum value;

[0043] Stereo and surround sound evaluation: Quantitative indicators: sound and image positioning accuracy, sound field width, sound and image separation; Conversion method: Use stereo or surround sound test signals to measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound and image positioning and separation.

[0044] Furthermore, the specific operation steps of the audio quality evaluation module to correct the evaluation score obtained by using the subjective parameters of the subjective evaluation are as follows:

[0045] Through listening tests, we collect users’ subjective evaluations of sound quality and analyze subjective parameters, including:

[0046] Clarity evaluation: The user's perception of details in the audio, whether each note or dialogue can be heard clearly, and the user's ratings are collected based on this standard, and the average is calculated as the clarity evaluation value, which is used as a standard to measure the user's evaluation of audio clarity;

[0047] Tone evaluation: The sound quality characteristics of audio, including the color and texture of musical instruments or sounds, are collected based on this standard, and the average is calculated as the color evaluation value, which is used as a standard to measure users' evaluation of the tone of the audio;

[0048] Balance evaluation: the balance between different frequency components, whether the audio maintains a balance between high and low frequencies, and collect user ratings based on this standard, calculate the average value and record it as the balance value, and use this balance value as a standard to measure users' evaluation of the balance of the audio;

[0049] Immersion evaluation: The degree of immersion of the user in the listening process, whether the audio can make the user feel immersive, and collect user ratings based on this standard, calculate the average value and record it as the immersion judgment value, and use this immersion judgment value as a standard to measure the user's evaluation of the balance of the audio;

[0050] Then the obtained clarity evaluation value, color evaluation value, balance value and sedimentation judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are entered into the following formula: To get the final score ZPP, where PCF is the evaluation score, They are the preset weight coefficients for clarity, color, balance and sedimentation respectively, and the final score is used as the standard for measuring audio quality evaluation.

[0051] Furthermore, the specific operation steps of the adaptive adjustment module to automatically adjust the parameters of the audio stream according to the evaluation results are as follows:

[0052] The audio quality evaluation module receives the evaluation results of the audio single parameters and the final score of the comprehensive evaluation. When the adjustment signaling is received, the automatic adjustment process is triggered, including:

[0053] First, the evaluation results of each single parameter analyzed in the audio quality evaluation module are analyzed, and the percentage of each single parameter below the preset standard is calculated, so as to determine the specific reason why the audio quality does not meet the standard;

[0054] Based on the analysis results, determine the audio parameters that need to be adjusted, including gain, equalization, compression, and expansion, and apply the corresponding audio processing algorithm to adjust the audio stream in real time;

[0055] During the adjustment process, the quality changes of the audio stream are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. According to the evaluation results, if the analyzed audio quality still does not meet the standard, the adjustment strategy is continued or optimized;

[0056] Record the specific information of each adjustment, including the adjusted parameters and the adjusted effects. Based on the historical adjustment data, continuously optimize the adaptive adjustment algorithm to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, notify the user through the user interaction module and provide the final audio quality evaluation report.

[0057] Furthermore, the specific operation steps of the audio acquisition module to automatically extract audio data from multimedia content and perform preprocessing are as follows:

[0058] Identify the formats of multimedia files, including MP4, AVI, MOV, MKV, and extract audio stream information from the files, including encoding format, sampling rate, bit rate, and number of channels;

[0059] Separate audio streams from multimedia files using decoding and decapsulation techniques;

[0060] The resulting audio stream is then preprocessed, including: noise reduction: applying algorithms to reduce background noise; gain control: adjusting the volume to a preset level; equalization: using an equalizer to optimize frequency response; sampling rate conversion: unifying the sampling rate to a standard value;

[0061] Convert audio data into PCM format for subsequent processing needs, establish a buffer mechanism to ensure the continuity of audio streams and delay in processing time;

[0062] Detect and resolve errors during extraction and preprocessing, including file corruption or unsupported formats.

[0063] Furthermore, the specific operation steps of the audio acquisition module to separate the audio stream from the multimedia file using decoding and decapsulation technology are as follows:

[0064] Open multimedia files, read file header information, locate audio data streams, and use tools such as FFmpeg or GStreamer to parse multimedia containers of different formats;

[0065] Determine the encoding format, type, and language attributes of the audio stream, select a decoder that matches the encoding format of the audio stream, and convert the compressed data into the original audio signal in PCM format;

[0066] For files containing multiple audio streams, repeat the decapsulation and decoding steps and ensure that the streams are synchronized on the timeline;

[0067] Manage the resources used in the decapsulation and decoding process, and obtain an asset valuation by performing a comprehensive evaluation through monitoring usage parameters. Compare the obtained asset valuation with the preset asset valuation threshold. When the asset valuation is greater than the preset asset valuation threshold, it is determined that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued. Optimization is performed by reducing the audio streams processed simultaneously to avoid system overload.

[0068] Furthermore, the specific process of the audio acquisition module performing comprehensive evaluation by monitoring the usage parameters to obtain the asset valuation is as follows:

[0069] The parameters used include:

[0070] CPU usage: measures the processor usage during the decapsulation and decoding process and is expressed as a percentage;

[0071] Memory occupancy rate: monitors the ratio of physical memory consumed during decapsulation and decoding to the total amount of original physical memory;

[0072] Disk I / O: Evaluates the disk input / output operation rate when reading multimedia files and writing decoded data, using the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The preset standard input operation speed and output operation speed are respectively used; and the obtained magnetic judgment value CP is used as a standard for measuring the input / output operation rate of the disk when reading multimedia files and writing decoded data;

[0073] GPU usage: for cases where hardware accelerated decoding is used;

[0074] Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the sum of CPU usage and GPU usage is used as the radius to establish the bottom circle. A cone model is established with a high memory occupancy. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of ​​the alien body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as a standard to measure the resources used in the decapsulation and decoding process.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] The present invention provides a comprehensive audio quality evaluation method by integrating multiple audio evaluation parameters, such as signal-to-noise ratio, distortion, frequency response, etc. The system not only performs automatic objective evaluation of audio signals, but also combines the user's subjective listening evaluation to ensure the accuracy and practicality of the evaluation results. This comprehensive evaluation mechanism makes the judgment of audio quality more accurate, helps to timely discover and solve audio quality problems, thereby significantly improving the overall sound quality of multimedia content;

[0077] The present invention realizes automatic audio parameter adjustment based on the evaluation results through the adaptive adjustment module, can intelligently analyze the evaluation data, identify the reasons why the audio quality does not meet the standards, and automatically apply the corresponding audio processing algorithm for optimization. This automated processing flow not only improves work efficiency and reduces manual intervention, but also ensures the timeliness and effectiveness of audio adjustment through intelligent adjustment strategies, further improving the quality of audio processing and the user experience of the system;

[0078] The user interaction module of the present invention provides a friendly user interface for the system of the present invention, so that the user can easily monitor the synchronization and evaluation process of the audio. The real-time data display and evaluation result feedback functions of the module enable the user to intuitively understand the status of the audio quality. At the same time, a parameter adjustment tool is provided, so that the user can manually adjust the audio stream parameters as needed. By optimizing the user interaction design, the audio test and adjustment process is made more intuitive, flexible and user-friendly. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0080] Figure 1 This is the overall system block diagram of the present invention. DETAILED DESCRIPTION

[0081] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0083] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0084] like Figure 1 As shown, a multimedia multi-audio test system includes an audio acquisition module, an audio quality evaluation module, an adaptive adjustment module and a user interaction module;

[0085] The audio acquisition module is used to automatically extract audio data from multimedia content and perform preprocessing;

[0086] Determine the multimedia file format, including MP4, AVI, MOV, MKV, etc., detect the audio stream information in the file, including the encoding format, sampling rate, bit rate and number of channels; separate the audio stream from the multimedia file through decoding and decapsulation process. The specific process is as follows:

[0087] Read multimedia files, which may involve opening files, reading file header information, and locating data streams. Use corresponding decapsulation tools or libraries (such as FFmpeg, GStreamer, etc.) to process different container formats and separate different data streams (such as audio streams, video streams, and subtitle streams) in multimedia container files; identify the audio stream contained in the file, including checking the type, encoding format, language and other information of the stream. According to the encoding format of the audio stream, select the corresponding decoder to decode the audio data, and decode the compressed audio data stream into the original audio signal or some intermediate format, such as PCM. The decoding process includes steps such as entropy decoding, inverse quantization, and filtering.

[0088] When a multimedia file contains multiple audio streams, repeat the above decapsulation and decoding process for each audio stream, manage multiple audio streams to ensure that they can be processed synchronously, and when processing multiple audio streams, ensure that they are correctly aligned on the timeline to facilitate subsequent audio processing; manage the resources used in the decapsulation and decoding process, and conduct a comprehensive evaluation by monitoring the usage parameters, including:

[0089] CPU usage: measures the processor usage during the decapsulation and decoding process, and is expressed as a percentage; Memory occupancy: monitors the ratio of physical memory (RAM) consumed during the decapsulation and decoding process to the original total physical memory; Disk I / O: evaluates the input / output operation rate of the disk when reading multimedia files and writing decoded data, through the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The preset standard input operation speed and output operation speed are respectively; and the obtained magnetic judgment value CP is used as a standard for measuring the input / output operation rate of the disk when reading multimedia files and writing decoded data; GPU utilization rate: For the case of using hardware accelerated decoding, the GPU utilization rate is also a measurement indicator;

[0090] Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the sum of CPU usage and GPU usage is used as the radius to establish a bottom circle, and a cone model is established with a high memory occupancy. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of ​​the alien body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as a standard for measuring the resources used in the decapsulation and decoding process; the correction index a is set between 0.53 and 1.33; the obtained asset valuation is compared with the preset asset valuation threshold. When the asset valuation is greater than the preset asset valuation threshold, it is judged that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued. Optimization is performed by reducing the audio streams processed simultaneously to avoid system overload.

[0091] The obtained audio stream is then preprocessed, including:

[0092] Noise reduction: Use algorithms to reduce or eliminate background noise and improve the quality of audio signals; Gain control: Adjust the volume level of the audio signal to ensure that the audio signal is within the appropriate level range; Equalization: Apply an equalizer to adjust the frequency response of the audio signal to improve the sound quality; Sampling rate conversion: Convert the sampling rate of the audio stream to a unified standard sampling rate for subsequent processing;

[0093] The preprocessed audio data is then converted into a format suitable for further processing, such as PCM (Pulse Code Modulation), ensuring that the data format matches the requirements of subsequent modules (such as the audio quality assessment module); a buffering mechanism is implemented to handle the discontinuity and time delay of the audio stream, and to detect and handle errors that occur during the extraction and preprocessing process, including file corruption, unsupported format, or incomplete data.

[0094] The audio quality evaluation module is used to evaluate the quality of the extracted audio stream;

[0095] Set the evaluation criteria and parameters, and configure the degree and focus of the evaluation according to user needs or presets. The evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation; specifically:

[0096] Audio signal analysis: Perform signal analysis in the time domain and frequency domain on the pre-processed audio data, and calculate the basic parameters of the signal, including amplitude, frequency and phase; Signal-to-noise ratio: Measure the ratio of useful signal to background noise in the audio signal, calculate the signal-to-noise ratio, and compare it with the preset threshold to evaluate the clarity of the audio; Distortion: Measure the distortion generated by the audio signal during amplification or processing, including harmonic distortion, intermodulation distortion, etc., and evaluate its impact on the sound quality; Frequency response: Analyze the response of the audio signal at different frequencies to ensure that the audio device or content provides balanced sound over the entire frequency range; Dynamic range: Measure the difference between the maximum and minimum audible levels of the audio signal to evaluate the dynamic expressiveness of the audio content; Transient response: Evaluate the audio signal's ability to respond to rapid changes, which is particularly important for percussion instruments and electronic music; Stereo and surround sound evaluation: For stereo or surround sound audio, evaluate the positioning and separation of the sound image to ensure that the distribution of the sound in the space is as expected;

[0097] The evaluation parameters are visualized as follows:

[0098] Audio waveform: Displays the time domain waveform of the audio signal, which allows you to intuitively see the amplitude changes of the audio signal. The continuity, breakpoints and noise of the audio signal can be observed through the waveform. Spectrum: Displays the frequency domain distribution of the audio signal, usually with frequency as the horizontal axis and amplitude or energy as the vertical axis. The spectrum can help identify the frequency components and possible frequency distortion in the audio. Signal-to-noise ratio (SNR) chart: Displays the measurement results of the signal-to-noise ratio through a bar chart or a line chart, and compares it with the preset threshold to show the changes in the signal-to-noise ratio in different time periods or different audio segments. Distortion curve: Harmonic distortion and intermodulation distortion can be displayed through a bar chart or a curve chart, showing the degree of distortion at different harmonics or frequencies, as well as the difference with the standard. Frequency response curve: Draw a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve can clearly show the uniformity of frequency response and whether there are peaks and valleys. Dynamic range bar graph: Use a bar graph to show the maximum and minimum audible levels of the audio signal, so you can intuitively see the dynamic range of the audio content. Transient response graph: Use a time-amplitude graph to show the response of the audio signal to transient changes, and you can observe the rise time and attenuation characteristics of the signal. Stereo and surround sound positioning graph: Use a polar coordinate graph or a three-dimensional sound field graph to show the position of each sound source in stereo or surround sound audio, so you can intuitively see the distribution and direction of the sound source in space.

[0099] After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. The specific process is as follows:

[0100] Convert the evaluation results of each evaluation parameter into quantitative values, including:

[0101] Audio signal analysis: Quantitative indicators: amplitude (dB), frequency (Hz), phase (degrees or radians); Conversion method: Use frequency domain analysis methods such as Fourier transform to extract the frequency components of the signal and calculate its amplitude, frequency and phase; Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio (dB); Conversion method: Calculate the ratio of signal power to noise power, and then convert it to decibels (dB) through the formula: ,in is the signal power, is the noise power; distortion: quantitative indicators: total harmonic distortion plus noise (THD+N), intermodulation distortion (IMD); conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave; is the total harmonic distortion, For noise; Frequency response: Quantitative index: frequency response curve (dB); Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve; Dynamic range: Quantitative index: dynamic range (dB); Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level; Transient response: Quantitative index: rise time (trise), overshoot (Overshoot), steady-state error; Conversion method: analyze the behavior of the audio signal during transient changes, such as measuring the time required for the signal to go from 10% to 90% of the maximum value (rise time); Stereo and surround sound evaluation: Quantitative index: sound and image positioning accuracy, sound field width, sound and image separation; Conversion method: use stereo or surround sound test signals (such as pink noise, sine wave scanning, etc.) to measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound and image positioning and separation;

[0102] Then, weights are assigned to each parameter based on its importance. The weights are determined based on experience, user needs or industry standards, where the weight assignment reflects the degree of influence of each parameter on the overall audio quality. In order to ensure that the quantized values ​​of different parameters are comparable, they need to be normalized so that they are at the same magnitude, where normalization can be linear or a nonlinear transformation based on a certain function. The quantized value of each parameter is multiplied by its weight, and then all products are summed to obtain a comprehensive score, which is recorded as the evaluation score.

[0103] Then conduct a subjective listening test and correct the evaluation score based on the subjective parameters of the subjective evaluation. Specifically, through the listening test, collect the user's subjective evaluation of the sound quality and analyze the subjective parameters, including:

[0104] Clarity evaluation: the user's perception of details in the audio, whether each note or dialogue can be heard clearly, and the user's ratings are collected based on this standard, and the average is calculated as the clarity rating value, which is used as a standard to measure the user's evaluation of the clarity of the audio; Tone evaluation: the sound quality characteristics of the audio, including the color and texture of the instrument or sound, and the user's ratings are collected based on this standard, and the average is calculated as the color rating value, which is used as a standard to measure the user's evaluation of the tone of the audio; Balance evaluation: the balance between different frequency components, whether the audio maintains a balance between high and low frequencies A good balance is achieved, and the user ratings are collected based on this standard. The average value is calculated and recorded as the balance value, and this balance value is used as a standard to measure the user's evaluation of the balance of the audio; immersion evaluation: the degree of immersion of the user in the listening process, whether the audio can make the user feel immersive, and the user ratings are collected based on this standard, and the average value is calculated and recorded as the immersion judgment value, and this immersion judgment value is used as a standard to measure the user's evaluation of the balance of the audio; the clarity evaluation value, color evaluation value, balance value and immersion judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are entered into the following formula: To get the final score ZPP, where PCF is the evaluation score, The preset weight coefficients are respectively the clarity evaluation value, the color evaluation value, the balance value and the sedimentation evaluation value, and the final score is used as the standard for measuring the audio quality evaluation; the final score is compared with the preset final evaluation threshold score. When the final score is lower than the preset final evaluation threshold score, it is judged that the audio quality does not meet the standard, and an adjustment signal is sent to the adaptive adjustment module to improve the overall quality of the audio.

[0105] The adaptive adjustment module is used to automatically adjust the parameters of the audio stream according to the evaluation results;

[0106] The audio quality evaluation module receives the evaluation results of the audio single parameters and the final score of the comprehensive evaluation. When the adjustment signaling is received, the automatic adjustment process is triggered, including:

[0107] First, analyze the evaluation results of each single parameter analyzed in the audio quality evaluation module, calculate the percentage of each single parameter below the preset standard, and determine the specific reason for the poor audio quality; according to the analysis results, determine the audio parameters that need to be adjusted, including gain, equalization, compression, expansion, etc., and apply the corresponding audio processing algorithm to adjust the audio stream in real time. For example, if the signal-to-noise ratio is low, it may be necessary to increase the strength of the noise reduction algorithm; if the dynamic range is small, it may be necessary to adjust the threshold of the compressor;

[0108] During the adjustment process, the quality changes of the audio stream are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. Based on the evaluation results, if the audio quality still does not meet the standard, continue to adjust or optimize the adjustment strategy; record the detailed information of each adjustment, including the adjusted parameters and the adjusted effect, and continuously optimize the adaptive adjustment algorithm based on historical adjustment data to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, the user is notified through the user interaction module, and a final audio quality evaluation report is provided.

[0109] The user interaction module is used to provide a user interface, allowing the user to monitor the synchronization and evaluation process;

[0110] Determine user needs, including the type of information they want to monitor, ease of operation, and user-friendly interface, and design an intuitive and easy-to-use user interface to ensure that users can easily access and monitor the various functions of the system; plan the functions of the user interaction module, including real-time data display, evaluation result feedback, parameter setting adjustment, etc., develop real-time data display functions, use charts, dashboards, etc. to display the synchronization status and evaluation parameters of audio streams, and visualize audio quality evaluation results;

[0111] Develop parameter adjustment tools to allow users to manually adjust the parameters of the audio stream, such as gain, equalization, etc., provide clear user operation instructions and help documents to help users understand how to use the interactive module; implement a user feedback collection mechanism to allow users to report problems or make improvement suggestions, set different user permission levels to ensure that users can only access and operate the functions they are authorized to, and provide interactive controls such as sliders, buttons, and drop-down menus to enable users to control the audio processing flow; allow users to view the evaluation history and adjustment history of the audio stream, including timestamps and operation details, test the user interaction module, collect user feedback, and optimize based on the feedback, integrate the user interaction module with other system modules, and deploy it in the actual environment.

[0112] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A multimedia multi-audio test system, characterized in that: include: The audio acquisition module is used to automatically extract audio data from multimedia content and perform preprocessing. The specific process is as follows: Identify the formats of multimedia files, including MP4, AVI, MOV, MKV, and extract audio stream information from the files, including encoding format, sampling rate, bit rate, and number of channels; Separate audio streams from multimedia files using decoding and decapsulation techniques; The resulting audio stream is then preprocessed, including: noise reduction: applying algorithms to reduce background noise; gain control: adjusting the volume to a preset level; equalization: using an equalizer to optimize frequency response; sampling rate conversion: unifying the sampling rate to a standard value; Convert audio data into PCM format for subsequent processing needs, establish a buffer mechanism to ensure the continuity of audio streams and delay in processing time; Detect and resolve errors during extraction and preprocessing, including corrupted files or unsupported formats; The specific steps for separating audio streams from multimedia files using decoding and decapsulation technology are as follows: Open multimedia files, read file header information, locate audio data streams, and use tools such as FFmpeg or GStreamer to parse multimedia containers of different formats; Determine the encoding format, type, and language attributes of the audio stream, select a decoder that matches the encoding format of the audio stream, and convert the compressed data into the original audio signal in PCM format; For files containing multiple audio streams, repeat the decapsulation and decoding steps and ensure that the streams are synchronized on the timeline; Manage the resources used in the decapsulation and decoding process, and obtain an asset value by monitoring the usage parameters for comprehensive evaluation. The obtained asset value is compared with the preset asset value threshold. When the asset value is greater than the preset asset value threshold, it is determined that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued to optimize by reducing the audio streams processed simultaneously to avoid system overload; The specific process of the audio acquisition module to obtain the asset valuation by monitoring the usage parameters for comprehensive evaluation is as follows: The parameters used include: CPU usage: measures the processor usage during the decapsulation and decoding process and is expressed as a percentage; Memory occupancy rate: monitors the ratio of physical memory consumed during decapsulation and decoding to the total amount of original physical memory; Disk I / O: Evaluates the disk input / output operation rate when reading multimedia files and writing decoded data, using the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The preset standard input operation speed and output operation speed are respectively used; and the obtained magnetic judgment value CP is used as a standard for measuring the input / output operation rate of the disk when reading multimedia files and writing decoded data; GPU usage: for cases where hardware accelerated decoding is used; Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the sum of CPU usage and GPU usage is used as the radius to establish the bottom circle, and the memory occupancy is used to establish a cone model. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of ​​the heteromorphic body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as a standard for measuring the resources used in the decapsulation and decoding process; The audio quality evaluation module is used to evaluate the quality of the extracted audio stream. The specific process is as follows: Set evaluation criteria and parameters, and configure the degree and focus of evaluation according to user needs or presets. Evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation; Visualize the evaluation parameters; After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. Specifically, the evaluation results of each evaluation parameter are converted into quantitative values, and then weights are assigned according to the importance of each parameter. After normalization, the quantitative value of each parameter is multiplied by its weight, and then all products are summed to obtain a comprehensive score, which is recorded as the evaluation score. Then, a subjective listening evaluation is performed, and the evaluation score obtained is corrected by the subjective parameters of the subjective evaluation to obtain a final score, and the final score obtained is compared with the preset final evaluation threshold score. When the final score is lower than the preset final evaluation threshold score, it is judged that the audio quality does not meet the standard, and an adjustment signaling is sent to the adaptive adjustment module to improve the overall quality of the audio. The specific operation steps for correcting the evaluation score obtained by the subjective parameters of the subjective evaluation are as follows: Through listening tests, we collect users’ subjective evaluations of sound quality and analyze subjective parameters, including: Clarity evaluation: The user's perception of details in the audio, whether each note or dialogue can be heard clearly, and the user's ratings are collected based on this standard, and the average is calculated as the clarity evaluation value, which is used as a standard to measure the user's evaluation of audio clarity; Tone evaluation: The sound quality characteristics of audio, including the color and texture of musical instruments or sounds, are collected based on this standard, and the average is calculated as the color evaluation value, which is used as a standard to measure users' evaluation of the audio's tone quality; Balance evaluation: the balance between different frequency components, whether the audio maintains a balance between high and low frequencies, and collect user ratings based on this standard, calculate the average value and record it as the balance value, and use this balance value as a standard to measure users' evaluation of the balance of the audio; Immersion evaluation: The degree of immersion of the user in the listening process, whether the audio can make the user feel immersive, and collect user ratings based on this standard, calculate the average value and record it as the immersion judgment value, and use this immersion judgment value as a standard to measure the user's evaluation of the balance of the audio; Then the obtained clarity evaluation value, color evaluation value, balance value and sedimentation judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are entered into the following formula: To get the final score ZPP, where PCF is the evaluation score, The preset weight coefficients for clarity, color, balance and sink are used to evaluate the audio quality. Adaptive adjustment module, used to automatically adjust the parameters of the audio stream according to the evaluation results to optimize the audio quality; The user interaction module provides a user interface that allows users to monitor the synchronization and evaluation process.

2. A multimedia multi-audio test system according to claim 1, characterized in that: The process of the audio quality evaluation module evaluating different single parameters is as follows: Audio signal analysis: Perform signal analysis in the time domain and frequency domain on the pre-processed audio data, and calculate the basic parameters of the signal, including amplitude, frequency and phase; Signal-to-noise ratio: measures the ratio of useful signal to background noise in the audio signal, calculates the signal-to-noise ratio, and compares it with the preset threshold to evaluate the clarity of the audio; Distortion: Measures the distortion of audio signals during amplification or processing, including harmonic distortion and intermodulation distortion, and evaluates the impact on sound quality; Frequency response: Analyzes the response of audio signals at different frequencies to ensure that the audio device or content provides balanced sound across the entire frequency range; Dynamic range: measures the difference between the maximum and minimum audible levels of an audio signal, assessing the dynamic expressiveness of audio content; Transient response: evaluates the audio signal's ability to respond to rapid changes; Stereo and Surround Sound Evaluation: For stereo or surround sound audio, evaluate the positioning and separation of sound images to ensure that the distribution of sound in the space is as expected.

3. A multimedia multi-audio test system according to claim 1, characterized in that: The specific operation steps of the audio quality evaluation module to visualize the evaluation parameters are as follows: The specific visualization method is as follows: Audio waveform graph: Displays the time domain waveform of the audio signal, allowing you to intuitively see the amplitude changes of the audio signal. You can also observe the continuity, breakpoints, and noise of the audio signal through the waveform graph. Spectrogram: Displays the frequency domain distribution of an audio signal, with frequency as the horizontal axis and amplitude or energy as the vertical axis. The spectrogram is used to identify the frequency components and possible frequency distortion in the audio. Signal-to-noise ratio chart: Displays the measurement results of the signal-to-noise ratio through a bar chart or line chart, compares it with the preset threshold, and displays the changes in the signal-to-noise ratio in different time periods or different audio segments; Distortion curve: Harmonic distortion and intermodulation distortion are displayed through bar graphs or curves, showing the degree of distortion at different harmonics or frequencies, and the comparison with the standard or threshold; Frequency response curve: Draw a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve graph is used to show the uniformity of frequency response and whether there are peaks and valleys; Dynamic range bar graph: Use a bar graph to display the maximum and minimum audible levels of the audio signal, which is used to intuitively see the dynamic range of the audio content; Transient response graph: Displays the response of the audio signal to transient changes through a time-amplitude graph, which is used to observe the rise time and decay characteristics of the signal; Stereo and Surround Positioning Diagrams: Use polar coordinates or three-dimensional sound field diagrams to show the location of various sound sources in stereo or surround audio, allowing you to see the distribution and direction of sound sources in space.

4. A multimedia multi-audio test system according to claim 1, characterized in that: The specific operation steps of the audio quality evaluation module to convert the evaluation result of each evaluation parameter into a quantized value are as follows: Audio signal analysis: Quantitative indicators: amplitude, frequency, phase; Conversion method: Use Fourier transform frequency domain analysis method to extract the frequency components of the signal and calculate the amplitude, frequency and phase; Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio; Conversion method: Calculate the ratio of signal power to noise power, and then convert it to decibels using the formula: ,in is the signal power, is the noise power; Distortion: Quantitative indicators: total harmonic distortion plus noise, intermodulation distortion; Conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave, is the total harmonic distortion, for noise; Frequency response: Quantitative indicator: frequency response curve; Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve; Dynamic range: Quantitative indicator: dynamic range; Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level; Transient response: Quantitative indicators: rise time, overshoot, steady-state error; Conversion method: analyze the behavior of the audio signal during transient changes and measure the time required for the signal to go from 10% to 90% of the maximum value; Stereo and surround sound evaluation: Quantitative indicators: sound and image positioning accuracy, sound field width, sound and image separation; Conversion method: Use stereo or surround sound test signals to measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound and image positioning and separation.

5. A multimedia multi-audio test system according to claim 1, characterized in that: The specific operation steps of the adaptive adjustment module to automatically adjust the parameters of the audio stream according to the evaluation results are as follows: The audio quality evaluation module receives the evaluation results of the audio single parameters and the final score of the comprehensive evaluation. When the adjustment signaling is received, the automatic adjustment process is triggered, including: First, the evaluation results of each single parameter analyzed in the audio quality evaluation module are analyzed, and the percentage of each single parameter below the preset standard is calculated, so as to determine the specific reason why the audio quality does not meet the standard; Based on the analysis results, determine the audio parameters that need to be adjusted, including gain, equalization, compression, and expansion, and apply the corresponding audio processing algorithm to adjust the audio stream in real time; During the adjustment process, the quality changes of the audio stream are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. According to the evaluation results, if the analyzed audio quality still does not meet the standard, the adjustment strategy is continued or optimized; Record the specific information of each adjustment, including the adjusted parameters and the adjusted effects. Based on the historical adjustment data, continuously optimize the adaptive adjustment algorithm to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, notify the user through the user interaction module and provide the final audio quality evaluation report.

Images