Multimedia multi-audio test system
Through the design of the multimedia multi-audio test system, the problem of incomplete evaluation of existing audio test systems in multi-audio streaming environments is solved, and the automatic evaluation and real-time optimization of audio quality is realized, improving the overall quality and user experience of the audio signal.
Patent Information
- Application Number
- CN202411461335.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-10-18
AI Technical Summary
The existing audio testing system is incomplete in its evaluation, low operation efficiency, lacks automation adjustment capabilities when processing multi-audio streaming multimedia content, and is difficult to provide accurate sound quality evaluation and real-time optimization in a multimedia environment.
A multimedia multi-audio testing system was designed, including an audio acquisition module, an audio quality evaluation module, an adaptive adjustment module and a user interaction module. By automatically extracting audio data, comprehensively evaluate the signal-to-noise ratio, distortion, frequency response and other parameters, and Combined with subjective audio evaluation, automated adjustments are achieved and audio quality is improved.
It realizes comprehensive automated evaluation and real-time optimization of audio quality, improves the overall quality and user experience of audio signals, improves the sound quality of multimedia content, and reduces manual intervention and processing time.
Smart Images

Figure CN118984407A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of audio testing, in particular to a multimedia multi-audio testing system. Background Art
[0002] With the rapid development of multimedia technology, audio, as an important component of multimedia content, its quality directly affects the user experience.
[0003] During multimedia production, transmission, and playback, audio signals may be affected by various noise, distortion, and synchronization issues, resulting in degraded sound quality.
[0004] Therefore, accurate and comprehensive testing and evaluation of audio quality is particularly important. Existing audio testing methods often rely on single evaluation indicators or lack automated testing processes, making it difficult to comprehensively evaluate audio quality, especially in complex multimedia environments with multiple audio streams. Traditional audio test systems typically focus only on specific audio parameters, such as signal-to-noise ratio or distortion, while neglecting the overall performance of the audio signal. Furthermore, these systems often require manual operation and lack the ability to automatically process multiple audio streams, resulting in low efficiency and difficulty adapting to rapidly changing multimedia content. In the case of multiple audio streams, such as stereo and surround sound systems, existing test methods struggle to accurately assess the localization and separation of sound and image, which are critical to providing an immersive audio experience. Therefore, a test system that can automatically process and evaluate multiple audio streams is needed to meet the demands of modern multimedia applications.
[0005] In order to solve the above-mentioned defects, a technical solution is now provided. Summary of the Invention
[0006] The present invention aims to solve the problems of incomplete evaluation, low operating efficiency and lack of automatic adjustment capability in existing audio test systems when processing multi-audio stream multimedia content, and proposes a multimedia multi-audio test system.
[0007] The purpose of the present invention can be achieved through the following technical solutions: A multimedia multi-audio test system, comprising: Audio acquisition module, used to automatically extract audio data from multimedia content and perform preprocessing; The audio quality evaluation module is used to evaluate the quality of the extracted audio stream. The specific process is as follows: Set evaluation criteria and parameters, and configure the degree and focus of evaluation based on user needs or presets. Evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation; Visualize the evaluation parameters; After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. Specifically, the evaluation results of each evaluation parameter are converted into quantitative values, and then weights are assigned according to the importance of each parameter. After normalization, the quantitative value of each parameter is multiplied by its weight, and then all the products are summed to obtain a comprehensive score, which is recorded as the evaluation score. A subjective listening evaluation is then conducted, and the evaluation score obtained is corrected using the subjective parameters of the subjective evaluation to obtain a final score. The final score obtained is compared with a preset final evaluation threshold score. If the final score is lower than the preset final evaluation threshold score, it is determined that the audio quality does not meet the standard, and an adjustment signaling is sent to the adaptive adjustment module to improve the overall audio quality; Adaptive adjustment module, used to automatically adjust the parameters of the audio stream according to the evaluation results to optimize the audio quality; The user interaction module provides a user interface that allows users to monitor the synchronization and evaluation process.
[0008] Furthermore, the audio quality evaluation module evaluates different single parameters as follows: Audio signal analysis: Perform signal analysis in the time and frequency domains on pre-processed audio data to calculate the basic parameters of the signal, including amplitude, frequency, and phase; Signal-to-noise ratio: measures the ratio of useful signal to background noise in an audio signal, calculates the signal-to-noise ratio, and compares it with a preset threshold to evaluate the clarity of the audio. Distortion: Measures the distortion generated during audio signal amplification or processing, including harmonic distortion and intermodulation distortion, and evaluates the impact on sound quality; Frequency response: Analyzes the response of audio signals at different frequencies to ensure that the audio device or content provides balanced sound across the entire frequency range; Dynamic range: measures the difference between the maximum and minimum audible levels of an audio signal, assessing the dynamic expressiveness of audio content; Transient response: evaluates the audio signal's ability to respond to rapid changes; Stereo and Surround Sound Evaluation: For stereo or surround sound audio, evaluate the positioning and separation of the sound image to ensure that the sound is distributed in the space as expected.
[0009] Furthermore, the specific operation steps of the audio quality evaluation module to visualize the evaluation parameters are as follows: The specific visualization method is as follows: Audio waveform graph: Displays the time domain waveform of the audio signal, allowing you to intuitively see the amplitude changes of the audio signal. You can also observe the continuity, breakpoints, and noise of the audio signal through the waveform graph. Spectrogram: Displays the frequency domain distribution of an audio signal, with frequency as the horizontal axis and amplitude or energy as the vertical axis. Spectrogram is used to identify frequency components and possible frequency distortion in audio. Signal-to-noise ratio chart: Displays signal-to-noise ratio measurement results through bar charts or line charts, compares them with preset thresholds, and displays changes in signal-to-noise ratio over different time periods or audio segments. Distortion curve: Harmonic distortion and intermodulation distortion are displayed through bar graphs or curves, showing the degree of distortion at different harmonics or frequencies, and comparing with standards or thresholds; Frequency response curve: Draw a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve graph is used to show the uniformity of the frequency response and whether there are peaks and valleys. Dynamic Range Bar Graph: Uses a bar graph to display the maximum and minimum audible levels of the audio signal, allowing you to intuitively see the dynamic range of the audio content. Transient response graph: Displays the response of an audio signal to transient changes through a time-amplitude graph, which is used to observe the rise time and decay characteristics of the signal; Stereo and Surround Positioning Diagrams: Use polar coordinates or three-dimensional sound field diagrams to show the position of each sound source in stereo or surround audio, allowing you to see the distribution and direction of sound sources in space.
[0010] Furthermore, the specific operation steps of the audio quality evaluation module for converting the evaluation result of each evaluation parameter into a quantitative value are as follows: Audio signal analysis: Quantitative indicators: amplitude, frequency, phase; Conversion method: Use Fourier transform frequency domain analysis method to extract the frequency components of the signal and calculate the amplitude, frequency and phase; Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio; Conversion method: Calculate the ratio of signal power to noise power, and then convert it to decibels using the formula: ,in is the signal power, is the noise power; Distortion: Quantitative indicators: total harmonic distortion plus noise, intermodulation distortion; Conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave; is the total harmonic distortion, For noise; Frequency response: Quantitative indicator: frequency response curve; Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve; Dynamic range: Quantitative indicator: dynamic range; Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level; Transient response: Quantitative indicators: rise time, overshoot, steady-state error; Conversion method: analyzes the behavior of the audio signal during transient changes and measures the time required for the signal to go from 10% to 90% of its maximum value; Stereo and surround sound evaluation: Quantitative indicators: sound image positioning accuracy, sound field width, sound image separation; Conversion method: Using stereo or surround sound test signals, measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound image positioning and separation.
[0011] Furthermore, the specific operation steps of the audio quality evaluation module to correct the evaluation score obtained by using the subjective parameters of the subjective evaluation are as follows: Through listening tests, we collect users' subjective evaluations of sound quality and analyze subjective parameters, including: Clarity evaluation: The user's perception of details in the audio, such as whether each note or dialogue can be clearly heard, is used to collect user ratings based on this criterion. The average of these scores is recorded as the clarity rating, which is used as a metric to measure the user's evaluation of audio clarity. Tone evaluation: The sound quality characteristics of audio, including the color and texture of instruments or sounds, are collected based on this standard. The average value is calculated and recorded as the tone evaluation value, which is used as a standard to measure users' evaluation of the audio's tone quality. Balance evaluation: The balance between different frequency components, whether the audio maintains a balance between high and low frequencies, and collect user ratings based on this standard. The average is calculated and recorded as the balance value, which is used as a criterion for measuring users' evaluation of audio balance. Immersion evaluation: The user's level of immersion during the listening process, whether the audio can make the user feel immersive, is collected based on this standard. The average of the scores is calculated as the immersion judgment value, which is used as a criterion for measuring the user's evaluation of the balance of the audio. The obtained clarity evaluation value, color evaluation value, balance value and sedimentation judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are input into the following formula: To get the final score ZPP, where PCF is the evaluation score, The preset weight coefficients are respectively for clarity, color, balance and sink, and the final score is used as the standard for measuring audio quality evaluation.
[0012] Furthermore, the specific operation steps of the adaptive adjustment module to automatically adjust the parameters of the audio stream according to the evaluation results are as follows: The audio quality evaluation module receives the evaluation results of individual audio parameters and the final score of the comprehensive evaluation. When an adjustment signal is received, the automatic adjustment process is triggered, including: First, analyze the evaluation results of each single parameter analyzed in the audio quality evaluation module and calculate the percentage of each single parameter below the preset standard to determine the specific reason why the audio quality does not meet the standard; Based on the analysis results, determine the audio parameters that need to be adjusted, including gain, equalization, compression, and expansion, and apply the corresponding audio processing algorithms to make real-time adjustments to the audio stream; During the adjustment process, the audio stream quality changes are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. Based on the evaluation results, if the analyzed audio quality still does not meet the requirements, the adjustment strategy is further adjusted or optimized. Record the specific information of each adjustment, including the adjusted parameters and the effect after adjustment. Based on historical adjustment data, continuously optimize the adaptive adjustment algorithm to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, notify the user through the user interaction module and provide a final audio quality evaluation report.
[0013] Furthermore, the specific operation steps of the audio acquisition module to automatically extract audio data from multimedia content and perform preprocessing are as follows: Identify multimedia file formats, including MP4, AVI, MOV, and MKV, and extract audio stream information from the file, including encoding format, sampling rate, bit rate, and number of channels; Separate audio streams from multimedia files using decoding and decapsulation techniques; The resulting audio stream is then pre-processed, including: noise reduction: applying algorithms to reduce background noise; gain control: adjusting the volume to a preset level; equalization: using an equalizer to optimize the frequency response; sampling rate conversion: unifying the sampling rate to a standard value; Convert audio data into PCM format for subsequent processing needs, establish a buffer mechanism to ensure the continuity of audio stream and processing time delay; Detect and resolve errors during extraction and pre-processing, including file corruption or unsupported formats.
[0014] Furthermore, the specific operation steps of the audio acquisition module using decoding and decapsulation technology to separate the audio stream from the multimedia file are as follows: Open multimedia files, read file header information, locate audio data streams, and use tools such as FFmpeg or GStreamer to parse multimedia containers of different formats; Determine the encoding format, type, and language attributes of the audio stream, select a decoder that matches the audio stream encoding format, and convert the compressed data into the original audio signal in PCM format; For files containing multiple audio streams, repeat the decapsulation and decoding steps and ensure that the streams are synchronized on the timeline; Manage the resources used in the decapsulation and decoding process, and conduct a comprehensive evaluation by monitoring usage parameters to obtain an estimated resource value. The estimated resource value is compared with the preset resource estimation threshold. When the estimated resource value is greater than the preset resource estimation threshold, it is determined that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued. Optimization is performed by reducing the number of audio streams processed simultaneously to avoid system overload.
[0015] Furthermore, the specific process of the audio acquisition module performing comprehensive evaluation by monitoring usage parameters to obtain asset valuation is as follows: The parameters used include: CPU usage: measures the processor usage during the decapsulation and decoding process and is expressed as a percentage; Memory occupancy: monitors the ratio of physical memory consumed during decapsulation and decoding to the total amount of original physical memory; Disk I / O: Evaluates the disk input / output operation rate when reading multimedia files and writing decoded data, using the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The obtained magnetic judgment value CP is used as a standard for measuring the input / output operation rate of the disk when reading multimedia files and writing decoded data; GPU usage: for hardware accelerated decoding; Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the sum of CPU usage and GPU usage is used as the radius to establish the base circle. A cone model is established with a high memory occupancy. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of the alien body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as the standard for measuring the resources used in the decapsulation and decoding process.
[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a comprehensive audio quality assessment method by integrating multiple audio evaluation parameters, such as signal-to-noise ratio, distortion, and frequency response. The system not only performs automated objective evaluation of audio signals but also incorporates user subjective listening evaluations to ensure the accuracy and practicality of the evaluation results. This comprehensive evaluation mechanism makes audio quality judgments more precise, helps to promptly identify and resolve audio quality issues, and thus significantly improves the overall sound quality of multimedia content. The present invention uses an adaptive adjustment module to achieve automatic audio parameter adjustment based on evaluation results. It can intelligently analyze evaluation data, identify the reasons for substandard audio quality, and automatically apply corresponding audio processing algorithms for optimization. This automated processing flow not only improves work efficiency and reduces manual intervention, but also ensures the timeliness and effectiveness of audio adjustments through intelligent adjustment strategies, further improving the quality of audio processing and the user experience of the system. The user interaction module of the present invention provides a friendly user interface for the system of the present invention, allowing users to easily monitor the synchronization and evaluation process of audio. The real-time data display and evaluation result feedback functions of the module allow users to intuitively understand the status of audio quality. At the same time, a parameter adjustment tool is provided, allowing users to manually adjust audio stream parameters as needed. By optimizing the user interaction design, the audio testing and adjustment process is made more intuitive, flexible and user-friendly. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is the overall system block diagram of the present invention. DETAILED DESCRIPTION
[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0019] It should be understood that the terms “include” and “comprising” used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.
[0020] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.
[0021] like Figure 1 As shown, a multimedia multi-audio testing system includes an audio acquisition module, an audio quality evaluation module, an adaptive adjustment module and a user interaction module; The audio acquisition module is used to automatically extract audio data from multimedia content and perform preprocessing; Determine the multimedia file format, including MP4, AVI, MOV, MKV, etc., and detect the audio stream information in the file, including the encoding format, sampling rate, bit rate, and number of channels; separate the audio stream from the multimedia file through decoding and decapsulation. The specific process is as follows: Reading multimedia files, which may involve opening the file, reading file header information, and locating data streams, using corresponding decapsulation tools or libraries (such as FFmpeg, GStreamer, etc.) to process different container formats, and separating different data streams (such as audio streams, video streams, and subtitle streams) in the multimedia container file; identifying the audio stream contained in the file, including checking the stream type, encoding format, language, and other information; selecting the corresponding decoder to decode the audio data based on the encoding format of the audio stream, and decoding the compressed audio data stream into the original audio signal or some intermediate format, such as PCM. The decoding process includes steps such as entropy decoding, inverse quantization, and filtering; When a multimedia file contains multiple audio streams, repeat the above decapsulation and decoding process for each audio stream, manage multiple audio streams, ensure they can be processed synchronously, and when processing multiple audio streams, ensure they are correctly aligned on the timeline to facilitate subsequent audio processing; manage the resources used in the decapsulation and decoding process, and conduct comprehensive evaluation by monitoring usage parameters, including: CPU usage: measures the processor usage during the decapsulation and decoding process and is expressed as a percentage. Memory occupancy: monitors the ratio of physical memory (RAM) consumed during the decapsulation and decoding process to the original physical memory. Disk I / O: evaluates the disk input / output operation rate when reading multimedia files and writing decoded data, using the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The default input and output operation speeds are respectively; the obtained magnetic judgment value CP is used as a standard to measure the disk input / output operation rate when reading multimedia files and writing decoded data; GPU usage rate: When using hardware accelerated decoding, the GPU usage rate is also a measurement indicator; Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the base circle is established with the sum of CPU usage and GPU usage as the radius. A cone model is established with high memory occupancy. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of the alien body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as a standard for measuring the resources used in the decapsulation and decoding process; the correction index a is set between 0.53-1.33; the obtained asset valuation is compared with the preset asset valuation threshold. When the asset valuation is greater than the preset asset valuation threshold, it is judged that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued. Optimization is performed by reducing the audio streams processed simultaneously to avoid system overload.
[0022] The obtained audio stream is then preprocessed, including: Noise reduction: Uses algorithms to reduce or eliminate background noise and improve the quality of audio signals. Gain control: Adjusts the volume level of audio signals to ensure that the audio signals are within the appropriate level range. Equalization: Applying an equalizer to adjust the frequency response of audio signals to improve sound quality. Sampling rate conversion: Converts the sampling rate of audio streams to a unified standard sampling rate for subsequent processing. The preprocessed audio data is then converted into a format suitable for further processing, such as PCM (Pulse Code Modulation), ensuring that the data format matches the requirements of subsequent modules (such as the audio quality assessment module); a buffering mechanism is implemented to handle discontinuities and time delays in the audio stream, and errors that occur during the extraction and preprocessing process are detected and handled, including file corruption, unsupported format, or incomplete data.
[0023] The audio quality evaluation module is used to evaluate the quality of the extracted audio stream; Set evaluation criteria and parameters, and configure the degree and focus of the evaluation based on user needs or presets. Evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation. Specifically: Audio signal analysis: Perform time domain and frequency domain signal analysis on the pre-processed audio data, and calculate the basic parameters of the signal, including amplitude, frequency and phase; Signal-to-noise ratio: Measure the ratio of useful signal to background noise in the audio signal, calculate the signal-to-noise ratio, and compare it with the preset threshold to evaluate the clarity of the audio; Distortion: Measure the distortion generated by the audio signal during amplification or processing, including harmonic distortion, intermodulation distortion, etc., and evaluate its impact on the sound quality; Frequency response: Analyze the response of the audio signal at different frequencies to ensure that the audio equipment or content provides balanced sound across the entire frequency range; Dynamic range: Measure the difference between the maximum and minimum audible levels of the audio signal to evaluate the dynamic expressiveness of the audio content; Transient response: Evaluate the audio signal's ability to respond to rapid changes, which is particularly important for percussion instruments and electronic music; Stereo and surround sound evaluation: For stereo or surround sound audio, evaluate the positioning and separation of sound and image to ensure that the distribution of sound in space is as expected; The evaluation parameters are visualized as follows: Audio waveform: displays the time domain waveform of the audio signal, which allows you to intuitively see the amplitude changes of the audio signal. The continuity, breakpoints and noise of the audio signal can be observed through the waveform. Spectrum: displays the frequency domain distribution of the audio signal, usually with frequency as the horizontal axis and amplitude or energy as the vertical axis. The spectrum can help identify the frequency components and possible frequency distortion in the audio. Signal-to-noise ratio (SNR) chart: displays the measurement results of the signal-to-noise ratio through a bar chart or line chart, and compares it with the preset threshold to show the changes in the signal-to-noise ratio in different time periods or different audio segments. Distortion curve: Harmonic distortion and intermodulation distortion can be displayed through a bar chart or a curve chart, showing the degree of distortion at different harmonics or frequencies, as well as the difference between the distortion and the standard. Frequency Response Curve: Draws a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve can clearly show the uniformity of the frequency response and whether there are peaks and valleys. Dynamic Range Bar Chart: Uses a bar chart to show the maximum and minimum audible levels of the audio signal, allowing you to intuitively see the dynamic range of the audio content. Transient Response Graph: Uses a time-amplitude graph to show the response of the audio signal to transient changes, allowing you to observe the rise time and decay characteristics of the signal. Stereo and Surround Sound Positioning Graph: Uses polar coordinate graphs or three-dimensional sound field graphs to show the position of each sound source in stereo or surround sound audio, allowing you to intuitively see the distribution and direction of the sound source in space. After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. The specific process is as follows: Convert the evaluation results of each evaluation parameter into quantitative values, including: Audio signal analysis: Quantitative indicators: amplitude (dB), frequency (Hz), phase (degrees or radians); Conversion method: Use frequency domain analysis methods such as Fourier transform to extract the frequency components of the signal and calculate its amplitude, frequency, and phase; Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio (dB); Conversion method: Calculate the ratio of signal power to noise power and then convert it to decibels (dB) using the formula: ,in is the signal power, is the noise power; distortion: quantitative indicators: total harmonic distortion plus noise (THD+N), intermodulation distortion (IMD); conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave; is the total harmonic distortion, Noise; Frequency response: Quantitative indicator: frequency response curve (dB); Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve; Dynamic range: Quantitative indicator: dynamic range (dB); Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level; Transient response: Quantitative indicators: rise time (trise), overshoot (overshoot), steady-state error; Conversion method: analyze the behavior of the audio signal during transient changes, such as measuring the time required for the signal to go from 10% to 90% of the maximum value (rise time); Stereo and surround sound evaluation: Quantitative indicators: sound image positioning accuracy, sound field width, sound image separation; Conversion method: use stereo or surround sound test signals (such as pink noise, sine wave sweep, etc.) to measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound image positioning and separation; Then, weights are assigned to each parameter based on its importance. The weights are determined based on experience, user needs, or industry standards, where the weight assignment reflects the degree of influence of each parameter on the overall audio quality. To ensure that the quantized values of different parameters are comparable, they need to be normalized so that they are at the same magnitude. Normalization can be linear or a nonlinear transformation based on a certain function. The quantized value of each parameter is multiplied by its weight, and then all products are summed to obtain a comprehensive score, which is recorded as the evaluation score.
[0024] Then conduct a subjective listening test and use the subjective parameters of the subjective evaluation to correct the evaluation score. Specifically, through the listening test, collect users' subjective evaluation of the sound quality and analyze the subjective parameters, including: Clarity evaluation: The user's perception of details in the audio, whether each note or dialogue can be heard clearly, and based on this standard, the user's ratings are collected, the average is calculated and recorded as the clarity rating value, and this clarity rating value is used as a standard to measure the user's evaluation of the clarity of the audio; Tone evaluation: The sound quality characteristics of the audio, including the color and texture of the instrument or sound, and based on this standard, the user's ratings are collected, the average is calculated and recorded as the color rating value, and this color rating value is used as a standard to measure the user's evaluation of the tone of the audio; Balance evaluation: The balance between different frequency components, whether the audio maintains a balance between high and low frequencies A good balance is achieved, and user ratings are collected based on this standard. The average value is calculated and recorded as the balance value, and this balance value is used as a standard to measure the user's evaluation of the balance of the audio; immersion evaluation: the degree of immersion of the user in the listening process, whether the audio can make the user feel immersive, and the user ratings are collected based on this standard. The average value is calculated and recorded as the immersion judgment value, and this immersion judgment value is used as a standard to measure the user's evaluation of the balance of the audio; the obtained clarity evaluation value, color evaluation value, balance value and immersion judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are entered into the following formula: To get the final score ZPP, where PCF is the evaluation score, The preset weight coefficients are respectively the clarity evaluation value, the color evaluation value, the balance value and the sinking evaluation value, and the final score is used as the standard for measuring the audio quality evaluation; the final score is compared with the preset final evaluation threshold score. When the final score is lower than the preset final evaluation threshold score, it is judged that the audio quality does not meet the standard, and an adjustment signaling is sent to the adaptive adjustment module to improve the overall quality of the audio.
[0025] The adaptive adjustment module is used to automatically adjust the parameters of the audio stream according to the evaluation results; The audio quality evaluation module receives the evaluation results of individual audio parameters and the final score of the comprehensive evaluation. When an adjustment signal is received, the automatic adjustment process is triggered, including: First, the evaluation results of each single parameter analyzed in the audio quality assessment module are analyzed, and the percentage of each single parameter below the preset standard is calculated to determine the specific cause of poor audio quality. Based on the analysis results, the audio parameters that need to be adjusted are determined, including gain, equalization, compression, expansion, etc., and the corresponding audio processing algorithms are applied to make real-time adjustments to the audio stream. For example, if the signal-to-noise ratio is low, the strength of the noise reduction algorithm may need to be increased; if the dynamic range is small, the compressor threshold may need to be adjusted. During the adjustment process, the quality changes of the audio stream are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. Based on the evaluation results, if the audio quality still does not meet the standard, continue to adjust or optimize the adjustment strategy; record the detailed information of each adjustment, including the adjusted parameters and the adjusted effect; based on historical adjustment data, continuously optimize the adaptive adjustment algorithm to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, notify the user through the user interaction module and provide a final audio quality evaluation report.
[0026] The user interaction module is used to provide a user interface, allowing users to monitor the synchronization and evaluation process; Determine user needs, including the type of information they wish to monitor, ease of operation, and user-friendly interface. Design an intuitive and easy-to-use user interface to ensure users can easily access and monitor various system functions. Plan the functions of the user interaction module, including real-time data display, evaluation result feedback, and parameter adjustment. Develop real-time data display capabilities, using charts and dashboards to display the synchronization status and evaluation parameters of audio streams, and visualize audio quality evaluation results. Develop parameter adjustment tools to allow users to manually adjust audio stream parameters such as gain and equalization, provide clear user operation instructions and help documentation to help users understand how to use the interactive module; implement a user feedback collection mechanism to allow users to report problems or make improvement suggestions, set different user permission levels to ensure that users can only access and operate the functions they are authorized to, and provide interactive controls such as sliders, buttons, and drop-down menus to enable users to control the audio processing flow; allow users to view the evaluation history and adjustment history of the audio stream, including timestamps and operation details, test the user interaction module, collect user feedback, and optimize based on the feedback, integrate the user interaction module with other system modules, and deploy it in the actual environment.
[0027] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. A multimedia multi-audio test system, It is characterized in that include: Audio acquisition module, used to automatically extract audio data from multimedia content and perform preprocessing; The audio quality evaluation module is used to evaluate the quality of the extracted audio stream. The specific process is as follows: Set evaluation criteria and parameters, and configure the degree and focus of evaluation according to user needs or presets. Evaluation parameters include audio signal analysis, signal-to-noise ratio, distortion, frequency response, dynamic range, transient response, and stereo and surround sound evaluation; Visualize the evaluation parameters; After completing the single parameter evaluation, a comprehensive evaluation is conducted on the evaluation parameters. Specifically, the evaluation results of each evaluation parameter are converted into quantitative values, and then weights are assigned according to the importance of each parameter. After normalization, the quantitative value of each parameter is multiplied by its weight, and then all products are summed to obtain a comprehensive score, which is recorded as the evaluation score. Then, a subjective listening evaluation is performed, and the evaluation score obtained is corrected by the subjective parameters of the subjective evaluation to obtain a final score, and the final score obtained is compared with a preset final evaluation threshold score. When the final score is lower than the preset final evaluation threshold score, it is determined that the audio quality does not meet the standard, and an adjustment signaling is sent to the adaptive adjustment module to improve the overall audio quality; Adaptive adjustment module, used to automatically adjust the parameters of the audio stream according to the evaluation results to optimize the audio quality; The user interaction module provides a user interface that allows users to monitor the synchronization and evaluation process.
2. A multimedia multi-audio test system according to claim 1, It is characterized in that The process of the audio quality evaluation module evaluating different single parameters is as follows: Audio signal analysis: Perform signal analysis in the time domain and frequency domain on the pre-processed audio data, and calculate the basic parameters of the signal, including amplitude, frequency and phase; Signal-to-noise ratio: measures the ratio of useful signal to background noise in the audio signal, calculates the signal-to-noise ratio, and compares it with the preset threshold to evaluate the clarity of the audio; Distortion: Measures the distortion of audio signals during amplification or processing, including harmonic distortion and intermodulation distortion, and evaluates the impact on sound quality; Frequency response: Analyzes the response of audio signals at different frequencies to ensure that the audio device or content provides balanced sound across the entire frequency range; Dynamic range: measures the difference between the maximum and minimum audible levels of an audio signal, assessing the dynamic expressiveness of audio content; Transient response: evaluates the audio signal's ability to respond to rapid changes; Stereo and Surround Sound Evaluation: For stereo or surround sound audio, evaluate the positioning and separation of sound images to ensure that the distribution of sound in the space is as expected.
3. A multimedia multi-audio test system according to claim 1, It is characterized in that The specific operation steps of the audio quality evaluation module to visualize the evaluation parameters are as follows: The specific visualization method is as follows: Audio waveform graph: Displays the time domain waveform of the audio signal, allowing you to intuitively see the amplitude changes of the audio signal. You can also observe the continuity, breakpoints, and noise of the audio signal through the waveform graph. Spectrogram: Displays the frequency domain distribution of an audio signal, with frequency as the horizontal axis and amplitude or energy as the vertical axis. The spectrogram is used to identify the frequency components and possible frequency distortion in the audio. Signal-to-noise ratio chart: Displays the measurement results of the signal-to-noise ratio through a bar chart or line chart, compares it with the preset threshold, and displays the changes in the signal-to-noise ratio in different time periods or different audio segments; Distortion curve: Harmonic distortion and intermodulation distortion are displayed through bar graphs or curves, showing the degree of distortion at different harmonics or frequencies, and the comparison with the standard or threshold; Frequency response curve: Draw a frequency response curve to show the gain or attenuation of audio equipment or content at different frequencies. The curve graph is used to show the uniformity of frequency response and whether there are peaks and valleys; Dynamic range bar graph: Use a bar graph to display the maximum and minimum audible levels of the audio signal, which is used to intuitively see the dynamic range of the audio content; Transient response graph: Displays the response of the audio signal to transient changes through a time-amplitude graph, which is used to observe the rise time and decay characteristics of the signal; Stereo and Surround Positioning Diagrams: Use polar coordinates or three-dimensional sound field diagrams to show the location of various sound sources in stereo or surround audio, allowing you to see the distribution and direction of sound sources in space.
4. A multimedia multi-audio test system according to claim 1, It is characterized in that The specific operation steps of the audio quality evaluation module to convert the evaluation result of each evaluation parameter into a quantized value are as follows: Audio signal analysis: Quantitative indicators: amplitude, frequency, phase; Conversion method: Use Fourier transform frequency domain analysis method to extract the frequency components of the signal and calculate the amplitude, frequency and phase; Signal-to-noise ratio: Quantitative indicator: signal-to-noise ratio; Conversion method: Calculate the ratio of signal power to noise power, and then convert it to decibels using the formula: ,in is the signal power, is the noise power; Distortion: Quantitative indicators: total harmonic distortion plus noise, intermodulation distortion; Conversion method: calculate the ratio of the harmonic component of the signal to the fundamental wave, convert it to percentage or decibel, through the formula: ,in It is The amplitude of the order harmonic, is the amplitude of the fundamental wave; is the total harmonic distortion, for noise; Frequency response: Quantitative indicator: frequency response curve; Conversion method: measure the output amplitude of the audio device at different frequencies, compare it with the input amplitude, and draw a frequency response curve; Dynamic range: Quantitative indicator: dynamic range; Conversion method: calculate the difference between the maximum undistorted output level of the audio signal and the noise level; Transient response: Quantitative indicators: rise time, overshoot, steady-state error; Conversion method: analyze the behavior of the audio signal during transient changes and measure the time required for the signal to go from 10% to 90% of the maximum value; Stereo and surround sound evaluation: Quantitative indicators: sound and image positioning accuracy, sound field width, sound and image separation; Conversion method: Use stereo or surround sound test signals to measure the phase difference, time difference and amplitude ratio between different channels to evaluate sound and image positioning and separation.
5. A multimedia multi-audio test system according to claim 1, It is characterized in that The specific operation steps of the audio quality evaluation module to correct the evaluation score obtained by using the subjective parameters of the subjective evaluation are as follows: Through listening tests, we collect users’ subjective evaluations of sound quality and analyze subjective parameters, including: Clarity evaluation: The user's perception of details in the audio, whether each note or dialogue can be heard clearly, and the user's ratings are collected based on this standard, and the average is calculated as the clarity evaluation value, which is used as a standard to measure the user's evaluation of audio clarity; Tone evaluation: The sound quality characteristics of audio, including the color and texture of musical instruments or sounds, are collected based on this standard, and the average is calculated as the color evaluation value, which is used as a standard to measure users' evaluation of the audio's tone quality; Balance evaluation: the balance between different frequency components, whether the audio maintains a balance between high and low frequencies, and collect user ratings based on this standard, calculate the average value and record it as the balance value, and use this balance value as a standard to measure users' evaluation of the balance of the audio; Immersion evaluation: The degree of immersion of the user in the listening process, whether the audio can make the user feel immersive, and collect user ratings based on this standard, calculate the average value and record it as the immersion judgment value, and use this immersion judgment value as a standard to measure the user's evaluation of the balance of the audio; Then the obtained clarity evaluation value, color evaluation value, balance value and sedimentation judgment value are calibrated as qf, sf, ph and cp respectively, and after normalization, they are entered into the following formula: To get the final score ZPP, where PCF is the evaluation score, They are the preset weight coefficients for clarity, color, balance and sedimentation respectively, and the final score is used as the standard for measuring audio quality evaluation.
6. A multimedia multi-audio test system according to claim 1, It is characterized in that The specific operation steps of the adaptive adjustment module to automatically adjust the parameters of the audio stream according to the evaluation results are as follows: The audio quality evaluation module receives the evaluation results of the audio single parameters and the final score of the comprehensive evaluation. When the adjustment signaling is received, the automatic adjustment process is triggered, including: First, the evaluation results of each single parameter analyzed in the audio quality evaluation module are analyzed, and the percentage of each single parameter below the preset standard is calculated, so as to determine the specific reason why the audio quality does not meet the standard; Based on the analysis results, determine the audio parameters that need to be adjusted, including gain, equalization, compression, and expansion, and apply the corresponding audio processing algorithm to adjust the audio stream in real time; During the adjustment process, the quality changes of the audio stream are monitored in real time, and the audio quality is repeatedly comprehensively evaluated. According to the evaluation results, if the analyzed audio quality still does not meet the standard, the adjustment strategy is continued or optimized; Record the specific information of each adjustment, including the adjusted parameters and the adjusted effects. Based on the historical adjustment data, continuously optimize the adaptive adjustment algorithm to improve the accuracy and efficiency of the adjustment. After the adjustment is completed, notify the user through the user interaction module and provide the final audio quality evaluation report.
7. A multimedia multi-audio test system according to claim 1, It is characterized in that The specific operation steps of the audio acquisition module to automatically extract audio data from multimedia content and perform preprocessing are as follows: Identify the formats of multimedia files, including MP4, AVI, MOV, MKV, and extract audio stream information from the files, including encoding format, sampling rate, bit rate, and number of channels; Separate audio streams from multimedia files using decoding and decapsulation techniques; The resulting audio stream is then preprocessed, including: noise reduction: applying algorithms to reduce background noise; gain control: adjusting the volume to a preset level; equalization: using an equalizer to optimize frequency response; sampling rate conversion: unifying the sampling rate to a standard value; Convert audio data into PCM format for subsequent processing needs, establish a buffer mechanism to ensure the continuity of audio streams and delay in processing time; Detect and resolve errors during extraction and preprocessing, including file corruption or unsupported formats.
8. A multimedia multi-audio test system according to claim 7, It is characterized in that The specific operation steps of the audio acquisition module using decoding and decapsulation technology to separate the audio stream from the multimedia file are as follows: Open multimedia files, read file header information, locate audio data streams, and use tools such as FFmpeg or GStreamer to parse multimedia containers of different formats; Determine the encoding format, type, and language attributes of the audio stream, select a decoder that matches the encoding format of the audio stream, and convert the compressed data into the original audio signal in PCM format; For files containing multiple audio streams, repeat the decapsulation and decoding steps and ensure that the streams are synchronized on the timeline; Manage the resources used in the decapsulation and decoding process, and obtain an asset valuation by performing a comprehensive evaluation through monitoring usage parameters. Compare the obtained asset valuation with the preset asset valuation threshold. When the asset valuation is greater than the preset asset valuation threshold, it is determined that the resources used in the decapsulation and decoding process exceed the system load, and an early warning is issued. Optimization is performed by reducing the audio streams processed simultaneously to avoid system overload.
9. A multimedia multi-audio test system according to claim 8, It is characterized in that The specific process of the audio acquisition module performing comprehensive evaluation by monitoring the usage parameters to obtain the asset valuation is as follows: The parameters used include: CPU usage: measures the processor usage during the decapsulation and decoding process and is expressed as a percentage; Memory occupancy rate: monitors the ratio of physical memory consumed during decapsulation and decoding to the total amount of original physical memory; Disk I / O: Evaluates the disk input / output operation rate when reading multimedia files and writing decoded data, using the formula: To obtain the magnetic judgment value CP, where and are the actual input operation speed and output operation speed respectively, and The preset standard input operation speed and output operation speed are respectively used; and the obtained magnetic judgment value CP is used as a standard for measuring the input / output operation rate of the disk when reading multimedia files and writing decoded data; GPU usage: for cases where hardware accelerated decoding is used; Then, the obtained CPU usage, memory occupancy, magnetic judgment value and GPU usage are normalized respectively, and the sum of CPU usage and GPU usage is used as the radius to establish the bottom circle. A cone model is established with a high memory occupancy. The vertex of the cone model is selected as the center of the sphere, and the product of the magnetic judgment value and the correction index a is used as the diameter to establish a spherical model. The surface area of the alien body formed by the fusion of the cone model and the spherical model is analyzed and recorded as the asset valuation. This asset valuation is used as a standard to measure the resources used in the decapsulation and decoding process.
Citation Information
Patent Citations
Audio processing method and device
CN115966198A
Sound effect adjusting method and device, electronic equipment and storage medium
CN117479076A
Audio test analysis method
CN117746907A
Sound environment adaptive USB audio optimization method and system
CN118230767A
Intelligent tuning audio output method and system based on MEMS
CN118354257A
Cited By
Multifunctional partitioned physiotherapy cabin with five-element music
CN119157716A
Audio and video acquisition equipment data management system based on artificial intelligence
CN119513355A
An artificial intelligence-based data management system for audio and video acquisition equipment
CN119513355B
Sound dynamic gain equalization method and system
CN119545247A
An audio dynamic gain equalization method and system
CN119545247B
