Audio automatic detection system and method based on multi-dimensional calibration

Abstract

The invention discloses an audio automatic detection system and method based on multi-dimensional calibration, and the method comprises the steps: arranging recording equipment through a standardized recording position rule, carrying out the calibration of space parameters through a range finder, an angle ruler and other tools, and guaranteeing the consistency of collected signals; a multi-channel acquisition architecture is constructed to support synchronous recording and detection of multiple devices; by comparing multi-dimensional acoustic feature extraction of standard audio and recorded audio and adopting a two-stage time alignment mechanism, similarity is accurately calculated to identify abnormities such as sound break and sound break, and a detection result is formed. And finally, analyzing and generating a report based on a detection result, and performing visual display and automatic pushing. The accuracy, efficiency and reliability of audio detection are remarkably improved, and the method is particularly suitable for a batch automatic detection scene before consumer electronics leave a factory.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing and automated detection, and in particular to an automated audio detection system and method based on multi-dimensional calibration. Background Technology

[0002] Current television audio testing methods rely on traditional manual methods that require real-time monitoring of audio output. Testing a single device for extended periods (e.g., 12 hours) is not only costly but also prone to causing operator fatigue. Simultaneous testing with multiple devices necessitates frequent switching between monitoring targets, further increasing the risk of missed detections. This method relies on subjective human judgment, which is affected by the operator's hearing sensitivity and fatigue levels. It is ineffective at identifying subtle anomalies such as millisecond-level silences and slight distortion below -60dB, and cannot quantify the specific time and frequency of anomalies. It struggles to identify momentary sound quality issues like interruptions or distortions. Audio comparison analysis lacks a timeline alignment mechanism, resulting in significant errors when comparing 12-hour audio clips, making it difficult to accurately pinpoint anomalies. Test results require manual paper recording or Excel compilation, and report generation depends on manual entry of anomaly data, which is inefficient and prone to errors and omissions due to negligence. Furthermore, it lacks the capability for remote monitoring and real-time result delivery.

[0003] Existing audio comparison technologies suffer from low detection efficiency, lack of standardized recording location definition, poor scene adaptability, insufficient audio source compatibility, low degree of process automation, and limited anomaly identification. Therefore, there is an urgent need for an audio detection solution that features standardized recording specifications, multi-scene adaptability, the ability to analyze one-to-many recorded audio (with the function of recording multiple products simultaneously), parallel analysis of multiple audio, multi-dimensional comparison capabilities, multiple detection modes, high degree of automation, and complex anomaly identification capabilities, in order to solve the pain points of efficiency and accuracy in batch detection scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide an automated audio detection system and method based on multi-dimensional calibration, which achieves accurate audio detection of devices such as televisions through standardized recording location definition, multi-scene parameter calibration, core algorithm optimization and full-process automation design.

[0005] The technical solution adopted in this invention is:

[0006] The automated audio detection method based on multi-dimensional calibration includes the following steps:

[0007] System setup and calibration: A multi-channel acquisition architecture is constructed using a uniform model of recording equipment, with each device under test corresponding to one independent channel; the recording equipment is arranged according to standardized specifications, a rangefinder is used to calibrate the distance between the recording equipment and the device under test, an angle gauge is used to calibrate the installation angle, and a noise tester is used to detect ambient noise to confirm that it meets the testing environment requirements; the device under test is loaded with a uniform test sound source and a uniform volume is set so that the recorded audio can be obtained;

[0008] Audio preprocessing: When importing audio, a standard audio and recorded audio are loaded at a uniform fixed sampling rate, and the audio track of the recorded audio is extracted; based on the ambient noise, spectral subtraction is used to eliminate ambient noise (such as music noise in the interference environment) of the recorded audio to improve signal purity; amplitude normalization is performed on recorded audio from different recording devices, distances and angles to eliminate signal differences caused by factors other than the device itself; audio segments are extracted in intervals of one second to lay the foundation for fine-grained analysis.

[0009] Specifically, assuming the current peak value of the audio is 0.5 and the target peak value is 0.9, then the normalization coefficient = 0.9 / 0.5 = 1.8; multiplying all sample values ​​by 1.8, the new peak value = 0.5 × 1.8 = 0.9; the corresponding formula is:

[0010] ;

[0011] Where i represents the index of the audio sample; This is the target peak value (default 0.9). It is the maximum absolute value of the audio signal;

[0012] Audio feature extraction and time alignment: The first processor extracts audio alignment processing, first calculates the cross-correlation function of the standard audio and the recorded audio to obtain the time offset, and adjusts the audio length by padding with zeros or truncating to achieve coarse alignment between the standard audio and the recorded audio. Then, the time warping algorithm of the first processor is used to achieve fine alignment between the standard audio and the recorded audio by elastically stretching or compressing the recorded audio.

[0013] Specifically, `shift` (offset) calculates the time offset of the recorded segment relative to the standard segment by finding the location of the maximum cross-correlation value. A positive `shift` indicates that the recorded segment is "lagging" behind the standard segment (needing to be moved forward, padded with zeros). A negative `shift` indicates that the recorded segment is "ahead" of the standard segment (needing to be moved backward, truncated). The correlation function formula is as follows:

[0014] time_shift_ms=(np.argmax(corr_window)-(max_shift_ms×sr / 1000)) / sr×1000;

[0015] Where ime_shift_ms is the final time offset; np.argmax(corr_window) is the sample index corresponding to the offset position where the cross-correlation value is the largest (the signal is most similar); max_shift_ms is the maximum search offset time; and Sr is the sampling rate.

[0016] Multi-dimensional similarity calculation: The second processor extracts at least 13-dimensional features of the audio, calculates the number of audio sampling points per second, and extracts standard audio segments and aligned recorded audio segments from the standard audio and recorded audio respectively according to the minimum audio length; the second processor extracts the coefficient features of each audio segment and performs standardization processing to eliminate interference from signal strength differences, and then calculates the audio similarity between the standard audio and the recorded audio.

[0017] The 13-dimensional feature consists of one logarithmic energy coefficient and 12 Mel frequency cepstral coefficients. The logarithmic energy coefficient corresponds to the 0th cepstral coefficient after DCT transformation, reflecting the energy level of the audio frame. The 12 Mel frequency cepstral coefficients correspond to the 1st to 12th cepstral coefficients after DCT transformation. They are the core coefficients obtained from the Mel-filtered spectrum through logarithmic compression and DCT transformation, reflecting the frequency distribution characteristics of the audio.

[0018] Detection result processing and output: The audio similarity is compared with the preset similarity threshold to generate the detection result. The detection result records the number of detections, the number of anomalies, the time of anomaly occurrence, audio waveform graph and similarity line graph data.

[0019] Furthermore, the uniform volume setting of the device under test is 50% of the maximum volume.

[0020] Furthermore, the standardized setup of recording equipment includes:

[0021] When testing a single device under test, place the recording device in the center of the front of the device under test, maintain a preset distance between the recording device and the device under test, and install the recording device at an angle perpendicular to the front of the device under test.

[0022] When testing multiple devices under test, the recording device is placed at the geometric center of all devices under test and directly below the corresponding device under test, and the positioning is calibrated by a dedicated audio file containing device identification marks.

[0023] Furthermore, the recording equipment is arranged according to standardized specifications, following these principles: consistency principle: the recording positions of all recording equipment to be tested are completely uniform and standardized to ensure the consistency of the spatial reference of the recorded audio; sound source pointing principle: the recording equipment is pointed precisely at the core area of ​​the device's audio output; isolation principle: the recording position is far away from the edge of the device, reflective surfaces and interference sources, in accordance with the requirements for phase interference avoidance; traceability principle: the recording position of each device is traceable through "reference mark + coordinate record".

[0024] Furthermore, the testing environment includes both quiet and interfering environments. The noise floor in the quiet environment is 36.8 dBA, and the safe distance between the recording device and the device under test is set to 10 cm according to the evaluation standard.

[0025] Furthermore, the recording device supports different types of recording devices. By adjusting the recording level of the recording software, the recording differences between different devices can be reduced, thus avoiding the impact of the recording device on the similarity.

[0026] Furthermore, for high-level loudspeaker equipment, the features can be dynamically expanded to 16 dimensions, that is, adding spectral bandwidth, spectral flux and zero-crossing rate features on the basis of 13-dimensional features.

[0027] Furthermore, the detection results are obtained through parallel analysis of audio recorded by multiple devices in the detection result processing and output steps.

[0028] Furthermore, the test results are visualized, including the test status per second, a comparison graph of standard audio and recorded audio waveforms, a similarity line graph, and an anomaly distribution heatmap. The test results are converted into an HTML report containing basic information, quantitative results, and detailed data, and the report is pushed through the communication unit. The report can be filtered by all results, qualified results, or unqualified results, and unqualified areas and anomaly types are marked. Anomaly types include broken sound, distorted sound, and no sound.

[0029] Furthermore, the present invention also discloses an automated audio detection system based on multi-dimensional calibration. Employing the aforementioned automated audio detection method based on multi-dimensional calibration, the system includes:

[0030] The recording calibration module includes a recording device, a rangefinder, an angle gauge, and a noise tester. It is used to calibrate the placement of the recording device according to standardized recording specifications to obtain consistent and effective recorded audio. The recording device uses a uniform model, with multiple devices forming a multi-channel acquisition architecture, and each device under test corresponding to one independent channel. The rangefinder calibrates the straight-line distance between the recording device and the speaker of the device under test. The angle gauge calibrates the installation angle of the recording device. The noise tester detects ambient noise to confirm that the environment meets preset testing requirements.

[0031] The audio preprocessing module is used to load standard audio and recorded audio at a uniform fixed sampling rate, extract the audio track of the recorded audio, eliminate environmental noise of the recorded audio using spectral subtraction based on environmental noise, normalize the amplitude of recorded audio from different recording devices, distances and angles, and extract audio segments in intervals of one second.

[0032] Specifically, assuming the current peak value of the audio is 0.5 and the target peak value is 0.9, then the normalization coefficient = 0.9 / 0.5 = 1.8; multiplying all sample values ​​by 1.8, the new peak value = 0.5 × 1.8 = 0.9; the corresponding formula is:

[0033] ;

[0034] Where i represents the index of the audio sample; This is the target peak value (default 0.9). It is the maximum absolute value of the audio signal;

[0035] The audio feature extraction and time alignment module is used to extract audio segments per second from standard audio and recorded audio respectively. First, the cross-correlation function of standard audio and recorded audio is calculated to obtain the time offset. The audio length is adjusted by padding with zeros or truncating to achieve coarse alignment between standard audio and recorded audio. Then, the time warping algorithm is used to elastically stretch or compress the recorded audio to achieve fine alignment between standard audio and recorded audio.

[0036] The multi-dimensional similarity calculation module extracts 13-dimensional audio features through the second processor to calculate the number of audio sampling points per second; according to the minimum audio length, it extracts standard audio segments and aligned recorded audio segments from the standard audio and the recorded audio respectively; it extracts the coefficient features of each audio segment and performs standardization processing to eliminate interference from signal strength differences, and then calculates the audio similarity between the standard audio and the recorded audio.

[0037] The detection result processing and output module compares the audio similarity with a preset similarity threshold to generate detection results. The detection results record the number of detections, the number of anomalies, the time of anomaly occurrence, audio waveform graphs, and similarity line graphs.

[0038] Furthermore, the detection result processing and output module visualizes the detection results, including the detection status per second, a comparison graph of standard audio and recorded audio waveforms, a similarity line graph, and an anomaly distribution heatmap.

[0039] Furthermore, the test result processing and output module converts the test results into an HTML format report containing basic information, quantitative results and detailed data, and pushes the report through the communication unit. The report supports filtering by all results, qualified results or unqualified results, and marks unqualified areas and abnormality types. Abnormality types include broken sound, cracked sound and no sound.

[0040] The present invention, employing the above technical solutions, has the following beneficial effects: 1) Through standardized recording calibration, multi-dimensional feature extraction, and dual-stage time alignment, it accurately identifies subtle anomalies such as millisecond-level dropouts and slight distortions, significantly improving detection accuracy; 2) The multi-channel architecture supports parallel detection by multiple devices, automating the entire process to replace manual listening, significantly improving detection efficiency and adapting to batch production needs; 3) Customized standardized specifications enable differentiated recording specifications for single / multiple devices to adapt to various scenarios, compatible with different recording devices and quiet / interference environments, demonstrating strong compatibility; 4) Detection data is automatically stored, reports are automatically generated, and multi-dimensional charts intuitively present the results, avoiding human error and ensuring objectivity; 5) Fault tracing and remote monitoring are convenient, with anomaly information recorded in seconds and playback supported, and reports automatically pushed via email, assisting in quality control.

[0041] This invention achieves accurate audio detection for devices such as televisions by defining standardized recording positions, calibrating parameters for multiple recording scenarios, and optimizing audio processing algorithms. Attached Figure Description

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0043] Figure 1 This is a schematic diagram illustrating the principle and flow of the automated audio detection method based on multi-dimensional calibration of the present invention.

[0044] Figure 2 This is a schematic diagram of the connection relationship for automated audio detection based on multi-dimensional calibration according to the present invention;

[0045] Figure 3 This is a comparative schematic diagram showing the arrangement of the recording device when it collects data from a single device under test versus multiple devices under test, according to the present invention.

[0046] Figure 4 This is a comparative diagram showing the setup of different recording environments for this invention;

[0047] Figure 5 A schematic diagram illustrating the principle of constructing a multi-channel acquisition architecture for the recording device of this invention;

[0048] Figure 6 A multi-dimensional chart diagram illustrating the visualization of the output results of this invention;

[0049] Figure 7 This is a schematic diagram illustrating the working principle of the first processor of the present invention;

[0050] Figure 8 This is a schematic diagram illustrating the working principle of the second processor of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0052] like Figures 1 to 8 As shown in the figure, the present invention discloses an automated audio detection method based on multi-dimensional calibration, which includes the following steps:

[0053] System setup and calibration: A multi-channel acquisition architecture is constructed using a uniform model of recording equipment, with each device under test corresponding to one independent channel; the recording equipment is arranged according to standardized specifications, a rangefinder is used to calibrate the distance between the recording equipment and the device under test, an angle gauge is used to calibrate the installation angle, and a noise tester is used to detect ambient noise to confirm that it meets the testing environment requirements; the device under test is loaded with a uniform test sound source and a uniform volume is set so that the recorded audio can be obtained;

[0054] Specifically, the hardware in the system hardware setup includes: 1) Recording equipment: supporting audio recording devices; 2) Auxiliary tools: rangefinder, angle ruler, noise tester; 3) Test computer: configured with ≥ Intel i5 processor, 8GB memory, and pre-installed environment;

[0055] The setup steps are as follows: 1) Arrange the recording equipment according to the recording location specifications: such as Figure 3 As shown, for a single device, place it in the center of the front; for multiple devices, place it in the center of the bottom. Calibrate the distance using a rangefinder and the installation angle using an angle gauge. 2) Use a noise meter to test the ambient noise level and confirm that it meets the requirements. Figure 4 The requirements for a quiet or noisy environment are specified. The distance between the interference source in the noisy environment and the device under test is set according to the standard, as shown in Table 2. Different sizes of the test equipment correspond to different recording distances. Connect the recording equipment to the test computer to ensure stable data transmission. 3) As Figure 5 As shown, a "multi-channel acquisition" architecture is constructed to realize different inputs from multiple recording devices and recording environments with different audio frequencies; each device under test corresponds to one independent channel to avoid signal interference between channels; all recording devices use the same model to ensure consistency in sensitivity and frequency response. 4) Load a unified detection sound source to the device under test and set the volume to 50% of the maximum volume. Specifically, the recording scene calibration rules are shown in Table 1.

[0056] Table 1 - Multi-Scene Recording Calibration Rules Table

[0057]

[0058] Table 2 - Recording Distances Corresponding to Different Sizes

[0059]

[0060] Furthermore, the recording equipment is arranged according to standardized specifications, following these principles: consistency principle: the recording positions of all recording equipment to be tested are completely uniform and standardized to ensure the consistency of the spatial reference of the recorded audio; sound source pointing principle: the recording equipment is pointed precisely at the core area of ​​the device's audio output; isolation principle: the recording position is far away from the edge of the device, reflective surfaces and interference sources, in accordance with the requirements for phase interference avoidance; traceability principle: the recording position of each device is traceable through "reference mark + coordinate record".

[0061] Specifically, the standardized definition of microphone recording position is shown in Table 3.

[0062] Table 3 - Standardized Definitions of Microphone Recording Position

[0063]

[0064] Audio preprocessing: When importing audio, a standard audio and recorded audio are loaded at a uniform fixed sampling rate, and the audio track of the recorded audio is extracted; based on the ambient noise, spectral subtraction is used to eliminate ambient noise (such as music noise in the interference environment) of the recorded audio to improve signal purity; amplitude normalization is performed on recorded audio from different recording devices, distances and angles to eliminate signal differences caused by factors other than the device itself; audio segments are extracted in intervals of one second to lay the foundation for fine-grained analysis.

[0065] Specifically, assuming the current peak value of the audio is 0.5 and the target peak value is 0.9, then the normalization coefficient = 0.9 / 0.5 = 1.8; multiplying all sample values ​​by 1.8, the new peak value = 0.5 × 1.8 = 0.9; the corresponding formula is:

[0066] ;

[0067] Where i represents the index of the audio sample; This is the target peak value (default 0.9). It is the maximum absolute value of the audio signal;

[0068] Audio feature extraction and time alignment: The first processor extracts audio alignment data. First, the cross-correlation function between the standard audio and the recorded audio is calculated to obtain the time offset shift. The audio length is adjusted by padding with zeros (shift>0) or truncating (shift<0) to ensure consistent duration and achieve coarse alignment between the standard audio and the recorded audio. Then, the time warping algorithm of the first processor is used to find the optimal alignment path with the standard audio by elastically stretching or compressing the recorded audio, thus solving the similarity calculation deviation caused by time axis distortion and achieving fine alignment between the standard audio and the recorded audio.

[0069] Specifically, such as Figure 7As shown, the first processor mainly performs audio alignment processing on standard audio and recorded audio, calculates the cross-correlation time offset, applies offset, trimming, and zero-padding to match the length, ensuring that the time dimension of the two audio segments matches, and calculates the number of samples per second based on the sampling rate after alignment, and processes the audio second by second in a loop until completion;

[0070] The `shift` function calculates the time offset of a recorded segment relative to a standard segment by finding the location of the maximum cross-correlation value. A positive `shift` indicates that the recorded segment is "lagging" behind the standard segment (needing to be moved forward, padded with zeros). A negative `shift` indicates that the recorded segment is "ahead" of the standard segment (needing to be moved backward, truncated). The correlation function formula is as follows:

[0071] time_shift_ms=(np.argmax(corr_window)-(max_shift_ms×sr / 1000)) / sr×1000;

[0072] Where ime_shift_ms is the final time offset; np.argmax(corr_window) is the sample index corresponding to the offset position where the cross-correlation value is the largest (the signal is most similar); max_shift_ms is the maximum search offset time; and Sr is the sampling rate.

[0073] Multi-dimensional similarity calculation: The second processor extracts 13-dimensional audio features to simulate non-linear human auditory perception and adapt to feature capture of different audio types; for high-level speaker devices, it can be dynamically expanded to 16-dimensional features to improve fine-grained difference recognition capabilities; the number of audio sampling points per second is calculated, and standard audio segments and aligned recorded audio segments are extracted from standard audio and recorded audio respectively according to the minimum audio length; the second processor extracts the coefficient features of each audio segment and performs standardization processing to eliminate signal strength difference interference, and then calculates the audio similarity between standard audio and recorded audio;

[0074] Specifically, as shown in Figure 8, the second processor extracts audio segments second by second, extracts human auditory perception feature values, calculates the average vector cosine similarity, extracts spectral feature values, performs normalization processing after alignment, and then calculates spectral similarity and audio energy; and calculates the total duration of the audio based on the longest duration of the standard audio and the recorded audio.

[0075] The number of audio sampling points per second is calculated, and the loop variable i ranges from 0 to the minimum audio length (in seconds). The start and end indices of each segment are determined to extract standard audio segments and aligned recorded audio segments. The coefficient features of each audio segment are extracted by the second processor to capture key information in the audio frequency domain and cepstral domain. The coefficient features are then processed by the second processor to standardize (mean is 0, standard deviation is 1) to eliminate the interference of signal strength differences on similarity calculation. Finally, the audio similarity is calculated.

[0076] Specifically, the coefficient features of each audio segment include human auditory perception, spectrum, and energy. In particular, audio similarity is measured by a comprehensive average of these three dimensions: human auditory perception, spectrum, and energy.

[0077] Human auditory perception is calculated by extracting auditory feature values ​​from audio segments one second at a time and then calculating the average vector cosine similarity.

[0078] ;

[0079] ;

[0080] Where A and B are the auditory characteristic values ​​of the left and right ears of a person, respectively; Cosine similarity; This represents the normalized similarity.

[0081] The spectrum similarity is calculated by extracting spectral feature values, aligning them, normalizing them, and then performing spectral similarity calculations.

[0082] ;

[0083] in, For vectors The average value; Let be the average value of vector b; n is the length of vectors a and b. This refers to the i-th observation of the first variable (e.g., the i-th sample data of variable a). For the i-th observation of the second variable (and) The i-th sample data corresponding to variable b).

[0084] To calculate audio energy, the energy of a standard audio segment and a recorded audio segment are calculated separately to obtain two values: "standard audio energy" and "recorded audio energy". Energy = sum of squares of all samples within a frame;

[0085] The energy similarity is calculated by dividing the minimum of the two values, "standard audio energy" and "recorded audio energy", by the maximum value. The result is the energy similarity. Energy similarity = min(standard audio energy, recorded audio energy) / max(standard audio energy, recorded audio energy).

[0086] Detection result processing and output: The audio similarity is compared with the preset similarity threshold to generate the detection result. The detection result records the number of detections, the number of anomalies, the time of anomaly occurrence, audio waveform graph and similarity line graph data.

[0087] Specifically, the detection result processing and output provides status management, supporting playback of standard / recorded audio, pause playback, and analysis / stop data modes. This invention records the number of detections, the number of anomalies, the time of anomaly occurrence, detailed data, audio waveform graphs, and similarity line graphs; it supports parallel analysis of detection results from multiple devices recording audio, and improves batch detection efficiency through multi-process scheduling.

[0088] Furthermore, the uniform volume setting of the device under test is 50% of the maximum volume.

[0089] Furthermore, the standardized setup of recording equipment includes:

[0090] When testing a single device under test, place the recording device in the center of the front of the device under test, maintain a preset distance between the recording device and the device under test, and install the recording device at an angle perpendicular to the front of the device under test.

[0091] When testing multiple devices under test, the recording device is placed at the geometric center of all devices under test and directly below the corresponding device under test, and the positioning is calibrated by a dedicated audio file containing device identification marks.

[0092] Specifically, taking five televisions as the test equipment as an example, the five televisions are arranged from left to right on the same horizontal plane. Take the geometric center point of the screen of the third television (counting from the left), and draw a vertical line downwards from this point (i.e., the vertical center line of the television). Place a recording device directly below the center point on this vertical line, below the bottom edge of the television screen (or directly on the ground / placement surface). The horizontal position of the recording device is consistent with the horizontal position of the center point of the television screen, and its vertical position is lower than the bottom edge of the television.

[0093] Furthermore, the testing environment includes both quiet and interfering environments. The noise floor in the quiet environment is 36.8 dBA, and the safe distance between the recording device and the device under test is set to 10 cm according to the evaluation standard.

[0094] Furthermore, the recording device supports different types of recording devices. By adjusting the recording level of the recording software, the recording differences between different devices can be reduced, thus avoiding the impact of the recording device on the similarity.

[0095] Furthermore, for high-level loudspeaker equipment, the features can be dynamically expanded to 16 dimensions, as shown in Table 4. This involves adding spectral bandwidth, spectral flux, and zero-crossing rate features to the 13-dimensional features. The 13-dimensional features consist of one logarithmic energy coefficient and 12 Mel-frequency cepstral coefficients. The logarithmic energy coefficient corresponds to the 0th cepstral coefficient after DCT transformation, reflecting the energy level of the audio frame. The 12 Mel-frequency cepstral coefficients correspond to the 1st to 12th cepstral coefficients after DCT transformation. These are core coefficients obtained from the Mel-filtered spectrum through logarithmic compression and DCT transformation, reflecting the frequency distribution characteristics of the audio.

[0096] Table 4 - Definition of Extracted Multidimensional Audio Features

[0097]

[0098] Furthermore, the detection results are obtained through parallel analysis of audio recorded by multiple devices in the detection result processing and output steps.

[0099] Furthermore, the test results are visualized, such as... Figure 6 As shown, the visualization includes the detection status per second, a comparison chart of standard audio and recorded audio waveforms, a similarity line graph, and an anomaly distribution heatmap.

[0100] The test results are converted into an HTML report containing basic information, quantitative results (number of tests, number of abnormalities, pass rate) and detailed data (time per second - similarity - test result), and the report is pushed through the communication unit. The report can be filtered by all results, qualified results or unqualified results, and the unqualified areas and abnormality types are marked. Abnormality types include broken sound, cracked sound and no sound.

[0101] Furthermore, the test results or reports accurately record the specific time (in seconds) when the anomaly occurred and its duration, and support audio clip playback, which facilitates fault location and tracing.

[0102] Furthermore, the communication unit enables automatic email push of reports, which include a test summary (number of anomalies, pass rate, similarity, etc.) and report attachments, and supports configuration of multiple recipients.

[0103] Furthermore, the present invention also discloses an automated audio detection system based on multi-dimensional calibration. Employing the aforementioned automated audio detection method based on multi-dimensional calibration, the system includes:

[0104] The recording calibration module, including a recording device, a rangefinder, an angle gauge, and a noise tester, is used to calibrate the placement of the recording device according to standardized recording specifications to obtain consistent and valid recorded audio. The recording device uses a uniform model, with multiple devices forming a multi-channel acquisition architecture, each device under test corresponding to one independent channel. The rangefinder calibrates the straight-line distance between the recording device and the speaker of the device under test. The angle gauge calibrates the installation angle of the recording device. The noise tester detects ambient noise to confirm that the environment meets preset testing requirements. Specifically, the recording calibration module defines standardized recording specifications and performs multi-dimensional calibration for device size, number of tests, and environmental scenarios to ensure the consistency and validity of the recorded data.

[0105] Specifically, the standardized recording position is defined as follows: 1) Single-device recording position: The recording device is placed in the center of the front of the device under test, avoiding the edges of the device and reflective surfaces (walls / desktops) at a certain distance; different devices under test are kept at a certain distance from the recording device, and the installation angle is perpendicular to the front of the device (0° angle); 2) Multi-device recording position: The recording device is placed in the center of the bottom of all devices under test, and a "dedicated audio file" (including device identification marks) is used for calibration and positioning to ensure that the audio sources of all devices can be effectively recorded. The recording device of this invention supports recording with different types of recording devices, and reduces the recording differences between different devices by adjusting the recording level of the recording software, avoiding the influence of the recording device on the similarity.

[0106] The audio preprocessing module is used to load standard audio and recorded audio at a uniform fixed sampling rate when importing audio, extract the audio track of the recorded audio; eliminate environmental noise of the recorded audio using spectral subtraction based on environmental noise, normalize the amplitude of recorded audio from different recording devices, distances and angles, and extract audio segments in intervals of one second.

[0107] Specifically, assuming the current peak value of the audio is 0.5 and the target peak value is 0.9, then the normalization coefficient = 0.9 / 0.5 = 1.8; multiplying all sample values ​​by 1.8, the new peak value = 0.5 × 1.8 = 0.9; the corresponding formula is:

[0108] ;

[0109] Where i represents the index of the audio sample; This is the target peak value (default 0.9). It is the maximum absolute value of the audio signal;

[0110] The audio feature extraction and time alignment module is used to extract audio segments per second from standard audio and recorded audio respectively. First, the cross-correlation function of standard audio and recorded audio is calculated to obtain the time offset. The audio length is adjusted by padding with zeros or truncating to achieve coarse alignment between standard audio and recorded audio. Then, the time warping algorithm is used to elastically stretch or compress the recorded audio to achieve fine alignment between standard audio and recorded audio.

[0111] Specifically, `shift` (offset) calculates the time offset of the recorded segment relative to the standard segment by finding the location of the maximum cross-correlation value. A positive `shift` indicates that the recorded segment is "lagging" behind the standard segment (needing to be moved forward, padded with zeros). A negative `shift` indicates that the recorded segment is "ahead" of the standard segment (needing to be moved backward, truncated). The correlation function formula is as follows:

[0112] time_shift_ms=(np.argmax(corr_window)-(max_shift_ms×sr / 1000)) / sr×1000;

[0113] Where ime_shift_ms is the final time offset; np.argmax(corr_window) is the sample index corresponding to the offset position where the cross-correlation value is the largest (the signal is most similar); max_shift_ms is the maximum search offset time; and Sr is the sampling rate.

[0114] The multi-dimensional similarity calculation module extracts 13-dimensional audio features through the second processor to calculate the number of audio sampling points per second. Based on the minimum audio length, it extracts standard audio segments and aligned recorded audio segments from the standard audio and the recorded audio respectively. It extracts the coefficient features of each audio segment and performs standardization processing to eliminate interference from signal strength differences, and then calculates the audio similarity between the standard audio and the recorded audio.

[0115] The 13-dimensional feature consists of one logarithmic energy coefficient and 12 Mel frequency cepstral coefficients. The logarithmic energy coefficient corresponds to the 0th cepstral coefficient after DCT transformation, reflecting the energy level of the audio frame. The 12 Mel frequency cepstral coefficients correspond to the 1st to 12th cepstral coefficients after DCT transformation. They are the core coefficients obtained from the Mel-filtered spectrum through logarithmic compression and DCT transformation, reflecting the frequency distribution characteristics of the audio.

[0116] Specifically, the number of audio sampling points per second is calculated, and the loop variable i ranges from 0 to the minimum audio length (in seconds). The start and end indices of each segment are determined to extract standard audio segments and aligned recorded audio segments. The coefficient features of each audio segment are extracted by the second processor to capture key information in the audio frequency domain and cepstral domain. The coefficient features are then processed by the second processor to standardize (mean is 0, standard deviation is 1) to eliminate the interference of signal strength differences on similarity calculation. Finally, the audio similarity is calculated.

[0117] Specifically, the coefficient features of each audio segment include human auditory perception, spectrum, and energy. In particular, audio similarity is measured by a comprehensive average of these three dimensions: human auditory perception, spectrum, and energy.

[0118] Human auditory perception is calculated by extracting auditory feature values ​​from audio segments one second at a time and then calculating the average vector cosine similarity.

[0119] ;

[0120] ;

[0121] Where A and B are the auditory characteristic values ​​of the left and right ears of a person, respectively; Cosine similarity; This represents the normalized similarity.

[0122] The spectrum similarity is calculated by extracting spectral feature values, aligning them, normalizing them, and then performing spectral similarity calculations.

[0123] ;

[0124] in, For vectors The average value; Let be the average value of vector b; n is the length of vectors a and b. This refers to the i-th observation of the first variable (e.g., the i-th sample data of variable a). For the i-th observation of the second variable (and) The i-th sample data corresponding to variable b).

[0125] To calculate audio energy, the energy of a standard audio segment and a recorded audio segment are calculated separately to obtain two values: "standard audio energy" and "recorded audio energy". Energy = sum of squares of all samples within a frame;

[0126] The energy similarity is calculated by dividing the minimum of the two values, "standard audio energy" and "recorded audio energy", by the maximum value. The result is the energy similarity. Energy similarity = min(standard audio energy, recorded audio energy) / max(standard audio energy, recorded audio energy).

[0127] The detection result processing and output module compares the audio similarity with a preset similarity threshold to generate detection results. The detection results record the number of detections, the number of anomalies, the time of anomaly occurrence, audio waveform graphs, and similarity line graphs.

[0128] Furthermore, the detection result processing and output module visualizes the detection results, including the detection status per second, a comparison graph of standard audio and recorded audio waveforms, a similarity line graph, and an anomaly distribution heatmap.

[0129] Furthermore, the test result processing and output module converts the test results into an HTML format report containing basic information, quantitative results and detailed data, and pushes the report through the communication unit. The report supports filtering by all results, qualified results or unqualified results, and marks unqualified areas and abnormality types. Abnormality types include broken sound, cracked sound and no sound.

[0130] This invention can effectively identify anomalies such as dropped sounds, distorted sounds, and silence. The system innovatively proposes mechanisms such as "distance calibration for device size adaptation," "differentiated recording positions for single / multiple devices," "multi-source fusion detection," and "simultaneous recording of different sound sources by multiple groups in the same environment." Combined with time warping of the first processor to complete parallel alignment and feature extraction of multiple audio frequencies, and feature extraction and multi-dimensional similarity calculation by the second processor, it solves the problems of recording position interference, environmental noise influence, and low detection efficiency in traditional detection. It is suitable for batch testing and long-term stability testing scenarios for consumer electronics.

[0131] The present invention, employing the above technical solutions, has the following beneficial effects: 1) Through standardized recording calibration, multi-dimensional feature extraction, and dual-stage time alignment, it accurately identifies subtle anomalies such as millisecond-level dropouts and slight distortions, significantly improving detection accuracy; 2) The multi-channel architecture supports parallel detection by multiple devices, automating the entire process to replace manual listening, significantly improving detection efficiency and adapting to batch production needs; 3) Customized standardized specifications enable differentiated recording specifications for single / multiple devices to adapt to various scenarios, compatible with different recording devices and quiet / interference environments, demonstrating strong compatibility; 4) Detection data is automatically stored, reports are automatically generated, and multi-dimensional charts intuitively present the results, avoiding human error and ensuring objectivity; 5) Fault tracing and remote monitoring are convenient, with anomaly information recorded in seconds and playback supported, and reports automatically pushed via email, assisting in quality control.

[0132] This invention achieves accurate audio detection for devices such as televisions by defining standardized recording positions, calibrating parameters for multiple recording scenarios, and optimizing audio processing algorithms.

[0133] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method for audio automated detection based on multi-dimension calibration, characterized in that: It comprises the following steps: System building and calibration: adopt a unified model of recording equipment to build a multi-channel acquisition architecture, each device to be tested corresponds to an independent channel; arrange the recording equipment according to the standardized specification, use the range finder to calibrate the distance between the recording equipment and the device to be tested, use the angle ruler to calibrate the installation angle, and use the noise tester to detect the environmental noise to confirm that it meets the detection environment requirements; Load the uniform detection sound source and set the uniform volume for the device to be tested, so as to record the recorded audio; Audio preprocessing: load the standard audio and recorded audio with a fixed sampling rate when importing the audio, and extract the audio track of the recorded audio; According to the environmental noise, the spectral subtraction method is used to eliminate the environmental noise of the recorded audio, the amplitude of the recorded audio of different recording devices, distances and angles is normalized, and the audio segment is intercepted according to the per second interval; Audio feature extraction and time alignment: extract the audio alignment processing through the first processor, calculate the cross-correlation function of the standard audio and the recorded audio to obtain the time offset, adjust the audio length by zero padding or truncation to realize the coarse alignment of the standard audio and the recorded audio, and then use the time normalization algorithm of the first processor to realize the fine alignment of the standard audio and the recorded audio by stretching or compressing the recorded audio; Multi-dimensional similarity calculation: extract at least 13-dimensional features of the audio through the second processor, calculate the number of audio samples per second, and according to the minimum length of the audio, the standard audio segment and the aligned recorded audio segment are correspondingly intercepted from the standard audio and the recorded audio respectively; extract the coefficient features of each audio segment through the second processor and perform standardization processing to eliminate the interference of signal intensity difference, and then calculate the audio similarity of the standard audio and the recorded audio; Detection result processing and output: compare the audio similarity with the preset similarity threshold to generate the detection result, and the detection result records the detection times, the number of abnormalities, the time of abnormality, the audio waveform graph and the similarity line graph data.

2. The method of claim 1, wherein: The set uniform volume of the device to be tested is 50% of the maximum volume.

3. The method of claim 1, wherein: Arranging the recording equipment according to the standardized specification includes: When detecting a single device to be tested, place the recording equipment in the middle of the front of the device to be tested, and keep the recording equipment and the device to be tested at a preset distance and the installation angle perpendicular to the front of the device to be tested; When detecting multiple devices to be tested, place the recording equipment at the geometric center of all devices to be tested and directly below the corresponding device to be tested, and calibrate and position through a special audio file containing device differentiation marks.

4. The multi-dimension calibration based audio automation detection method of claim 1, wherein: Arranging the recording equipment according to the standardized specification follows the following principles: consistency principle, the recording position of the recording equipment of all devices to be tested is completely unified and standardized, ensuring the consistency of the spatial reference of the recorded audio; Sound source pointing principle, the recording equipment is accurately pointed to the core area of the device audio output; Isolation principle, the recording position is far away from the edge of the device, the reflecting surface and the interference source, and meets the phase interference avoidance requirements; Traceability principle, the recording position of each device is realized through "reference mark + coordinate record".

5. The method of claim 1, wherein: The detection environment includes quiet environment and interference environment, the bottom noise of the quiet environment is 36.8dBA, and the safe distance between the recording equipment and the device to be tested is set to 10CM according to the evaluation standard.

6. The multi-dimension calibration based audio automation detection method of claim 1, wherein: The recording device supports different types of recording devices, and the recording level size is adjusted by adjusting the recording software to reduce the recording differences of different devices, so as to avoid the influence of the recording device on the similarity.

7. The multi-dimension calibration based audio automation detection method of claim 1, wherein: The detection results are visualized and displayed, including the detection status per second, the waveform comparison chart of the standard audio and the recorded audio, the similarity line chart and the abnormal distribution heat map. The detection results are converted into an HTML format report containing basic information, quantitative results and detailed data, and the report is pushed through the communication unit; the report can be filtered according to all results, qualified results or unqualified results, and the unqualified area and abnormal type are marked, and the abnormal type includes broken sound, broken sound and no sound.

8. The audio automated detection system based on multi-dimension calibration, using the audio automated detection method based on multi-dimension calibration of any one of claims 1 to 7, characterized in that: The system comprises: The recording calibration module comprises a recording device, a range finder, a protractor and a noise tester, which is used to calibrate the arrangement position of the recording device according to the standardized recording specification, so as to obtain the recording audio with consistency and effectiveness; The recording device adopts a unified model, and multiple recording devices form a multi-channel acquisition architecture, with each device corresponding to an independent channel; the range finder calibrates the straight-line distance between the recording device and the speaker of the device under test; the protractor calibrates the installation angle of the recording device; and the noise tester detects the environmental noise to confirm that the environment meets the preset detection requirements; The audio preprocessing module is used to uniformly fix the sampling rate to load the standard audio and the recorded audio when importing the audio, and extract the audio track of the recorded audio; According to the environmental noise, the spectral subtraction method is used to eliminate the environmental noise of the recorded audio, the amplitude of the recorded audio of different recording devices, distances and angles is normalized, and the audio segment is intercepted per second; The audio feature extraction and time alignment module is used to intercept the audio segment per second from the standard audio and the recorded audio respectively; the cross-correlation function of the standard audio and the recorded audio is calculated to obtain the time offset, the audio length is adjusted by zero padding or truncation to realize the coarse alignment of the standard audio and the recorded audio, and the time normalization algorithm is used to realize the fine alignment of the standard audio and the recorded audio by stretching or compressing the recorded audio; The multi-dimensional similarity calculation module extracts the 13-dimensional features of the audio through the second processor, calculates the number of audio sampling points per second, and correspondingly intercepts the standard audio segment and the aligned recorded audio segment from the standard audio and the recorded audio according to the minimum length of the audio; the coefficient features of each audio segment are extracted and standardized to eliminate the interference of signal intensity difference, and then the audio similarity of the standard audio and the recorded audio is calculated. The detection result processing and output module is used to compare the audio similarity with the preset similarity threshold to generate the detection result, which records the detection times, the abnormal times, the abnormal time, the audio waveform chart and the similarity line chart data.

9. The multi-dimension calibration based audio automation detection system of claim 8, wherein: The detection result processing and output module visualizes and displays the detection results, including the detection status per second, the waveform comparison chart of the standard audio and the recorded audio, the similarity line chart and the abnormal distribution heat map.

10. The multi-dimension calibration based audio automation detection system of claim 8, wherein: The detection result processing and output module converts the detection result to generate an HTML format report containing basic information, quantization result and detailed data, and pushes the report through the communication unit; the report supports filtering according to all results, qualified results or unqualified results, and marking unqualified areas and abnormal types, the abnormal types including broken sound, broken sound and no sound.

Images