Delay automatic calibration method, device and equipment of intelligent loudspeaker box system and storage medium

Abstract

The invention discloses an automatic delay calibration method, device and equipment for an intelligent loudspeaker box system and a storage medium. The method comprises the steps that the intelligent loudspeaker box system is initialized, and environment detection is carried out to preliminarily configure system parameters; exciting the target sound channel and playing the target test signal; positioning a first test signal based on a VAD voice activity detection algorithm, judging the start and end of the first test signal, and identifying the time difference of the first test signal to obtain the delay time of the main sound channel; positioning the arrival time of the second test signal based on a peak velocity search algorithm, and identifying the delay time of the subwoofer; and performing delay time calibration on the subwoofer and each main sound channel, and dynamically adjusting the delay calibration parameter of each sound channel according to the delay time calibration amount, thereby realizing accurate time alignment of the subwoofer and the main sound channel. The problem of low-frequency signal delay which is difficult to process in the prior art is solved, and a breakthrough progress is achieved especially in coordination of the subwoofer and other sound channels.

Description

Technical Field

[0001] The present invention relates to the field of audio technology, and in particular to a method, device, equipment and storage medium for automatic delay calibration of an intelligent speaker system. Background Art

[0002] At present, there is a common problem of time alignment in multi-channel speaker systems on the market. Due to the difference in physical distance between the speaker units and the inconsistency of signal processing delay, there is a deviation in the time when the sound arrives at the listening position. This time deviation will cause inaccurate sound field positioning, reduced sound restoration, and seriously affect the user's listening experience. It is difficult to achieve accurate time alignment with existing technologies, especially in the coordination of subwoofers and other channels. Traditional time alignment methods usually rely on manual measurement and adjustment, which is not only time-consuming and labor-intensive, but also difficult to achieve ideal accuracy. Although some automatic calibration systems can improve the time alignment problem to a certain extent, they often perform poorly when processing low-frequency signals, especially in the coordination of subwoofers and other channels. There are great difficulties. This is because the low-frequency signal has a long period and complex phase characteristics, making it difficult to accurately detect the signal arrival time. In addition, the existing automatic calibration methods often do not take into account the influence of different room acoustic environments, resulting in unsatisfactory results in practical applications. Although audio processing technologies such as frequency response correction and dynamic range control can improve some sound quality problems, they cannot fundamentally solve the sound field positioning and phase problems caused by time alignment. Therefore, there is an urgent need for an intelligent calibration method that can accurately calculate the delay time of each channel and adapt to different environments to improve the overall performance and user experience of the multi-channel speaker system. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for automatic delay calibration of an intelligent speaker system. The method utilizes voice activity detection (VAD) technology and a peak rate search algorithm to automatically calculate and calibrate the delay time of each speaker unit in a multi-channel system. The method can accurately calculate the delay time of a subwoofer and achieve precise time alignment with other channels. By adopting an adaptive algorithm, the delay parameters can be dynamically adjusted according to different room acoustic environments and listening positions to achieve the best sound field restoration effect.

[0004] A first aspect of the present invention provides a method for automatically calibrating a delay of an intelligent speaker system, comprising:

[0005] Initialize the smart speaker system and detect the environment to preliminarily configure system parameters;

[0006] Exciting a target channel and playing a target test signal, wherein the target test signal includes a first test signal for testing a main channel and a second test signal for testing a subwoofer;

[0007] Locating the first test signal based on a VAD voice activity detection algorithm and determining the start and end of the first test signal, identifying a time difference between when the first test signal is sent and when it is detected, and obtaining a delay time of the main channel;

[0008] Locating the arrival time of the second test signal based on a peak rate search algorithm, and identifying the time difference between the second test signal being emitted and being detected as the delay time of the subwoofer;

[0009] The delay time of the subwoofer and each main channel is calibrated according to the delay time of the main channel and the delay time of the subwoofer, and the delay calibration parameters of each channel are dynamically adjusted according to the delay time calibration amount, so as to achieve precise time alignment between the subwoofer and the main channel.

[0010] Preferably, the initializing the smart speaker system and detecting the environment to preliminarily configure system parameters includes:

[0011] Start the smart speaker system and enter calibration mode;

[0012] Play white noise or a frequency sweep signal containing various frequencies and collect the room's acoustic response through a microphone array;

[0013] Using fast Fourier transform to analyze the acoustic characteristics of the collected acoustic signal to obtain the room environment analysis result, which includes a frequency response curve and a reverberation time parameter;

[0014] The system parameters are preliminarily configured according to the environmental analysis results. The preliminarily configured operations include, when it is detected that the reverberation time of the room is greater than the reverberation time threshold, correspondingly increasing the time window length to capture longer sound clips for subsequent processing.

[0015] Preferably, the method of performing acoustic characteristic analysis on the collected signal by using fast Fourier transform includes:

[0016] Perform FFT fast Fourier transform on the collected acoustic signal x(n), convert the collected time domain signal into a frequency domain signal, and obtain the energy distribution of the signal at each frequency. The FFT implementation formula is as follows:

[0017] X(k)=Σ[n=0toN-1]x(n)*e^(-j2πkn / N);

[0018] Calculate the power spectral density: P(k) = |X(k)|^2 / N;

[0019] The energy attenuation curve is calculated using the Schroeder integral method to obtain the energy attenuation of the room at different frequencies. The target linear region is extracted from the energy attenuation curve, and a linear fit is performed on the target linear region. The time required for the energy to decay by 60 dB is calculated based on the fitted straight line to estimate the reverberation time T60.

[0020] Where X(k) represents the output frequency domain signal, N is the number of sampling points, k is the frequency index, |X(k)| represents the amplitude of the kth frequency component, P(k) represents the power spectrum density of the kth frequency component, and the target linear region ranges from -5dB to -35dB.

[0021] Preferably, the VAD voice activity detection algorithm is used to locate the first test signal and determine the start and end of the first test signal, identify the time difference between the first test signal being sent and being detected, and obtain the delay time of the main channel, including:

[0022] Excite each main channel in the smart speaker system in turn, and play the first test signal, where the first test signal includes a target pulse signal or a sine wave envelope signal of a preset frequency, and the preset frequency range covers 200 Hz-20 kHz;

[0023] Dividing the input first test signal into a plurality of short frames according to a preset length;

[0024] Acquire the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames, and compare the acquired short-time energy and zero-crossing rate with a preset energy threshold and a preset zero-crossing rate threshold to determine whether the frame signal is a voice activity;

[0025] If both the short-time energy and the zero-crossing rate exceed the corresponding thresholds, the result of the judgment is that the frame signal is a voice activity, thereby determining the start time of the test signal;

[0026] If the short-time energy and zero-crossing rate do not exceed the corresponding thresholds, the judgment result is that the frame signal is non-speech activity;

[0027] If the judgment result is a misjudgment result, the misjudgment result is eliminated by a smoothing algorithm.

[0028] Preferably, the step of obtaining the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames includes:

[0029] The short-time energy of each frame signal is calculated, that is, the sum of the squares of the signal samples, which is used to evaluate the strength of the signal. The formula for calculating the short-time energy E of each frame is as follows:

[0030] E=Σ(x[n]^2), n=0, 1,..., N-1,

[0031] By calculating the difference between the sign functions of adjacent sample points and taking the absolute value, the zero crossing rate ZCR of each frame signal is calculated:

[0032] ZCR=(1 / 2)*Σ|sgn(x[n])-sgn(x[n-1])|, n=1, 2,..., N-1,

[0033] Among them, sgn() is the sign function, x[n] is the signal sample, and N is the frame length.

[0034] Preferably, locating the arrival time of the second test signal based on the peak rate search algorithm and identifying the time difference between the second test signal being emitted and being detected as the delay time of the subwoofer comprises:

[0035] Playing the second test signal, where the second test signal is a swept frequency sine wave signal in the range of 20-200 Hz;

[0036] Locating the second test signal based on a peak rate search algorithm, and preprocessing the collected signal to remove noise and filter the signal;

[0037] Set a threshold for identifying valid peaks: threshold = μ + k * σ, where μ is the signal mean, σ is the standard deviation, and k is the adjustment coefficient;

[0038] Use sliding window technology to scan the signal, set the window size W, for each sampling point i, if x[i]>threshold and x[i]=max(x[iW / 2:i+W / 2]), then mark it as a potential peak, x[i] represents the value of the i-th sampling point of the signal;

[0039] Use parabola interpolation to accurately locate the peak position. For each potential peak point i, query the adjacent points i-1 and i+1 on the left and right of the peak point i, and fit a parabola based on the adjacent points and the potential peak point.

[0040] The arrival time of the second test signal is identified by calculating the time interval between adjacent peaks. If the interval is close to the expected signal period, the first valid peak is confirmed to be the signal arrival time.

[0041] Preferably, the delay time calibration of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjusting the delay calibration parameters of each channel according to the delay time calibration amount includes:

[0042] Selecting a channel with the longest delay time as a reference channel, calculating delay differences of other channels except the reference channel relative to the reference channel, and performing a first delay adjustment according to the delay differences;

[0043] Calculate the phase difference between the subwoofer and the main channel: Δφ = φ_sub - φ_main;

[0044] Convert the calculated phase difference into time delay: Δt = Δφ / (2π*f_crossover);

[0045] Performing a second delay adjustment according to the time delay, thereby ensuring a smooth phase transition between the subwoofer and the main channel near the crossover point;

[0046] Among them, Δφ represents the phase difference, φ_sub represents the phase of the subwoofer, φ_main represents the phase of the main channel, Δt represents the time delay, and f_crossover is the crossover frequency.

[0047] As a preferred embodiment, it also includes:

[0048] Based on the adaptive calibration mechanism, the parameters are dynamically adjusted according to the acoustic environment of different rooms, and periodic fast calibration checks are performed while the system is running, further including:

[0049] The background music or ambient sound being played is used as the calibration signal, and the signal characteristics of each channel are extracted from the mixed signal through blind source separation technology;

[0050] The ICA algorithm is used to separate the multi-channel mixed signal: X = AS, where X is the observed mixed signal, A is the mixing matrix, and S is the source signal;

[0051] The FastICA algorithm is used to estimate the separation matrix W and iteratively optimize the parameters until convergence, where S ≈ WX;

[0052] After obtaining the separated channel signals, time delay estimation is performed, and delay calibration parameters are updated according to the estimation results;

[0053] If a change in the delay calibration parameters is detected, the user is prompted whether a full calibration process is required.

[0054] As a preferred embodiment, it also includes:

[0055] The automatic calibration process is further optimized using machine learning based on user latency-adjusted behavioral feedback, including:

[0056] Define the state space S as the current delay parameter settings;

[0057] Define the action space A as possible delay adjustment operations;

[0058] Define a reward function R to give positive or negative rewards based on whether the user accepts the automatic calibration result or makes manual adjustments;

[0059] Use the Q-learning algorithm to update the value function: Q(s, a)←Q(s, a)+α[r+γ*max(Q(s', a'))-Q(s, a)];

[0060] Select the optimal calibration strategy based on the updated Q value;

[0061] Where α is the learning rate, γ is the discount factor, s and a are the current state and action respectively, and s' is the next state.

[0062] As a preferred embodiment, it also includes:

[0063] Building a database of room acoustic characteristics for smart speaker systems in multiple rooms, including room dimensions, reverberation time, and frequency response;

[0064] Based on the distributed consistency algorithm, the Raft consensus algorithm, version vector and anti-entropy process are used to synchronize the calibration data of multiple devices, thus ensuring the collaborative work of multiple rooms and multiple devices.

[0065] As a preferred embodiment, it also includes:

[0066] Integrate multiple preset audio processing algorithms to optimize audio, further including:

[0067] Room response correction is achieved by using adaptive filters to compensate for the room acoustics to the frequency response;

[0068] Configure a multi-threshold compressor to apply different compression ratios at different volumes to ensure the listening experience of low and high volumes;

[0069] A psychoacoustic model is introduced, and the calculated delay time is delayed and adjusted according to the masking effect of the human ear and the critical bandwidth characteristics. The acquired signal is subjected to STFT short-time Fourier transform to obtain the time-frequency spectrum X(k, m), and the energy of each frequency band k is calculated: E(k)=Σ|X(k, m)|^2, m=1, 2, ..., M. The delay sensitivity of different frequency bands is weighted according to the preset loudness curve standard, and an equal loudness filter H(k) is applied to each frequency band E'(k)=E(k)*H(k). The delay time is delayed and adjusted within the critical bandwidth to minimize the phase difference, where M represents the window function length or the number of frames of the short-time Fourier transform.

[0070] As a preferred embodiment, it also includes:

[0071] The steps to evaluate and optimize the performance of the smart speaker system include:

[0072] Calculate the cross-correlation function between the reference signal x(n) and each channel output signal y(n):

[0073] R_xy(τ)=Σx(n)y(n+τ), n=0, 1,..., N-1,

[0074] Find the peak position of the cross-correlation function, and the τ value corresponding to this position is the time offset:

[0075] τmax=argmax(R_xy(τ));

[0076] Compare the calculated actual time offset with the theoretical offset to assess the calibration accuracy:

[0077] Error = |τmax-τtheory|;

[0078] Where N represents the length of the signal, that is, the number of sampling points, τmax is the actual time offset, and τtheory is the theoretical offset.

[0079] A second aspect of the present invention provides a delay automatic calibration device for an intelligent speaker system, comprising:

[0080] An initialization module is used to initialize the smart speaker system and detect the environment to preliminarily configure system parameters;

[0081] A test module, used to stimulate a target channel and play a target test signal, wherein the target test signal includes a first test signal for testing a main channel and a second test signal for testing a subwoofer;

[0082] A main channel module, used to locate the first test signal based on a VAD voice activity detection algorithm and determine the start and end of the first test signal, identify the time difference from when the first test signal is sent to when it is detected, and obtain a delay time of the main channel;

[0083] A subwoofer module, configured to locate the arrival time of the second test signal based on a peak rate search algorithm, and identify the time difference between the second test signal being emitted and being detected as a delay time of the subwoofer;

[0084] The calibration module is used to calibrate the delay time of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjust the delay calibration parameters of each channel according to the delay time calibration amount, so as to achieve accurate time alignment between the subwoofer and the main channel.

[0085] A third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned automatic delay calibration method of the smart speaker system when executed by the processor.

[0086] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned automatic delay calibration method of the smart speaker system.

[0087] With respect to the prior art, the present invention has the following beneficial effects:

[0088] The present invention proposes a delay automatic calibration method for an intelligent speaker system, which realizes the precise time alignment of a multi-channel audio system by innovatively combining voice activity detection (VAD) technology and peak rate search algorithm. This solution not only solves the low-frequency signal delay problem that is difficult to handle with traditional technologies, especially in the coordination between the subwoofer and other channels, but also introduces a real-time adaptive calibration mechanism that can dynamically adjust parameters according to the acoustic environment of different rooms. The system integrates advanced audio processing technologies such as room correction, psychoacoustic model and dynamic range optimization, which comprehensively improves the sound restoration quality. Its modular design and open API interface enhance the scalability and compatibility of the system, and provide third-party developers with the possibility of function customization. By introducing machine learning algorithms for continuous optimization, combined with user feedback and A / B testing, the system can continuously improve performance. In terms of security and privacy protection, end-to-end encryption and differential privacy technologies are adopted to fully guarantee the security of user data. In addition, the solution also takes into account multi-room, multi-device collaboration and advanced user functions, providing a comprehensive solution. These innovations make this technology of great significance in improving the performance of multi-channel speaker systems, enhancing user experience and promoting technological progress in the industry. It has brought new technological breakthroughs to the field of smart speakers and is expected to be widely used in the market.

[0089] The present invention ensures that all channels can be accurately aligned during playback to improve the sound quality experience by accurately calibrating the main channel and subwoofer in time; automatically configures system parameters according to the acoustic characteristics of different rooms, so that the speaker system can adapt to a variety of listening environments and optimize the sound performance; dynamically adjusts the delay calibration parameters through periodic fast calibration checks and real-time signal processing based on blind source separation technology to cope with problems such as environmental changes or equipment aging; uses machine learning algorithms to further optimize the automatic calibration process based on user feedback to make the system more intelligent and personalized; establishes a room acoustic characteristic database and synchronizes the calibration data of multiple devices through a distributed consistency algorithm to achieve seamless collaboration between multiple rooms and multiple devices; integrates multiple preset audio processing algorithms, such as adaptive filters, multi-threshold compressors, and psychoacoustic models, to further improve audio quality and listening experience; evaluates the calibration accuracy by calculating the cross-correlation function of the reference signal and the output signal, and optimizes the system performance based on the comparison result of the actual time offset and the theoretical offset. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 Schematic diagram of the process of the automatic delay calibration method of the smart speaker system in this embodiment;

[0091] Figure 2 This is a schematic diagram of the steps of initializing the smart speaker system and detecting the environment to preliminarily configure system parameters in S1 of this embodiment;

[0092] Figure 3 FIG. 4 is a schematic diagram of an embodiment of a computer device in this embodiment. DETAILED DESCRIPTION

[0093] The present invention provides a method, device, equipment and storage medium for automatic delay calibration of an intelligent speaker system, which effectively solves the problem of reopening capture when the audio input player switches modes through a persistent audio capture session and a dynamic audio processing pipeline.

[0094] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0095] Embodiment 1

[0096] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1-2 , an embodiment of the present invention provides a method for automatic delay calibration of an intelligent speaker system, comprising:

[0097] S1: Initialize the smart speaker system and detect the environment to preliminarily configure system parameters, including the following:

[0098] S11: Start the smart speaker system and enter the calibration mode to ensure that the system can operate normally and prepare for the subsequent calibration process;

[0099] S12: Play white noise or swept frequency signals containing various frequencies, and collect the acoustic response of the room through the microphone array. The built-in microphone array is used here to analyze the acoustic characteristics of the surrounding environment;

[0100] S13: using fast Fourier transform (FFT) to perform acoustic characteristic analysis on the collected acoustic signal to obtain an environmental analysis result of the room, wherein the environmental analysis result includes a frequency response curve and a reverberation time parameter;

[0101] S14: Perform preliminary configuration operations on system parameters according to the environmental analysis results. The preliminary configuration operations include increasing the time window length accordingly to capture longer sound clips for subsequent processing when it is detected that the reverberation time of the room is greater than the reverberation time threshold; if obvious standing waves are found in certain frequency bands, the system will adopt more sophisticated analysis methods in these frequency bands.

[0102] Specifically, in step S13, using fast Fourier transform to analyze the acoustic characteristics of the collected signal includes:

[0103] Perform FFT fast Fourier transform on the collected acoustic signal x(n), convert the collected time domain signal into a frequency domain signal, and obtain the energy distribution of the signal at each frequency. The FFT implementation formula is as follows:

[0104] X(k)=Σ[n=0toN-1]x(n)*e^(-j2πkn / N) that is Through FFT, we can get the energy distribution of the signal at each frequency.

[0105] Calculate the power spectral density: P(k) = |X(k)|^2 / N; the power spectral density is the energy distribution of the signal at each frequency.

[0106] The energy attenuation curve is calculated using the Schroeder integral method to obtain the energy attenuation of the room at different frequencies. The target linear region is extracted from the energy attenuation curve, and a linear fit is performed on the target linear region. The time required for the energy to decay by 60 dB is calculated based on the fitted straight line to estimate the reverberation time T60.

[0107] Where X(k) represents the output frequency domain signal, N is the number of sampling points, k is the frequency index, |X(k)| represents the amplitude of the kth frequency component, P(k) represents the power spectrum density of the kth frequency component, and the target linear region ranges from -5dB to -35dB.

[0108] In this embodiment, the specific steps for estimating the reverberation time T60 are as follows: Calculate the energy decay curve using the Schroeder integral method: The Schroeder integral method is a method based on the room acoustic model for calculating the energy decay curve. Through this method, we can obtain the energy decay of the room at different frequencies. Fit a straight line in the range of -5dB to -35dB: In order to more accurately estimate the reverberation time T60, we need to find a linear region on the energy decay curve, that is, the range of -5dB to -35dB, and perform a linear fit on it. This can eliminate the influence of nonlinear factors and improve the accuracy of the estimate. Calculate the time required for energy decay of 60dB: According to the linear equation obtained by fitting, we can calculate the time required for energy decay of 60dB, which is the reverberation time T60. This time reflects the speed at which sound decays in the room, which is of great significance for the design and optimization of the smart speaker system. Among them, the time required for energy decay of 60dB can be calculated according to the linear equation obtained by fitting, which can be achieved by the following steps: Determine the linear equation: First, we need to determine the linear equation obtained by fitting in the range of -5dB to -35dB. This equation can usually be expressed in the form of y=mx+b, where y represents energy (in dB), x represents time (in seconds), m represents slope, and b represents intercept. Calculate the time point corresponding to 60dB: Next, we need to calculate the time point when the energy decays to 60dB. Substitute 60 into the y value of the linear equation and solve for the corresponding x value, which is the required time. The specific calculation process is as follows: Substitute 60 into the linear equation: 60=mx+b, and solve for the x value: x=(60-b) / m. Consider the actual situation: In actual applications, some actual situations may need to be considered, such as the impact of room reflection, absorption and other factors on energy attenuation. Therefore, some corrections or adjustments may need to be made during the calculation process.

[0109] S2: Excite the target channel and play the target test signal, wherein the target test signal includes a first test signal for testing the main channel and a second test signal for testing the subwoofer, and the target channel includes the main channel such as left front, right front, center, left surround, right surround, etc. and the subwoofer;

[0110] S3: locating the first test signal based on the VAD voice activity detection algorithm and determining the start and end of the first test signal, identifying the time difference between the first test signal being sent and being detected, and obtaining the delay time of the main channel;

[0111] In this embodiment, the specific steps of calculating the delay time of the main channel are as follows:

[0112] S31: Excite each main channel in the smart speaker system in turn and play the first test signal, where the first test signal includes a target pulse signal or a sine wave envelope signal of a preset frequency, where the preset frequency range covers 200 Hz-20 kHz, and the target pulse signal is a short pulse signal;

[0113] S32: Divide the first test signal input into multiple short frames according to a preset length. The short frames here are the input signal framing. The preset length is 10-30ms per frame. Since the speech signal changes over time, in order to analyze this change, it is necessary to divide the continuous signal into shorter time segments, i.e., frames. In this way, the local characteristics of the speech signal can be captured according to the preset length while maintaining sufficient time resolution.

[0114] S33: Obtain the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames, and compare the obtained short-time energy and zero-crossing rate with the preset energy threshold and the preset zero-crossing rate threshold to determine whether the frame signal is a voice activity; short-time energy refers to the sum of the energy of all sample points in a frame signal. This feature can reflect the strength of the signal. If the energy in a frame is high, it may indicate that the frame contains a voice signal. The zero-crossing rate refers to the number of times the signal passes through the zero point. It is an indicator of the frequency content of the signal and can be used to distinguish between voice and non-voice such as noise. Voice signals usually have a high zero-crossing rate.

[0115] S34: If both the short-time energy and the zero-crossing rate exceed the corresponding thresholds, the judgment result is that the frame signal is voice activity, thereby determining the start time of the test signal; by comparing the calculated short-time energy and zero-crossing rate with the preset threshold, it can be judged whether each frame contains voice activity. If both the energy and the zero-crossing rate of a frame exceed the corresponding thresholds, then the frame is considered to be active, that is, it contains voice.

[0116] S35: If the short-time energy and zero-crossing rate do not exceed the corresponding threshold value, the judgment result is that the frame signal is non-speech activity; if the judgment result is a false judgment result, the false judgment result is eliminated by a smoothing algorithm. Since the judgment of a single frame may be affected by noise, resulting in isolated false judgments, a smoothing algorithm is needed to eliminate these isolated judgment results. The smoothing algorithm can be a simple moving average or other more complex filters, the purpose of which is to reduce false judgments and obtain more accurate VAD results. The VAD technology used above analyzes the short-time energy and zero-crossing rate of the audio signal, combined with a preset threshold to determine the presence of speech, which can effectively detect speech segments from a continuous audio stream and provide a basis for subsequent speech processing tasks. Record the time difference from the test signal being sent to being detected, which is the delay time of the channel, and repeat steps S31-S35 to complete the delay time calculation of all main channels.

[0117] Specifically, in step S33, obtaining the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames includes:

[0118] The short-time energy of each frame signal is calculated, that is, the sum of the squares of the signal samples, which is used to evaluate the strength of the signal. The formula for calculating the short-time energy E of each frame is as follows:

[0119] E=Σ(x[n]^2), n=0, 1,..., N-1,

[0120] By calculating the difference between the sign functions of adjacent sample points and taking the absolute value, the zero crossing rate ZCR of each frame signal is calculated:

[0121] ZCR=(1 / 2)*Σ|sgn(x[n])-sgn(x[n-1])|, n=1, 2,..., N-1,

[0122] Among them, sgn() is the sign function, x[n] is the signal sample, and N is the frame length.

[0123] The code example is as follows:

[0124] if(E>E_threshold&&ZCR>ZCR_threshold)

[0125] frameisactive

[0126] else

[0127] frameisinactive.

[0128] Those skilled in the art can understand that if the short-time energy E of a frame is greater than the preset energy threshold Ethreshold and the zero-crossing rate ZCR of the frame is greater than the preset zero-crossing rate threshold ZCRthreshold, the frame is considered to be active, i.e., contains speech. Otherwise, the frame is considered to be inactive, i.e., does not contain speech.

[0129] S4: locating the arrival time of the second test signal based on the peak rate search algorithm, and identifying the time difference between the second test signal being sent and being detected as the delay time of the subwoofer; specifically comprising the following steps:

[0130] S41: Play the second test signal, where the second test signal is a swept frequency sine wave signal in the range of 20-200 Hz, and when implemented, the duration is 2-5 seconds;

[0131] S42: locating the second test signal based on a peak rate search algorithm, preprocessing the collected signal to remove noise and filter the signal, using a median filter to remove impulse noise, applying a bandpass filter, and retaining the signal within a target frequency range;

[0132] S43: Set a threshold for identifying valid peaks: threshold = μ+k*σ, where μ is the signal mean, σ is the standard deviation, and k is the adjustment coefficient, usually 2-3; use sliding window technology to scan the signal, set the window size W to 1 / 4 to 1 / 2 of the signal period, for each sampling point i, if x[i]>threshold and x[i]=max(x[iW / 2:i+W / 2]), then mark it as a potential peak, x[i] represents the value of the i-th sampling point of the signal; use parabolic interpolation to accurately locate the peak position, for each potential peak point i, query the left and right adjacent points i-1 and i+1 of the peak point i, and fit a parabola based on the adjacent points and the potential peak point, where a, b and c are the coefficients of the polynomial, x is the independent variable (usually representing time or spatial position), and y is the dependent variable (such as the strength or height of the signal). The vertex of the parabola (i.e., the maximum or minimum point) can be obtained by taking the derivative and setting it to zero to obtain the peak position: x_peak = -b / (2a), where the x coordinate of the vertex of the parabola is the peak position x_peak. In order to quickly calculate the peak position without solving the quadratic equation, the following simplified formula can be used: x_peak = i + (x[i-1] - x[i+1]) / (2*(x[i-1] - 2x[i] + x[i+1])), which is derived by considering the symmetry of the parabola and the offset of the vertex position relative to the center point. Here, i is the index of the center point, and x[i-1] and x[i+1] are the x values ​​of the left and right adjacent points, respectively. In this way, by calculating the time interval between adjacent peaks, the arrival time of the second test signal is identified. If the interval is close to the expected signal period, the first valid peak is confirmed to be the signal arrival time. Use median filtering to remove outliers and improve the stability of the results. Finally, record the time difference from the subwoofer test signal being emitted to being detected, which is the delay time of the subwoofer.

[0133] S5: Calibrate the delay time of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjust the delay calibration parameters of each channel according to the delay time calibration amount, so as to achieve accurate time alignment between the subwoofer and the main channel. Specifically, the steps are as follows:

[0134] Select the channel with the longest delay time as the reference channel, calculate the delay difference of other channels relative to the reference channel except the reference channel, and perform the first delay adjustment according to the delay difference; during implementation: assuming that the right front channel has the longest delay time of 30ms, and the left front channel delay time is 25ms, then the calibration delay of the left front channel is 5ms.

[0135] This embodiment performs special processing on the delay of the subwoofer: Since the wavelength of the low-frequency signal is long, the phase alignment problem needs to be considered. The phase difference between the subwoofer and the main channel is calculated, and appropriate delay adjustment is performed. The detailed description is as follows:

[0136] Calculate the phase difference between the subwoofer and the main channel: Δφ = φ_sub - φ_main;

[0137] Convert the calculated phase difference into time delay: Δt = Δφ / (2π*f_crossover);

[0138] A second delay adjustment is performed according to the time delay to ensure a smooth phase transition between the subwoofer and the main channel near the crossover point; wherein Δφ represents the phase difference, φ_sub represents the phase of the subwoofer, φ_main represents the phase of the main channel, Δt represents the time delay, and f_crossover is the crossover point frequency.

[0139] In this embodiment, the real-time adaptive calibration of the system includes: dynamically adjusting parameters according to different room acoustic environments based on the adaptive calibration mechanism, and performing periodic rapid calibration checks when the system is running. In this embodiment, the system automatically performs a rapid calibration check every 30 minutes or after the user adjusts the speaker position. It further includes the following steps:

[0140] Use the background music or ambient sound being played as the calibration signal to avoid interrupting the user's listening experience, and use blind source separation technology to extract the signal characteristics of each channel from the mixed signal;

[0141] The ICA algorithm is used to separate the multi-channel mixed signal: X = AS, where X is the observed mixed signal, A is the mixing matrix, and S is the source signal;

[0142] The FastICA algorithm is used to estimate the separation matrix W and iteratively optimize the parameters until convergence, where S ≈ WX;

[0143] FastICA iterative process: 1. Initialize w as a random unit vector, 2. w+ = E{xg(w^Tx)}-E{g'(w^Tx)}w, 3. w = w+ / ||w+||, 4. If not converged, return to step 2, where g() is a nonlinear function, such as g(u) = tanh(au).

[0144] After obtaining the separated channel signals, time delay estimation is performed, and delay calibration parameters are updated according to the estimation results;

[0145] If a change in the delay calibration parameters is detected, the user is prompted whether a full calibration process is required.

[0146] In this embodiment, the steps of user interaction and optimization of the system include: 1. Providing a user interface to allow the user to manually fine-tune the calibration result. Embodiment: Design an intuitive graphical interface, and the user can adjust the relative delay of each channel through a slider. 2. According to the user's adjustment habits, the automatic calibration process is optimized using a machine learning algorithm, and a reinforcement learning method is adopted to use the user's adjustment behavior as feedback to continuously optimize the parameters of the calibration algorithm, that is, according to the user's delay adjustment behavior feedback, the automatic calibration process is further optimized using machine learning, including the following steps: define the state space S as the current delay parameter setting; define the action space A as the possible delay adjustment operation; define the reward function R, and give positive and negative rewards according to whether the user accepts the automatic calibration result or makes manual adjustments; use the Q-learning algorithm to update the value function: Q(s, a)←Q(s, a)+α[r+γ*max(Q(s', a'))-Q(s, a)]; select the optimal calibration strategy according to the updated Q value; where α is the learning rate, γ is the discount factor, s and a are the current state and action respectively, and s' is the next state. 3. Provide A / B testing function to allow users to select a better calibration effect. Example: The system generates two sets of slightly different calibration parameters. The user selects the better effect through a blind listening test, and the system records and learns this choice.

[0147] In this embodiment, the steps of the system performing multi-room and multi-device collaboration include:

[0148] For smart speaker systems in multiple rooms, a room acoustic characteristic database is established. The acoustic characteristics include room size, reverberation time and frequency response. An embodiment: an acoustic fingerprint is created for each room, including parameters such as room size, reverberation time, frequency response, etc., and a machine learning algorithm (such as support vector machine SVM) is used to classify the rooms so as to quickly call the most suitable initial calibration parameters.

[0149] Based on the distributed consistency algorithm, the Raft consensus algorithm, version vector and anti-entropy process are used to synchronize the calibration data of multiple devices, thereby ensuring the collaborative work of multiple rooms and multiple devices. The specific implementation steps are as follows: a) Use the Raft consensus algorithm to elect the master node: All nodes are initially in the Follower state. If they do not receive the heartbeat message after the timeout, they will switch to the Candidate state and initiate an election. The Candidate that obtains the majority of votes becomes the Leader. The Leader regularly sends heartbeat messages to the Follower to maintain its leadership position. b) The master node is responsible for coordinating the calibration process and data updates: The Leader receives the client request, copies the log to the Follower, waits for the majority of Followers to confirm, submits the log and applies it to the state machine, and notifies the client that the operation is completed. c) Use the version vector (VersionVector) to track the data status of each node: VV = {(Node1, Clock1), (Node2, Clock2), ...}, each node maintains its own logical clock, which is incremented when the data is updated. d) When data inconsistency is detected, the data is synchronized through the anti-entropy process: nodes regularly exchange version vectors, compare version vectors, identify data that needs to be synchronized, and transmit differential data to achieve final consistency. Automatically switch to the preset calibration mode according to the user's usage scenario (such as party mode, movie mode, etc.).

[0150] In this embodiment, the system performs advanced audio processing including: integrating multiple preset audio processing algorithms to optimize audio, and the specific implementation steps are as follows:

[0151] The first method is to achieve room response correction by compensating the room acoustics on the frequency response through an adaptive filter, that is, combining room correction technology to compensate for the influence of room acoustics on the frequency response. When using an adaptive filter to achieve room response correction: a) Measure the impulse response h(n) of the room: Use a maximum length sequence (MLS) or a swept sine signal as excitation, calculate the cross-correlation function Rxy(τ)=Σx(n)y(n+τ) of the input signal x(n) and the output signal y(n), and the room impulse response h(n)≈Rxy(n) / Rxx(0). b) Design the target response d(n), which is usually a flat frequency response. c) Use the minimum mean square error (LMS) algorithm to iteratively update the filter coefficients w(n): e(n)=d(n)-w^T(n)x(n), w(n+1)=w(n)+2μe(n)x(n), where μ is the step size, e(n) is the error signal, and x(n) is the input signal. d) Combine correction filters with delay calibration for comprehensive audio optimization.

[0152] The second method is to configure a multi-threshold compressor that applies different compression ratios at different volumes to ensure the listening experience at low and high volumes, that is, to achieve dynamic range compression and expansion, and optimize the listening experience at different volumes. Example: Design a multi-threshold compressor that applies different compression ratios at different signal levels to ensure clear details at low volumes and rich dynamics at high volumes.

[0153] Implementation instructions: a) Define multiple threshold levels T1 < T2 <... < Tn. b) For the input signal x(n), calculate its level L = 20 * log10(|x(n)|). c) Select the corresponding compression ratio R according to the level L: If L ≤ T1, R = R1; if T1 < L ≤ T2, R = R2... if L > Tn, R = Rn. d) Calculate the gain control signal: G(n) = T + (L - T) / R when L > T, and G(n) = L when L ≤ T. e) Apply the gain: y(n) = x(n) * 10^((G(n) - L) / 20).

[0154] The third method is to consider the masking effect and critical bandwidth characteristics of the human ear, fine-tune the calculated delay time, introduce a psychoacoustic model, and adjust the delay of the calculated delay time according to the masking effect and critical bandwidth characteristics of the human ear. Perform a short-time Fourier transform (STFT) on the acquired signal to obtain the time-frequency spectrum X(k, m), and calculate the energy of each frequency band k: E(k) = Σ|X(k, m)|^2, where m = 1, 2,..., M. Weight the delay sensitivity of different frequency bands according to a preset loudness curve standard such as the ISO226:2003 standard, apply the equal-loudness filter H(k) to each frequency band E'(k) = E(k) * H(k), and adjust the delay time within the critical bandwidth to minimize the phase difference. Here, M represents the window function length or the number of frames of the short-time Fourier transform. For each critical band: Calculate the average phase φavg within the band, and adjust the delay so that the phases of all frequency components within the band are close to φavg, Δt(k) = (φavg - φ(k)) / (2πf(k)).

[0155] In this embodiment, the steps for the system to perform performance evaluation and optimization include:

[0156] Step 1: Design objective evaluation indicators and regularly detect the system calibration effect. Example: Use the cross-correlation function to evaluate the time alignment accuracy:

[0157] Calculate the cross-correlation function of the reference signal x(n) and the output signal y(n) of each channel:

[0158] R_xy(τ) = Σx(n)y(n + τ), where n = 0, 1,..., N - 1,

[0159] Find the peak position of the cross-correlation function, and the τ value corresponding to this position is the time offset:

[0160] τmax=argmax(R_xy(τ));

[0161] Compare the calculated actual time offset with the theoretical offset to assess the calibration accuracy:

[0162] Error = |τmax-τtheory|;

[0163] Where N represents the length of the signal, that is, the number of sampling points, τmax is the actual time offset, and τtheory is the theoretical offset.

[0164] Step 2: Implement an automated testing process to simulate different usage scenarios. This embodiment uses the Monte Carlo method to generate a large number of test cases: a) Define the input parameter space, such as room size, speaker position, listening position, etc. b) Use the Latin Hypercube Sampling method to uniformly sample in the parameter space: divide the range of each parameter into N equal parts, randomly select N non-repeating segments in each dimension, and combine these selections to form N sample points. c) Perform simulation tests on each set of sampling parameters to evaluate the performance of the system under different conditions. d) Statistically analyze the test results to identify the performance bottlenecks and improvement directions of the system: calculate the mean and variance of each performance indicator, perform sensitivity analysis, and identify key influencing factors.

[0165] Step 3: Establish a performance database to track the long-term performance of the system. Example: Design a time series database to record the results of each calibration and user feedback. Use time series analysis methods (such as ARIMA model) to predict system performance trends and promptly identify and resolve potential problems. Among them, the algorithm used is described as follows: ARIMA (p, d, q) model: a) Perform d-order differences on the original time series to make it a stationary series. b) Fit AR (p) and MA (q) models:

[0166] y(t)=c+φ1y(t-1)+...+φpy(tp)+θ1ε(t-1)+...+θqε(tq)+ε(t).

[0167] c) Estimate parameters using maximum likelihood estimation or conditional least squares. d) Perform model diagnosis and use it for prediction if it is appropriate, otherwise adjust p, d, q values ​​and re-model.

[0168] In this implementation, the system also integrates advanced user functions, which are described in detail as follows: 1. Provide professional mode to allow audio enthusiasts to make more refined adjustments. Example: Open low-level parameter access rights, such as filter coefficients, dynamic range processing parameters, etc. Provide real-time spectrum analysis and phase response visualization tools to assist users in professional-level debugging. 2. Implement personalized audio profiles. Application algorithm description: Use collaborative filtering algorithm to recommend audio settings for users: a) Construct a user-configuration matrix to record each user's preference for different audio configurations. b) Use singular value decomposition (SVD) to reduce the matrix dimension and extract potential features. R≈U*Σ*V^T, where R is the original rating matrix, U and V are the feature matrices of users and items respectively, and Σ is the singular value matrix. c) Based on user similarity, predict the target user's possible preference for untried configurations. For user u and item i: predicted score = μ+bu+bi+Σ(pu, k*qi, k), k=1, 2,..., K, where μ is the global average score, bu and bi are user and item biases, and pu and qi are feature vectors of users and items. d) Recommend the highest-scoring audio configuration to the user. 3. Integrate music streaming services to automatically optimize audio settings based on different music types. Example: Access the streaming platform through the API to obtain metadata of the song being played. Establish a mapping relationship between music types and optimal audio parameters to achieve seamless sound quality optimization.

[0169] In this implementation, the system also integrates security and privacy protection functions, which are described in detail as follows: 1. Implement end-to-end encryption to protect the user's audio data and configuration information. Application algorithm description: Use the AES-256 encryption algorithm to encrypt sensitive data stored locally: a) Generate a random 256-bit key. b) Use the PBKDF2 (Password-BasedKeyDerivationFunction2) algorithm to derive the encryption key from the user password. DK = PBKDF2 (Password, Salt, c, dkLen), where c is the number of iterations (usually ≥ 10000) and dkLen is the desired key length. c) Use the GCM (Galois / CounterMode) mode of the AES-256 algorithm for encryption to ensure the confidentiality and integrity of the data. C = AES-GCM-Encrypt (K, IV, P, A), where K is the key, IV is the initialization vector, P is the plaintext, and A is the additional authentication data. 2. Design an anonymization mechanism to protect user privacy when uploading system performance data. Embodiment: Use differential privacy technology to process uploaded data: a) Add Laplace noise to the original data: f'(x) = f(x) + Lap(Δf / ε), where f(x) is a function or query result of the original data, f'(x) is the data after adding noise, Δf is the sensitivity, and ε is the privacy budget. b) Use k-anonymization technology to generalize user features to ensure that each feature combination has at least k users. 3. Implement a secure firmware update mechanism to prevent malware intrusion. Application algorithm description: Use digital signature and secure boot (SecureBoot) mechanism: a) Use the manufacturer's private key to sign the firmware. Signature = Sign(PrivateKey, Hash(Firmware)). b) When the device starts, use the pre-stored public key to verify the firmware signature. isValid = Verify(PublicKey, Signature, Hash(Firmware)). c) Execute the firmware only after the signature verification is passed to ensure system integrity.

[0170] Through the above detailed technical solutions, the present invention realizes the automatic calibration of the delay of the smart speaker system, which greatly improves the sound restoration and user experience of the multi-channel audio system. This solution not only solves the problem of precise time alignment between the subwoofer and other channels, but also takes into account the influence of the room acoustic environment and realizes adaptive dynamic optimization. The modular design and open API of the system ensure good scalability and compatibility, while the continuous optimization mechanism ensures the long-term stability and improvement of system performance. This innovative solution has brought new technological breakthroughs to the field of smart speakers and is expected to be widely used in the market.

[0171] Embodiment 2

[0172] Based on the same inventive concept, this embodiment provides a delay automatic calibration device for an intelligent speaker system, comprising:

[0173] An initialization module is used to initialize the smart speaker system and detect the environment to preliminarily configure system parameters;

[0174] A test module, used to stimulate a target channel and play a target test signal, wherein the target test signal includes a first test signal for testing a main channel and a second test signal for testing a subwoofer;

[0175] A main channel module, used to locate the first test signal based on a VAD voice activity detection algorithm and determine the start and end of the first test signal, identify the time difference from when the first test signal is sent to when it is detected, and obtain a delay time of the main channel;

[0176] A subwoofer module, configured to locate the arrival time of the second test signal based on a peak rate search algorithm, and identify the time difference between the second test signal being emitted and being detected as a delay time of the subwoofer;

[0177] The calibration module is used to calibrate the delay time of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjust the delay calibration parameters of each channel according to the delay time calibration amount, so as to achieve accurate time alignment between the subwoofer and the main channel. The principle is shown in Example 1 and will not be repeated here.

[0178] Based on the same inventive concept, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the computer program is executed by the processor, the steps of the automatic delay calibration method of the smart speaker system described in any one of the above items are implemented.

[0179] Based on the same inventive concept, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the automatic delay calibration method of the smart speaker system described in any one of the above items are implemented.

[0180] The computer device in this embodiment is described in detail from the perspective of hardware processing.

[0181] See also Figure 3 As shown, the computer device includes a processor 100 and a memory 101, wherein the memory 101 stores machine executable instructions that can be executed by the processor 100, and the processor 100 executes the machine executable instructions to implement the automatic delay calibration method of the above-mentioned smart speaker system.

[0182] further, Figure 3 The computer device shown further includes a bus 102 and a communication interface 103 , and the processor 100 , the communication interface 103 and the memory 101 are connected via the bus 102 .

[0183] Among them, the memory 101 may include a high-speed random access memory (Random Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 102 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0184] The processor 100 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 100 or an instruction in the form of software. The above processor 100 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and completes the method steps of the above-mentioned embodiment in combination with its hardware.

[0185] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the automatic delay calibration method of the smart speaker system.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0188] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic delay calibration of an intelligent speaker system, characterized in that: include: Initialize the smart speaker system and detect the environment to preliminarily configure system parameters; Exciting a target channel and playing a target test signal, wherein the target test signal includes a first test signal for testing a main channel and a second test signal for testing a subwoofer; Locating the first test signal based on a VAD voice activity detection algorithm and determining the start and end of the first test signal, identifying a time difference between when the first test signal is sent and when it is detected, and obtaining a delay time of the main channel; Locating the arrival time of the second test signal based on a peak rate search algorithm, and identifying the time difference between the second test signal being emitted and being detected as the delay time of the subwoofer; The delay time of the subwoofer and each main channel is calibrated according to the delay time of the main channel and the delay time of the subwoofer, and the delay calibration parameters of each channel are dynamically adjusted according to the delay time calibration amount, so as to achieve precise time alignment between the subwoofer and the main channel.

2. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Initializing the smart speaker system and detecting the environment to preliminarily configure system parameters includes: Start the smart speaker system and enter calibration mode; Play white noise or a frequency sweep signal containing various frequencies and collect the room's acoustic response through a microphone array; Using fast Fourier transform to analyze the acoustic characteristics of the collected acoustic signal to obtain the room environment analysis result, which includes a frequency response curve and a reverberation time parameter; The system parameters are preliminarily configured according to the environmental analysis results. The preliminarily configured operations include, when it is detected that the reverberation time of the room is greater than the reverberation time threshold, correspondingly increasing the time window length to capture longer sound clips for subsequent processing.

3. The automatic delay calibration method of the smart speaker system according to claim 2, characterized in that: The method of analyzing the acoustic characteristics of the collected signal by using fast Fourier transform includes: Perform FFT fast Fourier transform on the collected acoustic signal x(n), convert the collected time domain signal into a frequency domain signal, and obtain the energy distribution of the signal at each frequency. The FFT implementation formula is as follows: X(k)=Σ[n=0toN-1]x(n)*e^(-j2πkn / N); Calculate the power spectral density: P(k) = |X(k)|^2 / N; The energy attenuation curve is calculated using the Schroeder integral method to obtain the energy attenuation of the room at different frequencies. The target linear region is extracted from the energy attenuation curve, and a linear fit is performed on the target linear region. The time required for the energy to decay by 60 dB is calculated based on the fitted straight line to estimate the reverberation time T60. Where X(k) represents the output frequency domain signal, N is the number of sampling points, k is the frequency index, |X(k)| represents the amplitude of the kth frequency component, P(k) represents the power spectrum density of the kth frequency component, and the target linear region ranges from -5dB to -35dB.

4. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: The VAD-based voice activity detection algorithm is used to locate the first test signal and determine the start and end of the first test signal, identify the time difference between the first test signal being sent and being detected, and obtain the delay time of the main channel, including: Excite each main channel in the smart speaker system in turn, and play the first test signal, where the first test signal includes a target pulse signal or a sine wave envelope signal of a preset frequency, and the preset frequency range covers 200 Hz-20 kHz; Dividing the input first test signal into a plurality of short frames according to a preset length; Acquire the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames, and compare the acquired short-time energy and zero-crossing rate with a preset energy threshold and a preset zero-crossing rate threshold to determine whether the frame signal is a voice activity; If both the short-time energy and the zero-crossing rate exceed the corresponding thresholds, the result of the judgment is that the frame signal is a voice activity, thereby determining the start time of the test signal; If the short-time energy and zero-crossing rate do not exceed the corresponding thresholds, the judgment result is that the frame signal is non-speech activity; If the judgment result is a misjudgment result, the misjudgment result is eliminated by a smoothing algorithm.

5. The automatic delay calibration method of the smart speaker system according to claim 4, characterized in that: The method of obtaining the short-time energy and zero-crossing rate characteristics of each frame signal according to the divided short frames includes: The short-time energy of each frame signal is calculated, that is, the sum of the squares of the signal samples, which is used to evaluate the strength of the signal. The formula for calculating the short-time energy E of each frame is as follows: E=Σ(x[n]^2), n=0, 1,..., N-1, By calculating the difference between the sign functions of adjacent sample points and taking the absolute value, the zero crossing rate ZCR of each frame signal is calculated: ZCR=(1 / 2)*Σ|sgn(x[n])-sgn(x[n-1])|, n=1, 2,..., N-1, Among them, sgn() is the sign function, x[n] is the signal sample, and N is the frame length.

6. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: The method of locating the arrival time of the second test signal based on the peak rate search algorithm and identifying the time difference between the second test signal being emitted and being detected as the delay time of the subwoofer includes: Playing the second test signal, where the second test signal is a swept frequency sine wave signal in the range of 20-200 Hz; Locating the second test signal based on a peak rate search algorithm, and preprocessing the collected signal to remove noise and filter the signal; Set a threshold for identifying valid peaks: threshold = μ + k * σ, where μ is the signal mean, σ is the standard deviation, and k is the adjustment coefficient; Use sliding window technology to scan the signal, set the window size W, for each sampling point i, if x[i]>threshold and x[i]=max(x[iW / 2:i+W / 2]), then mark it as a potential peak, x[i] represents the value of the i-th sampling point of the signal; Use parabola interpolation to accurately locate the peak position. For each potential peak point i, query the adjacent points i-1 and i+1 on the left and right of the peak point i, and fit a parabola based on the adjacent points and the potential peak point. The arrival time of the second test signal is identified by calculating the time interval between adjacent peaks. If the interval is close to the expected signal period, the first valid peak is confirmed to be the signal arrival time.

7. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: The delay time calibration of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjusting the delay calibration parameters of each channel according to the delay time calibration amount includes: Selecting a channel with the longest delay time as a reference channel, calculating delay differences of other channels except the reference channel relative to the reference channel, and performing a first delay adjustment according to the delay differences; Calculate the phase difference between the subwoofer and the main channel: Δφ = φ_sub - φ_main; Convert the calculated phase difference into time delay: Δt = Δφ / (2π*f_crossover); Performing a second delay adjustment according to the time delay, thereby ensuring a smooth phase transition between the subwoofer and the main channel near the crossover point; Among them, Δφ represents the phase difference, φ_sub represents the phase of the subwoofer, φ_main represents the phase of the main channel, Δt represents the time delay, and f_crossover is the crossover frequency.

8. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Also includes: Based on the adaptive calibration mechanism, the parameters are dynamically adjusted according to the acoustic environment of different rooms, and periodic fast calibration checks are performed while the system is running, further including: The background music or ambient sound being played is used as the calibration signal, and the signal characteristics of each channel are extracted from the mixed signal through blind source separation technology; The ICA algorithm is used to separate the multi-channel mixed signal: X = AS, where X is the observed mixed signal, A is the mixing matrix, and S is the source signal; The FastICA algorithm is used to estimate the separation matrix W and iteratively optimize the parameters until convergence, where S ≈ WX; After obtaining the separated channel signals, time delay estimation is performed, and delay calibration parameters are updated according to the estimation results; If a change in the delay calibration parameters is detected, the user is prompted whether a full calibration process is required.

9. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Also includes: The automatic calibration process is further optimized using machine learning based on user latency-adjusted behavioral feedback, including: Define the state space S as the current delay parameter settings; Define the action space A as possible delay adjustment operations; Define a reward function R to give positive or negative rewards based on whether the user accepts the automatic calibration result or makes manual adjustments; Use the Q-learning algorithm to update the value function: Q(s, a)←Q(s, a)+α[r+γ*max(Q(s', a'))-Q(s, a)]; Select the optimal calibration strategy based on the updated Q value; Where α is the learning rate, γ is the discount factor, s and a are the current state and action respectively, and s' is the next state.

10. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Also includes: Building a database of room acoustic characteristics for smart speaker systems in multiple rooms, including room dimensions, reverberation time, and frequency response; Based on the distributed consistency algorithm, the Raft consensus algorithm, version vector and anti-entropy process are used to synchronize the calibration data of multiple devices, thus ensuring the collaborative work of multiple rooms and multiple devices.

11. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Also includes: Integrate multiple preset audio processing algorithms to optimize audio, further including: Room response correction is achieved by using adaptive filters to compensate for the room acoustics to the frequency response; Configure a multi-threshold compressor to apply different compression ratios at different volumes to ensure the listening experience of low and high volumes; A psychoacoustic model is introduced, and the calculated delay time is delayed and adjusted according to the masking effect of the human ear and the critical bandwidth characteristics. The acquired signal is subjected to STFT short-time Fourier transform to obtain the time-frequency spectrum X(k, m), and the energy of each frequency band k is calculated: E(k)=Σ|X(k, m)|^2, m=1, 2, ..., M. The delay sensitivity of different frequency bands is weighted according to the preset loudness curve standard, and an equal loudness filter H(k) is applied to each frequency band E'(k)=E(k)*H(k). The delay time is delayed and adjusted within the critical bandwidth to minimize the phase difference, where M represents the window function length or the number of frames of the short-time Fourier transform.

12. The automatic delay calibration method of the smart speaker system according to claim 1, characterized in that: Also includes: The steps to evaluate and optimize the performance of the smart speaker system include: Calculate the cross-correlation function between the reference signal x(n) and each channel output signal y(n): R_xy(τ)=Σx(n)y(n+τ), n=0, 1,..., N-1, Find the peak position of the cross-correlation function, and the τ value corresponding to this position is the time offset: τmax=argmax(R_xy(τ)); Compare the calculated actual time offset with the theoretical offset to assess the calibration accuracy: Error = |τmax-τtheory|; Where N represents the length of the signal, that is, the number of sampling points, τmax is the actual time offset, and τtheory is the theoretical offset.

13. A delay automatic calibration device for an intelligent speaker system, characterized in that: include: An initialization module is used to initialize the smart speaker system and detect the environment to preliminarily configure system parameters; A test module, used to stimulate a target channel and play a target test signal, wherein the target test signal includes a first test signal for testing a main channel and a second test signal for testing a subwoofer; A main channel module, used to locate the first test signal based on a VAD voice activity detection algorithm and determine the start and end of the first test signal, identify the time difference from when the first test signal is sent to when it is detected, and obtain a delay time of the main channel; A subwoofer module, configured to locate the arrival time of the second test signal based on a peak rate search algorithm, and identify the time difference between the second test signal being emitted and being detected as a delay time of the subwoofer; The calibration module is used to calibrate the delay time of the subwoofer and each main channel according to the delay time of the main channel and the delay time of the subwoofer, and dynamically adjust the delay calibration parameters of each channel according to the delay time calibration amount, so as to achieve accurate time alignment between the subwoofer and the main channel.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the method steps for automatic delay calibration of the smart speaker system as described in any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method steps for automatic delay calibration of the smart speaker system as described in any one of claims 1 to 12 are implemented.

Images