Audio change calculation method and terminal

Abstract

The invention discloses an audio change calculation method and a terminal, and the method comprises the steps: carrying out the filtering of collected music audio, extracting music beats, and facilitating the subsequent adaption of a more accurate light effect for the audio; according to the method, the audio signal is subjected to normalization and windowing processing and converted into the sound pressure level data, and then index prediction and calculation of the change value of the index prediction are performed, so that the change value of the audio can be obtained by processing, converting and predicting the audio signal, the calculation speed is increased, and the calculation cost is reduced.

Description

[0001] This case is a divisional application based on the invention patent with application date of July 20, 2023, application number 202310891726.8, and name "A lighting effect display method and terminal based on music beat" as the parent case. Technical Field

[0002] The present invention relates to the technical field of audio calculation, and in particular to a method and a terminal for calculating audio changes. Background Art

[0003] Light strings are usually used in home decoration, party and festival decorations and other occasions. The current smart LED light strings have built-in controllers and wireless modules that communicate with mobile phones. They can not only adjust the color according to the user's selection on the mobile phone APP, but also change the lighting effect according to the rhythm of the surrounding music signals. The brightness and flashing mode of the LED light body can change and flash according to the rhythm of the music, increasing the user experience and festive atmosphere.

[0004] At present, the smart lighting components on the market have poor noise filtering effects when acquiring audio signals and are easily affected by various types of noise. They simply light up the lights continuously according to the intensity of the music. In continuous high-intensity melodies, the characteristics of being ahead of the beat are not obvious. The lighting effects may appear continuously bright, or jump randomly due to noise and clutter, which can easily lead to a visual feeling of unclear rhythmic effects and a weak correlation between audio rhythm points and lighting effects. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and a terminal for calculating audio changes, which can improve the correlation between music rhythm points and lighting effects.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for calculating audio changes, comprising the steps of:

[0008] Receiving a regularly collected music audio signal, filtering the audio signal to extract the music beat, and converting the audio signal into sound pressure level data;

[0009] Exponential prediction is performed on the sound pressure level data of the audio signal, a quadratic discrete signal of the standardized sound pressure level data is calculated, a cubic exponential prediction is performed based on the quadratic discrete signal, and a change value of the exponential prediction result is calculated by first-order difference.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A computing terminal for audio changes comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned computing method for audio changes is implemented.

[0012] The beneficial effects of the present invention are as follows: the collected music audio is filtered and the music beat is extracted, so as to facilitate the subsequent adaptation of more accurate lighting effects to the audio; the audio signal is normalized and windowed, and converted into sound pressure level data, and then the index prediction and the calculation of its change value are performed, so the change value of the audio can be obtained by processing, converting and predicting the audio signal, thereby improving the calculation speed and reducing the calculation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of a method for calculating audio changes according to an embodiment of the present invention;

[0014] Figure 2 A schematic diagram of a computing terminal for audio changes according to an embodiment of the present invention;

[0015] Figure 3 A module diagram of a lighting effect display method based on music beat according to an embodiment of the present invention;

[0016] Figure 4 A flowchart of specific steps of a lighting effect display method based on music beat according to an embodiment of the present invention;

[0017] Figure 5 is a loudness function curve diagram of an embodiment of the present invention;

[0018] Figure 6 is a peak threshold diagram of an embodiment of the present invention;

[0019] Figure 7 is a graph of an original audio signal according to an embodiment of the present invention;

[0020] Figure 8 is a graph of a normalized audio signal according to an embodiment of the present invention;

[0021] Fig. 9 is a graph of a normalized and windowed audio signal according to an embodiment of the present invention;

[0022] Fig.10 is a curve diagram of sound pressure level data and sensitivity threshold value according to an embodiment of the present invention;

[0023] Fig.11 It is a curve diagram of the index prediction result value and the peak threshold value according to an embodiment of the present invention.

[0024] Description of labels:

[0025] 1. A computing terminal for audio changes; 2. A memory; 3. A processor. DETAILED DESCRIPTION

[0026] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0027] The embodiment of the present invention provides a lighting effect display method based on music beat, comprising the steps of:

[0028] Receive the audio signal of the music collected at a fixed time, filter the audio signal to extract the music beat, normalize and window the audio signal, and convert the audio signal into sound pressure level data;

[0029] An exponential prediction is performed on the sound pressure level data, and a change value of the exponential prediction result is calculated by first-order difference. When the change value reaches a trigger condition, a dynamically matched lighting effect is output in combination with the music beat.

[0030] From the above description, it can be seen that the beneficial effects of the present invention are: filtering the collected music audio and extracting the music beat, so as to facilitate the subsequent adaptation of more accurate lighting effects to the audio; normalizing and windowing the audio signal and converting it into sound pressure level data, and then performing index prediction and calculation of its change value, so that the change value of the audio can be obtained by processing, converting and predicting the audio signal, which improves the calculation speed and reduces the calculation cost. When the change value reaches the trigger condition, the dynamic matching lighting effect is output in combination with the music beat, which can increase the correlation between the music beat and the lighting effect and improve the lighting rhythm experience.

[0031] Furthermore, normalizing the audio signal includes:

[0032] Subtract the average value of the elements of each frame from the individual element values ​​of each frame in the audio signal:

[0033]

[0034] x'[i]=x v0 [i]-x ave [t];

[0035] In the formula, t represents a frame of data in each t-time period, m represents the number of data points of the elements collected in each frame of data, 1≤i≤m, x v0 [i] represents the i-th element value of each frame, x ave [t] represents the element average of a frame of data in every t period, and x'[i] represents the normalized audio signal.

[0036] From the above description, it can be seen that the audio signal is normalized to ensure that the mean value of each frame is 0, which is convenient for subsequent data processing.

[0037] Further, windowing the audio signal includes:

[0038] Add the Hamming window function to each frame of the normalized audio signal and multiply the normalized audio signal by the window function:

[0039]

[0040] x v [i] = x'[i] × ω(i);

[0041] Where ω(i) represents the window function, x v [i] represents the windowed audio signal.

[0042] It can be seen from the above description that processing such as framing and windowing of audio signals can prevent spectrum leakage.

[0043] Furthermore, converting the audio signal into sound pressure level data includes:

[0044] Calculate the sound pressure level data x of the processed audio signal spl [t]:

[0045]

[0046]

[0047] In the formula, x v [i] represents the normalized and windowed audio signal, P represents the sound pressure to be measured, and P0 represents the reference sound pressure;

[0048] The sound pressure level data are normalized:

[0049] x' spl [t] = Mx spl [t] / D peak ;

[0050] In the formula, x spl '[t] represents the standardized sound pressure level data, M represents the standardized preset upper limit value, and Dpeak represents the preset relative silence value.

[0051] From the above description, we can see that converting audio signals into sound pressure levels can more intuitively show the changes in sound compared to the usual sound intensity level data. In reality, differences in distance, environment, etc. will lead to unstable volume, which will affect speech recognition analysis. Therefore, volume standardization is performed to facilitate subsequent prediction analysis.

[0052] Furthermore, the sound pressure level data of the audio signal is exponentially predicted:

[0053] Calculate the quadratic discrete signal of the normalized sound pressure level data: t [t] = x spl ' 2 [t];

[0054] Perform cubic exponential prediction based on the quadratic discrete signal:

[0055] Set the initial value of an exponential smoothing forecast to t gain [0] = 1e-1, the initial value of the quadratic exponential smoothing forecast is t smooth [0] = 1e-1, the initial value of the triple exponential smoothing forecast is t sthrength [0] = 0.01, through iteration, we get the exponential smoothing forecast value t for the t+1 period. gain [t+1], quadratic exponential smoothing forecast value t smooth [t+1], triple exponential smoothing forecast value t sthrength [t+1]:

[0056]

[0057] x gain [t] = x t [t] / t[t+1];

[0058]

[0059] x smooth [t] = t smooth [t] 2 ;

[0060]

[0061] x sthrength [t] = x smooth [t] / t sthrength [t+1];

[0062] In the formula, represents the smoothing coefficient, x sthrength [t] represents the index prediction result value.

[0063] From the above description, we can see that the audio signal is a set of data collected in chronological order, which has a stable development and change trend and law in a short period of time. Therefore, the time series prediction method can be considered to predict future changes, and the use of cubic exponential smoothing prediction has a higher prediction effect on data with macro-periodicity.

[0064] Further, calculating the change value of the audio signal by first-order difference includes:

[0065] Take the first-order difference between the index forecast result value of the current period and the index forecast result value of the previous period to obtain the change value

[0066]

[0067] From the above description, we can see that the first-order difference can eliminate some fluctuations, make the data stable, more intuitively reflect the change effect of the peak value, and is more conducive to peak threshold filtering.

[0068] Furthermore, the lighting effect of outputting dynamic matching in combination with the music beat includes:

[0069] The lighting change range A is calculated according to the exponential prediction value of the sound pressure level data of the audio signal:

[0070] L max =x sthrength [t]×100;

[0071] A=L max -Y;

[0072]

[0073] Where, L max Indicates the highest brightness reached by the lighting rhythm, x sthrength [t] represents the exponential prediction result value of the sound pressure level data, Y represents the brightness maintained at the end of each beat in the music, A represents the light change range corresponding to each beat, ω represents the light change response speed corresponding to each beat, and τ represents the τ time of the change process after the start of the tth frame data.

[0074] From the above description, it can be seen that the lighting effects that are dynamically matched to the music beat output can be changed according to the changes in each frame. Different lighting effects can be transformed through parameters such as amplitude, speed, and bottom brightness, thereby increasing the correlation between the music beat and the lighting effects.

[0075] Please refer to Figure 2 Another embodiment of the present invention provides an audio change calculation terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned audio change calculation method when executing the computer program.

[0076] The above-mentioned audio change calculation method and terminal of the present invention are suitable for improving the correlation between music rhythm points and lighting effects when LED lights are displayed. The following is an explanation through specific implementation methods:

[0077] Embodiment 1

[0078] Please refer to Figure 1 , Figure 3 and Figure 4 , a lighting effect display method based on music beat, comprising the steps of:

[0079] S1. Receive a music audio signal collected at a fixed time, filter the audio signal to extract the music beat, normalize and window the audio signal, and convert the audio signal into sound pressure level data.

[0080] In this embodiment, the audio signal of the music can be collected regularly by the audio collection device module, and the audio signal can be processed by the audio signal processing module.

[0081] An audio signal needs to go through signal processing methods, that is, framing, windowing, filtering, and other additional calculations to extract audio features, in order to obtain the most critical information and remove redundant information.

[0082] S11. Consider a mean function approximated by a Gaussian function for filter design to extract music beats.

[0083] For details, please refer to Figure 7 , audio signals are usually continuous signals, and the signal of a piece of music needs to be divided into several frames for analysis. The key to the melody of music lies in the beat, which is mainly generated by bass equipment. The low-frequency component is an important component of the music signal. The low-pass filter based on the Gaussian function can extract the beat very well, but the Gaussian function is difficult to implement in the filter and the cost is high. Therefore, a mean function approximated by a Gaussian function is considered for filter design to extract the music beat. The mean function can be x ave [t],x ave [t] represents the element average of a frame of data in every t period of time.

[0084] The interference period of 50HZ frequency signal is 20ms. In order to suppress 50HZ interference, sampling is adopted every 20ms, and 256 points of raw data are sampled at a time. The music audio signal is received through the module ADC detection.

[0085] S12: Normalize the audio signal.

[0086] For details, please refer to Figure 8 , normalize the data sampled each time by the mean, and subtract the element average of each frame from the element values ​​of each frame in the audio signal to ensure that the mean of each frame is 0:

[0087]

[0088] x'[i]=x v0 [i]-x ave [t];

[0089] In the formula, t represents a frame of data in each t-time period, m represents the number of data points of the elements collected in each frame of data, 1≤i≤m, x v0 [i] represents the i-th element value of each frame, x ave [t] represents the element average of a frame of data in every t period, and x'[i] represents the normalized audio signal.

[0090] S13, adding a window to the audio signal.

[0091] For details, please refer to Fig. 9 In order to prevent spectrum leakage, a Hamming window function is added to each frame of normalized audio signal, and the normalized audio signal is multiplied by the window function, which is equivalent to convolution in the frequency domain, to obtain a better sidelobe reduction amplitude:

[0092]

[0093] x v [i] = x'[i] × ω(i);

[0094] Where ω(i) represents the window function, x v [i] represents the windowed audio signal.

[0095] S14: Convert the audio signal into sound pressure level data.

[0096] For details, please refer to Fig.10 , calculate the sound pressure level data x of the processed audio signal spl [t]:

[0097]

[0098]

[0099] In the formula, x v [i] represents the normalized and windowed audio signal, P represents the sound pressure to be measured, that is, the root mean square value of the time domain sampling point, P0 represents the reference sound pressure, and the maximum voltage value is 4095;

[0100] The sound pressure level data is peak normalized so that the data is within the range of 0-1 and the data indicators are consistent. Subtraction is used for consistency without changing the dispersion of the data, which is relatively stable:

[0101] x' spl [t] = Mx spl [t] / D peak ;

[0102] In the formula, x spl '[t] represents the standardized sound pressure level data, M represents the standardized preset upper limit value (M=1), D peak Indicates the preset relative silence value.

[0103] S2. Perform exponential prediction on the sound pressure level data, and calculate the change value of the exponential prediction result by first-order difference. When the change value reaches the trigger condition, a dynamically matched lighting effect is output in combination with the music beat.

[0104] In this embodiment, the LED light effect matching module can be used to determine whether to generate light according to the current audio signal, and the LED driving display light effect module can be used to output a light effect that dynamically matches the audio signal.

[0105] S21. Perform exponential prediction on the sound pressure level data of the audio signal.

[0106] Specifically, the audio signal is a set of data collected in chronological order. It has a stable development trend and regularity in a short period of time. Therefore, the time series prediction method can be considered to predict future changes. The triple exponential smoothing prediction has a higher prediction effect on macro-periodic data. Considering comprehensively, we use the triple exponential prediction and the smoothing coefficient In the prediction calculation formula, the actual value is obtained by comparing the predicted value with the actual collection value for multiple simulations to obtain the optimal smoothing coefficient for each stage and dynamically adjust the calculation.

[0107] In this embodiment, the quadratic discrete signal of the standardized sound pressure level data is: t [t] = x spl ' 2 [t];

[0108] Perform cubic exponential prediction based on the quadratic discrete signal:

[0109] Set the initial value of an exponential smoothing forecast to t gain [0] = 1e-1, the initial value of the quadratic exponential smoothing forecast is t smooth [0] = 1e-1, the initial value of the triple exponential smoothing forecast is t sthrength [0] = 0.01, through iteration, we get the exponential smoothing forecast value t for the t+1 period. gain [t+1], quadratic exponential smoothing forecast value t smooth [t+1], triple exponential smoothing forecast value t sthrength [t+1]:

[0110]

[0111] xgain [t] = x t [t] / t[t+1];

[0112]

[0113] x smooth [t] = t smooth [t] 2 ;

[0114]

[0115] x sthrength [t] = x smooth [t] / t sthrength [t+1];

[0116] In the formula, represents the smoothing coefficient, x sthrength [t] represents the index prediction result value.

[0117] S22. Calculate the change value of the audio signal through first-order difference.

[0118] Specifically, the index forecast result value of the current period is taken into account by taking the first-order difference with the index forecast result value of the previous period to obtain the change value. The range is [-1.0, +1.0]. When it is greater than 0, it indicates the music beat, and different thresholds are set for different music types:

[0119]

[0120] Finally, according to the multi-dimensional judgment threshold, the hardware lights are driven to turn off and display the lighting effects. For different music types, the dynamic range is different. The LRA (Loudness Range) calculation method is based on the loudness changes at different time points in a track, and then statistically calculated. Please refer to Figure 6 , pop music, rock music, and classical music have different dynamic range performances. Based on the experimental debugging experience data, we defined pop music, classical music, rock music, and jazz music. The change threshold is dynamically adjusted in the range of [0-1.0]. The specific value is positively correlated with the dynamic range. Figure 5 The loudness curve gives a nonlinear relationship to the threshold of the music type.

[0121] Please refer to Fig.10 and Fig.11The sensitivity sensi_threshold can be selected by the user according to the distance of the use scenario and the volume of the sound source. The general range of 1%-100% sensitivity corresponds to the range of -50db and -10db. The range of different microphone models will be different:

[0122]

[0123] S23, calculating the lighting change range A according to the exponential prediction value of the sound pressure level data of the audio signal:

[0124] L max =x sthrength [t]×100;

[0125] A=L max -Y;

[0126]

[0127] Where, L max Indicates the highest brightness reached by the lighting rhythm, x sthrength [t] represents the exponential prediction result value of the sound pressure level data, Y represents the final brightness maintained after each beat in the music, A represents the light change range corresponding to each beat, ω represents the light change response speed corresponding to each beat, τ represents the τ moment of the change process after the start of the t-th frame data, and can be adjusted for different music types. The color can be changed in a cycle according to the change of τ per frame, and different lighting effects can be transformed through parameters such as amplitude, speed, and bottom brightness.

[0128] Therefore, in this embodiment, the above-mentioned lighting effect display method based on music beat is used, which has fast calculation speed, consumes less system resources and has low implementation cost; and enhances the correlation between music rhythm and lighting effects, improves the rhythmic experience, and is suitable for use in multiple scenarios; a more easily implemented filtering method is used in combination with exponential smoothing prediction to improve accuracy, and different judgment threshold values ​​of differential peaks are matched for different music types, and the lighting effects dynamically match the calculation and display results.

[0129] Embodiment 2

[0130] Please refer to Figure 2 A computing terminal 1 for audio changes includes a memory 2, a processor 3, and a computer program stored in the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, each step of a computing method for audio changes in embodiment 1 is implemented.

[0131] In summary, the present invention provides a method and terminal for calculating audio changes, which filters the collected music audio and extracts the music beat, so as to facilitate the subsequent adaptation of more accurate lighting effects to the audio; the audio signal is normalized and windowed, and converted into sound pressure level data, and then the index prediction and calculation of its change value are performed. Therefore, the change value of the audio can be obtained by processing, converting and predicting the audio signal, which improves the calculation speed and reduces the calculation cost. When the change value reaches the trigger condition, the dynamic matching lighting effect is output in combination with the music beat, which can increase the correlation between the music beat and the lighting effect and improve the lighting rhythm experience.

[0132] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for calculating audio changes, characterized in that: Includes steps: Receiving a regularly collected music audio signal, filtering the audio signal to extract the music beat, and converting the audio signal into sound pressure level data; Exponential prediction is performed on the sound pressure level data of the audio signal, a quadratic discrete signal of the standardized sound pressure level data is calculated, a cubic exponential prediction is performed based on the quadratic discrete signal, and a change value of the exponential prediction result is calculated by first-order difference.

2. The method for calculating audio changes according to claim 1, characterized in that: The performing cubic exponential prediction according to the quadratic discrete signal comprises: Set the initial value of an exponential smoothing forecast to t gain [0] = 1e-1, the initial value of the quadratic exponential smoothing forecast is t smooth [0] = 1e-1, the initial value of the triple exponential smoothing forecast is t sthrength [0] = 0.01, through iteration, we get the exponential smoothing forecast value t for the t+1 period. gain [t+1], quadratic exponential smoothing forecast value t smooth [t+1], triple exponential smoothing forecast value t sthrength [t+1]: x gain [t]=x t [t] / t[t+1]; x smooth [t]=t smooth [t] 2 ; x sthrength [t]=x smooth [t] / t sthrength [t+1]; In the formula, represents the smoothing coefficient, x sthrength [t] represents the index prediction result value, x t [t] represents a quadratic discrete signal of the normalized sound pressure level data.

3. The method for calculating audio changes according to claim 1, characterized in that: The converting of the audio signal into sound pressure level data comprises: Calculate the sound pressure level data x of the processed audio signal spl [t]: In the formula, x v [i] represents the normalized and windowed audio signal, P represents the sound pressure to be measured, and P0 represents the reference sound pressure; The sound pressure level data are normalized: x' spl [t]=M-x spl [t] / D peak ; In the formula, x spl '[t] represents the standardized sound pressure level data, M represents the standardized preset upper limit value, D peak Indicates the preset relative silence value.

4. The method for calculating audio changes according to claim 1, characterized in that: Calculating the change value of the audio signal by first-order difference includes: Take the first-order difference between the index prediction result value of the current period and the index prediction result value of the previous period to obtain the change value Δx sthrength : Δx sthrength =x sthrength [t]-x sthrength [t-1]。 5. The method for calculating audio changes according to claim 1, characterized in that: Converting the audio signal into sound pressure level data includes: Subtract the average value of the elements of each frame from the individual element values ​​of each frame in the audio signal: x'[i]=x v0 [i]-x ave [t]; In the formula, t represents a frame of data in each t-time period, m represents the number of data points of the elements collected in each frame of data, 1≤i≤m, x v0 [i] represents the i-th element value of each frame, x ave [t] represents the element average of a frame of data in every t period, and x'[i] represents the normalized audio signal.

6. A computing terminal for audio variation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: Receiving a regularly collected music audio signal, filtering the audio signal to extract the music beat, and converting the audio signal into sound pressure level data; Exponential prediction is performed on the sound pressure level data of the audio signal, a quadratic discrete signal of the standardized sound pressure level data is calculated, a cubic exponential prediction is performed based on the quadratic discrete signal, and a change value of the exponential prediction result is calculated by first-order difference.

7. The audio-changing computing terminal according to claim 6, characterized in that: The performing cubic exponential prediction according to the quadratic discrete signal comprises: Set the initial value of an exponential smoothing forecast to t gain [0] = 1e-1, the initial value of the quadratic exponential smoothing forecast is t smooth [0] = 1e-1, the initial value of the triple exponential smoothing forecast is t sthrength [0] = 0.01, through iteration, we get the exponential smoothing forecast value t for the t+1 period. gain [t+1], quadratic exponential smoothing forecast value t smooth [t+1], triple exponential smoothing forecast value t sthrength [t+1]: x gain [t]=x t [t] / t[t+1]; x smooth [t]=t smooth [t] 2 ; x sthrength [t]=x smooth [t] / t sthrength [t+1]; In the formula, represents the smoothing coefficient, x sthrength [t] represents the index prediction result value, x t [t] represents a quadratic discrete signal of the normalized sound pressure level data.

8. The audio-changing computing terminal according to claim 6, characterized in that: The converting of the audio signal into sound pressure level data comprises: Calculate the sound pressure level data x of the processed audio signal spl [t]: In the formula, x v [i] represents the normalized and windowed audio signal, P represents the sound pressure to be measured, and P0 represents the reference sound pressure; The sound pressure level data are normalized: x' spl [t]=M-x spl [t] / D peak ; In the formula, x spl '[t] represents the standardized sound pressure level data, M represents the standardized preset upper limit value, D peak Indicates the preset relative silence value.

9. The audio-changing computing terminal according to claim 6, characterized in that: Calculating the change value of the audio signal by first-order difference includes: Take the first-order difference between the index forecast result value of the current period and the index forecast result value of the previous period to obtain the change value 10. The audio-changing computing terminal according to claim 6, characterized in that: Converting the audio signal into sound pressure level data includes: Subtract the average value of the elements of each frame from the individual element values ​​of each frame in the audio signal: x'[i]=x v0 [i]-x ave [t]; In the formula, t represents a frame of data in each t-time period, m represents the number of data points of the elements collected in each frame of data, 1≤i≤m, x v0 [i] represents the i-th element value of each frame, x ave [t] represents the element average of a frame of data in every t period, and x'[i] represents the normalized audio signal.

Images