Intelligent sound effect adjustment speaker

Abstract

The present invention relates to the field of audio and sound effect processing, and in particular to an intelligent sound effect adjustment speaker. It includes a speaker body and an intelligent sound effect adjustment system. The intelligent sound effect adjustment system analyzes the type of music to which the audio information to be played belongs, and selects sound effect setting parameters according to the music type to play the audio information. The intelligent sound effect adjustment system includes a communication module, a machine learning discrimination module, a storage module and a playback module. The communication module receives the audio information to be played; the machine learning discrimination module analyzes the type of music to which the audio information to be played belongs, including a feature extraction submodule, a classification evaluation submodule, and a voting decision submodule; the storage module stores the sound effect setting parameters; the playback module selects the sound effect setting parameters according to the music type to play the audio information. There is no need to manually select the sound effect settings according to the played audio, which greatly improves the quality of the music played by the speaker, eliminates the inconvenience of manually adjusting the sound effect, and makes the speaker more intelligent and humane.

Description

Technical Field

[0001] The present invention relates to the field of audio and sound effect processing, and in particular to an intelligent sound effect adjustment speaker. Background Art

[0002] With the development of science and technology and the continuous innovation of digital products, the speaker system has also evolved and is no longer limited to a single sound function. For example, the size has been reduced, and radios, card readers, displays, and alarm clocks have been integrated. The connection method has expanded from traditional interfaces to USB interfaces, wireless interfaces, and even direct connections to the network. In terms of sound quality improvement, from the early 2.0 channels to today's 7.1 channels, as well as innovations such as stereo sound technology and surround sound technology. Among the many technologies for improving sound quality, amplifying or reducing the electrical signals of sounds in different frequency bands is a common means. In the prior art, various video and audio players have similar functions. However, the sound effect adjustment relies on manual adjustment. Once changed, the subsequent playback uses the changed sound effect until the next manual adjustment. However, music playback can automatically switch songs. Different types of music have different ways of adjusting the sound effect, and the sound effect cannot be adjusted synchronously with the music being played. Summary of the invention

[0003] The purpose of the present invention is to solve the above-mentioned related problems and design an intelligent sound effect adjustment speaker. To achieve the above-mentioned purpose, the present invention provides the following solutions:

[0004] An intelligent sound effect adjustment speaker, comprising:

[0005] A speaker body, and an intelligent sound effect adjustment system arranged on the speaker body, wherein the intelligent sound effect adjustment system analyzes the music type to which the audio information to be played belongs, and selects sound effect setting parameters according to the music type to which the audio information to be played belongs to play the audio information;

[0006] The intelligent sound effect adjustment system includes a communication module, a machine learning discrimination module, a storage module and a playback module;

[0007] A communication module, used to communicate with the network and receive audio information to be played;

[0008] A machine learning discrimination module, used for analyzing the music genre to which the audio information to be played belongs, wherein the music genre includes classical, jazz, rock, pop, and hip-hop;

[0009] The machine learning discrimination module includes a feature extraction submodule, a classification evaluation submodule, and a voting decision submodule;

[0010] The feature extraction submodule extracts the audio features of the audio information to be played, and forms an audio feature vector according to the values ​​extracted from each audio feature; the classification evaluation submodule makes a first judgment on the music type of the audio information to be played according to the feature vector; the voting decision submodule makes a second judgment on the music type of the audio information to be played by voting according to the judgment result of the classification evaluation submodule, and the result of the second judgment on the music type of the audio information to be played is the music type of the audio information to be played;

[0011] A storage module, used for storing sound effect setting parameters, wherein the sound effect setting parameters correspond to the music type;

[0012] The playing module is used to select sound effect setting parameters to play the audio information according to the music type to which the audio information to be played belongs analyzed by the machine learning discrimination module.

[0013] As a further improvement of the technical solution, the communication module uses WIFI, Bluetooth or WLAN communication mode to connect to the network to obtain the audio information to be played.

[0014] As a further improvement of the present technical solution, the audio features include musical range, average note duration, zero crossing rate, spectral centroid, short-time spectral energy, Mel-frequency cepstral coefficients, rhythm histogram, and playing speed; the importance of the audio features is ranked from high to low as Mel-frequency cepstral coefficients, spectral centroid, rhythm histogram, musical range, short-time spectral energy, average note duration, playing speed, and zero crossing rate.

[0015] As a further improvement of the present technical solution, the classification evaluation submodule determines the music type of the audio information to be played for the first time according to the following formula:

[0016] ;

[0017] in, is a vector composed of a certain audio feature quantity of the audio information to be played, A parameter vector of the same audio feature of a certain music type obtained by machine learning;

[0018] y=1, then the classification evaluation submodule will first determine the audio information to be played as the music type according to the characteristics of the audio information to be played;

[0019] y=0, then the classification evaluation submodule will first determine that the audio information to be played does not belong to the music type according to the characteristics of the audio information to be played.

[0020] As a further improvement of the present technical solution, the voting decision submodule adopts the highest vote strategy to determine the music type to which the audio information to be played belongs for the second time, and counts the results of the music type to which the audio information to be played belongs determined by the classification evaluation submodule based on different audio features, and the music type with the most calculated y=1 is determined as the final music type of the audio information to be played; if the number of y=1 calculated for two music types is equal, the final music type of the audio information to be played is determined by sorting them according to the importance of the audio features.

[0021] As a further improvement of the present technical solution, the voting decision submodule adopts a weighted voting strategy to determine for a second time the type of music to which the audio information to be played belongs. The audio features are assigned weight values ​​from large to small according to their importance. The weight values ​​are multiplied by the judgment value y of the corresponding music type and then summed to obtain the weighted vote number of the type. The music type with the largest weighted vote number is determined as the final music type of the audio information to be played.

[0022] As a further improvement of the technical solution, the parameter vector of the same audio feature of a certain music type Machine learning training, select classical, jazz, rock, pop, hip-hop music to form a set, extract the audio features of each element in the set to form an audio feature vector ={ ,···, }, the set of audio feature vectors is the training set of the audio feature, and the parameter vector to be trained is , the objective function is as follows:

[0023] ;

[0024] Where n is the number of samples in the training set, ( , ) is the value of the i-th sample, f(x)= ;

[0025] Pair Vector The i-th quantity Find the partial derivative, ;

[0026] Iterate using stochastic gradient descent , := +η ;

[0027] Where η is the learning rate, and 0<η<1; ( , ) is the randomly selected sample data, and k is the data index of the selected sample data.

[0028] As a further improvement of the present technical solution, the sound effect setting parameters can be manually adjusted and set. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Various other advantages and benefits will become apparent to those skilled in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention;

[0030] Figure 1 This is a structural schematic diagram of an intelligent sound effect adjustment speaker of the present invention;

[0031] Figure 2 An evaluation result table of a classification evaluation submodule of an intelligent sound effect adjustment speaker of the present invention;

[0032] Figure 3 This is a determination weight table of a classification evaluation submodule of an intelligent sound effect adjustment speaker of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0034] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0035] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0036] Example 1: Figure 1 As shown, a structural schematic diagram of an intelligent sound effect adjustment speaker.

[0037] The intelligent sound effect adjustment speaker is internally equipped with an intelligent sound effect adjustment system. The intelligent sound effect adjustment system can autonomously analyze the music type of the audio information to be played through machine learning. The system stores sound effect adjustment schemes for different music types, and each scheme has different sound effect setting parameters. The intelligent sound effect adjustment system determines the type of the audio to be played as one of classical, jazz, rock, pop, and hip-hop, and selects sound effect setting parameters according to the music type of the audio to be played to play the audio information. The sound effect setting parameters of each audio type can be manually set and adjusted by the user.

[0038] The communication module of the intelligent sound system receives the audio information to be played. The audio information to be played can be transmitted to the intelligent sound adjustment speaker via WIFI, Bluetooth or WLAN communication. The machine learning discrimination module of the intelligent sound system performs type judgment on the audio learning to be played. The machine learning discrimination module includes a feature extraction submodule, a classification evaluation submodule, and a voting decision submodule.

[0039] The feature extraction submodule extracts the audio features of the audio to be played, and forms an audio feature vector according to the values ​​extracted from each audio feature.

[0040] Audio can be divided into two parts: melody and text. According to popular understanding, melody is the music part of audio, and text is the lyrics part of audio. The present invention classifies the music to be played by the characteristics of the audio represented by the melody, and can also classify the audio by adding text features. It can be shown that the present invention selects the following audio features for judging the audio type: range, average note duration, zero crossing rate, spectral centroid, short-time spectrum energy, Mel-frequency cepstral coefficients, rhythm histogram, and playing speed. According to the importance of different audio features for judging the audio type, the audio features are sorted in order from high to low as follows: Mel-frequency cepstral coefficients, spectral centroid, rhythm histogram, range, short-time spectrum energy, average note duration, playing speed, and zero crossing rate.

[0041] To extract the audio features of the audio to be played, the feature extraction submodule obtains the MIDI format file of the audio to be played, samples, frames and adds windows to the MIDI audio signal. For each audio feature, calculate and extract it as follows

[0042] Read the pitch sequence P={p1,p2···,p n}, the audio range is:

[0043] ;

[0044] In the formula, Indicates the maximum value of the audio pitch sequence. Represents the minimum value of the audio pitch sequence.

[0045] Read the duration of each note and calculate the average note duration as follows:

[0046] ;

[0047] Where n is the total number of notes and Di is the duration of a single note.

[0048] The zero-crossing rate is the number of times the value of the audio signal passes through the zero point in each frame of the audio signal, and is calculated as follows:

[0049] ;

[0050] Where sign(x) is the sign function, is the signal of the nth frame in the time domain.

[0051] The spectral centroid is the center of gravity of the amplitude spectrum of the short-time Fourier transform of the audio signal, which reflects the shape and brightness of the audio. The value of the spectral centroid is proportional to the bright texture and frequency of the audio signal. The spectral centroid is calculated as follows:

[0052] ;

[0053] Where k is the sampling interval in the frequency domain, N is the frame length, is the amplitude of the corresponding frequency after short-time Fourier transform.

[0054] The short-time spectrum energy reflects the spectrum energy of the audio and is calculated as follows:

[0055] ;

[0056] In the formula, k is the number of frames, N is the total number of frames, is the amplitude of the corresponding frequency after short-time Fourier transform.

[0057] Mel frequency cepstral coefficients model the human auditory system. When the frequency perceived by the human ear is below 1000Hz, the two are linearly distributed and have the same increase pattern as the Mel frequency. For frequencies above 1000Hz, the frequency perceived by the human ear is logarithmically related to the actual frequency. The relationship between the two is calculated as follows:

[0058] ;

[0059] Where f is the actual frequency of the audio.

[0060] The speed of performance is an important musical component and has a significant impact on genre classification;

[0061] ;

[0062] Where n is the total number of notes, Di is the duration of a single note, It is the unit playing speed of a single note. The overall playing speed uses the weighted average of single notes to express the playing speed of the audio.

[0063] The audio type is closely related to the rhythm. The rhythm actually reflects the beat of the audio. The periodicity of the beat in the audio signal is the most significant, which is the rhythm feature of the audio. The wavelet coefficients of the audio are obtained by the Mallat algorithm, where the sampling process is calculated as follows:

[0064] ;

[0065] ;

[0066] Where t(n) is the time domain signal of the audio to be played, N is the number of samples, is a low-pass filter, is a high-pass filter. Perform wavelet decomposition on the low-pass and high-pass filters to obtain the sequence signal α and the interference signal β, and obtain the reconstructed signal as follows:

[0067] ;

[0068] ;

[0069] Further extract the energy peak coefficient in the frequency domain:

[0070] ;

[0071] By obtaining the number of frequency bands of the signal, the beat distribution histogram can be obtained, which is the rhythm histogram of the audio signal.

[0072] The classification evaluation submodule makes a first judgment on the music type of the audio information according to the feature vector. If the feature of the audio information has only one value, it is regarded as a one-dimensional feature vector. The classification evaluation module substitutes each feature value of the audio to be played into the classification judgment formula of the corresponding feature value of each type of music to obtain a judgment result of whether it belongs to this type of music. If the feature judgment formula of this type of music determines that the music to be played belongs to this type of music according to the feature, the output is 1, otherwise it is 0. The parameters of the feature classification judgment formula are obtained through machine learning. For example, the range of the audio to be played is a one-dimensional vector, which is substituted into the judgment formula of each type of music according to the feature of the range. The output of classical, jazz, and pop is 1, and the output of hip-hop and rock is 0. According to the range of the audio to be played, classical, jazz, and pop believe that the audio to be played belongs to this type, while hip-hop and rock do not belong to this type. Select the first 13 Mel-frequency cepstral coefficients of the Mel-frequency spectrum of the audio to be played to form a vector, ={mel1,···mel13}, substitute the characteristic judgment formula for each type of music, the output of jazz, pop, and hip-hop is 1, and the output of classical and rock is 0. Then, based on the Mel-frequency cepstral coefficient, jazz, pop, and hip-hop are considered to be the audio to be played, while classical and rock are not considered to be of this type. The classification of the played audio obtained by judging the characteristic information of each audio is the first judgment result. The voting decision submodule judges the audio type to which the audio information belongs for the second time by voting based on the first judgment result of the classification evaluation submodule, and the voting result is finally confirmed as the type of the audio. The intelligent sound effect system retrieves the sound effect setting parameters of this type of music in the storage module according to the final result of the classification, and uses the sound effect setting of the parameters to play the audio.

[0073] Example 2: Figure 2 and Figure 3 shown.

[0074] An intelligent sound effect adjustment speaker, the audio to be played is subjected to a first judgment of the music type of the audio information by the classification evaluation submodule according to the feature vector. Figure 2 As shown, the voting decision submodule determines the audio type to which the audio information belongs for the second time by voting according to the first judgment result of the classification evaluation submodule.

[0075] If the highest vote strategy is adopted, the number of votes for each type is counted. Judging from the four features of Mel frequency cepstral coefficient, short-time spectrum energy, average note duration and zero-crossing rate, the audio belongs to the classical type, so the classical type has 4 votes. By analogy, jazz has 6 votes, rock has 4 votes, pop has 6 votes, and hip-hop has 4 votes. Since the number of votes for jazz and pop is the same, it is necessary to further determine the type based on the importance of the features, and compare each feature from high to low in order of importance. The Mel frequency cepstral coefficients of the most important feature are both 1, and the second most important spectral centroid is 1 for jazz and 0 for pop, so the audio played with this band is finally determined to be jazz.

[0076] If the weighted voting strategy is adopted, each feature is assigned a weight value according to its importance, and the total weight value is 1. Figure 3 , calculate the votes of each type. Taking the classical type as an example, the votes of the classical type = 0.2*1+0.18*0+0.16*0+0.13*0+0.12*1+0.09*1+0.07*0+0.05*1=0.46;

[0077] Calculate the weighted votes for each type in turn, and the highest value is the type of the audio.

[0078] Embodiment 3: An intelligent sound effect adjustment speaker, in order to obtain the judgment function of a certain audio feature of each type, a machine learning method is used to determine the various parameters of the judgment function. Select 1000 pieces of classical, jazz, rock, pop, and hip-hop music, and extract the Mel-frequency cepstral coefficients, spectral centroid, rhythm histogram, range, short-time spectrum energy, average note duration, playing speed and zero-crossing rate of these 5000 pieces of music. Use the Mel-frequency cepstral coefficients of the 5000 pieces of music as the training set. Take the first 13 values ​​of the Mel-frequency cepstral coefficients to form a vector ={mel1,···mel 13 Taking the machine learning of Mel-frequency cepstral coefficients of classical music as an example, the parameter vector of Mel-frequency cepstral coefficients of classical music is ={θ1,···θ 13}, the objective function is as follows:

[0079] ;

[0080] Where n is the number of samples in the training set, ( , ) is the value of the i-th sample, = ,If the sample is classical music, then =1, if it is not classical music, =0,f(x)= ;

[0081] Pair Vector The i-th quantity Find the partial derivative, ;

[0082] Iterate using stochastic gradient descent from the training set , := +η ;

[0083] Where η is the learning rate, and 0<η<1; ( , ) is the randomly selected sample data, k is the data index, and iterates to Once convergence occurs, the parameter value can be obtained.

[0084] If the audio feature to be learned is a single value, it is considered as a one-dimensional vector.

[0085] In addition to using stochastic gradient descent iteratively , or the steepest descent method or the mini-batch gradient descent method can also be used.

[0086] To sum up, the intelligent sound effect adjustment speaker proposed in the present invention can automatically adjust the sound effects of the audio to be played according to the music being played, without the need for manual selection of sound effect settings based on the played audio. This greatly improves the quality of music played by the speaker, eliminates the inconvenience of manually adjusting the sound effects, and makes the speaker more intelligent and user-friendly.

Claims

1. An intelligent sound effect adjustment speaker, characterized in that: include: A speaker body, and an intelligent sound effect adjustment system arranged on the speaker body, wherein the intelligent sound effect adjustment system analyzes the music type of the audio information to be played, and selects sound effect setting parameters according to the music type of the audio information to be played to play the audio information; The intelligent sound effect adjustment system includes a communication module, a machine learning discrimination module, a storage module and a playback module; A communication module, used to communicate with the network and receive audio information to be played; A machine learning discrimination module, used for analyzing the music type of the audio information to be played, wherein the music type includes classical, jazz, rock, pop and hip-hop; The machine learning discrimination module includes a feature extraction submodule, a classification evaluation submodule and a voting decision submodule; The feature extraction submodule extracts the audio features of the audio information to be played, and forms an audio feature vector according to the values ​​extracted from each audio feature; the classification evaluation submodule makes a first judgment on the music type of the audio information to be played according to the feature vector; the voting decision submodule makes a second judgment on the music type of the audio information to be played by voting according to the judgment result of the classification evaluation submodule, and the result of the second judgment on the music type of the audio information to be played is the music type of the audio information to be played; A storage module, used for storing sound effect setting parameters, wherein the sound effect setting parameters correspond to the music type; The playing module is used to select sound effect setting parameters to play the audio information according to the music type of the audio information to be played analyzed by the machine learning discrimination module.

2. The intelligent sound effect adjustment speaker according to claim 1, characterized in that: The communication module uses WIFI, Bluetooth or WLAN communication mode to connect to the network and obtain the audio information to be played.

3. The intelligent sound effect adjustment speaker according to claim 1, characterized in that: The audio features include range, average note duration, zero crossing rate, spectral centroid, short-time spectral energy, Mel-frequency cepstral coefficients, rhythm histogram and performance speed; the importance of the audio features is ranked from high to low as Mel-frequency cepstral coefficients, spectral centroid, rhythm histogram, range, short-time spectral energy, average note duration, performance speed, and zero crossing rate.

4. The intelligent sound effect adjustment speaker according to claim 3, characterized in that: The classification evaluation submodule first determines the music type of the audio information to be played according to the following formula: ； in, is an audio feature vector composed of a certain audio feature of the audio information to be played, A parameter vector of the same audio feature of a certain music type obtained by machine learning; y=1, then the classification evaluation submodule will first determine the audio information to be played as the music type according to the audio features of the audio information to be played; y=0, then the classification evaluation submodule will first determine that the audio information to be played does not belong to the music type according to the audio features of the audio information to be played.

5. The intelligent sound effect adjustment speaker according to claim 4, characterized in that: The voting decision submodule adopts the highest vote strategy to determine the music type of the audio information to be played for the second time, and counts the results of the music type of the audio information to be played determined by the classification evaluation submodule according to different audio features, and the music type with the most y=1 calculated is determined as the final music type of the audio information to be played; if the number of y=1 calculated for two music types is equal, the final music type of the audio information to be played is determined by sorting according to the importance of the audio features.

6. The intelligent sound effect adjustment speaker according to claim 4, characterized in that: The voting decision submodule adopts a weighted voting strategy to determine the music type of the audio information to be played for the second time. The audio features are assigned weight values ​​from large to small according to their importance. The weight values ​​are multiplied by the judgment value y of the corresponding music type and then summed to obtain the weighted vote number of the music type. The music type with the largest weighted vote number is determined as the final music type of the audio information to be played.

7. The intelligent sound effect adjustment speaker according to claim 4, characterized in that: The parameter vector of the same audio feature of the certain music type Machine learning training is performed to select classical, jazz, rock, pop and hip-hop music to form a set, and the audio features of each element in the set are extracted to form an audio feature vector ={ ,···, }, the set of audio feature vectors is the training set of the audio feature, and the parameter vector to be trained is , the objective function is as follows: ； Where n is the number of samples in the training set, ( , ) is the value of the i-th sample, f(x)= ; For parameter vector The i-th quantity Find the partial derivative, ; Iterate using stochastic gradient descent , := +η ; Where η is the learning rate, and 0<η<1; ( , ) is the randomly selected sample data, and k is the data index of the selected sample data.

8. The intelligent sound effect adjustment speaker according to claim 1, characterized in that: The sound effect setting parameters can be manually adjusted and set.

Images