Sound effect adjustment method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a sound effect adjustment method, electronic equipment and a storage medium. The method comprises the following steps: establishing a style classification model based on a song data set; in response to a sound box playing song, acquiring song data of the song, and identifying style information of the song based on the song data through the style classification model; determining a target sound effect mode corresponding to the song based on the style information and a corresponding relationship between the multiple pieces of style information and multiple sound effect modes; and controlling the sound equipment to play the song based on the target sound effect mode. According to the embodiment of the invention, when the song is played, the song style can be identified through the style classification model, the song is played based on the sound effect mode conforming to the song style, and the auditory experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a sound effect adjustment method, an electronic device and a storage medium. Background Art

[0002] In order to improve the user's listening experience in the car, car audio has become the research and development focus of various electronic equipment manufacturers. In order to meet the needs of different users, car audio can usually provide a variety of sound effect modes for users to manually select, such as whole car, main driving, theater, vocals, custom, etc. However, different songs have different styles, and the sound effect mode selected by the user cannot adapt to all songs. If the sound effect mode does not match the song style, the playback effect of the car audio is poor, which affects the user's listening experience. Summary of the invention

[0003] In view of the above, it is necessary to provide a sound effect adjustment method, an electronic device and a storage medium to solve the problem that the above sound effect mode does not match the song style, resulting in poor playback effect of the car audio and affecting the user's listening experience.

[0004] In a first aspect, an embodiment of the present application provides a sound effect adjustment method, the method comprising: establishing a style classification model based on a song data set; in response to a speaker playing a song, obtaining song data of the song, and identifying style information of the song based on the song data through the style classification model; determining a target sound effect mode corresponding to the song based on the style information and the correspondence between multiple style information and multiple sound effect modes; and controlling the speaker to play the song based on the target sound effect mode.

[0005] In one possible implementation, the song data set includes audio data of multiple songs, and establishing a style classification model based on the song data set includes: extracting target features of each song based on the audio data of each song, the target features including at least one of rhythm features, melody features, and chord features; inputting the target features into a preset deep learning model to predict style information of each song; calculating a loss function value of the preset deep model based on the style information of each song, and adjusting parameters of the preset deep model based on the loss function value to obtain the style classification model.

[0006] In a possible implementation, the target features of each song are extracted based on the audio data of each song, including: dividing the audio data of each song into main song audio data and accompaniment audio data; analyzing the main song audio data using a beat tracking algorithm to obtain the rhythm features; extracting the audio features of the main song audio data to obtain the melody features; extracting the audio features of the accompaniment audio data to obtain the chord features.

[0007] In one possible implementation, the extracting of audio features of the main song audio data includes: framing and windowing the main song audio data; performing Fourier transform on the framed and windowed main song audio data to obtain frequency domain features of the main song audio data; and calculating Mel-frequency cepstral coefficients of the main song audio data based on the frequency domain features as the audio features of the main song audio data.

[0008] In a possible implementation, the song data set also includes text data of the multiple songs, and establishing a style classification model based on the song data set includes: extracting text features of each song based on the text data of each song; inputting the text features and the target features into the preset deep learning model to predict the style information of each song; calculating the loss function value of the preset deep model based on the style information of each song, and adjusting the parameters of the preset deep model based on the loss function value to obtain the style classification model.

[0009] In a possible implementation, the extracting of text features of each song based on the text data of each song includes: performing word segmentation on the text data of each song, and constructing a vocabulary based on the text data after the word segmentation processing; and generating a feature vector of the text data based on the vocabulary using a word embedding model.

[0010] In a possible implementation, the song data of the song is obtained, and the style information of the song is identified based on the song data through the style classification model, including: collecting the song information of the song, and obtaining the audio data and text data of the song based on the song information; inputting the audio data and the text data into the style classification model, and using the style classification model to extract target features of the song based on the audio data, wherein the target features include at least one of rhythm features, melody features, and chord features; extracting text features of the song based on the text data; fusing the target features and the text features to obtain fused features; and linearly mapping the fused features to obtain the style information of the song.

[0011] In a possible implementation, identifying the style information of the song based on the song data through the style classification model also includes: dividing the audio data of the song into multiple segments of sub-audio data, inputting each segment of sub-audio data into the style classification model, and extracting target features of each segment of sub-audio data using the style classification model; and linearly mapping the target features of each segment of sub-audio data to obtain style information of each segment of sub-audio data.

[0012] In a second aspect, an embodiment of the present application provides a sound effect adjustment device, the device comprising: an establishment module for establishing a style classification model based on a song data set; an identification module for acquiring song data of the song in response to a speaker playing a song, and identifying style information of the song based on the song data through the style classification model; a determination module for determining a target sound effect mode corresponding to the song based on the style information and the correspondence between multiple style information and multiple sound effect modes; and a playing module for controlling the speaker to play the song based on the target sound effect mode.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device executes the above-mentioned sound effect adjustment method.

[0014] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores program instructions. When the program instructions are executed on an electronic device, a processor of the electronic device executes the above-mentioned sound effect adjustment method.

[0015] The sound effect adjustment method, device, electronic device and storage medium provided in the embodiments of the present application can identify the style of a song through a style classification model when playing a song, determine a sound effect mode that matches the style of the song, and play the song based on the determined sound effect mode, thereby effectively improving the user's auditory experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0017] Figure 1 It is a schematic diagram of an application scenario of a sound effect adjustment method provided in an embodiment of the present application.

[0018] Figure 2 It is a flow chart of a sound effect adjustment method provided in one embodiment of the present application.

[0019] Figure 3 This is a flowchart of establishing a style classification model provided by an embodiment of the present application.

[0020] Figure 4 It is a correspondence table between style information and sound effect modes provided in an embodiment of the present application.

[0021] Figure 5 It is a flow chart of a sound effect adjustment method provided by another embodiment of the present application.

[0022] Figure 6 It is a structural schematic diagram of a sound effect adjustment device provided in one embodiment of the present application.

[0023] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The terms "first" and "second" involved in the embodiments of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" refers to one or more. "Multiple" refers to two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c. In the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0026] In order to improve the user's listening experience in the car, car audio has become the research and development focus of various electronic equipment manufacturers. In order to meet the needs of different users, car audio can usually provide a variety of sound effect modes for users to manually select, such as whole car, main driving, theater, vocals, custom, etc. However, different songs have different styles, and the sound effect mode selected by the user cannot adapt to all songs. If the sound effect mode does not match the song style, the playback effect of the car audio is poor, which affects the user's listening experience.

[0027] The embodiment of the present application provides a sound effect adjustment method, which can identify the style of a song through a style classification model when playing a song, determine a sound effect mode that matches the style of the song, and play the song based on the determined sound effect mode, thereby effectively improving the user's auditory experience.

[0028] See also Figure 1 , which is a schematic diagram of the application environment of the sound effect adjustment method provided in one embodiment of the present application. Figure 1 The sound effect adjustment system 100 shown includes a control unit 10, a collection unit 20, a communication unit 30, a cloud 40 and an execution unit 50. In an embodiment of the present application, the control unit 10 may be a processor in an electronic device, and the electronic device may be a vehicle-mounted terminal. The control unit 10 may deploy a computer program product (such as software code, computer-readable instructions, etc.) programmed and implemented according to the sound effect adjustment method provided in an embodiment of the present application, thereby providing a sound effect adjustment service. The collection unit 20 includes a screen information collection unit, a content end information collection unit and a CAN (Controller Area Network) bus. The screen information collection unit is used to collect song information of the song being played by the audio system displayed on the display screen of the electronic device. The content end information collection unit is used to collect corresponding song data based on the song information. The song data includes the audio data of the song and the text data of the lyrics. The CAN bus is used to send the song information and song data to the control unit 10. The communication unit 30 can realize mobile communication. The control unit 10 can send song information and / or song data to the cloud 40 through the communication unit 30, identify the style information of the song based on the song data through the big data model (for example, the style classification model of the song) in the cloud 40, and feed back the identified style information to the control unit 10 through the communication unit 30. The control unit 10 determines the corresponding target sound effect mode based on the style information, and sends a CAN signal to the execution unit 50 based on the target sound effect mode. The execution unit 50 responds to the CAN signal and plays the song based on the target sound effect mode.

[0029] See also Figure 2 FIG. 1 is a flow chart of a sound effect adjustment method provided in an embodiment of the present application. The method is applied in an electronic device, and the sound effect adjustment method includes: S101, establishing a style classification model based on the song dataset.

[0030] In one embodiment of the present application, a song data set includes audio data of multiple songs of different styles (i.e., the song style information is known), the audio data of each song is extracted, and a preset deep model is trained based on the audio data of multiple songs to obtain a song style classification model. The song can be pure music without lyrics, or it can be music with lyrics. The preset deep model can be a clustering model, a decision tree model, or a neural network model. For example, the clustering model can be a K-means clustering model, a density-based clustering algorithm (DBSCAN) model, etc. The decision tree model can be a random forest model, XGBoost, etc. The neural network model can be a transformer model, a long short-term memory model LSTM, etc.

[0031] See also Figure 3 As shown, it is a flowchart of establishing a style classification model provided by an embodiment of the present application.

[0032] S1011, extracting a target feature of each song based on the audio data of each song, wherein the target feature includes at least one of a rhythm feature, a melody feature, and a chord feature.

[0033] In one embodiment of the present application, for a song with lyrics, the audio data includes verse audio data and accompaniment audio data, the verse audio data includes vocals, and the accompaniment audio data includes but is not limited to vocals (e.g., harmony) and instrumental sounds. The audio data of each song is divided into verse audio data and accompaniment audio data based on audio frequency or audio intensity. Since the verse audio data is mainly vocals, and the accompaniment audio data is mainly instrumental sounds, the audio data of the song can be divided into verse audio data and accompaniment audio data based on the difference in audio frequency corresponding to the vocals and the audio frequency corresponding to the instrumental sounds. In addition, the audio intensity of the vocals in the song is usually greater than the audio intensity of the accompaniment, so the audio data of the song can also be divided into verse audio data and accompaniment audio data based on the difference in audio intensity corresponding to the verse and the audio intensity corresponding to the accompaniment.

[0034] In one embodiment of the present application, the beat tracking algorithm is used to analyze the main song audio data to obtain the rhythm feature. The main song audio data is framed, and the short-time Fourier transform is performed on the framed main song audio data to obtain the spectrum of the main song audio data. The spectrum amplitude is logarithmically processed, and the phase information of the endpoint intensity curve and the peak value of the endpoint intensity curve are output through half-wave rectification to determine the starting beat point. The beats per minute (BPM) feature is extracted based on the autocorrelation characteristics of the endpoint intensity curve. The peak points of the endpoint intensity curve are clustered using the maximum and minimum distance method to obtain the beat point position of the main song audio data, and the beat point position is determined as the rhythm feature.

[0035] In one embodiment of the present application, the audio features of the main song audio data are extracted to obtain the melody features. The main song audio data is framed and windowed, and the short-time Fourier transform is performed on the framed and windowed main song audio data to obtain the frequency domain features of the main song audio data. The Mel cepstral coefficients of the main song audio data are calculated based on the frequency domain features, including: calculating the square of the amplitude of each frame of the main song audio data to obtain the power spectrum density of each frame of the main song audio data, filtering the power spectrum density of each frame of the main song audio data through a filter group to obtain the filtered energy, taking the logarithm of the filtered energy value, performing an inverse Fourier transform on the logarithmic energy spectrum to obtain the cepstral coefficients, and selecting the cepstral coefficients of the first preset number (for example, 12 or 13) as the final Mel cepstral coefficients. The Mel cepstral coefficients of the main song audio data are determined as the melody features. In other embodiments of the present application, the linear prediction cepstral coefficients of the main song audio data can also be calculated based on the frequency domain features of the main song audio data, and the linear prediction cepstral coefficients are determined as the melody features.

[0036] In one embodiment of the present application, audio features of the accompaniment audio data are extracted to obtain chord features. The mel-cepstral coefficients or linear prediction cepstral coefficients of the accompaniment audio data can be calculated using the above-mentioned method for calculating the mel-cepstral coefficients of the main song audio data, and the mel-cepstral coefficients or linear prediction cepstral coefficients of the accompaniment audio data are determined as chord features.

[0037] S1012, input the target features into a preset deep learning model to predict the style information of each song.

[0038] In one embodiment of the present application, if the target feature includes at least one of a rhythm feature, a melody feature, and a chord feature, the target feature is input into a preset deep learning model to predict the style information of each song. Taking the preset deep learning model as a convolutional neural network model as an example, the preset deep learning model linearly maps the target feature through a classification network to obtain the probability distribution of the style information, and determines the style information with the highest probability as the style information of the song. Among them, the classification network is a fully connected layer. If the target feature includes multiple of the rhythm feature, the melody feature, and the chord feature, the preset deep learning model fuses multiple target features through a feature fusion network to obtain a fused target feature, and linearly maps the fused target feature through a classification network to obtain the probability distribution of the style information, and determines the style information with the highest probability as the style information of the song.

[0039] S1013, calculating the loss function value of the preset deep model based on the style information of each song, adjusting the parameters of the preset deep model based on the loss function value, and obtaining a style classification model.

[0040] In one embodiment of the present application, the loss function of the preset deep model can be a cross-entropy loss function, a mean absolute error loss function (MAE), or a mean squared error loss function (MSE). The style information of each predicted song and the true style information are input into the loss function to obtain the loss function value. It is determined whether the loss function value is less than or equal to a preset threshold. If the loss function value is greater than the preset threshold, the parameters of the preset deep model are adjusted, and then S1012 is returned. The target features of the song are continuously input into the preset deep learning model to predict the corresponding style information, and then S1013 is executed until the loss function value is less than or equal to the preset threshold, determining that the preset deep learning model converges, thereby determining that the training of the preset deep learning model is completed, and the trained preset deep learning model is determined as the style classification model. If the loss function value is less than or equal to the preset threshold, it is determined that the training of the preset deep learning model is completed, and the trained preset deep learning model is determined as the style classification model. For example, the parameters of the preset deep model include, but are not limited to: the size and number of convolutional kernels, the weights and bias values of each layer in the neural network, and the learning rate.

[0041] In another embodiment of the present application, the song dataset further includes the text data of multiple songs, and the text data is the lyrics corresponding to the songs. Establishing a style classification model based on the song dataset may include: extracting the text features of each song based on the text data of each song, inputting the text features and the target features into the preset deep learning model to predict the style information of each song, calculating the loss function value of the preset deep model based on the style information of each song, and adjusting the parameters of the preset deep model based on the loss function value to obtain the style classification model.

[0042] In the above-mentioned another embodiment of the present application, the text data of each song is preprocessed, and the preprocessing includes removing punctuation marks, removing stop words (such as "de", "shi", etc.), unifying the case, word segmentation processing, etc. A vocabulary is constructed based on the text data after word segmentation processing, and a feature vector of the text data is generated based on the vocabulary using a word embedding model (Word2Vec), and the feature vector is the text feature of the text data.

[0043] S102. In response to the sound player playing a song, obtain the song data of the song, and identify the style information of the song based on the song data through the style classification model.

[0044] In one embodiment of the present application, in response to a song being played on a speaker, song information of the song is collected, the song information including but not limited to: song title, singer, album title, and release date; audio data and text data of the song are obtained based on the song information; the audio data and text data are input into a style classification model; the style classification model is used to extract target features of the song based on the audio data; the target features include at least one of rhythm features, melody features, and chord features; text features of the song are extracted based on the text data; the target features and text features are fused to obtain fused features; the fused features are linearly mapped to obtain style information of the song.

[0045] In one embodiment of the present application, in the scenario of playing songs using the car audio system, the car terminal display screen displays song information, the screen information collection unit collects the song information displayed on the display screen, and sends the song information to the cloud through the communication unit. The cloud obtains song data based on the song information, and uses the style classification model to identify the style information of the song based on the song data. In another embodiment of the present application, the song data and the style classification model can also be stored locally in the car terminal.

[0046] In another embodiment of the present application, the audio data of the song is divided into multiple sections of sub-audio data, the text data of the song is divided into multiple sections of sub-text data, the multiple sections of sub-audio data are aligned with the multiple sections of sub-text data, each section of sub-audio data and each section of sub-text data are input into a style classification model, the style classification model is used to extract the target features of each section of sub-audio data, and the text features of each section of sub-text data are extracted, the target features and the text features are fused to obtain fused features, and the fused features are linearly mapped to obtain the style information of each section of sub-audio data. Each section of sub-audio data has a preset duration, for example, the preset duration is 20 seconds, 25 seconds, 30 seconds or other time.

[0047] S103, determining a target sound effect mode corresponding to the song based on the style information and the correspondence between the plurality of style information and the plurality of sound effect modes.

[0048] See also Figure 4 As shown, it is a table of correspondence between style information and sound effect modes provided by an embodiment of the present application. For example, if the style classification model recognizes that the style information of a song is a piano style, then the target sound effect mode corresponding to the song is determined to be a piano mode, in which the frequency and harmonics of the piano sound can be enhanced; if the style classification model recognizes that the style information of a song is a rock style, then the target sound effect mode corresponding to the song is determined to be a rock mode, in which the volume and dive of the low-frequency sound can be optimized.

[0049] In another embodiment of the present application, if the style classification model identifies the style information of a segment of audio data, the target sound effect mode corresponding to the segment of audio data is determined, so that when the segment of audio data is played, the sound effect mode is switched to the target sound effect mode.

[0050] S104, controlling the audio system to play songs based on the target sound effect mode.

[0051] In one embodiment of the present application, the target sound effect mode is sent to the digital signal processor DSP in the execution unit, and the DSP controls the audio to switch the sound effect mode when playing the song to the target sound effect mode, thereby playing the song based on the target sound effect mode.

[0052] The embodiment of the present application uses deep learning technology to achieve that during the process of playing songs, the sound effect mode automatically changes as the style of the song being played changes. The user does not need to actively switch the sound effects to listen to the song played with sound effects that match the style of the song. This also improves driving safety and comfort in the in-vehicle scenario, effectively enhancing the user experience.

[0053] See also Figure 5 FIG. 1 is a flow chart of a sound effect adjustment method provided by another embodiment of the present application. The method is applied in an electronic device, and the sound effect adjustment method includes: S201, establishing a style classification model based on the song dataset.

[0054] S202, in response to the sound system playing a song, determining whether the song tag of the song includes style information. If the song tag of the song does not include style information, the process proceeds to S203; if the song tag of the song includes style information, the process proceeds to S204.

[0055] In one embodiment of the present application, the song tag is an ID3 tag (Identification, Tagging, and Digitized Information), which can store descriptive information about the audio content in the audio file, such as the song name, style information, singer, album, release date, etc. ID3 tags can be stored in text form, and can also contain other types of information such as images and URLs, so that audio files can carry more additional information, which is convenient for audio management, classification and display.

[0056] S203, obtaining song data of the song, and identifying style information of the song based on the song data through a style classification model.

[0057] S204, determining a target sound effect mode corresponding to the song based on the style information and the correspondence between the plurality of style information and the plurality of sound effect modes.

[0058] S205, controlling the audio system to play songs based on the target sound effect mode.

[0059] The specific implementations of S201 and S203 - S205 are the same as those of S101 - S104 and are not described in detail here.

[0060] In another embodiment of the present application, when the user plays a song through the audio system, it is determined whether the song is played for the first time based on the ID3 information. If the song is played for the first time, since many songs do not have style information, or the style information does not match the style of the song, it is necessary to identify the style of the song during the song playback process. A segment of audio data of the song is obtained every preset time (for example, 30 seconds), and the audio features of the audio data are extracted. After the audio features are input into the style classification model based on deep learning, the style information is obtained. After the entire song is played, multiple style information can be obtained, and the style information with the most occurrences is determined to be the style information of the song. The style information can be mapped to the corresponding target sound effect mode, and the style information or the target sound effect mode and the ID3 information are stored together in the memory of the vehicle terminal. If the song is not played for the first time, the style information is obtained through the ID3 information, the target sound effect mode corresponding to the style information is selected, and the target sound effect mode is set to the DSP to take effect.

[0061] See also Figure 6 , which is a schematic diagram of the structure of a sound effect adjustment device provided in one embodiment of the present application. In one embodiment of the present application, the sound effect adjustment device 200 may include a plurality of functional modules composed of computer program segments. The computer program segments in the sound effect adjustment device 200 may be stored in a memory of an electronic device and executed by at least one processor to perform the sound effect adjustment function.

[0062] In one embodiment of the present application, the sound effect adjustment device 200 can be divided into multiple functional modules according to the functions it performs. The functional modules of the sound effect adjustment device 200 may include: a creation module 201, an identification module 202, a determination module 203, and a playback module 204. The module in the embodiment of the present application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory.

[0063] The building module 201 is used to build a style classification model based on the song data set.

[0064] The identification module 202 is used to obtain song data of the song in response to the sound system playing the song, and to identify the style information of the song based on the song data through a style classification model.

[0065] The determination module 203 is used to determine the target sound effect mode corresponding to the song based on the style information and the correspondence between multiple style information and multiple sound effect modes.

[0066] The playing module 204 is used to control the audio system to play songs based on a target sound effect mode.

[0067] The embodiment of the present application also provides an electronic device 1, which can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device 1.

[0068] See also Figure 7 The figure is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The sound effect adjustment method provided in the embodiment of the present application is applied to an electronic device 1, and the electronic device 1 includes, but is not limited to, a processor 110 and a memory 120 connected via a communication bus 130. Figure 7 This is merely an example of an electronic device and does not constitute a corresponding limitation. In other embodiments, the electronic device may include more components than those shown in the figure.

[0069] The memory 120 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data. The random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0070] The non-volatile memory may also store executable programs and user and application data, etc., and may be pre-loaded into the random access memory for direct reading and writing by the processor 110. The non-volatile memory may include a disk storage device and a flash memory.

[0071] The memory 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include a plurality of instructions, and when the plurality of instructions are executed by the processor 110, the sound effect adjustment method executed on the electronic device 1 can be implemented.

[0072] In other embodiments, the electronic device 1 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 1 .

[0073] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0074] The processor 110 provides computing and control capabilities. For example, the processor 110 is used to execute a computer program stored in the memory 120 to implement the above-mentioned sound effect adjustment method.

[0075] The communication bus 130 is at least used to provide a channel for the memory 120 and the processor 110 in the electronic device 1 to communicate with each other.

[0076] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 1. In other embodiments of the present application, the electronic device 1 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0077] The embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the electronic device 1, the electronic device 1 executes the above-mentioned related method steps to implement the sound effect adjustment method in the above-mentioned embodiment.

[0078] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the sound effect adjustment method in the above-mentioned embodiment.

[0079] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the sound effect adjustment method in the above-mentioned method embodiments.

[0080] Among them, the server, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0081] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0082] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program code.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A sound effect adjustment method, applied to electronic equipment, characterized in that: The method comprises: Build a style classification model based on the song dataset; In response to the sound system playing a song, obtaining song data of the song, and identifying style information of the song based on the song data by using the style classification model; Determining a target sound effect mode corresponding to the song based on the style information and the correspondence between the plurality of style information and the plurality of sound effect modes; The speaker is controlled to play the song based on the target sound effect mode.

2. The sound effect adjustment method according to claim 1, wherein: The song data set includes audio data of multiple songs, and the method of establishing a style classification model based on the song data set includes: Extracting a target feature of each song based on the audio data of each song, wherein the target feature includes at least one of a rhythm feature, a melody feature, and a chord feature; Input the target features into a preset deep learning model to predict the style information of each song; The loss function value of the preset deep model is calculated based on the style information of each song, and the parameters of the preset deep model are adjusted based on the loss function value to obtain the style classification model.

3. The sound effect adjustment method according to claim 2, wherein: The step of extracting target features of each song based on the audio data of each song includes: The audio data of each song is divided into main song audio data and accompaniment audio data; Analyzing the main song audio data using a beat tracking algorithm to obtain the rhythm features; Extracting audio features of the main song audio data to obtain the melody features; The audio feature of the accompaniment audio data is extracted to obtain the chord feature.

4. The sound effect adjustment method according to claim 3, characterized in that: The step of extracting the audio features of the main song audio data comprises: Performing frame division and windowing on the main song audio data; Performing Fourier transform on the verse song audio data after frame division and windowing to obtain frequency domain features of the verse song audio data; The Mel-frequency cepstral coefficients of the main song audio data are calculated based on the frequency domain features as the audio features of the main song audio data.

5. The sound effect adjustment method according to claim 2, wherein: The song data set also includes text data of the plurality of songs, and establishing a style classification model based on the song data set includes: Extracting text features of each song based on the text data of each song; Inputting the text features and the target features into the preset deep learning model to predict the style information of each song; The loss function value of the preset deep model is calculated based on the style information of each song, and the parameters of the preset deep model are adjusted based on the loss function value to obtain the style classification model.

6. The sound effect adjustment method according to claim 5, characterized in that: The extracting of text features of each song based on the text data of each song includes: Performing word segmentation processing on the text data of each song, and constructing a vocabulary based on the text data after the word segmentation processing; A word embedding model is used to generate a feature vector of the text data based on the vocabulary.

7. The sound effect adjustment method according to claim 1, wherein: The step of acquiring the song data of the song and identifying the style information of the song based on the song data by using the style classification model includes: Collecting song information of the song, and acquiring audio data and text data of the song based on the song information; Inputting the audio data and the text data into the style classification model, and extracting target features of the song based on the audio data using the style classification model, wherein the target features include at least one of rhythm features, melody features, and chord features; Extracting text features of the song based on the text data; Fusing the target feature and the text feature to obtain a fused feature; Linear mapping is performed on the fused features to obtain the style information of the song.

8. The sound effect adjustment method according to claim 7, wherein: The step of identifying the style information of the song based on the song data by the style classification model further includes: The audio data of the song is divided into a plurality of sub-audio data segments, each sub-audio data segment is input into the style classification model, and the target feature of each sub-audio data segment is extracted using the style classification model; Linear mapping is performed on the target features of each sub-audio data to obtain the style information of each sub-audio data.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor: Wherein, the memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the electronic device executes the sound effect adjustment method as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium stores program instructions, and when the program instructions are executed on an electronic device, a processor of the electronic device executes the sound effect adjustment method according to any one of claims 1 to 8.

Images