Music denoising method and system based on channel attention model

Abstract

The application provides a music denoising method and system based on a channel attention model, belongs to the technical field of intelligent music processing, forms a noisy music data set suitable for music denoising, and trains an effective channel attention-based feature learning network and a noise filtering network based on the noisy music data set; the effective channel attention-based feature learning network is used to obtain enhanced attention features, and then the music after denoising is obtained according to the enhanced attention features. The application can improve the distortion of the music recorded by non-professional users in a natural scene caused by various background noises, solve the problems of expensive music denoising equipment and complex operation, and effectively capture enough noise music information by fusing an efficient channel attention model, thereby improving the generalization ability of the music denoising system based on the channel attention model to different noises and music.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent music processing, and particularly relates to a music denoising method and system based on a channel attention model, an electronic device, and a storage medium. BACKGROUND

[0002] With the rapid development of multimedia technology, people can record and share their music recordings on social platforms. Compared with professional recordings, the music recordings of ordinary users are usually affected by background noise and recording equipment, resulting in reduced quality. Therefore, there is an urgent need for an automatic music denoising method for natural environment recordings in practical applications.

[0003] The traditional music denoising process is a labor-intensive process that relies on excellent sound engineers. The music denoising task belongs to the category of audio denoising. In the field of audio denoising, Boll disclosed a spectral subtraction algorithm in the prior art to reduce the spectral effects of acoustic noise in speech. Ephraim and Van Trees proposed a perceptually-based linear signal estimator for enhancing speech signals. The above methods estimate parameters from noise and establish different noise models, but due to the simplification of the assumption conditions in the noise model, these methods have poor processing effects on non-stationary noise.

[0004] With the development of deep learning, there are technologies in the prior art that use deep learning to improve the performance of denoising models. Among them, Hu et al. proposed a deep complex convolution recurrent network (DCCRN) for speech enhancement, which uses a complex network to model complex-valued spectra. In addition, Xu et al. introduced a learning model based on observations of human speech, which can automatically denoise speech using the silent intervals of human speech. Although the processing effect on non-stationary noise is improved, there are still disadvantages: compared with speech signals, music has more complex excitation signals and spectral shapes, resulting in poor performance of current speech denoising methods in processing noisy music recordings.

[0005] Therefore, there is an urgent need for a music denoising method based on a channel attention model suitable for music recordings. SUMMARY

[0006] The present application provides a music denoising method and system based on a channel attention model, an electronic device, and a storage medium to overcome at least one technical problem in the prior art.

[0007] To achieve the above-mentioned purpose, the present application provides a music denoising method based on a channel attention model, the method comprising:

[0008] obtain the noisy music data to be processed, and perform format conversion and channel addition preprocessing on the noisy music data;

[0009] input the preprocessed noisy music data into a preset effective channel attention-based feature learning network for feature extraction, to obtain enhanced attention features corresponding to the noisy music data; the effective channel attention-based feature learning network is trained by a noisy music data set, and the noisy music data in the noisy music data set is randomly generated from noise data and music data; the noise data includes five categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise; the music data includes nine categories of piano, drum set, harp, cello, pipa, trumpet, zither, multi-instrument and song;

[0010] input the enhanced attention features into a preset noise filtering network for noise filtering, to obtain denoised music features; and convert the music features to the time domain through inverse short-time Fourier transform, to obtain denoised music data.

[0011] Further, preferably, the method for performing format conversion and channel addition preprocessing on the noisy music data to be processed comprises:

[0012] perform sampling frequency conversion on the noisy music data to be processed, to obtain single-channel audio data with a set sampling frequency;

[0013] perform short-time Fourier transform on the single-channel audio data with the set sampling frequency and add 10 frequency-position embedding vectors, to obtain preprocessed noisy music data with a set number of channels.

[0014] Further, preferably, the training method of the effective channel attention-based feature learning network comprises: obtaining a noisy music data set; the noisy music data set is composed of pure music and noisy music data pairs;

[0015] perform format conversion and channel addition preprocessing on the music data in the noisy music data set, to obtain preprocessed music data with a set number of channels;

[0016] perform convolution layer and exponential linear unit processing on the preprocessed music data with the set number of channels, to obtain early convolution features; and perform feature extraction on the early convolution features through a U-net network; the U-net network includes four encoder modules and four decoder modules, an effective channel attention module arranged between the encoder modules and the decoder modules, and a supervised attention module arranged after the decoder modules;

[0017] The parameters of the effective channel attention based feature learning network are trained and constrained by using a loss function, and then the parameter values of the effective channel attention based feature learning network are determined; and the training of the effective channel attention based feature learning network is completed.

[0018] Further, preferably, each encoder is composed of a three-layer DenseNet block with a residual connection; the encoder and the decoder share the same hyperparameters, and the jump connection is between each encoder and the corresponding mirror decoder.

[0019] Further, preferably, the enhanced attention feature is input into a preset noise filtering network for noise filtering to obtain denoised music features; and the music features are converted to the time domain through inverse short-time Fourier transform to obtain denoised music data, which is achieved by the following formula:

[0020] R DM (t)=ISTFT(Filtering(Learning(cat[STFT(R NM (t)),FP])))

[0021] Where, ISTFT represents the inverse short-time Fourier transform process, Filtering represents the noise filtering module processing process, Learing represents the feature learning module processing process, cat[·] represents the channel connection operation, and FP is 10 frequency-position embedding vectors.

[0022] Further, preferably, the noisy music data is obtained by the following formula:

[0023] R NM (t)=R CM (t)+αR N (t)

[0024] Where, R CM (t) is randomly selected pure music, alpha is a proportion factor of signal-to-noise ratio, R N (t) is randomly selected noise; and R N (t) is copied in time by using the overlap-add synthesis method, so that R N (t) and R CM (t) have the same length.

[0025] Further, preferably, the training method of the noise filtering network comprises:

[0026] The music data of the noisy music data set composed of pure music and noisy music data pairs is extracted by using the effective channel attention based feature learning network to obtain the enhanced attention features corresponding to the music data;

[0027] The noise filtering network is trained by using the enhanced attention features corresponding to the music data, wherein the noise filtering network is a U-net network structure.

[0028] The noise filtering network is trained by using the average absolute error loss function, the parameter value of the noise filtering network is obtained, and the training of the noise filtering network is completed.

[0029] To solve the above problems, the application further provides a music denoising system based on a channel attention model, comprising:

[0030] An acquisition unit is configured to acquire noisy music data to be processed, and convert the format and add a channel preprocessing to the noisy music data;

[0031] A feature enhancement unit is configured to input the preprocessed noisy music data into a preset feature learning network based on effective channel attention to extract features, and obtain enhanced attention features corresponding to the noisy music data; wherein the feature learning network based on effective channel attention is obtained by training a noisy music data set, and the noisy music data in the noisy music data set is randomly generated by noise data and music data; the noise data includes five categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise; and the music data includes nine categories of piano, drum set, harp, cello, pipa, trumpet, zither, multi-instrument and song.

[0032] A denoising unit is configured to input the enhanced attention features into a preset noise filtering network to filter noise and obtain denoised music features; and the music features are converted to the time domain by inverse short-time Fourier transform to obtain denoised music data.

[0033] To solve the above problems, the application further provides an electronic device, which comprises:

[0034] At least one processor; and,

[0035] A memory in communication connection with the at least one processor; wherein,

[0036] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps in the music denoising method based on the channel attention model.

[0037] 10. A computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the music denoising method based on the channel attention model according to any one of claims 1 to 7.

[0038] To solve the above problems, the application further provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the music denoising method based on the channel attention model.

[0039] The music denoising method, system, electronic device and storage medium based on the channel attention model can improve the distortion of music recorded by non-professional users in a natural scene caused by various background noises, solve the problems of expensive music denoising equipment and complex operation, and effectively capture sufficient noise music information by fusing the efficient channel attention model, thereby improving the generalization ability of the music denoising system based on the channel attention model to different noises and music. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0041] Figure 1 The flowchart of the music denoising method based on the channel attention model provided by an embodiment of the present application;

[0042] Figure 2 The principle diagram of the music denoising method based on the channel attention model provided by an embodiment of the present application;

[0043] Figure 3 The denoising effect diagram of different types of noisy music simulation provided by an embodiment of the present application; (a) is the video picture corresponding to the cello solo music, the audio signal spectrogram after mixing the noise and the audio signal spectrogram after denoising; (b) is the video picture corresponding to the harp and piano ensemble music, the audio signal spectrogram after mixing the noise and the audio signal spectrogram after denoising;

[0044] Figure 4 The denoising effect diagram of different types of noisy music in actual life provided by an embodiment of the present application; (a) is a diagram for denoising the street violin solo performance recorded by using a common mobile device; (b) is a diagram for denoising the multi-instrument ensemble performance recorded in a non-professional recording studio;

[0045] Figure 5 This is a schematic diagram of a music denoising system based on a channel attention model according to an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the internal structure of an electronic device that implements a music denoising method based on a channel attention model according to an embodiment of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] Figure 1 This is a flowchart illustrating a music denoising method based on a channel attention model according to an embodiment of the present invention. The method can be executed by a system, which can be implemented in software and / or hardware.

[0050] like Figure 1 As shown, in this embodiment, the music denoising method based on the channel attention model includes steps S110 to S130.

[0051] S110. Obtain the noise-containing music data to be processed, and perform preprocessing such data by converting the format and adding channels.

[0052] The method for preprocessing the noise-containing music data to be processed by converting its format and adding channels includes: S111, converting the sampling frequency of the noise-containing music data to obtain mono audio data with a set sampling frequency; S112, performing a short-time Fourier transform on the mono audio data with the set sampling frequency and adding 10 frequency-position embedding vectors to obtain preprocessed noise-containing music data with a set number of channels. In the specific implementation process, the sampling frequency and number of channels are first converted to ensure that the input audio data pairs all meet the set sampling frequency of 44100Hz and are mono audio; then, a short-time Fourier transform (STFT) is performed and 10 frequency-position embedding vectors are added to make it conform to the set number of channels. In this embodiment, the number of channels of the input data is 12.

[0053] The existing music denoising model only relies on coding and decoding operations to extract relevant information of the noisy music, and it is difficult to fully aggregate the features, thereby reducing the denoising effect to a certain extent. In view of the shortcomings of the existing music denoising model, the present application proposes an automatic music denoising method fusing an efficient channel attention model, which can realize effective denoising of music recorded by non-professional users in a natural environment.

[0054] S120, input the preprocessed noisy music data into the preset feature learning network based on effective channel attention for feature extraction, and obtain enhanced attention features corresponding to the noisy music data; wherein the feature learning network based on effective channel attention is obtained by training the noisy music data set, and the noisy music data in the noisy music data set is randomly generated by noise data and music data; the noise data includes 5 categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise; the music data includes 9 categories of piano, drum, harp, cello, pipa, trumpet, zither, multi-instrument and song.

[0055] That is, in order to solve the problem that the existing music denoising model involves limited types of musical instruments and insufficient diversity of processable noise scenes, the present application follows two principles of music form and recording environment, collects 63 noise records containing 5 categories with a total length of 1283 seconds and 1100 music records containing 9 categories with a total length of 11000 seconds to train the music denoising network based on the effective channel attention model of the present application, effectively simulates the possible environment of music performance with different background noise, and improves the richness of the types of musical instruments involved in the model and the diversity of processable noise scenes. It should be noted that the music records used are natural and smooth, and match the music form, and no rhythm or melody, randomly spliced instrument playing fragments are used. The noise records used are the types of noise often contained when recording audio in a natural scene, and noise types that usually do not appear in a recording scene, such as gun noise, are not used.

[0056] The training method of the feature learning network based on effective channel attention includes: S121, obtaining a noisy music data set; wherein the noisy music data set is composed of pure music and noisy music data pairs.

[0057] Specifically, the existing music denoising model in the prior art mainly learns the noise characteristics in historical music recordings for denoising, and the types of instruments involved are limited, and the types of noise that can be processed are single; however, the noise components contained in the music recorded in the natural scene are complex, such as crowd noise, traffic noise, etc.; therefore, there is a problem that the existing music denoising model is difficult to generalize to the music recorded in the natural scene. The present application trains an effective channel attention-based feature learning network by using 63 noise records containing 5 categories and 1100 music records containing 9 categories to randomly generate <pure, noisy> audio data pairs; it should be noted that the noise categories include circuit noise, which is used to simulate the noise caused by the quality or operation problem of the device; crowd noise, which is used to simulate the crowd chatting, cheering and children's laughter that may occur when recording in public places; weather noise, which is used to simulate the noise caused by weather conditions when the user records; traffic noise, which is used to simulate the noise when the user records the audio near the lane; stationary noise, which is used to simulate the potential noise when the user processes the recorded audio in the time-frequency domain. The music data includes nine categories of piano, drum set, harp, cello, pipa, trumpet, zither, multi-instrument and song. Among them, the multi-instrument category contains recordings of different instruments, and the song category contains both vocal music and musical accompaniment.

[0058] Since the noise records can be randomly arranged and recombined, these data are sufficient to well simulate the noise music scene. In a specific embodiment, the noisy music data is obtained by the following formula:

[0059] R NM (t)=R CM (t)+αR N (t)

[0060] Wherein, R CM (t) is a randomly selected pure music, alpha is a proportional factor of signal-to-noise ratio (SNR), R N (t) is a randomly selected noise; and R N (t) is copied in time by using the overlap-add synthesis method, so that R N (t) and R CM (t) have the same length. In the specific implementation process, by changing the value of alpha, noise music with different noise levels can be obtained.

[0061] S122, convert the format of the music data in the noisy music data set and add channel preprocessing to obtain preprocessed music data with a set number of channels.

[0062] Specifically, the processing method is the same as steps S111-S112; that is, firstly, the conversion of the sampling frequency and the number of channels is performed, to ensure that the input audio data meets the set sampling frequency of 44100Hz and is single-channel audio; then, short-time Fourier transform is performed and 10 frequency-position embedding vectors are added, so as to meet the set channel number of 12.

[0063] That is, the preprocessed noise-containing music data of the set channel number of the final output of step S112 can be expressed as:

[0064] cat[STFT(R NM (t)),FP]

[0065] Wherein, cat[·] represents a channel connection operation, STFT is Fourier transform, and FP is 10 frequency-position embedding vectors.

[0066] Figure 2 The principle schematic diagram of the music denoising method based on the channel attention model provided by an embodiment of the present application is shown in Fig. 1. Figure 2 As shown in Fig. 1, the feature learning network based on effective channel attention is an improved U-net network structure. By introducing an efficient channel attention-Efficient Channel Attention (ECA) mechanism into the traditional U-net and combining a supervised attention module (Supervised Attention Module, SAM), a feature learning network based on ECA-U-SAM is designed, which enhances the feature aggregation capability and improves the versatility of the entire algorithm to various music and noise environments.

[0067] S123, the preprocessed music data of the set channel number is processed through a convolution layer and an exponential linear unit to obtain early convolution features; the early convolution features are extracted through a U-net network; the U-net network includes four encoder modules and four decoder modules, an effective channel attention module arranged between the encoder modules and the decoder modules, and a supervised attention module arranged after the decoder modules.

[0068] Wherein, each encoder is composed of a three-layer DenseNet block with a residual connection; the encoder and the decoder share the same hyperparameters, and there is a skip connection between each encoder and the corresponding mirror decoder.

[0069] Specifically, the preprocessed 12-channel input cat[STFT(R NM (t)),FP] is input to a convolution layer, and an Exponential Linear Unit (ELU) is arranged behind the convolution layer to learn early convolution features FC . Wherein, the exponential linear unit combines the advantages of both Sigmoid and ReLU activation functions, which can keep soft saturation like Sigmoid, making ELU more robust to input changes, while also alleviating the gradient vanishing problem like ReLU.

[0070] Then, an improved U-net network structure is adopted to encode the output feature F C for noise estimation. Specifically, the feature learning network based on effective channel attention is a U-net network with four encoder modules and four decoder modules. Each encoder module is composed of a three-layer DenseNet Block with residual connection. The encoder block shares the same hyperparameters with its symmetrical decoder block. The DenseNet Block performs channel-level connection internally. Therefore, the feature representation obtained from the mapping can be reused continuously to improve the efficiency of feature extraction. In addition, the skip connection between each encoder block and its mirror decoder block can make the extracted features more comprehensive.

[0071] The ECA module is located between the encoder and the decoder, the output of the encoder is fed into the ECA module, and the output of the ECA module is fed into the first decoder block. The detailed architecture of the ECA module adopted is shown in Table 1.

[0072] Table 1. Detailed architecture table of ECA module

[0073]

[0074] In Table 1, the four-parameter group corresponding to the "output shape" column refers to the batch size, output height, output width, and output channel number of the current layer; the three-parameter group corresponds to the batch size, output channel number, and output dimension of the current layer; the two-parameter group corresponds to the batch size and output channel number of the current layer. K is the size of the convolution kernel, stride is the convolution step; padding represents the number of layers of 0 added to each side of the input; element-level intelligent product (Element-wise Product) is to multiply each input vector by a given "weight" vector. In other words, it is to scale each column of the data set by a multiplier.

[0075] In the specific implementation process, in order to send the most relevant features into the noise filtering network, the attention feature F A output by the ECA-U-net is fed into the SAM, which can filter out features containing useful information and remove features containing redundant information through a series of convolution operations.

[0076] Finally, the feature learning module outputs the enhanced attention feature FE This is achieved by the following formula:

[0077] F E = Learning(cat[STFT(R NM (t)),FP])

[0078] where Learning denotes the feature learning module processing process, cat[·] denotes the channel connection operation, and FP is the 10 frequency-location embedding vectors.

[0079] S124, the parameters of the feature learning network based on effective channel attention are constrained by using the loss function, and then the parameter values of the feature learning network based on effective channel attention are determined; the training of the feature learning network based on effective channel attention is completed.

[0080] S130, input the enhanced attention feature into the preset noise filtering network for noise filtering to obtain the denoised music feature; the music feature is converted to the time domain through inverse short-time Fourier transform to obtain the denoised music data.

[0081] It should be noted that the enhanced attention feature is input into the preset noise filtering network for noise filtering to obtain the denoised music feature; the music feature is converted to the time domain through inverse short-time Fourier transform. Specifically, the enhanced attention feature formed by the feature learning network based on effective channel attention is input into the noise filtering network to complete the training of the final denoising model. In the specific implementation process, a single U-net and a convolution structure with the same structure as the feature learning network are used to constitute the noise filtering network, and the specific steps are as follows: the enhanced attention feature F E output from the feature learning network is input into the single U-net, and after coding and decoding operations and convolution structure, the denoised music feature F o is learned and generated, and finally the feature can be converted to the time domain through inverse short-time Fourier transform (inverse Short-Time Fourier Transform, Fourier) to generate and output the denoised music recording R DM (t); this is achieved by the following formula:

[0082] R DM (t) = ISTFT(Filtering(Learning(cat[STFT(R NM (t)),FP])))

[0083] wherein, ISTFT represents an inverse short-time Fourier transform process, Filtering represents a noise filtering module processing process, Learing represents a feature learning module processing process, cat[·] represents a channel connection operation, and FP is 10 frequency-position embedding vectors.

[0084] The training method of the noise filtering network comprises the following steps: S131, using a feature learning network based on effective channel attention to extract features from music data in a noisy music data set composed of pure music and noisy music data pairs, to obtain enhanced attention features corresponding to the music data; S132, using the enhanced attention features corresponding to the music data to train the noise filtering network; wherein, the noise filtering network is a U-net network structure; S133, using an average absolute error loss function to constrain the training of the noise filtering network, to obtain the parameter value of the noise filtering network, and complete the training of the noise filtering network.

[0085] The optimal parameter estimation value is obtained by minimizing the average absolute error loss function. The average absolute error loss function optimization algorithm is used to automatically and iteratively obtain the optimal integration weight, so that the experimental results are greatly improved. After obtaining the trained noise filtering network, the noise filtering network needs to be tested using a test set.

[0086] Figure 3 and Figure 4 The principle of the music denoising method based on the channel attention model is described in the embodiment of the application; wherein, Figure 3 Fig. 1 is a schematic diagram of the denoising effect of different types of simulated noisy music provided by an embodiment of the application; (a) is a video screen corresponding to a cello solo music, an audio signal spectrogram after mixing noise, and an audio signal spectrogram after denoising; (b) is a video screen corresponding to a harp and piano ensemble music, an audio signal spectrogram after mixing noise, and an audio signal spectrogram after denoising; Figure 4 Fig. 2 is a schematic diagram of the denoising effect of different types of noisy music in actual life provided by an embodiment of the application; (a) is a schematic diagram of denoising a street cello solo performance recorded using a common mobile device; (b) is a schematic diagram of denoising a multi-instrument ensemble performance recorded in a non-professional recording studio.

[0087] from Figure 3 It can be seen that the application can well denoise the simulated instrument solo and multi-instrument performance in the laboratory; from Figure 4 It can be seen that the application can well denoise the instrument solo and multi-instrument performance in the actual life scene.

[0088] In summary, the music denoising method based on the channel attention model can improve the distortion of music recorded by non-professional users in a natural scene caused by various background noises, solve the problems of expensive music denoising equipment and complex operation, and effectively capture sufficient noise music information by fusing an efficient channel attention model, thereby improving the generalization ability of the music denoising method based on the channel attention model for different noises and music.

[0089] As shown in Figure 5 The music denoising system 500 based on the channel attention model can obtain a feature enhancement unit 520 and a denoising unit 530 according to the functions to be implemented. The units of the present application can also be referred to as modules, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0090] In this embodiment, the functions of each module / unit are as follows:

[0091] The obtaining unit 510 is configured to obtain noisy music data to be processed, and perform format conversion and channel preprocessing on the noisy music data.

[0092] The feature enhancement unit 520 is configured to input the preprocessed noisy music data into a preset feature learning network based on effective channel attention for feature extraction, to obtain enhanced attention features corresponding to the noisy music data. The feature learning network based on effective channel attention is obtained by training a noisy music data set. The noisy music data in the noisy music data set is randomly generated from noise data and music data. The noise data includes five categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise. The music data includes nine categories of piano, snare drum, harp, cello, pipa, trumpet, zither, multi-instrument and song.

[0093] The denoising unit 530 is configured to input the enhanced attention features into a preset noise filtering network for noise filtering, to obtain music features after denoising. The music features are converted to the time domain by inverse short-time Fourier transform, to obtain music data after denoising.

[0094] The music denoising system based on the channel attention model 500 of the present application can improve the distortion of the music recorded by non-professional users in a natural scene caused by various background noises, solve the problem of expensive music denoising equipment and complex operation, and effectively capture sufficient noise music information by fusing the efficient channel attention model, thereby improving the generalization ability of the music denoising system based on the channel attention model of the present application to different noises and music.

[0095] As shown in Figure 6 The present application provides an electronic device 6 for a music denoising method based on a channel attention model.

[0096] The electronic device 6 can include a processor 60, a memory 61 and a bus, and can further include a computer program stored in the memory 61 and executable on the processor 60, such as a music denoising program based on a channel attention model 62. The memory 61 can include both an internal storage unit of the music denoising system based on the channel attention model and an external storage device. The memory 61 can be used not only to store installed application software and various data, such as the code of the music denoising program based on the channel attention model, but also to temporarily store data that has been output or will be output.

[0097] The memory 61 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 61 can be an internal storage unit of the electronic device 6 in some embodiments, such as a mobile hard disk of the electronic device 6. The memory 61 can also be an external storage device of the electronic device 6 in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 61 can include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 can be used not only to store installed application software and various data of the electronic device 6, such as the code of the music denoising program based on the channel attention model, but also to temporarily store data that has been output or will be output.

[0098] The processor 60 may, in some embodiments, be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits of the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 60 is the control core of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 61 (such as a music denoising program based on a channel attention model, etc.), and calls data stored in the memory 61 to perform various functions and process data of the electronic device 6.

[0099] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 61 and the at least one processor 60, etc.

[0100] Figure 6 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 6 The structure shown does not constitute a limitation on the electronic device 6, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0101] For example, although not shown, the electronic device 6 can also include a power supply (such as a battery) for powering various components. Preferably, the power supply can be logically connected to the at least one processor 60 through a power management system, so as to realize functions such as charge management, discharge management, and power consumption management through the power management system. The power supply can also include one or more direct current or alternating current power supplies, a recharging system, a power supply fault detection circuit, a power supply converter or inverter, a power supply status indicator, and any other components. The electronic device 6 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0102] Further, the electronic device 6 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is usually used to establish a communication connection between the electronic device 6 and other electronic devices.

[0103] Optionally, the electronic device 6 can also include a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display information processed in the electronic device 6 and to display a visualized user interface.

[0104] It should be understood that the above embodiments are only for illustration and do not limit the scope of the patent application.

[0105] The music denoising program 62 based on the channel attention model stored in the memory 61 in the electronic device 6 is a combination of a plurality of instructions, which, when executed in the processor 60, can achieve: obtaining the to-be-processed noisy music data, and converting the format and adding channel preprocessing to the noisy music data; inputting the preprocessed noisy music data into a preset effective channel attention based feature learning network for feature extraction to obtain enhanced attention features corresponding to the noisy music data; wherein the effective channel attention based feature learning network is obtained by training a noisy music data set, and the noisy music data in the noisy music data set is randomly generated by noise data and music data; the noise data includes five categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise; the music data includes nine categories of piano, snare drum, harp, cello, pipa, trumpet, zither, multi-instrument and song; inputting the enhanced attention features into a preset noise filtering network for noise filtering to obtain denoised music features; and converting the music features to the time domain through inverse short-time Fourier transform to obtain denoised music data.

[0106] Specifically, the specific implementation method of the above instructions by the processor 60 can refer to Figure 1 The description of the related steps in the corresponding embodiments will not be repeated here. It should be emphasized that, in order to further ensure the privacy and security of the above music denoising program based on the channel attention model, the above database high-availability processing data is stored in the nodes of the blockchain where the server cluster is located.

[0107] Further, the modules / units integrated in the electronic device 6, if implemented in the form of software function units and sold or used as an independent product, can be stored in a computer readable storage medium. The computer readable medium can include any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0108] The embodiment of the present application also provides a computer readable storage medium, which can be non-volatile or volatile, and stores a computer program. The computer program is executed by a processor to realize the following steps: obtaining to-be-processed noisy music data, and performing format conversion and channel addition preprocessing on the noisy music data; inputting the preprocessed noisy music data into a preset effective channel attention-based feature learning network to perform feature extraction, and obtaining enhanced attention features corresponding to the noisy music data; the effective channel attention-based feature learning network is obtained by training a noisy music data set, and the noisy music data in the noisy music data set is randomly generated from noise data and music data; the noise data includes five categories of circuit noise, crowd noise, weather noise, traffic noise and stationary noise; the music data includes nine categories of piano, snare drum, harp, cello, pipa, trumpet, zither, multi-instrument and song; inputting the enhanced attention features into a preset noise filtering network to perform noise filtering, and obtaining denoised music features; performing inverse short-time Fourier transform conversion on the music features to the time domain, and obtaining denoised music data.

[0109] Specifically, the computer program is executed by the processor to specifically implement the method, which can refer to the description of the related steps in the music denoising method based on the channel attention model in the embodiment, and details are not described herein.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner.

[0111] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment scheme.

[0112] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function module.

[0113] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0114] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0115] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptography. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0116] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or systems stated in the system claims can also be implemented by one unit or system through software or hardware. The second word is used to indicate the name, and does not indicate any specific order.

[0117] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A music denoising method based on a channel attention model, characterized in that, include: Acquire the noise-containing music data to be processed, and perform preprocessing such data including format conversion and channel addition; The preprocessed noisy music data is input into a pre-defined feature learning network based on effective channel attention for feature extraction, obtaining enhanced attention features corresponding to the noisy music data. The feature learning network based on effective channel attention is trained using a noisy music dataset, where the noisy music data is randomly generated from noise and music data. The noise data includes five categories: circuit noise, crowd noise, weather noise, traffic noise, and steady noise. The music data includes nine categories: piano, drum kit, harp, cello, pipa, trumpet, guzheng, multi-instrument music, and songs. The training method for the feature learning network based on effective channel attention includes: acquiring a noisy music dataset; wherein the noisy music dataset is composed of clean music and noisy music data. The process involves: converting the format of the music data in the noisy music dataset and adding channels for preprocessing to obtain preprocessed music data with a set number of channels; processing the preprocessed music data with the set number of channels through convolutional layers and exponential linear units to obtain early convolutional features; extracting features from the early convolutional features using a U-net network; the U-net network includes four encoder modules and four decoder modules, an effective channel attention module set between the encoder modules and the decoder modules, and a supervised attention module set after the decoder modules; using a loss function to constrain the parameters of the feature learning network based on effective channel attention, thereby determining the parameter values ​​of the feature learning network based on effective channel attention; and completing the training of the feature learning network based on effective channel attention. The enhanced attention features are input into a preset noise filtering network for noise filtering to obtain denoised music features; the music features are then transformed to the time domain using inverse short-time Fourier transform to obtain denoised music data; the training method of the noise filtering network includes: using a feature learning network based on effective channel attention to extract features from music data in a noisy music dataset consisting of pairs of clean music and noisy music data to obtain enhanced attention features corresponding to the music data; training the noise filtering network using the enhanced attention features corresponding to the music data; wherein the noise filtering network is a U-net network structure; and using the mean absolute error loss function to constrain the training of the noise filtering network, obtaining the parameter values ​​of the noise filtering network, and completing the training of the noise filtering network.

2. The music denoising method based on the channel attention model according to claim 1, characterized in that, The method for preprocessing the noise-containing music data to be processed, including format conversion and channel addition, includes: The sampling frequency of the noise-containing music data to be processed is converted to obtain mono audio data with a set sampling frequency; The mono audio data with the set sampling frequency is subjected to a short-time Fourier transform and 10 frequency-position embedding vectors are added to obtain preprocessed noisy music data with a set number of channels.

3. The music denoising method based on the channel attention model according to claim 1, characterized in that, Each encoder consists of a three-layer DenseNet block with residual connections; the encoder and the decoder share the same hyperparameters, and there are skip connections between each encoder and its corresponding mirror decoder.

4. The music denoising method based on the channel attention model according to claim 1, characterized in that, The method of inputting the enhanced attention features into a preset noise filtering network for noise filtering to obtain denoised music features; and transforming the music features to the time domain through inverse short-time Fourier transform to obtain denoised music data, is implemented by the following formula: in, ISTFT This represents the inverse short-time Fourier transform process. This indicates the processing procedure of the noise filtering module. This indicates the processing procedure of the feature learning module. Indicates a channel connection operation. It consists of 10 frequency-position embedding vectors.

5. The music denoising method based on the channel attention model according to claim 1, characterized in that, The noisy music data is obtained using the following formula: in, For randomly selected pure music, α is the signal-to-noise ratio scaling factor. The noise is randomly selected; and it is temporally replicated using an overlapping additive synthesis method. , making and They have the same length.

6. A music denoising system based on a channel attention model, characterized in that, include: The acquisition unit is used to acquire the noise-containing music data to be processed, and to perform preprocessing such preprocessing as format conversion and channel addition on the noise-containing music data. A feature enhancement unit is used to input the preprocessed noisy music data into a preset feature learning network based on effective channel attention for feature extraction, thereby obtaining enhanced attention features corresponding to the noisy music data. The feature learning network based on effective channel attention is trained using a noisy music dataset, where the noisy music data is randomly generated from noise and music data. The noise data includes five categories: circuit noise, crowd noise, weather noise, traffic noise, and steady noise. The music data includes nine categories: piano, drum kit, harp, cello, pipa, trumpet, guzheng, multiple instruments, and songs. The training method for the feature learning network based on effective channel attention includes: acquiring a noisy music dataset; wherein the noisy music dataset consists of clean music, noisy music, and music... The music data is composed of several parts. The music data in the noisy music dataset undergoes format conversion and channel addition preprocessing to obtain preprocessed music data with a set number of channels. This preprocessed music data with the set number of channels is then processed through convolutional layers and exponential linear units to obtain early convolutional features. These early convolutional features are then extracted using a U-net network. The U-net network includes four encoder modules and four decoder modules, an effective channel attention module positioned between the encoder and decoder modules, and a supervised attention module positioned after the decoder modules. A loss function is used to constrain the parameters of the feature learning network based on effective channel attention during training, thereby determining the parameter values ​​of the feature learning network based on effective channel attention. Training of the feature learning network based on effective channel attention is then completed. A denoising unit is used to input the enhanced attention features into a preset noise filtering network for noise filtering to obtain denoised music features; the music features are then transformed to the time domain using an inverse short-time Fourier transform to obtain denoised music data; wherein, the training method of the noise filtering network includes: using a feature learning network based on effective channel attention to extract features from music data in a noisy music dataset consisting of pairs of clean music and noisy music data to obtain enhanced attention features corresponding to the music data; training the noise filtering network using the enhanced attention features corresponding to the music data; wherein, the noise filtering network is a U-net network structure; and using the mean absolute error loss function to constrain the training of the noise filtering network, obtaining the parameter values ​​of the noise filtering network, and completing the training of the noise filtering network.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps in the music denoising method based on the channel attention model as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the music denoising method based on the channel attention model as described in any one of claims 1 to 5.

Images