Pitch estimation method based on compressed excitation block

Abstract

The invention relates to a pitch estimation method based on a compressed excitation block, and the method comprises the steps: converting an audio file into a logarithmic Mel spectrogram, thereby converting an input audio into advanced representation which is easier to understand by a machine, screening redundant information, and reserving and strengthening information which is beneficial to pitch extraction. Carrying out feature extraction on the logarithm Mel spectrogram by adopting a compressed excitation block, and extracting audio global features; meanwhile, a self-attention mechanism is introduced to extract local important features, pitch representation content corresponding to the input audio is obtained through a coding and decoding structure, a pitch probability matrix is obtained through an estimation module, and a pitch estimation result of the audio file is finally determined according to the pitch probability matrix. The problem of difficulty in pitch estimation of the traditional opera audio is effectively solved, and the accuracy of the prediction result of the musical pitch of the human sound is improved.

Description

Technical Field

[0001] The present application relates to the field of pitch estimation, and in particular to a pitch estimation method based on a compressed excitation block. Background Art

[0002] As a treasure of Chinese traditional culture, opera art has distinct artistic characteristics and profound cultural connotations. It integrates multiple art forms such as music, performance, dance, and literature, and forms a unique art system through multiple performance techniques such as acting, fighting, singing, and reciting. In opera, music and singing are the core elements, running through the entire performance process, and play a vital role in narrative, emotional expression, and character creation. Compared with modern pop music or Western classical music, the melody of opera is complex and changeable, the range of singing is large, and it is rich in subtle pitch changes, such as glissando, ornaments, and vibrato. The singing in opera art has a complex structure and style. For example, Peking Opera, Yue Opera, Henan Opera, etc. have their own unique vocal cavity system, which is often accompanied by complex pitch changes. Pitch is not only the basic element of melody in opera music, but also a unique artistic tool. Through delicate pitch changes, opera performers can vividly express emotions, shape characters, and promote the development of the plot. In opera performances, the communication of emotions often depends on the fine control of pitch. Subtle changes in pitch, such as the rise, fall, and extension, can express the complex emotions of a character, such as excitement, sadness, anger, and helplessness. Through precise pitch estimation technology, researchers can explore in depth how opera performers use pitch changes to convey complex emotions, and reveal the pitch characteristics and style differences of different schools and types of opera.

[0003] In today's digital age, pitch estimation also plays a vital role in the digital preservation and inheritance of opera music. The inheritance of opera art is inseparable from the accurate recording of classic performances and singing styles. As one of the core elements of performance, accurate pitch estimation can better record, analyze and preserve the audio data of opera. This not only helps to digitize the traditional characteristics of opera singing styles, but also helps the new generation of opera lovers to better learn and understand these singing techniques through technical means. However, the pitch estimation methods involved in the prior art have the problem of inaccurate estimation. Summary of the invention

[0004] In order to overcome at least one deficiency in the prior art, the present application provides a treble estimation method based on a compressed excitation block.

[0005] In a first aspect, a pitch estimation method based on a compressed excitation block is provided, comprising:

[0006] Constructing a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; the residual codec structure includes an encoder, a decoder, and an intermediate feature transformation module; the encoder includes multiple self-attention residual encoding layers and multiple residual convolution blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolution blocks;

[0007] Convert the audio file into a logarithmic Mel spectrogram, input the logarithmic Mel spectrogram into the pitch estimation model, and obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a pitch;

[0008] The compressed excitation block extracts features from the logarithmic Mel spectrum graph to obtain a preliminary feature graph; the preliminary feature graph is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder encode the preliminary feature graph to obtain a high-dimensional representation of the feature. The output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; the transformed feature is input into the decoder, and the decoder performs decoding operation based on the transformed feature and the output of each residual convolution block to obtain the decoded feature; the decoded feature is input into the estimation module for feature extraction to obtain a pitch probability matrix;

[0009] The pitch corresponding to the maximum value of the element in the pitch probability matrix is ​​selected as the pitch estimation result of the audio file.

[0010] In one embodiment, the compression excitation block includes a deconvolution layer, a residual convolution unit, a convolution layer, and a BN+ReLU layer connected in sequence; the residual convolution unit includes multiple residual convolution layers; the logarithmic Mel spectrum map is input into the deconvolution layer for upsampling to obtain the upsampled features; the upsampled features are input into the residual convolution unit, and each residual convolution layer performs a residual convolution operation on the upsampled features. After the output of each residual convolution layer is added, it is input into the convolution layer for convolution operation to obtain the convolution result; the convolution result is input into the BN+ReLU layer for batch normalization to obtain a preliminary feature map.

[0011] In one embodiment, the residual convolution layer includes a residual layer, a global pooling layer, a first fully connected layer, a second fully connected layer and a scale transformation layer; the residual layer performs a residual operation on the upsampled features to obtain a residual result; the residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer and the scale transformation layer to obtain the scale-transformed features; the scale-transformed features, the residual results and the upsampled features are added to obtain the output of the residual convolution layer.

[0012] In one embodiment, a plurality of self-attention residual coding layers are connected in sequence, and each self-attention residual coding layer includes an average pooling layer, a residual convolution block, a self-attention layer, a residual convolution block, a self-attention layer, and a residual convolution block connected in sequence;

[0013] The preliminary feature map is input into the encoder and processed by multiple self-attention residual coding layers in sequence. The last self-attention residual coding layer outputs a high-dimensional representation of the feature.

[0014] In one embodiment, multiple self-attention residual decoding layers are connected in sequence, and the output of the last self-attention residual decoding layer is the decoded feature. Each self-attention residual decoding layer includes a self-attention layer, 4 residual convolution blocks, a BN+ReLU layer and a deconvolution layer connected in sequence.

[0015] In one embodiment, the residual convolution block includes a convolution layer, a BN+ReLU layer, a convolution layer, and a BN+ReLU layer connected in sequence;

[0016] The input of the residual convolution block passes through the convolution layer, BN+ReLU layer, convolution layer, and BN+ReLU layer to obtain the first output; after the input is convolved, the second output is obtained; the first output and the second output are added to obtain the output of the residual convolution block.

[0017] In one embodiment, multiple intermediate convolution blocks are connected in sequence, and the output of the last intermediate convolution block is the transformed feature;

[0018] Each intermediate convolution block consists of a residual convolution block, a self-attention layer, a residual convolution block, and a residual convolution block connected in sequence.

[0019] In one embodiment, the estimation module includes a convolutional layer, an activation function layer, and a fully connected layer.

[0020] In one embodiment, constructing the pitch estimation model includes training the pitch estimation model, and the loss function used in the training is:

[0021]

[0022] Among them, L is the loss function, ω is the weight, i is the number of the pitch, y i is the true probability value of the pitch i of the training sample, is the probability value of pitch i estimated by the model.

[0023] In a second aspect, a pitch estimation device based on a compression excitation block is provided, comprising:

[0024] A model building module, used for building a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; the residual codec structure includes an encoder, a decoder, and an intermediate feature transformation module; the encoder includes a plurality of self-attention residual encoding layers and a plurality of residual convolution blocks, the decoder includes a plurality of self-attention residual decoding layers, and the intermediate feature transformation module includes a plurality of intermediate convolution blocks;

[0025] A pitch estimation module is used to convert the audio file into a logarithmic Mel-spectrogram, and input the logarithmic Mel-spectrogram into a pitch estimation model to obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a pitch;

[0026] The compressed excitation block extracts features from the logarithmic Mel spectrum graph to obtain a preliminary feature graph; the preliminary feature graph is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder encode the preliminary feature graph to obtain a high-dimensional representation of the feature. The output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; the transformed feature is input into the decoder, and the decoder performs decoding operation based on the transformed feature and the output of each residual convolution block to obtain the decoded feature; the decoded feature is input into the estimation module for feature extraction to obtain a pitch probability matrix;

[0027] The pitch determination module is used to select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file.

[0028] Compared with the prior art, the present application has the following beneficial effects: The pitch estimation method based on the compressed excitation block of the present application converts the audio file into a logarithmic Mel spectrogram, thereby converting the input audio into a high-level representation that is easier for the machine to understand, filtering out redundant information, and retaining and strengthening information that is helpful for pitch extraction. The compressed excitation block is used to extract features from the logarithmic Mel spectrogram to extract global audio features; at the same time, the self-attention mechanism is introduced to extract important local features, and the pitch representation content corresponding to the input audio is obtained through the codec structure. The pitch probability matrix is ​​obtained through the estimation module, and the pitch estimation result of the audio file is finally determined according to the pitch probability matrix. The present application effectively solves the problem of difficulty in estimating the pitch of opera audio and improves the accuracy of the pitch prediction results of vocal music. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present application may be better understood by referring to the following description given in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification. In the drawings:

[0030] Figure 1 shows a schematic diagram of a pitch estimation model;

[0031] Figure 2 shows a schematic diagram of a compressed excitation block;

[0032] Figure 3 shows a schematic diagram of a self-attention residual encoding layer;

[0033] Figure 4 shows a schematic diagram of the self-attention residual decoding layer;

[0034] Figure 5 A schematic diagram of a residual convolution block is shown. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present application will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions can be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.

[0036] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, only the device structure closely related to the scheme according to the present application is shown in the drawings, while other details that are not closely related to the present application are omitted.

[0037] It should be understood that the present application is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.

[0038] The present application provides a pitch estimation method based on a compressed excitation block, comprising:

[0039] Step S1, constructing a pitch estimation model; Figure 1 A schematic diagram of the pitch estimation model is shown in Figure 1 The pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; the residual codec structure includes an encoder (EncoderLayer), a decoder (Decoder Layer) and an intermediate feature transformation module (ICBs); the encoder includes multiple self-attention residual encoding layers (AREBs) and multiple residual convolution blocks (RCBs), the decoder includes multiple self-attention residual decoding layers (ARDBs), and the intermediate feature transformation module includes multiple intermediate convolution blocks (ICBs).

[0040] Step S2, converting the audio file into a logarithmic Mel-spectrogram, inputting the logarithmic Mel-spectrogram into a pitch estimation model to obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a pitch.

[0041] Here, the audio file is first processed, including resampling the input audio file to 16kHz, and then slicing the audio file. The length of each slice is controlled to 2s. If it is less than 2s, the audio is repeatedly padded to 2s. After slicing, the librosa.feature.melspectrogram and librosa.power_to_db functions in the Librosa library are used to generate the Mel-spectrogram and convert it into a log Mel-spectrogram. The purpose of the processing is to convert the input audio into a high-level representation that is easier for the machine to understand, filter out redundant information, and retain and enhance information that helps extract pitch.

[0042] The compressed excitation block extracts features from the logarithmic Mel spectrum graph to obtain a preliminary feature graph; the preliminary feature graph is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder encode the preliminary feature graph to obtain a high-dimensional representation of the feature. The output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; the transformed feature is input into the decoder, and the decoder performs decoding operation based on the transformed feature and the output of each residual convolution block to obtain the decoded feature; the decoded feature is input into the estimation module for feature extraction to obtain a pitch probability matrix;

[0043] Step S3, selecting the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file. Here, the pitch probability matrix includes 360 pitch probabilities.

[0044] After obtaining the pitch estimation result, this step can determine the corresponding estimated frequency according to the high estimation result. Specifically, the following formula can be used:

[0045]

[0046] Among them, f ref is the set frequency, which is 10Hz, and confi is the sound confidence. is the probability value of pitch i in the estimated pitch probability matrix, is the pitch of the audio file; for The maximum value of; th is the threshold, which is set to 0.5.

[0047] The above embodiment converts the audio file into a logarithmic Mel spectrogram, thereby converting the input audio into a high-level representation that is easier for the machine to understand, filtering out redundant information, and retaining and strengthening information that is helpful for pitch extraction. The compressed excitation block is used to extract features from the logarithmic Mel spectrogram to extract global audio features; at the same time, the self-attention mechanism is introduced to extract important local features, and the pitch representation content corresponding to the input audio is obtained through the codec structure. The pitch probability matrix is ​​obtained through the estimation module, and the pitch estimation result of the audio file is finally determined according to the pitch probability matrix. This embodiment effectively solves the problem of difficulty in estimating the pitch of opera audio and improves the accuracy of the pitch prediction results of vocal music.

[0048] In one embodiment, Figure 2 A schematic diagram of the compressed excitation block is shown in FIG. Figure 2 The compressed excitation block includes a deconvolution layer (ConvTranspose2d), a residual convolution unit (SEConvBlock), a convolution layer (Conv2d), and a BN+ReLU layer connected in sequence; the residual convolution unit includes multiple residual convolution layers; the logarithmic Mel spectrum map is input to the deconvolution layer for upsampling to obtain the upsampled features; the upsampled features are input to the residual convolution unit, and each residual convolution layer performs a residual convolution operation on the upsampled features. After the output of each residual convolution layer is added, it is input to the convolution layer for convolution operation to obtain the convolution result; the convolution result is input to the BN+ReLU layer for batch normalization to obtain a preliminary feature map.

[0049] Specifically, the residual convolution layer includes a residual layer (Residual), a global pooling layer (Global pooling), a first fully connected layer (FC+ReLU), a second fully connected layer (FC+Sigmoid) and a scale transformation layer (Scale); the residual layer performs a residual operation on the upsampled features to obtain a residual result; the residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer and the scale transformation layer to obtain the scale-transformed features; the scale-transformed features, the residual results and the upsampled features are added to obtain the output of the residual convolution layer.

[0050] In this embodiment, the input logarithmic Mel spectrum graph is first upsampled by the deconvolution layer, and its channel number is increased from 1 to 16, the data dimension is expanded, and the model can capture and represent feature information of more dimensions; the upsampled features are respectively input into 16 SEConvBlocks to extract effective content. In each SEConvBlock, the input upsampled features will compress the global spatial information into the channel descriptor through the global pooling layer, and embed the global information into the signal of each feature channel. Then, in order to utilize the information aggregated in the compression operation, two fully connected layers are used and different activation functions ReLU and Sigmoid are used respectively. The use of ReLU can reduce the computational cost, and the use of Sigmoid can fit complex curves more conveniently. The scale transformation is used to obtain the scaled features, and the scaled features, the residual results and the upsampled features are added to obtain the output of the residual convolution layer to achieve the residual effect, so that the model will not lose information.

[0051] In one embodiment, Figure 3 shows a schematic diagram of the self-attention residual encoding layer, see Figure 3 , multiple self-attention residual coding layers (AREB) are connected in sequence, and 5 or 4 layers can be set. Each self-attention residual coding layer includes an average pooling layer (Avgpool), a residual convolution block (RCB), a self-attention layer (Attention), a residual convolution block (RCB), a self-attention layer (Attention), and a residual convolution block (RCB) connected in sequence;

[0052] The preliminary feature map is input into the encoder and processed by multiple self-attention residual coding layers in sequence. The last self-attention residual coding layer outputs a high-dimensional representation of the feature.

[0053] In one embodiment, Figure 4 A schematic diagram of the self-attention residual decoding layer is shown in Figure 4 , multiple self-attention residual decoding layers (ARDB) are connected in sequence, and 5 or 4 can be set. The output of the last self-attention residual decoding layer is the decoded feature. Each self-attention residual decoding layer includes a self-attention layer (Attention), 4 residual convolution blocks (RCB), a BN+ReLU layer, and a deconvolution layer (ConvTranspose) connected in sequence. Here, the deconvolution layer kernel size is 3×3 and the stride is 2×2.

[0054] Specifically, Figure 5 shows a schematic diagram of the residual convolution block, see Figure 5 , the residual convolution block (RCB) includes a convolutional layer (Conv2D), a BN+ReLU layer, a convolutional layer (Conv2D), and a BN+ReLU layer connected in sequence;

[0055] The input of the residual convolution block passes through the convolution layer, BN+ReLU layer, convolution layer, and BN+ReLU layer to obtain the first output; after the input is convolved, the second output is obtained; the first output and the second output are added to obtain the output of the residual convolution block.

[0056] Specifically, multiple intermediate convolution blocks (ICB) are connected in sequence, and the output of the last intermediate convolution block is the transformed feature; each intermediate convolution block includes a residual convolution block, a self-attention layer, a residual convolution block, and a residual convolution block connected in sequence. Here, the obtained high-dimensional representation of the feature is further transformed through multiple intermediate convolution blocks.

[0057] In one embodiment, the estimation module includes a convolution layer, an activation function layer, and a fully connected layer. The decoded features are input to the convolution layer for feature extraction. The number of input channels of the convolution layer is 16, the number of output channels is 3, and the convolution kernel size is 1. The result after convolution processing is then passed through the activation function layer and the fully connected layer to obtain a pitch probability matrix, which can accurately reflect the pitch changes in the input audio and provide a basis for further audio analysis and application.

[0058] In one embodiment, constructing the pitch estimation model includes training the pitch estimation model, and the loss function used in the training is:

[0059]

[0060] Among them, L is the loss function, ω is the weight, i is the number of the pitch, y i is the true probability value of the pitch i of the training sample, is the probability value of pitch i estimated by the model.

[0061] Here, the training data sets used are MIR-1K and MDB-stem-synth data sets. The pitch estimation model is trained based on the training data sets to obtain a trained pitch estimation model.

[0062] Table 1 shows the prediction accuracy of the pitch estimation method based on compressed excitation block of the present application and the existing method. The estimation accuracy of the method of the present application was verified by calculating the two indicators of raw pitch accuracy (RPA) and raw chroma accuracy (RCA). RPA calculates the proportion of melody frames in the reference and predicts the pitch within ±50 divisions of the original true pitch. RCA calculates the original pitch accuracy ignoring the octave error. According to Table 1, the method of the present application has a higher estimation accuracy than the prior art.

[0063] Table 1

[0064]

[0065] Using the same inventive concept as the pitch estimation method based on the compressed excitation block, this embodiment also provides a corresponding pitch estimation device based on the compressed excitation block, including:

[0066] A model building module, used for building a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; the residual codec structure includes an encoder, a decoder, and an intermediate feature transformation module; the encoder includes a plurality of self-attention residual encoding layers and a plurality of residual convolution blocks, the decoder includes a plurality of self-attention residual decoding layers, and the intermediate feature transformation module includes a plurality of intermediate convolution blocks;

[0067] A pitch estimation module is used to convert the audio file into a logarithmic Mel-spectrogram, and input the logarithmic Mel-spectrogram into a pitch estimation model to obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a pitch;

[0068] The compressed excitation block extracts features from the logarithmic Mel spectrum graph to obtain a preliminary feature graph; the preliminary feature graph is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder encode the preliminary feature graph to obtain a high-dimensional representation of the feature. The output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; the transformed feature is input into the decoder, and the decoder performs decoding operation based on the transformed feature and the output of each residual convolution block to obtain the decoded feature; the decoded feature is input into the estimation module for feature extraction to obtain a pitch probability matrix;

[0069] The pitch determination module is used to select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file.

[0070] The pitch estimation device based on the compressed excitation block of this embodiment has the same inventive concept as the pitch estimation method based on the compressed excitation block mentioned above. Therefore, the specific implementation method of the device can be seen in the implementation example part of the pitch estimation method based on the compressed excitation block in the previous text, and its technical effect corresponds to the technical effect of the above method, which will not be repeated here.

[0071] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A pitch estimation method based on a compressed excitation block, characterized in that: include: Constructing a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; The residual encoding and decoding structure includes an encoder, a decoder and an intermediate feature transformation module; the encoder includes multiple self-attention residual encoding layers and multiple residual convolution blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolution blocks; Converting an audio file into a logarithmic Mel-spectrogram, and inputting the logarithmic Mel-spectrogram into the pitch estimation model to obtain a pitch probability matrix, wherein each element in the pitch probability matrix corresponds to a probability of a pitch; The compression excitation block performs feature extraction on the logarithmic Mel spectrum map to obtain a preliminary feature map; the preliminary feature map is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder perform encoding operations on the preliminary feature map to obtain a high-dimensional representation of the feature, and the output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; The transformed features are input to the decoder, and the decoder performs a decoding operation according to the transformed features and the output of each residual convolution block to obtain decoded features; The decoded features are input into the estimation module for feature extraction to obtain the pitch probability matrix; The pitch corresponding to the maximum value of the elements in the pitch probability matrix is ​​selected as the pitch estimation result of the audio file.

2. The method according to claim 1, characterized in that The compressed excitation block includes a deconvolution layer, a residual convolution unit, a convolution layer, and a BN+ReLU layer connected in sequence; the residual convolution unit includes multiple residual convolution layers; the logarithmic Mel spectrum map is input into the deconvolution layer for upsampling to obtain the upsampled features; the upsampled features are input into the residual convolution unit, and each residual convolution layer performs a residual convolution operation on the upsampled features. After the outputs of each residual convolution layer are added, they are input into the convolution layer for convolution operation to obtain the convolution result; the convolution result is input into the BN+ReLU layer for batch normalization to obtain the preliminary feature map.

3. The method according to claim 2, characterized in that The residual convolution layer includes a residual layer, a global pooling layer, a first fully connected layer, a second fully connected layer and a scale transformation layer; the residual layer performs a residual operation on the upsampled features to obtain a residual result; the residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer and the scale transformation layer to obtain the scale-transformed features; The scale-transformed features, the residual results, and the up-sampled features are added to obtain the output of the residual convolution layer.

4. The method according to claim 1, characterized in that The multiple self-attention residual coding layers are connected in sequence, and each self-attention residual coding layer includes an average pooling layer, a residual convolution block, a self-attention layer, a residual convolution block, a self-attention layer, and a residual convolution block connected in sequence; The preliminary feature map is input into the encoder, and after being processed by multiple self-attention residual coding layers in sequence, the last self-attention residual coding layer outputs the high-dimensional representation of the feature.

5. The method according to claim 1, characterized in that The multiple self-attention residual decoding layers are connected in sequence, and the output of the last self-attention residual decoding layer is the decoded feature. Each of the self-attention residual decoding layers includes a self-attention layer, 4 residual convolution blocks, a BN+ReLU layer and a deconvolution layer connected in sequence.

6. The method according to claim 4 or 5, characterized in that The residual convolution block includes a convolution layer, a BN+ReLU layer, a convolution layer, and a BN+ReLU layer connected in sequence; The input of the residual convolution block passes through the convolution layer, the BN+ReLU layer, the convolution layer, and the BN+ReLU layer to obtain a first output; after the input is convolved, a second output is obtained; the first output and the second output are added to obtain the output of the residual convolution block.

7. The method according to claim 1, characterized in that The multiple intermediate convolution blocks are connected in sequence, and the output of the last intermediate convolution block is the transformed feature; Each of the intermediate convolution blocks includes a residual convolution block, a self-attention layer, a residual convolution block, and a residual convolution block connected in sequence.

8. The method according to claim 1, characterized in that The estimation module includes a convolutional layer, an activation function layer, and a fully connected layer.

9. The method according to claim 1, characterized in that The construction of the pitch estimation model includes training the pitch estimation model, and the loss function used in the training is: Among them, L is the loss function, ω is the weight, i is the number of the pitch, y i is the true probability value of the pitch i of the training sample, is the probability value of pitch i estimated by the model.

10. A pitch estimation device based on a compressed excitation block, characterized in that: include: A model building module, used to build a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual codec structure, and an estimation module; The residual encoding and decoding structure includes an encoder, a decoder and an intermediate feature transformation module; the encoder includes multiple self-attention residual encoding layers and multiple residual convolution blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolution blocks; A pitch estimation module, used for converting an audio file into a logarithmic Mel-spectrogram, inputting the logarithmic Mel-spectrogram into the pitch estimation model to obtain a pitch probability matrix, wherein each element in the pitch probability matrix corresponds to a probability of a pitch; The compression excitation block performs feature extraction on the logarithmic Mel spectrum map to obtain a preliminary feature map; the preliminary feature map is input into the residual codec structure, and the multiple self-attention residual coding layers of the encoder perform encoding operations on the preliminary feature map to obtain a high-dimensional representation of the feature, and the output of each self-attention residual coding layer is input into the corresponding residual convolution block for residual convolution operation, and the output of each residual convolution block is input into the decoder; the high-dimensional representation of the feature is input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed feature; The transformed features are input to the decoder, and the decoder performs a decoding operation according to the transformed features and the output of each residual convolution block to obtain decoded features; The decoded features are input into the estimation module for feature extraction to obtain the pitch probability matrix; The pitch determination module is used to select the pitch corresponding to the maximum value of the elements in the pitch probability matrix as the pitch estimation result of the audio file.

Images