A pitch estimation method based on compressed excitation blocks

Abstract

The application relates to a pitch estimation method based on a compression excitation block, which converts an audio file into a log-mel spectrogram, so that the input audio is converted into a high-level representation easier for a machine to understand, redundant information is screened out, and information helpful to pitch extraction is retained and strengthened. A compression excitation block is used to extract features of the log-mel spectrogram, and global features of the audio are extracted; meanwhile, a self-attention mechanism is introduced to extract important local features, a pitch representation corresponding to the input audio is obtained through a coding-decoding structure, a pitch probability matrix is obtained through an estimation module, and finally, a pitch estimation result of the audio file is determined according to the pitch probability matrix. The application effectively solves the problem of difficult pitch estimation of opera audio, and improves the accuracy of the pitch prediction result of human voice music.

Description

Technical Field

[0001] This application relates to the field of pitch estimation, and more specifically, to a pitch estimation method based on compressed excitation blocks. Background Technology

[0002] As a treasure of traditional Chinese culture, traditional Chinese opera possesses distinct artistic characteristics and profound cultural connotations. It integrates various art forms such as music, performance, dance, and literature, forming a unique artistic system through multiple expressive techniques including acting, martial arts, singing, and recitation. In traditional Chinese opera, music and singing are the core elements, permeating the entire performance and playing a crucial role in narration, emotional expression, and character portrayal. Compared to modern popular music or Western classical music, traditional Chinese opera melodies are complex and varied, with a wide vocal range and rich in subtle pitch changes, such as glissando, ornamentation, and vibrato. The singing styles in traditional Chinese opera have complex structures and styles; for example, Peking Opera, Yue Opera, and Henan Opera each have their own unique vocal systems, often accompanied by complex pitch variations. Pitch in traditional Chinese opera music is not merely a basic element of melody; it is also a unique artistic tool. Through subtle pitch variations, performers can vividly express emotions, shape characters, and drive the plot forward. In traditional Chinese opera performance, the transmission of emotion often relies on precise control of pitch. Subtle changes in pitch, such as rising, falling, and extending, can express complex emotions like excitement, sadness, anger, and helplessness. Through precise pitch estimation techniques, researchers can delve into how opera performers use pitch variations to convey complex emotions and reveal the pitch characteristics and stylistic differences between different schools and genres.

[0003] In today's digital age, pitch estimation plays a crucial role in the digital preservation and transmission of traditional Chinese opera music. The transmission of traditional opera art relies heavily on the accurate recording of classic performances and vocal techniques. Pitch, as one of the core elements of performance, allows for better recording, analysis, and preservation of audio data through accurate pitch estimation. This not only helps digitize the traditional characteristics of traditional opera vocal techniques but also enables a new generation of opera enthusiasts to better learn and understand these techniques through technological means. However, existing pitch estimation methods suffer from inaccuracies. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a pitch estimation method based on compressed excitation blocks.

[0005] Firstly, a pitch estimation method based on compressed excitation blocks is provided, including:

[0006] A pitch estimation model is constructed. The pitch estimation model includes a compressed excitation block, a residual encoding and decoding structure, and an estimation module. The residual encoding and decoding structure includes an encoder, a decoder, and an intermediate feature transformation module. The encoder includes multiple self-attention residual coding layers and multiple residual convolutional blocks. The decoder includes multiple self-attention residual decoding layers. The intermediate feature transformation module includes multiple intermediate convolutional blocks.

[0007] The audio file is converted into a log-Mel spectrogram. The log-Mel spectrogram is then input into the pitch estimation model to obtain a pitch probability matrix. Each element in the pitch probability matrix corresponds to the probability of a certain pitch.

[0008] The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is then input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional representation of the features. The output of each self-attention residual coding layer is input into the corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is then input into the decoder. This high-dimensional feature representation is input into an intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are then input into the decoder. The decoder performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are then input into an estimation module for feature extraction to obtain the pitch probability matrix.

[0009] The pitch corresponding to the maximum value of the element in the pitch probability matrix is ​​selected as the pitch estimation result of the audio file.

[0010] In one embodiment, the compressed excitation block includes a deconvolutional layer, a residual convolutional unit, a convolutional layer, and a BN+ReLU layer connected in sequence; the residual convolutional unit includes multiple residual convolutional layers; the log-Mel spectrum is input to the deconvolutional layer for upsampling to obtain upsampled features; the upsampled features are input to the residual convolutional unit, each residual convolutional layer performs a residual convolution operation on the upsampled features, the outputs of each residual convolutional layer are summed and then input to the convolutional layer for convolution to obtain the convolution result; the convolution result is input to the BN+ReLU layer for batch normalization to obtain a preliminary feature map.

[0011] In one embodiment, the residual convolutional layer includes a residual layer, a global pooling layer, a first fully connected layer, a second fully connected layer, and a scaling layer. The residual layer performs a residual operation on the upsampled features to obtain a residual result. The residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer, and the scaling layer to obtain scaled features. The scaled features, the residual result, and the upsampled features are added together to obtain the output of the residual convolutional layer.

[0012] In one embodiment, multiple self-attention residual coding layers are connected in sequence, and each self-attention residual coding layer includes an average pooling layer, a residual convolutional block, a self-attention layer, a residual convolutional block, a self-attention layer, and a residual convolutional block connected in sequence.

[0013] The initial feature map is input into the encoder and processed by multiple self-attention residual coding layers. The last self-attention residual coding layer outputs a high-dimensional representation of the features.

[0014] In one embodiment, multiple self-attention residual decoding layers are connected in sequence, and the output of the last self-attention residual decoding layer is the decoded feature. Each self-attention residual decoding layer includes a self-attention layer, four residual convolutional blocks, a BN+ReLU layer, and a deconvolutional layer connected in sequence.

[0015] In one embodiment, the residual convolutional block includes a convolutional layer, a BN+ReLU layer, another convolutional layer, and a BN+ReLU layer connected in sequence.

[0016] The input of the residual convolutional block passes through a convolutional layer, a BN+ReLU layer, another convolutional layer, and another BN+ReLU layer to obtain the first output; after the input undergoes a convolution operation, the second output is obtained; the first output and the second output are added together to obtain the output of the residual convolutional block.

[0017] In one embodiment, multiple intermediate convolutional blocks are connected sequentially, and the output of the last intermediate convolutional block is the transformed feature;

[0018] Each intermediate convolutional block consists of a residual convolutional block, a self-attention layer, a residual convolutional block, and another residual convolutional block connected in sequence.

[0019] In one embodiment, the estimation module includes convolutional layers, activation function layers, and fully connected layers.

[0020] In one embodiment, constructing a pitch estimation model includes training the pitch estimation model using a loss function:

[0021]

[0022] Where L is the loss function, ω is the weight, i is the pitch label, and y i This represents the true probability value of pitch i in the training samples. This represents the probability value of pitch i obtained from the model estimation.

[0023] Secondly, a pitch estimation device based on a compression excitation block is provided, comprising:

[0024] The model building module is used to build a pitch estimation model. The pitch estimation model includes a compressed excitation block, a residual encoding and decoding structure, and an estimation module. The residual encoding and decoding structure includes an encoder, a decoder, and an intermediate feature transformation module. The encoder includes multiple self-attention residual encoding layers and multiple residual convolutional blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolutional blocks.

[0025] The pitch estimation module is used to convert audio files into log-Mel spectrograms. The log-Mel spectrograms are then input into the pitch estimation model to obtain a pitch probability matrix. Each element in the pitch probability matrix corresponds to the probability of a certain pitch.

[0026] The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is then input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional representation of the features. The output of each self-attention residual coding layer is input into the corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is then input into the decoder. This high-dimensional feature representation is input into an intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are then input into the decoder. The decoder performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are then input into an estimation module for feature extraction to obtain the pitch probability matrix.

[0027] The pitch determination module is used to select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file.

[0028] Compared to existing technologies, this application offers the following advantages: The pitch estimation method based on compressed excitation blocks converts audio files into log-Mel spectrograms, thereby transforming the input audio into a higher-level representation more easily understood by machines, filtering out redundant information, and retaining and enhancing information helpful for pitch extraction. Compressed excitation blocks are used to extract features from the log-Mel spectrogram, extracting global audio features; simultaneously, a self-attention mechanism is introduced to extract important local features. The pitch representation corresponding to the input audio is obtained through an encoding / decoding structure, and the pitch probability matrix is ​​obtained through an estimation module. Based on the pitch probability matrix, the final pitch estimation result of the audio file is determined. This application effectively solves the problem of difficulty in pitch estimation for opera audio and improves the accuracy of pitch prediction results for vocal music. Attached Figure Description

[0029] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0030] Figure 1 A schematic diagram of the pitch estimation model is shown;

[0031] Figure 2 A schematic diagram of the compression excitation block is shown;

[0032] Figure 3 A schematic diagram of a self-attention residual coding layer is shown;

[0033] Figure 4 A schematic diagram of the self-attention residual decoding layer is shown;

[0034] Figure 5 A schematic diagram of a residual convolution block is shown. Detailed Implementation

[0035] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0036] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0037] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0038] This application provides a pitch estimation method based on a compressed excitation block, including:

[0039] Step S1: Construct a pitch estimation model; Figure 1 A schematic diagram of the pitch estimation model is shown; see [link / reference]. Figure 1 The pitch estimation model includes a compressed excitation block, a residual encoding / decoding structure, and an estimation module. The residual encoding / decoding structure includes an encoder layer, a decoder layer, and intermediate feature transformation modules (ICBs). The encoder includes multiple self-attention residual coding layers (AREBs) and multiple residual convolutional blocks (RCBs), the decoder includes multiple self-attention residual decoding layers (ARDBs), and the intermediate feature transformation modules include multiple intermediate convolutional blocks (ICBs).

[0040] Step S2: Convert the audio file into a log-Mel spectrogram, input the log-Mel spectrogram into the pitch estimation model to obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a certain pitch.

[0041] Here, the audio file is first processed, including resampling the input audio file at 16kHz, then slicing the audio file, with each slice being 2 seconds long. Slices shorter than 2 seconds are padded until they reach 2 seconds. After slicing, the librosa.feature.melspectrogram and librosa.power_to_db functions from the Librosa library are used to generate a Mel-spectrogram, which is then converted into a log Mel-spectrogram. The purpose of this processing is to transform the input audio into a higher-level representation that is easier for machines to understand, filtering out redundant information and retaining and enhancing information that helps with pitch extraction.

[0042] The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is then input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional representation of the features. The output of each self-attention residual coding layer is input into the corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is then input into the decoder. This high-dimensional feature representation is input into an intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are then input into the decoder. The decoder performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are then input into an estimation module for feature extraction to obtain the pitch probability matrix.

[0043] Step S3: Select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file. Here, the pitch probability matrix includes the probabilities of 360 pitches.

[0044] After obtaining the pitch estimation results, this step allows for the determination of the corresponding estimated frequencies based on the pitch estimation results. Specifically, the following formula can be used:

[0045]

[0046] Among them, f ref The frequency is set to 10Hz, and confi is the confidence level of the sound output. To estimate the probability value of pitch i in the obtained pitch probability matrix, The pitch of the audio file; for The maximum value; th is the threshold, set to 0.5.

[0047] The above embodiment converts the audio file into a log-Mel spectrogram, thereby transforming the input audio into a higher-level representation that is easier for machines to understand, filtering out redundant information, and retaining and enhancing information helpful for pitch extraction. A compressed excitation block is used to extract features from the log-Mel spectrogram, extracting global audio features; simultaneously, a self-attention mechanism is introduced to extract important local features. The pitch representation corresponding to the input audio is obtained through an encoding / decoding structure, and an estimation module obtains the pitch probability matrix. Based on the pitch probability matrix, the final pitch estimation result of the audio file is determined. This embodiment effectively solves the problem of difficulty in pitch estimation for opera audio and improves the accuracy of pitch prediction results for vocal music.

[0048] In one embodiment, Figure 2 A schematic diagram of the compression excitation block is shown; see [link / reference]. Figure 2 The compressed excitation block consists of a deconvolutional layer (ConvTranspose2d), a residual convolutional unit (SEConvBlock), a convolutional layer (Conv2d), and a BN+ReLU layer connected in sequence. The residual convolutional unit includes multiple residual convolutional layers. The log-Mel spectrum is input to the deconvolutional layer for upsampling to obtain upsampled features. The upsampled features are input to the residual convolutional unit, and each residual convolutional layer performs a residual convolution operation on the upsampled features. The outputs of each residual convolutional layer are summed and then input to the convolutional layer for convolution to obtain the convolution result. The convolution result is input to the BN+ReLU layer for batch normalization to obtain a preliminary feature map.

[0049] Specifically, the residual convolutional layer includes a residual layer, a global pooling layer, a first fully connected layer (FC+ReLU), a second fully connected layer (FC+Sigmoid), and a scale transformation layer. The residual layer performs residual operations on the upsampled features to obtain the residual result. The residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer, and the scale transformation layer to obtain the scale-transformed features. The scale-transformed features, the residual result, and the upsampled features are added together to obtain the output of the residual convolutional layer.

[0050] In this embodiment, the input log-Melogram spectrum is first upsampled through a deconvolution layer, increasing the number of channels from 1 to 16. This expands the data dimension, allowing the model to capture and represent more dimensional feature information. The upsampled features are then input into 16 SEConvBlocks to extract effective content. Within each SEConvBlock, the upsampled features are compressed into channel descriptors using a global pooling layer, embedding global information into the signal of each feature channel. To utilize the information aggregated during compression, two fully connected layers are used with different activation functions: ReLU and Sigmoid. ReLU reduces computational cost, while Sigmoid facilitates better fitting of complex curves. A scaling transformation is applied to obtain scaled features. The scaled features, residual results, and upsampled features are then added to obtain the output of the residual convolutional layer, achieving a residual effect and preventing information loss in the model.

[0051] In one embodiment, Figure 3 A schematic diagram of a self-attention residual coding layer is shown. See [link / reference] Figure 3 Multiple self-attention residual coding layers (AREBs) are connected in sequence. There can be 5 or 4. Each self-attention residual coding layer includes an average pooling layer (Avgpool), a residual convolutional block (RCB), a self-attention layer (Attention), a residual convolutional block (RCB), and a self-attention layer (Attention) and a residual convolutional block (RCB) connected in sequence.

[0052] The initial feature map is input into the encoder and processed by multiple self-attention residual coding layers. The last self-attention residual coding layer outputs a high-dimensional representation of the features.

[0053] In one embodiment, Figure 4 A schematic diagram of the self-attention residual decoding layer is shown. See [link / reference] Figure 4 Multiple self-attention residual decoding layers (ARDBs) are connected sequentially, and the number can be set to 5 or 4. The output of the last ARDB is the decoded feature. Each ARDB consists of a self-attention layer, 4 residual convolutional blocks (RCBs), a BN+ReLU layer, and a deconvolutional layer (ConvTranspose) connected sequentially. Here, the deconvolutional layer kernel size is 3×3 and the stride is 2×2.

[0054] Specifically, Figure 5 A schematic diagram of a residual convolution block is shown; see [link / reference]. Figure 5 The residual convolutional block (RCB) consists of a convolutional layer (Conv2D), a BN+ReLU layer, a convolutional layer (Conv2D), and a BN+ReLU layer connected in sequence.

[0055] The input of the residual convolutional block passes through a convolutional layer, a BN+ReLU layer, another convolutional layer, and another BN+ReLU layer to obtain the first output; after the input undergoes a convolution operation, the second output is obtained; the first output and the second output are added together to obtain the output of the residual convolutional block.

[0056] Specifically, multiple intermediate convolutional blocks (ICBs) are connected sequentially, with the output of the last intermediate convolutional block being the transformed feature. Each intermediate convolutional block includes a residual convolutional block, a self-attention layer, another residual convolutional block, and another residual convolutional block connected sequentially. Here, the high-dimensional representation of the obtained features is further transformed through multiple intermediate convolutional blocks.

[0057] In one embodiment, the estimation module includes a convolutional layer, an activation function layer, and a fully connected layer. The decoded features are input to the convolutional layer for feature extraction. This convolutional layer has 16 input channels, 3 output channels, and a kernel size of 1. The result of the convolutional processing then passes through the activation function layer and the fully connected layer to obtain a pitch probability matrix, which accurately reflects the pitch changes in the input audio, providing a foundation for further audio analysis and applications.

[0058] In one embodiment, constructing a pitch estimation model includes training the pitch estimation model using a loss function:

[0059]

[0060] Where L is the loss function, ω is the weight, i is the pitch label, and y i This represents the true probability value of pitch i in the training samples. This represents the probability value of pitch i obtained from the model estimation.

[0061] Here, the training datasets used are the MIR-1K and MDB-stem-synth datasets. The pitch estimation model is trained based on the training datasets to obtain the trained pitch estimation model.

[0062] Table 1 shows the prediction accuracy of the pitch estimation method based on compressed excitation blocks in this application compared with existing methods. The estimation accuracy of the method in this application was verified by calculating two metrics: raw pitch accuracy (RPA) and raw chroma accuracy (RCA). RPA calculates the proportion of melody frames in the reference, with the predicted pitch within ±50 divisions of the original true pitch. RCA calculates the raw pitch accuracy while ignoring octave errors. As shown in Table 1, the method in this application has a higher estimation accuracy compared to existing technologies.

[0063] Table 1

[0064]

[0065] Employing the same inventive concept as the pitch estimation method based on compression excitation blocks, this embodiment also provides a corresponding pitch estimation device based on compression excitation blocks, comprising:

[0066] The model building module is used to build a pitch estimation model. The pitch estimation model includes a compressed excitation block, a residual encoding and decoding structure, and an estimation module. The residual encoding and decoding structure includes an encoder, a decoder, and an intermediate feature transformation module. The encoder includes multiple self-attention residual encoding layers and multiple residual convolutional blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolutional blocks.

[0067] The pitch estimation module is used to convert audio files into log-Mel spectrograms. The log-Mel spectrograms are then input into the pitch estimation model to obtain a pitch probability matrix. Each element in the pitch probability matrix corresponds to the probability of a certain pitch.

[0068] The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is then input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional representation of the features. The output of each self-attention residual coding layer is input into the corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is then input into the decoder. This high-dimensional feature representation is input into an intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are then input into the decoder. The decoder performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are then input into an estimation module for feature extraction to obtain the pitch probability matrix.

[0069] The pitch determination module is used to select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file.

[0070] The pitch estimation device based on the compression excitation block in this embodiment has the same inventive concept as the pitch estimation method based on the compression excitation block described above. Therefore, the specific implementation of this device can be found in the embodiment section of the pitch estimation method based on the compression excitation block described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0071] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A pitch estimation method based on a compressed excitation block, characterized in that, include: A pitch estimation model is constructed; the pitch estimation model includes a compressed excitation block, a residual encoding / decoding structure, and an estimation module. The residual encoding / decoding structure includes an encoder, a decoder, and an intermediate feature transformation module; the encoder includes multiple self-attention residual encoding layers and multiple residual convolutional blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolutional blocks; The audio file is converted into a log-Mel spectrogram, and the log-Mel spectrogram is input into the pitch estimation model to obtain a pitch probability matrix, where each element in the pitch probability matrix corresponds to the probability of a certain pitch. The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional feature representation. The output of each self-attention residual coding layer is input into a corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is input into the decoder. The high-dimensional feature representation is then input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are input into the decoder, which performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are input into the estimation module for feature extraction to obtain the pitch probability matrix; The pitch corresponding to the maximum value of the element in the pitch probability matrix is ​​selected as the pitch estimation result of the audio file.

2. The method as described in claim 1, characterized in that, The compressed excitation block includes a deconvolutional layer, a residual convolutional unit, a convolutional layer, and a BN+ReLU layer connected in sequence. The residual convolutional unit includes multiple residual convolutional layers. The log-Melogram is input to the deconvolutional layer for upsampling to obtain upsampled features. The upsampled features are input to the residual convolutional unit, and each residual convolutional layer performs a residual convolution operation on the upsampled features. The outputs of each residual convolutional layer are summed and then input to the convolutional layer for convolution to obtain a convolution result. The convolution result is input to the BN+ReLU layer for batch normalization to obtain the preliminary feature map.

3. The method as described in claim 2, characterized in that, The residual convolutional layer includes a residual layer, a global pooling layer, a first fully connected layer, a second fully connected layer, and a scaling layer. The residual layer performs a residual operation on the upsampled features to obtain a residual result. The residual result is sequentially input into the global pooling layer, the first fully connected layer, the second fully connected layer, and the scaling layer to obtain scaled features. The features after scaling, the residual result, and the features after upsampling are added together to obtain the output of the residual convolutional layer.

4. The method as described in claim 1, characterized in that, The multiple self-attention residual coding layers are connected in sequence, and each self-attention residual coding layer includes an average pooling layer, a residual convolutional block, a self-attention layer, a residual convolutional block, a self-attention layer, and a residual convolutional block connected in sequence. The initial feature map is input into the encoder and processed by multiple self-attention residual coding layers in sequence. The last self-attention residual coding layer outputs the high-dimensional representation of the feature.

5. The method as described in claim 1, characterized in that, The multiple self-attention residual decoding layers are connected in sequence, and the output of the last self-attention residual decoding layer is the decoded feature. Each self-attention residual decoding layer includes a self-attention layer, four residual convolutional blocks, a BN+ReLU layer, and a deconvolutional layer connected in sequence.

6. The method as described in claim 4 or 5, characterized in that, The residual convolutional block includes a convolutional layer, a BN+ReLU layer, a convolutional layer, and a BN+ReLU layer connected in sequence. The input of the residual convolutional block passes through the convolutional layer, the BN+ReLU layer, the convolutional layer, and the BN+ReLU layer to obtain the first output; after the input undergoes a convolution operation, a second output is obtained; the first output and the second output are added together to obtain the output of the residual convolutional block.

7. The method as described in claim 1, characterized in that, The multiple intermediate convolutional blocks are connected sequentially, and the output of the last intermediate convolutional block is the transformed feature; Each of the intermediate convolutional blocks includes a residual convolutional block, a self-attention layer, a residual convolutional block, and another residual convolutional block connected in sequence.

8. The method as described in claim 1, characterized in that, The estimation module includes convolutional layers, activation function layers, and fully connected layers.

9. The method as described in claim 1, characterized in that, The construction of the pitch estimation model includes training the pitch estimation model, and the loss function used for training is: Where L is the loss function, ω is the weight, i is the pitch label, and y i This represents the true probability value of pitch i in the training samples. This represents the probability value of pitch i obtained from the model estimation.

10. A pitch estimation device based on a compression excitation block, characterized in that, include: A model building module is used to build a pitch estimation model; the pitch estimation model includes a compressed excitation block, a residual encoding / decoding structure, and an estimation module. The residual encoding / decoding structure includes an encoder, a decoder, and an intermediate feature transformation module; the encoder includes multiple self-attention residual encoding layers and multiple residual convolutional blocks, the decoder includes multiple self-attention residual decoding layers, and the intermediate feature transformation module includes multiple intermediate convolutional blocks; The pitch estimation module is used to convert the audio file into a log-Mel spectrogram, input the log-Mel spectrogram into the pitch estimation model, and obtain a pitch probability matrix, wherein each element in the pitch probability matrix corresponds to the probability of a certain pitch. The compressed excitation block extracts features from the log-Mel spectrum to obtain a preliminary feature map. This preliminary feature map is input into the residual encoder-decoder structure. Multiple self-attention residual coding layers of the encoder encode the preliminary feature map to obtain a high-dimensional feature representation. The output of each self-attention residual coding layer is input into a corresponding residual convolutional block for residual convolution. The output of each residual convolutional block is input into the decoder. The high-dimensional feature representation is then input into the intermediate feature transformation module for intermediate feature transformation to obtain the transformed features. The transformed features are input into the decoder, which performs a decoding operation based on the transformed features and the output of each residual convolutional block to obtain the decoded features. The decoded features are input into the estimation module for feature extraction to obtain the pitch probability matrix; The pitch determination module is used to select the pitch corresponding to the maximum value of the element in the pitch probability matrix as the pitch estimation result of the audio file.

Images