A folk song style recognition and classification method and system based on audio feature analysis

Abstract

The application relates to the technical field of speech recognition, and discloses a folk song style recognition and classification method and system based on audio feature analysis, which comprises the following steps: performing multi-window time-frequency analysis on unaccompanied solo audio to obtain multi-scale features, extracting shared features through time sequence-frequency double-path coding, predicting pitch and energy by using melody decoding, and adopting gated adaptive normalization and residual attention. In the prior art, global features are difficult to decouple melody contour and aria timing details, especially under the condition of unaccompanied solo singing of the same melody with different arias, so that the technical problem of classification confusion caused by the effective separation of aria style information and melody content cannot be solved. Due to the gated adaptive normalization and residual attention guided by melody decoding, the decoupling of aria microscopic ornamentation, dynamic change and other style elements and the melody pitch contour is realized, so that the problem that the aria style information is covered by the melody content is avoided, and the accuracy of aria style classification is improved.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a method and system for identifying and classifying folk song styles based on audio feature analysis. Background Technology

[0002] In the field of folk song research and preservation, the same folk song is often performed a cappella in different singing styles, such as flat melody, high melody, and vibrato. These singing styles contain key information such as regional culture, vocal techniques, and emotional expression, making accurate identification and classification of these styles of music of great value. Existing technologies typically extract global statistical features from a cappella audio, such as Mel-frequency cepstral coefficients or deep embedding vectors based on pooling of the entire audio segment, and then use a general classifier to determine the style. Such approaches indiscriminately encode the melody outline and vocal details, attempting to represent the acoustic signal containing information at different scales using a single global vector.

[0003] However, the acoustic signal of a cappella folk songs is composed of both melodic skeleton and vocal embellishments. Differences in vocal style are mainly reflected in micro-temporal variations such as short vibrato, glissando, appoggiatura, and dynamic gradations. Global features tend to capture the pitch trends of melodies with stronger energy and longer durations, causing the subtle stylistic information of the vocal style to be masked by melodic commonalities. When the same folk song melody is performed with different vocal styles, the differences in the overall frequency envelope are far smaller than the differences between different melodies. This makes it difficult for classifiers based on global features to resolve melodic-independent vocal style representations, easily leading to vocal style confusion and blurred classification boundaries in practical applications.

[0004] Therefore, there is an urgent need in this field for a method that can effectively decouple and independently encode vocal style components from melodic components under a cappella conditions, so as to improve the ability to distinguish vocal styles. Summary of the Invention

[0005] This invention aims to solve the technical problem in the prior art where global features are difficult to decouple from melodic contours and vocal temporal details, especially in the case of a cappella singing of the same melody performed in different vocal styles, which leads to classification confusion due to the inability to effectively separate vocal style information from melodic content.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a folk song style recognition and classification method based on audio feature analysis.

[0007] The folk song style recognition and classification method based on audio feature analysis includes:

[0008] Step S10: Obtain the audio signal of the a cappella folk song, and perform a preprocessing task based on the audio signal of the a cappella folk song using a multi-window time-frequency analysis mechanism to output a multi-scale input feature matrix;

[0009] Step S20: Based on the multi-scale input feature matrix, a time-frequency dual-path coding mechanism is used to perform the shared feature extraction task and output a frame-level shared feature sequence;

[0010] Step S30: Based on the frame-level shared feature sequence, a melody decoding mechanism is used to perform a melody component prediction task, and the melody decoding joint features are output;

[0011] Step S40: Based on the melody decoding joint features, a gated adaptive normalization and residual attention mechanism are used to perform the vocal style decoupling and encoding task, and output the global vocal style embedding;

[0012] Step S50: Based on the global vocal style embedding and the melody decoding joint features, a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism is used to perform joint optimization and style recognition tasks, and output the vocal style category.

[0013] Preferably, step S10, which involves acquiring the audio signal of an a cappella folk song, performing a preprocessing task based on the audio signal using a multi-window time-frequency analysis mechanism, and outputting a multi-scale input feature matrix, specifically includes:

[0014] Step S101: Convert the input audio to a uniform sampling rate and calculate the linear spectrum using multi-window short-time Fourier transform;

[0015] Step S102: Pass the linear spectrum through a Mel filter bank to obtain the Mel spectrogram, and take the logarithmic energy to form the logarithmic Mel time-frequency spectrum matrix. ,in For time frames, For Mel-band number;

[0016] Step S103: Calculate the first-order increment of the log-Mehr time spectrum. and second-order acceleration ,Will , , Multi-scale input feature matrices are formed by stacking along the frequency axis. .

[0017] Preferably, step S20, which involves performing a shared feature extraction task based on the multi-scale input feature matrix using a time-frequency dual-path coding mechanism and outputting a frame-level shared feature sequence, specifically includes:

[0018] Step S201: Construct a shared feature encoder consisting of a concatenated convolutional dimensionality reduction layer, a Conformer block, and a bidirectional gated recurrent unit (Bi-GRU);

[0019] Step S202: Input the multi-scale feature matrix The input shared feature encoder is subjected to convolutional dimensionality reduction and then the Conformer block extracts intermediate features containing local time-frequency texture and long-range context information.

[0020] Step S203: The intermediate features are fused with bidirectional temporal dependencies using Bi-GRU to output a frame-level shared feature sequence. , , , To share feature dimensions.

[0021] Preferably, in step S202, the Conformer block consists of a multi-head self-attention module, a convolution module, and a feedforward network module. It first uses multi-head self-attention to capture global temporal dependencies, then uses the depthwise separable convolution of the convolution module to extract local time-frequency textures, and finally performs feature transformation through the feedforward network.

[0022] Preferably, step S30, which involves performing a melody component prediction task based on the frame-level shared feature sequence using a melody decoding mechanism and outputting joint melody decoding features, specifically includes:

[0023] Step S301: Construct a melody encoder , It consists of a single-layer forward GRU and a fully connected layer;

[0024] Step S302: Share the feature sequence Every frame Feed melody encoder Output melody in hidden state ;

[0025] Step S303: Hide the melody The results are obtained by passing the softmax layer and the sigmoid layer respectively. Frame pitch category probability distribution and the Frame energy parameters ,in For pitch prediction, the weight matrix of the fully connected layer, To predict the weight matrix of the fully connected layer for energy. For pitch prediction, the bias vector of the fully connected layer, The bias scalar for predicting the energy of the fully connected layer. The sigmoid function is used, and the predicted pitch label is determined based on the pitch category probability distribution. ;

[0026] Step S304: Shared features of each frame Pitch category probability distribution Energy parameters Predicted pitch tags Melody in Hidden State The sub-features of the melody decoding of the frame are encapsulated, and the sub-features of all frames constitute the joint melody decoding features.

[0027] Preferably, in step S40, the step of performing the vocal style decoupling and encoding task based on the joint features of the melody decoding using a gated adaptive normalization and residual attention mechanism, and outputting the global vocal style embedding, specifically includes:

[0028] Step S401: Obtain shared features from each frame of the melody decoding joint features. Predicted pitch tags Melody in Hidden State Establish a learnable melody pattern library , , This represents the total number of pitch categories, with each pitch category corresponding to one. The steady-state vocal pattern vector is determined based on the predicted pitch label. from Extract the corresponding mode vector from and associate it with the melody's hidden state. The data is then concatenated and fed into the dynamic parameter generator. Output the scaling parameters of the current frame. Translation parameters ,Right now ;

[0029] Step S402: Utilize and Shared features Perform melodic adaptive instance normalization to obtain steady-state features. ,in This represents element-wise multiplication. The mean, Standard deviation, It is the stability constant;

[0030] Step S403: Calculate the original residuals of the original shared features and the steady-state features. And generate gate vectors through a gated network. ,in Here is the weight matrix of the gated network. Using the bias vector of the gated network, we obtain the vocal style residual. ;

[0031] Step S404: Perform residual sequence analysis on vocal style A multi-head self-attention mechanism is applied, and a global vocal style embedding is generated through attention pooling. The calculation method is as follows: , ,in and Here are the trainable parameters, and T is the transpose. For attention weights, .

[0032] Preferably, in step S50, the step of performing joint optimization and style recognition tasks based on the global vocal style embedding and the melody decoding joint features using a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism, and outputting the vocal style category, specifically includes:

[0033] Step S501: Obtain the pitch category probability distribution of each frame from the melody decoding joint features. Predicted pitch tags and energy parameters Based on the probability distribution of pitch categories One-hot encoding and energy parameters Generate melody condition vector And by the basic decoder Generate the basic spectral skeleton Embedding the overall singing style Generate globally consistent scaling and modulation parameters through a fully connected network. Translation modulation parameters , reconstruct the first The frame log-number Mel spectrum is And calculate the L1 reconstruction loss. ;

[0034] in Describing the L1 norm, The first of the original logarithmic Mel spectrum frame;

[0035] Step S502: Embed the global vocal style By mapping to the Lorenz hyperbolic space through the first exponential mapping layer, we obtain the style hyperbolic embedding. The global melody latent state is obtained by aggregating the joint features from the melody decoding. Mapping it to the same hyperbolic space yields a melodic hyperbolic embedding. Style cut vectors are obtained through logarithmic mapping. tangent vector of melody Construct hyperbolic orthogonal adversarial loss: And during training, it acts on the gradient inversion layer. To maximize ,in Indicates the inner product. Represents the L2 norm;

[0036] Step S503: Joint optimization total loss:

[0037] ;

[0038] in For pitch prediction, cross-entropy loss, This represents the mean square error loss in energy prediction. Embedded based on global vocal style The cross-entropy loss for vocal style classification The weighting coefficients for pitch prediction loss, Weighting coefficients for energy prediction loss Weighting coefficients for spectrum reconstruction loss, The weighting coefficients of the hyperbolic orthogonal adversarial loss, Weighting coefficients for loss in vocal style classification The weight coefficients for the regularization term; after training, the a cappella folk song audio to be identified is extracted and embedded through steps S10 to S40 to extract the global vocal style. The system uses a geodesic distance Softmax classifier in hyperbolic space to output vocal style categories.

[0039] This invention also provides a folk song style recognition and classification system based on audio feature analysis, comprising:

[0040] The multi-scale feature extraction module is used to acquire the audio signal of a cappella folk songs. Based on the audio signal of the a cappella folk songs, a multi-window time-frequency analysis mechanism is used to perform preprocessing tasks and output a multi-scale input feature matrix.

[0041] The shared feature encoding module is used to perform a shared feature extraction task based on the multi-scale input feature matrix using a time-frequency dual-path encoding mechanism, and output a frame-level shared feature sequence.

[0042] The melody component prediction module is used to perform a melody component prediction task based on the frame-level shared feature sequence using a melody decoding mechanism, and output melody decoding joint features.

[0043] The style decoupling encoding module is used to perform vocal style decoupling and encoding tasks based on the joint features of the melody decoding using a gated adaptive normalization and residual attention mechanism, and outputs a global vocal style embedding.

[0044] The joint optimization and recognition module is used to perform joint optimization and style recognition tasks based on the global vocal style embedding and the melody decoding joint features, using a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism, and outputs the vocal style category.

[0045] The present invention also provides a folk song style recognition and classification device based on audio feature analysis. The folk song style recognition and classification device based on audio feature analysis includes: a memory, a processor, and a folk song style recognition and classification program based on audio feature analysis stored in the memory and executable on the processor. When the folk song style recognition and classification program based on audio feature analysis is executed by the processor, it implements the above-mentioned method.

[0046] The present invention also provides a computer program product, the computer program product including a folk song style recognition and classification program based on audio feature analysis, the folk song style recognition and classification program based on audio feature analysis implementing the above method when executed by a processor.

[0047] The beneficial effects of this invention are as follows:

[0048] By designing a multi-window time-frequency analysis mechanism, the time-frequency details of the a cappella audio are preserved while spectral leakage is suppressed, enabling subsequent processing to obtain a more stable multi-scale input representation, thus laying the foundation for accurate decoupling of melody and singing style.

[0049] By jointly modeling temporal-frequency dual-path coding and melody decoding, melodic components such as pitch contours and energy envelopes are first explicitly predicted from the audio. Then, the predicted melody conditions guide gating adaptive normalization and residual attention mechanisms to separate the melody-dependent steady-state components from the melody-independent vocal residuals, thereby obtaining a global embedding that only reflects the differences in vocal style, avoiding interference of melody content on style representation.

[0050] By introducing a biconditional spectrum reconstruction and a hyperbolic orthogonal adversarial mechanism, the reconstruction process forces style embedding and melody conditions to co-generate a complete spectrum. At the same time, orthogonal loss is applied to the style and melody tangent vectors in the hyperbolic space, further strengthening the independence of style representation and melody information. This makes style embedding more discriminative and helps improve the accuracy of style classification. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the first embodiment of a folk song style recognition and classification method based on audio feature analysis according to the present invention.

[0052] Figure 2 This is a block diagram of a second embodiment of a folk song style recognition and classification system based on audio feature analysis according to the present invention.

[0053] Figure 3 This is a flowchart illustrating the style decoupling and recognition process of a folk song style recognition and classification method based on audio feature analysis according to the present invention. Detailed Implementation

[0054] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0055] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1: As Figure 1 The diagram shown is a flowchart of the first embodiment of a folk song style recognition and classification method based on audio feature analysis according to the present invention. The first embodiment of the folk song style recognition and classification method based on audio feature analysis according to the present invention is presented.

[0057] In the first embodiment, the folk song style recognition and classification method based on audio feature analysis includes:

[0058] Step S10: Obtain the audio signal of the a cappella folk song, and perform a preprocessing task based on the audio signal of the a cappella folk song using a multi-window time-frequency analysis mechanism to output a multi-scale input feature matrix;

[0059] It should be noted that the multi-window time-frequency analysis mechanism in this step refers to using multiple orthogonal window functions of different shapes to perform short-time Fourier transform on the audio signal and averaging its power spectrum to reduce spectral leakage and variance while maintaining time-frequency resolution. The output multi-scale input feature matrix is ​​formed by stacking the log-mel-time spectrum, its first-order increment, and second-order acceleration along the frequency axis, forming a three-dimensional tensor of T×F×3. The first-order and second-order dynamic information helps to characterize the transient changes in the singing style and will be used by the shared feature encoder later.

[0060] Understandably, by using multi-window spectrum estimation, this step can obtain a smoother and more reliable time-frequency representation from a cappella audio. At the same time, it packages the static spectrum and its dynamic change information together into multi-scale features, so that the subsequent network can not only perceive the frame-level spectrum shape, but also capture the temporal patterns of the onset, decay and timbre sliding of the sound in the singing, providing original information to distinguish the melody outline from the singing details.

[0061] It should be understood that, compared to the traditional single-window function short-time Fourier transform, which is prone to introducing spectral leakage and random fluctuations in a cappella signals, multi-window time-frequency analysis can effectively suppress sidelobe interference while maintaining the same frame length, making the harmonic structure in the Melancholic spectrogram clearer. By combining first-order and second-order dynamic feature inputs, compared to classification methods that only use static spectra, this step allows the model to encounter temporal change clues related to singing style earlier, which helps to alleviate the problem of insufficient information when decoupling melody and style later.

[0062] For example, in an a cappella folk song performed in a high-pitched style, the singer might use a short, breathy glissando on certain notes. This variation is reflected in the energy shifting to higher frequencies and then falling back down within a frame of the Mel spectrum. It's difficult to determine whether this is vocal embellishment or a pitch transition based solely on a static spectral frame; rather, it requires considering factors such as the frequency response and the overall tone. and With multi-scale input features, the model can clearly capture the brief impulse and fallback rate of energy, thus providing a basis for classifying such micro-decorations into vocal style representation.

[0063] Step S20: Based on the multi-scale input feature matrix, a time-frequency dual-path coding mechanism is used to perform the shared feature extraction task and output a frame-level shared feature sequence;

[0064] It should be noted that the temporal-frequency dual-path encoding mechanism in this step refers to compressing multi-scale input features through convolutional dimensionality reduction layers, and then feeding them into a shared feature encoder composed of Conformer blocks consisting of alternating multi-head self-attention, depthwise separable convolutions, and feedforward networks, as well as cascaded bidirectional gated recurrent units (Bi-GRUs). The output is a frame-level shared feature sequence. Each frame It simultaneously carries local time-frequency textures and long-range temporal contexts for subsequent melody prediction and style decoupling reuse.

[0065] Understandably, shared feature encoders, without explicit task guidance, extract general representations from audio that contain both precise local time-frequency variations and cross-frame dependencies. These representations are expressive of both melody and vocal style, but are not yet differentiated. The encoded frame-level features provide a unified information foundation for subsequent melody decoding and style decoupling, avoiding the redundancy and inconsistencies caused by designing separate encoders.

[0066] It should be understood that, compared to feature extraction using only convolution or pure Transformer, the cascaded design of Conformer and Bi-GRU balances the efficiency of local time-frequency texture extraction with the completeness of global temporal modeling. Especially in a cappella singing, where long-distance legato and rapid vibrato coexist, this dual-path structure avoids the bias of a single structure focusing only on melody or only on local details, allowing shared features to more evenly carry the melody outline and vocal texture, creating conditions for subsequent decoupling.

[0067] For example, suppose a cappella melody maintains the same pitch for several seconds, but the vocals incorporate vibrato that gradually increases in speed. The multi-head self-attention within the Conformer block can correlate frames spaced far apart, confirming overall pitch stability; depthwise separable convolution captures harmonic structure changes within a single frame; and Bi-GRU further encodes the cumulative effect of vibrato rate changes into frame-level features. This results in… It reflects both the "continuation of the same pitch" and the "gradual acceleration of the vibrato," with the two types of information coexisting without confusion, thus preparing for their separation.

[0068] Step S30: Based on the frame-level shared feature sequence, a melody decoding mechanism is used to perform a melody component prediction task, and the melody decoding joint features are output;

[0069] It should be noted that the melody decoding mechanism utilizes an Emel melody encoder consisting of a single-layer forward GRU and a fully connected layer, which decodes shared features frame by frame. Mapped to the melody hidden state Then by Two parallel linear layers predict the pitch category probability distribution of the current frame. and energy parameters Ultimately, , , Predicted pitch tags and the melody in a hidden state It is encapsulated as a joint feature for melody decoding, so that each frame explicitly carries the estimated information of melody attributes.

[0070] Understandably, this step forces the model to predict melodic components separately from shared features, explicitly separating pitch and energy—the two main variables constituting the melody skeleton. The joint features of melody decoding not only provide melodic conditions for subsequent style decoupling but also allow for the application of pitch and energy prediction losses during training, guiding the shared feature encoder to retain sufficient melodic information. This makes it possible for melodic information to be conditionally removed from the style branch.

[0071] It should be understood that, unlike methods that rely solely on autoencoders to implicitly reconstruct pitch, this invention explicitly establishes a melody prediction branch and utilizes pitch category cross-entropy and energy mean square error for supervision. This results in the melody latent state... It has a more direct correspondence with the pitch and dynamics of the actual singing. In subsequent style extraction, It can be explicitly used as a "known melody interference" for normalization or as a gating condition. Compared with the completely unsupervised stripping of melody in traditional blind separation, the label-guided melody decoding makes the extraction of style residuals more stable and interpretable.

[0072] Step S40: Based on the melody decoding joint features, a gated adaptive normalization and residual attention mechanism are used to perform the vocal style decoupling and encoding task, and output the global vocal style embedding;

[0073] It should be noted that this step first establishes a learnable melody pattern library. Each pitch category corresponds to a steady-state vocal vector. Based on the predicted pitch labels provided by the melody decoder. take out Connect it with the melody's hidden state After stitching, the scaling of the current frame is output through the dynamic parameter generator Gparam. and displacement For shared features Perform melody adaptive instance normalization to obtain the steady-state feature htnorm. Then calculate... The raw residuals from htnorm are used to generate gated vectors through a gated network. Residuals related to singing style are preserved. Finally, all frames Global vocal style embedding is achieved through multi-head self-attention and attention pooling aggregation. .

[0074] Understandably, the role of melody adaptive instance normalization is to normalize the components in the shared features that can be linearly explained by pitch and melody dynamics to a standard distribution using known melody patterns, thereby stripping away components inconsistent with the steady-state vocal pattern. The subsequent gating network further filters out residuals that are indeed related to vocal style rather than random fluctuations, and then uses attention pooling to compress the frame-level style residuals into a fixed-dimensional global representation. The whole process made The influence of the melody content is suppressed, and more attention is paid to the individual characteristics of the singing style, such as timbre, ornamentation, and subtle changes in dynamics.

[0075] It should be understood that, compared to the common global average pooling method that directly aggregates sequence features, this invention explicitly removes standard vocal components using melodic conditions and employs multi-head attention to aggregate residuals based on frame importance, making the final embedding more sensitive to short-term vocal effects. Without this decoupling step in traditional methods, style embedding is easily dominated by melody frames with higher loudness or longer duration, with vocal variations existing only as weak perturbations. This invention, through the approach of "correcting first and then extracting residuals," increases the proportion of vocal style information in the global embedding, which is beneficial for subsequently distinguishing fine-grained vocal styles.

[0076] For example, suppose a folk song has accurate pitch and stable dynamics when sung using the flat vocal technique, but when sung using the high vocal technique, the same melody has multiple short appoggiaturas accompanied by sudden dynamic spikes. In the flat vocal audio, the residual after adaptive normalization of the melody is very small, and the residual of the vocal style obtained after gating is almost zero, resulting in a converged... Approximately zero vectors; however, in high-cavity audio, the residuals at appoggiaturas and apexes are significant, and attention mechanisms tend to assign higher weights to these frames, ultimately... The amplitude and timing patterns of these sudden events will be encoded to create a clear separation between the two vocal styles in the embedded space.

[0077] Step S50: Based on the global vocal style embedding and the melody decoding joint features, a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism is used to perform joint optimization and style recognition tasks, and output the vocal style category.

[0078] It should be noted that this step includes two sub-mechanisms: bi-conditional reconstruction and hyperbolic orthogonal adversarial mechanisms, as well as the final identification process. Bi-conditional reconstruction utilizes the aforementioned melody condition vector. The driver base decoder generates a basic spectral skeleton, which is then used by style embedding. The global modulation parameters are used to perform a channel-level affine transformation on the basic skeleton of each frame, reconstructing the original log-Mel spectrum, and constrained by L1 loss. Hyperbolic orthogonal adversarial methods then map the style embedding and global melodic latent state to the Lorentz hyperbolic space to obtain style tangent vectors. tangent vector of melody The square of their normalized inner product is constructed as the orthogonality loss, and this loss is maximized through a gradient inversion layer on the style path to make the two tend to be orthogonal in the hyperbolic space. During training, the melody prediction loss, reconstruction loss, orthogonality loss, style classification loss, and regularization term are jointly optimized; after training, S10 to S40 are performed on unknown audio to obtain... The hyperbolic space geodesic distance Softmax classifier is used to output the vocal style category.

[0079] Understandably, the biconditional reconstruction-forced style embedding must ensure that the vocal information required for reconstruction is not omitted when providing modulation parameters, preventing the style embedding from degenerating into a useless constant. Hyperbolic orthogonal adversarial methods explicitly promote orthogonality between the style representation and the melody tangent vector, geometrically reducing the likelihood that the style representation contains melodic information. Together, these two methods ensure that the style embedding is both compact and effective, and unrelated to melody, thus enabling stable vocal category identification based solely on style embedding during the recognition phase.

[0080] It should be understood that, compared to simply adding L2 decoupling regularization to the feature space, the orthogonal constraints of the hyperbolic space leverage the characteristic of hyperbolic geometry to well preserve hierarchical structure, making it more suitable for representing pitch and style with hierarchical relationships. Simultaneously, by using gradient inversion layers for adversarial training instead of directly minimizing correlations, overly strict assumptions about the distribution of melodic features are avoided, preserving the continuity of the melodic space itself. This soft orthogonal adversarial approach is more effective in guiding style representations towards higher discriminativeness and without obscuring melodies in complex a cappella signals.

[0081] For example, when performing the same melody using both flat and vibrato techniques, the reconstruction module requires... To restore the stable spectrum of the flat tone and the periodic fluctuation spectrum of the vibrato, if Incorporating melodic information can cause pitch shifts in the decoded spectral skeleton, leading to increased reconstruction loss and thus inverse penalty. At the same time, hyperbolic orthogonal loss forces The direction of change is as perpendicular as possible to the direction of change of the melody's implicit state; when the pitch of the same melody rises, the melody tangent vector shifts, but is pushed to an orthogonal position. Adjustments are made only at the stylistic level, therefore the same singing style Embedded to maintain focus, different singing styles The shift in style direction makes the final category interface clearer.

[0082] Example 2: Figure 2 The diagram shown is a block diagram of a second embodiment of a folk song style recognition and classification system based on audio feature analysis according to the present invention. The folk song style recognition and classification system based on audio feature analysis provided by the present invention employs a folk song style recognition and classification method based on audio feature analysis as described in the above embodiments, and can solve the technical problem of folk song style recognition and classification based on audio feature analysis. The beneficial effects of the folk song style recognition and classification system based on audio feature analysis provided by the present invention are the same as those of the folk song style recognition and classification method based on audio feature analysis provided in the above embodiments, and other technical features of the folk song style recognition and classification system based on audio feature analysis are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0083] Example 3: This invention provides a folk song style recognition and classification device based on audio feature analysis. The device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which are then executed to enable the at least one processor to perform the folk song style recognition and classification method based on audio feature analysis described in Example 1. The folk song style recognition and classification device based on audio feature analysis in this invention can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. This folk song style recognition and classification device based on audio feature analysis is merely an example and should not limit the functionality or scope of the invention. A folk song style recognition and classification device based on audio feature analysis may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory or a program loaded from a storage device into a random access memory. The random access memory also stores various programs and data required for the operation of the folk song style recognition and classification device based on audio feature analysis. The processing unit, the read-only memory, and the random access memory are interconnected via a bus. An I / O interface is also connected to the bus. Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication device allows the folk song style recognition and classification device based on audio feature analysis to communicate wirelessly or wiredly with other devices to exchange data. Although a folk song style recognition and classification device based on audio feature analysis with various systems has been described, it should be understood that it is not required to implement or possess all the systems described. It can be implemented alternatively or with more or fewer systems.

[0084] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the folk song style recognition and classification method based on audio feature analysis as described above. The computer program product provided by this invention can solve the technical problem of folk song style recognition and classification based on audio feature analysis. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as those of the folk song style recognition and classification method based on audio feature analysis provided in the above embodiments, and will not be repeated here.

[0085] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a read-only memory. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0086] like Figure 3 The diagram shows the style decoupling and recognition flowchart of a folk song style recognition and classification method based on audio feature analysis according to the present invention. It utilizes the joint features of global vocal style embedding and melody decoding, calculates L1 reconstruction loss through a dual-condition reconstruction branch to force the style embedding to retain necessary vocal information, and maps style and melody to Lorentz hyperbolic space through a hyperbolic orthogonal adversarial branch to maximize its orthogonal loss, thereby promoting the geometric separation of style representation and melody information. Finally, the reconstruction loss, orthogonal loss, melody prediction loss and style classification loss are weighted and summed to obtain the total loss, which is used to update the network during the training phase and directly extracts the style embedding during the inference phase and outputs the vocal style category using a hyperbolic geodesic softmax classifier.

[0087] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0088] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A folk song style recognition and classification method based on audio feature analysis, characterized in that, The methods include: Step S10: Obtain the audio signal of the a cappella folk song, and perform a preprocessing task based on the audio signal of the a cappella folk song using a multi-window time-frequency analysis mechanism to output a multi-scale input feature matrix; Step S20: Based on the multi-scale input feature matrix, a time-frequency dual-path coding mechanism is used to perform the shared feature extraction task and output a frame-level shared feature sequence; Step S30: Based on the frame-level shared feature sequence, a melody decoding mechanism is used to perform a melody component prediction task, and the melody decoding joint features are output; Step S40: Based on the melody decoding joint features, a gated adaptive normalization and residual attention mechanism are used to perform the vocal style decoupling and encoding task, and output the global vocal style embedding; Step S50: Based on the global vocal style embedding and the melody decoding joint features, a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism is used to perform joint optimization and style recognition tasks, and output the vocal style category.

2. The folk song style recognition and classification method based on audio feature analysis according to claim 1, characterized in that, Step S10 involves acquiring the a cappella folk song audio signal, performing a preprocessing task based on the a cappella folk song audio signal using a multi-window time-frequency analysis mechanism, and outputting a multi-scale input feature matrix. Specifically, this includes: Step S101: Convert the input audio to a uniform sampling rate and calculate the linear spectrum using multi-window short-time Fourier transform; Step S102: linear spectrum is passed through a mel filter bank to obtain a mel spectrogram, and log energy is taken to form a log mel time-frequency spectrum matrix wherein is the number of time frames, is the number of mel frequency bands; Step S103: Calculate the first-order increment of the log-Mehr time spectrum. and second-order acceleration ,Will , , Multi-scale input feature matrices are formed by stacking along the frequency axis. .

3. The folk song style recognition and classification method based on audio feature analysis as described in claim 1, characterized in that, Step S20, which involves performing a shared feature extraction task based on the multi-scale input feature matrix using a time-frequency dual-path coding mechanism and outputting a frame-level shared feature sequence, specifically includes: Step S201: Construct a shared feature encoder consisting of a concatenated convolutional dimensionality reduction layer, a Conformer block, and a bidirectional gated recurrent unit (Bi-GRU); Step S202: Input the multi-scale feature matrix The input shared feature encoder is subjected to convolutional dimensionality reduction and then the Conformer block extracts intermediate features containing local time-frequency texture and long-range context information. Step S203: The intermediate features are fused with bidirectional temporal dependencies using Bi-GRU to output a frame-level shared feature sequence. , , , To share feature dimensions.

4. The folk song style recognition and classification method based on audio feature analysis as described in claim 3, characterized in that, In step S202, the Conformer block consists of a multi-head self-attention module, a convolution module, and a feedforward network module. It first uses multi-head self-attention to capture global temporal dependencies, then uses the depthwise separable convolution of the convolution module to extract local time-frequency textures, and finally performs feature transformation through the feedforward network.

5. The folk song style recognition and classification method based on audio feature analysis as described in claim 1, characterized in that, Step S30, which involves performing a melody component prediction task based on the frame-level shared feature sequence using a melody decoding mechanism and outputting joint melody decoding features, specifically includes: Step S301: Construct a melody encoder , It consists of a single-layer forward GRU and a fully connected layer; Step S302: Share the feature sequence Every frame Feed melody encoder Output melody in hidden state ; Step S303: Hide the melody The results are obtained by passing the softmax layer and the sigmoid layer respectively. Frame pitch category probability distribution and the Frame energy parameters ,in For pitch prediction, the weight matrix of the fully connected layer, To predict the weight matrix of the fully connected layer for energy. For pitch prediction, the bias vector of the fully connected layer, The bias scalar for predicting the energy of the fully connected layer. The sigmoid function is used, and the predicted pitch label is determined based on the pitch category probability distribution. ; Step S304: Shared features of each frame Pitch category probability distribution Energy parameters Predicted pitch tags Melody in Hidden State The sub-features of the melody decoding of the frame are encapsulated, and the sub-features of all frames constitute the joint melody decoding features.

6. The folk song style recognition and classification method based on audio feature analysis as described in claim 1, characterized in that, In step S40, the steps of performing vocal style decoupling and encoding tasks based on the joint features of melody decoding using gated adaptive normalization and residual attention mechanisms, and outputting the global vocal style embedding, specifically include: Step S401: Obtain shared features from each frame of the melody decoding joint features. Predicted pitch tags Melody in Hidden State Establish a learnable melody pattern library , , This represents the total number of pitch categories, with each pitch category corresponding to one. The steady-state vocal pattern vector is determined based on the predicted pitch label. from Extract the corresponding mode vector from and associate it with the melody's hidden state. The data is then concatenated and fed into the dynamic parameter generator. Output the scaling parameters of the current frame. Translation parameters ,Right now ; Step S402: Utilize and Shared features Perform melodic adaptive instance normalization to obtain steady-state features. ,in This represents element-wise multiplication. The mean, Standard deviation, It is the stability constant; Step S403: Calculate the original residuals of the original shared features and the steady-state features. And generate gate vectors through a gated network. ,in Here is the weight matrix of the gated network. Using the bias vector of the gated network, we obtain the vocal style residual. ; Step S404: Perform residual sequence analysis on vocal style A multi-head self-attention mechanism is applied, and a global vocal style embedding is generated through attention pooling. The calculation method is as follows: , ,in and Here are the trainable parameters, and T is the transpose. For attention weights, .

7. The folk song style recognition and classification method based on audio feature analysis as described in claim 1, characterized in that, Step S50, which involves performing joint optimization and style recognition tasks based on the global vocal style embedding and the melody decoding joint features using a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism to output the vocal style category, specifically includes: Step S501: Obtain the pitch category probability distribution of each frame from the melody decoding joint features. Predicted pitch tags and energy parameters Based on the probability distribution of pitch categories One-hot encoding and energy parameters Generate melody condition vector And by the basic decoder Generate the basic spectral skeleton Embedding the overall singing style Generate globally consistent scaling and modulation parameters through a fully connected network. Translation modulation parameters , reconstruct the first The frame log-number Mel spectrum is And calculate the L1 reconstruction loss. ; in Describing the L1 norm, The first of the original logarithmic Mel spectrum frame; Step S502: Embed the global vocal style By mapping to the Lorenz hyperbolic space through the first exponential mapping layer, we obtain the style hyperbolic embedding. The global melody latent state is obtained by aggregating the joint features from the melody decoding. Mapping it to the same hyperbolic space yields a melodic hyperbolic embedding. Style cut vectors are obtained through logarithmic mapping. tangent vector of melody Construct hyperbolic orthogonal adversarial loss: And during training, it acts on the gradient inversion layer. To maximize ,in Indicates the inner product. Represents the L2 norm; Step S503: Joint optimization total loss: ； in For pitch prediction, cross-entropy loss, This represents the mean square error loss in energy prediction. Embedded based on global vocal style The cross-entropy loss for vocal style classification The weighting coefficients for pitch prediction loss, Weighting coefficients for energy prediction loss Weighting coefficients for spectrum reconstruction loss, The weighting coefficients of the hyperbolic orthogonal adversarial loss, Weighting coefficients for loss in vocal style classification The weight coefficients for the regularization term; after training, the a cappella folk song audio to be identified is extracted and embedded through steps S10 to S40 to extract the global vocal style. The system uses a geodesic distance Softmax classifier in hyperbolic space to output vocal style categories.

8. A folk song style recognition and classification system based on audio feature analysis, applied to the folk song style recognition and classification method based on audio feature analysis as described in any one of claims 1 to 7, characterized in that, The system includes: The multi-scale feature extraction module is used to acquire the audio signal of a cappella folk songs. Based on the audio signal of the a cappella folk songs, a multi-window time-frequency analysis mechanism is used to perform preprocessing tasks and output a multi-scale input feature matrix. The shared feature encoding module is used to perform a shared feature extraction task based on the multi-scale input feature matrix using a time-frequency dual-path encoding mechanism, and output a frame-level shared feature sequence. The melody component prediction module is used to perform a melody component prediction task based on the frame-level shared feature sequence using a melody decoding mechanism, and output melody decoding joint features. The style decoupling encoding module is used to perform vocal style decoupling and encoding tasks based on the joint features of the melody decoding using a gated adaptive normalization and residual attention mechanism, and outputs a global vocal style embedding. The joint optimization and recognition module is used to perform joint optimization and style recognition tasks based on the global vocal style embedding and the melody decoding joint features, using a dual-condition reconstruction and hyperbolic orthogonal adversarial mechanism, and outputs the vocal style category.

9. A folk song style recognition and classification device based on audio feature analysis, characterized in that, The folk song style recognition and classification device based on audio feature analysis includes: a memory, a processor, and a folk song style recognition and classification program based on audio feature analysis stored in the memory and executable on the processor. When the folk song style recognition and classification program based on audio feature analysis is executed by the processor, it implements a folk song style recognition and classification method based on audio feature analysis as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a folk song style recognition and classification program based on audio feature analysis. When the folk song style recognition and classification program based on audio feature analysis is executed by a processor, it implements a folk song style recognition and classification method based on audio feature analysis as described in any one of claims 1 to 7.

Images