Dynamic emotion recognition method and device based on symbolic music structure
By using a dynamic emotion recognition method based on symbolic music structure, dynamic emotion curves are generated by utilizing chord change boundary slicing and multi-channel feature extraction. This solves the interpretability and accuracy problems of music emotion recognition in existing technologies and achieves high-precision music emotion prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-02-14
- Publication Date
- 2026-06-23
AI Technical Summary
Existing music emotion recognition methods are susceptible to recording quality, performance style and environmental noise, making it difficult to provide interpretable analysis based on music structure and to effectively capture the correspondence between the internal structure of music and emotional evolution.
A dynamic emotion recognition method based on symbolic music structure is adopted. By extracting chord sequences and using chord change boundaries as slicing points, music data is divided into structural segments. Multi-channel feature extraction and emotion recognition model input are then performed to generate dynamic emotion curves.
It achieves high-precision prediction of the evolution of music emotion values over time, enhances the model's ability to structurally align with emotion trends and model temporal sequences, and improves the accuracy, robustness and generalization of the recognition task.
Smart Images
Figure CN122266397A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of music information processing technology, and in particular to a dynamic emotion recognition method and device based on symbolic music structure. Background Technology
[0002] Music emotion recognition aims to automatically identify the emotional state evoked or expressed by music through computational models. It is a key technology in fields such as music information retrieval, emotion computing, and human-computer interaction.
[0003] Music emotion recognition methods in related technologies mainly extract acoustic features such as Mel spectrum and MFCC, and use machine learning or deep learning models to classify or regress audio files.
[0004] However, the effectiveness of such methods is easily affected by recording quality, performance style, and environmental noise, and it is difficult to provide interpretable analysis based on music structure. Summary of the Invention
[0005] The purpose of this application is to provide a dynamic emotion recognition method and device based on symbolic music structure. With the structure-aware slicing method and continuous emotion regression modeling mechanism as the core, it can achieve high-precision prediction of the evolution of music emotion value over time and enhance the model's ability to align the structure with emotion trends and its ability to model the temporal sequence.
[0006] This application provides a dynamic emotion recognition method based on symbolic music structure, including: Chord sequences are extracted from the input symbolic music data, and the data is divided into multiple continuous structural segments using the boundary of changes between adjacent chords as slicing points. Multi-channel feature extraction is performed on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data. The multi-dimensional structural feature sequence is input into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model. The emotion prediction values corresponding to each structural segment are arranged in order to generate a dynamic emotion curve corresponding to the symbolic music data.
[0007] Optionally, dividing the symbolic music data into multiple continuous structural segments using the boundary of changes in adjacent chords in the chord sequence as slicing points includes: dividing adjacent note sequences with the same chord into the same structural segment, and determining new structural segment boundaries and generating new structural segments when chord changes are detected; wherein each structural segment corresponds to a chord interval; and the boundary of each structural segment is determined by changes in adjacent chords.
[0008] Optionally, the step of performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data includes: extracting the chord body features and the transition features of adjacent chords for each structural segment as the chord channel features corresponding to each structural segment; extracting the pitch, duration, dynamics, and relative position features of all notes within each structural segment as the score channel features corresponding to each structural segment; and extracting statistical values such as note density, register span, and dynamics variance within each structural segment, as well as the difference features of each statistical value relative to the previous structural segment, as the statistical feature channel features corresponding to each structural segment.
[0009] Optionally, the step of inputting the multidimensional structural feature sequence into the emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model includes: inputting the chord channel features, the musical score channel features, and the statistical feature channel features into the sub-model corresponding to the emotion recognition model for encoding to obtain the chord channel feature vector, the musical score channel feature vector, and the statistical feature channel feature vector; performing feature fusion on the chord channel feature vector, the musical score channel feature vector, and the statistical feature channel feature vector, and outputting the emotion prediction value through regression.
[0010] Optionally, the step of fusing the chord channel feature vector, the music score channel feature vector, and the statistical feature channel feature vector, and outputting a sentiment prediction value through regression, includes: concatenating the chord channel feature vector, the music score channel feature vector, and the statistical feature channel feature vector, and performing linear projection and normalization processing on the concatenated feature vector; applying channel-level attention, or a channel-level gating mechanism, to the normalized feature vector, and then performing sequence regression through the output layer of the sentiment recognition model to output a sentiment prediction value.
[0011] This application also provides a dynamic emotion recognition device based on symbolic music structure, including: The system includes a chord extraction module for extracting chord sequences from input symbolic music data; a segmentation module for dividing the symbolic music data into multiple continuous structural segments using the boundary changes of adjacent chords in the chord sequence as slicing points; a feature extraction module for performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data; a sentiment prediction module for inputting the multi-dimensional structural feature sequence into a sentiment recognition model to obtain the sentiment prediction value corresponding to each structural segment output by the sentiment recognition model; and a curve generation module for arranging the sentiment prediction values corresponding to each structural segment in order to generate a dynamic sentiment curve corresponding to the symbolic music data.
[0012] Optionally, the segment division module is specifically used to divide adjacent note sequences with the same chord into the same structural segment, and when a chord change is detected, to determine the new structural segment boundary and generate a new structural segment; wherein, each structural segment corresponds to a chord interval; and the boundary of each structural segment is determined by the adjacent chord changes.
[0013] Optionally, the feature extraction module is specifically used to extract the chord body features and the transition features of adjacent chords for each structural segment, as the chord channel features corresponding to each structural segment; the feature extraction module is also specifically used to extract the pitch, duration, dynamics, and relative position features of all notes within each structural segment, as the score channel features corresponding to each structural segment; the feature extraction module is also specifically used to extract the statistical values of note density, register span, and dynamics variance within each structural segment, as well as the difference features of each statistical value relative to the previous structural segment, as the statistical feature channel features corresponding to each structural segment.
[0014] Optionally, the sentiment prediction module is specifically used to input the chord channel features, the score channel features, and the statistical feature channel features into the sub-model corresponding to the sentiment recognition model for encoding, to obtain the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector; the sentiment prediction module is also specifically used to perform feature fusion on the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector, and output the sentiment prediction value through regression.
[0015] Optionally, the sentiment prediction module is specifically used to concatenate the chord channel feature vector, the music score channel feature vector, and the statistical feature channel feature vector, and perform linear projection and normalization processing on the concatenated feature vector; the sentiment prediction module is also specifically used to apply channel-level attention to the normalized feature vector, or, after a channel-level gating mechanism, perform sequence regression through the output layer of the sentiment recognition model to output the sentiment prediction value.
[0016] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the dynamic emotion recognition method based on symbolic music structure as described above.
[0017] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the dynamic emotion recognition method based on symbolic music structure as described above.
[0018] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the dynamic emotion recognition method based on symbolic music structure as described above.
[0019] The dynamic emotion recognition method and apparatus based on symbolic music structure provided in this application first extracts chord sequences from the input symbolic music data, and divides the symbolic music data into multiple continuous structural segments using the boundary changes of adjacent chords in the chord sequence as slicing points. Then, multi-channel feature extraction is performed on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data. The multi-dimensional structural feature sequence is input into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model. Finally, the emotion prediction values corresponding to each structural segment are arranged in order to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, with a structure-aware slicing method and a continuous emotion regression modeling mechanism as its core, high-precision prediction of the evolution of musical emotion values over time is achieved, enhancing the model's structural alignment ability and temporal modeling ability for emotion trends. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is one of the flowcharts illustrating the dynamic emotion recognition method based on symbolic music structure provided in this application; Figure 2 This is the second flowchart of the dynamic emotion recognition method based on symbolic music structure provided in this application; Figure 3 This is a schematic diagram of the chord recognition and structure slicing results provided in this application; Figure 4 This is a schematic diagram of multi-channel modeling and feature fusion provided in this application; Figure 5 This is a schematic diagram of the dynamic emotion recognition device based on symbolic music structure provided in this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. All actions involving the acquisition of signal information or data in this application are performed in accordance with the relevant data protection laws and policies of the country where the application is located and with authorization from the owner of the relevant device.
[0024] Most symbolic music emotion recognition methods in related technologies are based on static emotion classification of the entire piece of music or use coarse-grained segment division, which cannot effectively capture the correspondence between the internal structure of the music and the evolution of emotions, resulting in a lack of temporal continuity and structural sensitivity in the emotion recognition results. At the same time, the few existing regression modeling methods often ignore the semantic boundaries of the music structure and use fixed-length sliding windows or manual segmentation to extract segments, making it difficult to model the dynamic changes in emotions in detail.
[0025] To address the aforementioned technical problems in related technologies, this application provides a dynamic emotion recognition method based on symbolic music structure. This method, with structure-aware slicing and continuous emotion regression modeling mechanism as its core, achieves high-precision prediction of the evolution of music emotion values over time, enhances the model's ability to align the structure of emotion trends with the time sequence, and thus improves the accuracy, robustness, and generalization ability in dynamic emotion recognition tasks.
[0026] The dynamic emotion recognition method based on symbolic music structure provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0027] like Figure 1 As shown in the embodiment of this application, a dynamic emotion recognition method based on symbolic music structure is provided. This method may include the following steps 101 to 103: Step 101: Extract chord sequences from the input symbolic music data, and divide the symbolic music data into multiple continuous structural segments using the boundary of changes between adjacent chords in the chord sequence as slicing points.
[0028] For example, such as Figure 2 The diagram shown illustrates the overall flowchart of the dynamic emotion recognition method based on symbolic music structure provided in this application embodiment. The method includes: firstly, parsing the input symbolic music data (such as MusicXML, MIDI, ABC, and other symbol formats). Then, performing chord recognition and structural slicing. Figure 3 The diagram shown is a schematic of chord recognition and structural slicing provided in an embodiment of this application. Chord sequences are extracted by a chord recognition algorithm, and the entire piece of music is divided into multiple structural segments using chord boundaries as slicing points. Each structural segment corresponds to a relatively stable chord interval (the boundary can be determined by changes in adjacent chords), analogous to a word in text, used to represent the basic unit of the structural segment.
[0029] Specifically, step 101 above may also include step 101a: Step 101a: Divide adjacent note sequences with the same chord into the same structural segment, and when a chord change is detected, determine the boundary of the new structural segment and generate a new structural segment.
[0030] Each structural segment corresponds to a chord interval; the boundary of each structural segment is determined by the variation of adjacent chords.
[0031] For example, each structural segment corresponds to a relatively stable chord interval, the boundaries of which can be determined by the changes of adjacent chords, and can be subdivided or merged in combination with a preset granularity, and can be adjusted within a tolerance range.
[0032] For example, since the characteristics of each structural segment are determined not only by the chords and local musical features of the segment itself, but also by the chord sequences of the preceding and following context, this slicing method preserves the structural meaning of the segment itself while supporting context-dependent modeling, which helps to capture the fine-grained dynamic changes in musical emotion as harmony evolves.
[0033] Step 102: Perform multi-channel feature extraction on each structural segment to obtain the multi-dimensional structural feature sequence corresponding to the symbolic music data.
[0034] The multidimensional structural feature sequence includes multidimensional structural features corresponding to each structural segment; the multidimensional structural features include: chord channel features, musical score channel features, and statistical feature channel features.
[0035] For example, after the structural segments are divided, multi-dimensional structural features are extracted and encoded. In this embodiment, features are extracted from three complementary channels for each chord structural segment.
[0036] Specifically, step 102 above may also include steps 102a1 to 102a3: Step 102a1: Extract the chord body features and the transition features of adjacent chords for each structural segment, and use them as the chord channel features corresponding to each structural segment.
[0037] Step 102a2: Extract the pitch, duration, dynamics, and relative position features of all notes within each structural segment as the corresponding musical score channel features for each structural segment.
[0038] Step 102a3: Extract the statistical values of note density, register span, and dynamic variance within each structural segment, as well as the difference feature of each statistical value relative to the previous structural segment, as the statistical feature channel feature corresponding to each structural segment.
[0039] The chord channel features include: chord body features and transition features; the chord body features include at least one of the following: chord structure, function label, tension level, and consonance; the transition features are used to characterize the degree of transition between the current chord and adjacent chords in the tonal space.
[0040] For example, such as Figure 2 As shown, the embodiments of this application can extract features from the chord channel, the score channel, and the statistics channel respectively (the order is not limited, and they can be extracted in parallel or interchanged).
[0041] For example, such as Figure 4 As shown, for chord channel feature extraction, chord ontology features (such as chord structure, functional labels, tension level, consonance, etc.) and chord transfer features (i.e. the degree of transfer between two chords in the tonal space, etc.) of structural segments can be extracted. The features are divided into two categories: discrete features (token form) and continuous features.
[0042] For example, such as Figure 4 As shown, for feature extraction of the music score channel, note-level classification features such as pitch, duration, dynamics, and position can be extracted from the note time sequence corresponding to each structural segment, which can be used to capture performance dynamics and melodic cues.
[0043] For example, such as Figure 4 As shown, for feature extraction of statistical feature channels, the local statistical characteristics of each structural segment in the overall structure can be statistically analyzed, including continuous statistical features such as note density, register span, and dynamic variance, as well as the differences between the above features, which are used to describe the changing trends between segments.
[0044] For example, after feature extraction is completed, all the above features can be converted into numerical inputs that the model can process through standardization, vectorization, embedding and other means.
[0045] Step 103: Input the multidimensional structural feature sequence into the emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model, and arrange the emotion prediction values corresponding to each structural segment in order to generate the dynamic emotion curve corresponding to the symbolic music data.
[0046] The emotion prediction values include arousal and valence.
[0047] For example, such as Figure 2 As shown, after extracting the three types of channel features, the extracted three types of channel features can be input into the corresponding sub-models for multi-channel feature modeling and fusion, and the sentiment value can be continuously predicted and output based on the fused features.
[0048] Specifically, step 103 above, which involves inputting the multidimensional structural feature sequence into the emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model, may further include the following steps 103a and 103b: Step 103a: Input the chord channel features, music score channel features, and statistical feature channel features into the sub-model corresponding to the emotion recognition model for encoding, to obtain the chord channel feature vector, music score channel feature vector, and statistical feature channel feature vector.
[0049] Step 103b: Perform feature fusion on the chord channel feature vector, the music score channel feature vector, and the statistical feature channel feature vector, and output the sentiment prediction value through regression.
[0050] For example, chord channel features can be input into the chord feature encoding sub-model of the emotion recognition model to perform temporal encoding, obtaining a first feature vector; then, the score channel features can be input into the score feature encoding sub-model of the emotion recognition model to perform temporal encoding, obtaining a second feature vector; finally, the statistical feature channel features can be input into the statistical feature encoding sub-model of the emotion recognition model to encode, obtaining a third feature vector. After obtaining the above feature vectors, the first feature vector, the second feature vector, and the third feature vector can be fused, and a sentiment prediction value can be output through regression.
[0051] For example, for the chord channel, a sequence modeling network (such as LSTM, Transformer, or equivalent structures) is used to perform temporal modeling of the embedded vector sequence within the chord segment; continuous features and embedding results can be fused within this channel and obtained as a fixed-dimensional representation through linear projection (or equivalent mapping). For the score channel, the note event sequence is multidimensionally embedded and its temporal performance dynamic features are extracted by the sequence modeling network. For the statistical channel, a feedforward network (such as MLP) is used to model local statistical trends and cross-segment variation information.
[0052] For example, taking the temporal coding and feature fusion of chord channels as an example, such as Figure 4 As shown, discrete features within a channel are embedded using a Multilayer Perceptron (MLP) and then fused with continuous features to form a fixed-dimensional vector for each structural segment. Subsequently, an LSTM / Transformer model is used for temporal modeling. Finally, the feature concatenation and fusion module achieves feature fusion through concatenation and a fully connected layer.
[0053] It should be noted that the aforementioned embedded vector sequence mainly refers to the vector sequence obtained by first embedding discrete features. However, the input of temporal modeling does not only include embedding, but at each time step, the embedding is concatenated and fused with continuous features (which can be aligned in dimensions using MLP / linear projection) to form a representation sequence, which is then fed into LSTM / Transformer. Therefore, both chord ontology features and connectivity features participate in temporal encoding.
[0054] Specifically, step 103b above may also include the following steps 103b1 and 103b2: Step 103b1: Concatenate the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector, and perform linear projection and normalization on the concatenated feature vector.
[0055] Step 103b2: Apply channel-level attention or channel-level gating mechanism to the normalized feature vector, and then perform sequence regression through the output layer of the emotion recognition model to output the emotion prediction value.
[0056] For example, the three channels output vectors respectively. After concatenation, they can be linearly projected and normalized sequentially, and channel-level gating or attention can be applied to achieve adaptive selection and noise suppression of multi-channel information. Then, they enter the output layer for sequence regression to generate a dynamic sentiment prediction in the form of [T×2] (T is the number of segments, and 2 corresponds to Valence and Arousal). The entire sentiment curve is then assembled according to the segment order. For details, please refer to [link / reference]. Figure 3 As shown. The above gating / attention and projection normalization steps are optional features, and can be replaced by equivalent mechanisms such as channel weighting and hybrid expert (MoE).
[0057] It should be noted that, in this embodiment, the specific training and evaluation process for the model is not limited. Model parameters can be obtained using existing supervised learning methods. The aforementioned emotion recognition model can be initialized with pre-trained parameters and fine-tuned on target symbolic music data. The loss function and metrics are not limited; commonly used metrics in dynamic emotion recognition, such as CCC, Pearson correlation coefficient, and RMSE, can be selected for performance measurement, as well as combinations or equivalent substitutions. During the inference stage, a continuous emotion curve is output according to the chord alignment slicing, multi-channel feature modeling, and channel-level fusion process provided in this embodiment.
[0058] The dynamic emotion recognition method based on symbolic music structure provided in this application proposes to dynamically divide symbolic music into structurally perceptual segments using chord changes as the basis for slicing, replacing existing methods that process entire pieces or long segments, thus aligning the granularity of slicing with the semantic structure of the music. This method, combined with a deep learning model, achieves continuous emotion numerical prediction based on chord units, enhancing the ability to model the evolution of musical emotions over time.
[0059] The dynamic emotion recognition method based on symbolic music structure provided in this application employs a multi-channel input structure consisting of chord channels, note event channels, and statistical feature channels. This structure extracts mid-level semantics, temporal details, and overall statistical attributes, which are then embedded and fused into the model to enhance the comprehensive perception of musical structure and emotional evolution. This structure is adaptable to various sequence modeling architectures such as LSTM and Transformer.
[0060] The dynamic emotion recognition method based on symbolic music structure provided in this application embodiment has good musical semantic interpretability based on chord-driven structured prediction method. It can support the development of downstream tasks such as emotion analysis, harmony control, and style guidance, and expand the practical application value of symbolic music emotion recognition in scenarios such as emotion recommendation, assisted creation, and music therapy.
[0061] The dynamic emotion recognition method based on symbolic music structure provided in this application first extracts chord sequences from the input symbolic music data, and divides the symbolic music data into multiple continuous structural segments using the boundary of changes between adjacent chords in the chord sequence as slicing points. Then, multi-channel feature extraction is performed on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data. The multi-dimensional structural feature sequence is input into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model. Finally, the emotion prediction values corresponding to each structural segment are arranged in order to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, with a structure-aware slicing method and a continuous emotion regression modeling mechanism as its core, high-precision prediction of the evolution of musical emotion values over time is achieved, enhancing the model's ability to structurally align and temporally model emotion trends.
[0062] It should be noted that the dynamic emotion recognition method based on symbolic music structure provided in this application embodiment can be executed by a dynamic emotion recognition device based on symbolic music structure, or by a control module within that device for executing the dynamic emotion recognition method based on symbolic music structure. This application embodiment uses the execution of the dynamic emotion recognition method based on symbolic music structure by a dynamic emotion recognition device based on symbolic music structure as an example to illustrate the dynamic emotion recognition device based on symbolic music structure provided in this application embodiment.
[0063] It should be noted that, in the embodiments of this application, the dynamic emotion recognition methods based on symbolic music structures shown in the accompanying drawings are all illustrated using one accompanying drawing from one of the embodiments of this application as an example. In specific implementation, the dynamic emotion recognition methods based on symbolic music structures shown in the accompanying drawings of the above methods can also be implemented in conjunction with any other accompanying drawings shown in the above embodiments, which will not be elaborated here.
[0064] The following describes the dynamic emotion recognition device based on symbolic music structure provided in this application. The description below corresponds to the dynamic emotion recognition method based on symbolic music structure described above.
[0065] Figure 5 This is a schematic diagram of the structure of the dynamic emotion recognition device based on symbolic music structure provided in the embodiments of this application, as shown below. Figure 5 As shown, it specifically includes: The system comprises: a chord extraction module 501 for extracting chord sequences from input symbolic music data; a segmentation module 502 for dividing the symbolic music data into multiple continuous structural segments using the boundary changes of adjacent chords in the chord sequence as slicing points; a feature extraction module 503 for performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data; an emotion prediction module 504 for inputting the multi-dimensional structural feature sequence into an emotion recognition model to obtain an emotion prediction value corresponding to each structural segment output by the emotion recognition model; and a curve generation module 505 for arranging the emotion prediction values corresponding to each structural segment in order to generate a dynamic emotion curve corresponding to the symbolic music data.
[0066] Optionally, the segment division module 502 is specifically used to divide adjacent note sequences with the same chord into the same structural segment, and when a chord change is detected, to determine the new structural segment boundary and generate a new structural segment; wherein, each structural segment corresponds to a chord interval; and the boundary of each structural segment is determined by the adjacent chord changes.
[0067] Optionally, the feature extraction module 503 is specifically used to extract the chord body features and the transition features of adjacent chords for each structural segment, as the chord channel features corresponding to each structural segment; the feature extraction module 503 is also specifically used to extract the pitch, duration, dynamics, and relative position features of all notes in each structural segment, as the score channel features corresponding to each structural segment; the feature extraction module 503 is also specifically used to extract the statistical values of note density, register span, and dynamic variance in each structural segment, as well as the difference features of each statistical value relative to the previous structural segment, as the statistical feature channel features corresponding to each structural segment.
[0068] Optionally, the sentiment prediction module 504 is specifically used to input the chord channel features, the score channel features, and the statistical feature channel features into the sub-model corresponding to the sentiment recognition model for encoding, to obtain the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector; the sentiment prediction module 504 is also specifically used to perform feature fusion on the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector, and output the sentiment prediction value through regression.
[0069] Optionally, the sentiment prediction module 504 is specifically used to concatenate the chord channel feature vector, the music score channel feature vector, and the statistical feature channel feature vector, and perform linear projection and normalization processing on the concatenated feature vector; the sentiment prediction module 504 is also specifically used to apply channel-level attention to the normalized feature vector, or, after a channel-level gating mechanism, perform sequence regression through the output layer of the sentiment recognition model to output the sentiment prediction value.
[0070] The dynamic emotion recognition device based on symbolic music structure provided in this application first extracts chord sequences from the input symbolic music data, and divides the symbolic music data into multiple continuous structural segments using the boundary of changes between adjacent chords in the chord sequence as slicing points. Then, multi-channel feature extraction is performed on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data. This multi-dimensional structural feature sequence is input into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model. Finally, the emotion prediction values corresponding to each structural segment are arranged in order to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, by using a structure-aware slicing method and a continuous emotion regression modeling mechanism as the core, high-precision prediction of the evolution of musical emotion values over time is achieved, enhancing the model's ability to structurally align and temporally model emotion trends.
[0071] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a dynamic emotion recognition method based on symbolic music structure. This method includes: first, extracting chord sequences from the input symbolic music data, and dividing the symbolic music data into multiple continuous structural segments using the boundary changes of adjacent chords in the chord sequence as slicing points; then, performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data; inputting the multi-dimensional structural feature sequence into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model; finally, arranging the emotion prediction values corresponding to each structural segment in order to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, by using a structure-aware slicing approach and a continuous emotion regression modeling mechanism as the core, we can achieve high-precision prediction of the evolution of music emotion values over time, and enhance the model's ability to structurally align with and model temporally the emotion trend.
[0072] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0073] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by the computer, the computer can execute the dynamic emotion recognition method based on symbolic music structure provided by the above methods. The method includes: first, extracting chord sequences from the input symbolic music data, and dividing the symbolic music data into multiple continuous structural segments using the boundary of changes between adjacent chords in the chord sequence as slicing points; then, performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data; inputting the multi-dimensional structural feature sequence into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model; finally, arranging the emotion prediction values corresponding to each structural segment in order to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, with the structure-aware slicing method and continuous emotion regression modeling mechanism as the core, high-precision prediction of the evolution of music emotion values over time is achieved, enhancing the model's structural alignment ability and temporal modeling ability for emotion trends.
[0074] Furthermore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned dynamic emotion recognition methods based on symbolic music structures. The method includes: first, extracting chord sequences from input symbolic music data, and dividing the symbolic music data into multiple continuous structural segments using the boundary changes of adjacent chords in the chord sequence as slicing points; then, performing multi-channel feature extraction on each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data; inputting the multi-dimensional structural feature sequence into an emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model; and finally, arranging the emotion prediction values corresponding to each structural segment in sequence to generate a dynamic emotion curve corresponding to the symbolic music data. Thus, by using a structure-aware slicing method and a continuous emotion regression modeling mechanism as the core, high-precision prediction of the evolution of music emotion values over time is achieved, enhancing the model's structural alignment and temporal modeling capabilities for emotion trends.
[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A dynamic emotion recognition method based on symbolic music structure, characterized in that, include: Chord sequences are extracted from the input symbolic music data, and the symbolic music data is divided into multiple continuous structural segments using the boundary of changes between adjacent chords in the chord sequence as slicing points. Multi-channel feature extraction is performed on each structural segment to obtain the multi-dimensional structural feature sequence corresponding to the symbolic music data; The multidimensional structural feature sequence is input into the emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model. The emotion prediction values corresponding to each structural segment are arranged in order to generate the dynamic emotion curve corresponding to the symbolic music data.
2. The method according to claim 1, characterized in that, The multidimensional structural feature sequence includes the multidimensional structural features corresponding to each structural segment; The multidimensional structural features include: chord channel features, musical score channel features, and statistical feature channel features; the emotion prediction values include: arousal and valence.
3. The method according to claim 1, characterized in that, The method of dividing the symbolic music data into multiple continuous structural segments using the boundary of changes in adjacent chords in the chord sequence as slicing points includes: The sequence of adjacent notes with the same chord is divided into the same structural segment, and when a chord change is detected, the boundary of the new structural segment is determined and a new structural segment is generated; Each structural segment corresponds to a chord interval; the boundary of each structural segment is determined by the variation of adjacent chords.
4. The method according to claim 2 or 3, characterized in that, The process of extracting multi-channel features from each structural segment to obtain a multi-dimensional structural feature sequence corresponding to the symbolic music data includes: Extract the chord body features and the transition features of adjacent chords for each structural segment, and use them as the chord channel features corresponding to each structural segment; Extract the pitch, duration, dynamics, and relative position of all notes within each structural segment as the corresponding musical score channel features for each structural segment; The statistical values of note density, register span, and dynamic variance within each structural segment, as well as the difference feature of each statistical value relative to the previous structural segment, are extracted and used as the statistical feature channel features corresponding to each structural segment.
5. The method according to claim 4, characterized in that, The chord channel features include: chord body features and transition features; the chord body features include at least one of the following: chord structure, function label, tension level, and consonance; the transition features are used to characterize the degree of transition between the current chord and adjacent chords in the tonal space.
6. The method according to claim 2, characterized in that, The step of inputting the multidimensional structural feature sequence into the emotion recognition model to obtain the emotion prediction value corresponding to each structural segment output by the emotion recognition model includes: The chord channel features, musical score channel features, and statistical feature channel features are respectively input into the sub-model corresponding to the emotion recognition model for encoding, to obtain the chord channel feature vector, musical score channel feature vector, and statistical feature channel feature vector; The chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector are fused together, and the sentiment prediction value is output through regression.
7. The method according to claim 6, characterized in that, The step of fusing the chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector, and outputting a sentiment prediction value through regression, includes: The chord channel feature vector, the score channel feature vector, and the statistical feature channel feature vector are concatenated, and the concatenated feature vector is subjected to linear projection and normalization. Channel-level attention or channel-level gating is applied to the normalized feature vector, and then sequence regression is performed through the output layer of the emotion recognition model to output the emotion prediction value.
8. A dynamic emotion recognition device based on symbolic music structure, characterized in that, The device includes: The chord extraction module is used to extract chord sequences from the input symbolic music data; The segmentation module is used to divide the symbolic music data into multiple continuous structural segments, using the boundary of the change of adjacent chords in the chord sequence as the slicing point; The feature extraction module is used to perform multi-channel feature extraction on each structural segment to obtain the multi-dimensional structural feature sequence corresponding to the symbolic music data; The sentiment prediction module is used to input the multidimensional structural feature sequence into the sentiment recognition model to obtain the sentiment prediction value corresponding to each structural segment output by the sentiment recognition model. The curve generation module is used to arrange the emotion prediction values corresponding to each structural segment in order to generate a dynamic emotion curve corresponding to the symbolic music data.
9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the dynamic emotion recognition method based on symbolic music structure as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the dynamic emotion recognition method based on symbolic music structure as described in any one of claims 1 to 7.