A song style analysis method, system, terminal and storage medium for fusing heterogeneous information

Abstract

The application discloses a song style analysis method and system fusing heterogeneous information, a terminal and a storage medium. The method comprises the following steps: obtaining the score data of a target song, performing feature extraction, obtaining a plurality of score features, and constructing a score knowledge graph according to the plurality of score features; obtaining a plurality of hierarchical geographical features corresponding to the target song, constructing a geographical context graph, and constructing a heterogeneous knowledge graph according to the score knowledge graph and the geographical context graph; performing node feature vectorization processing, heterogeneous graph aggregation processing and contrast learning processing on the heterogeneous knowledge graph to obtain a target embedding vector; performing unsupervised style clustering processing and explainability analysis on the target embedding vector to obtain a song style analysis result of the target song. The score data of the song and the corresponding geographical features are combined to construct a heterogeneous knowledge graph and generate a song style analysis result, so that the accuracy of the song style analysis can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, system, terminal, and computer-readable storage medium for analyzing song styles by integrating heterogeneous information. Background Technology

[0002] With the development of the times, a constant stream of songs have been created. Many songs carry rich historical information, regional characteristics and ethnic customs. The style of many songs is highly coupled with geographical space. The ecological environment, history, culture and ethnic composition of different regions have a significant impact on the formation of musical styles.

[0003] However, existing methods for analyzing song styles do not consider the relationship between songs and geographical regions, natural geography, and cultural geography, making it difficult to distinguish between regional commonalities and the individual styles of songs, thus leading to inaccurate style analysis of songs.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a song style analysis method, system, terminal, and computer-readable storage medium that integrates heterogeneous information. This invention aims to solve the problem that existing song style analysis methods do not consider the relationship between songs and geographical regions, natural geography, and cultural geography, making it difficult to support the distinction between regional commonalities and song individual styles, thus leading to inaccurate song style analysis.

[0006] To achieve the above objectives, the present invention provides a method for song style analysis that integrates heterogeneous information, the method comprising the following steps:

[0007] Obtain the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features;

[0008] Obtain the multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph;

[0009] The heterogeneous knowledge graph is subjected to node feature vectorization, heterogeneous graph aggregation, and contrastive learning to obtain the target embedding vector.

[0010] Unsupervised style clustering and interpretability analysis are performed on the target embedding vector to obtain the song style analysis results of the target song.

[0011] Optionally, in the song style analysis method that integrates heterogeneous information, the solfège features include a symbolic sequence, a relative pitch sequence, a relative time value sequence, and a contour morphology sequence.

[0012] The process involves acquiring the simplified musical notation data of the target song, extracting features from the simplified musical notation data to obtain multiple simplified musical notation features, specifically including:

[0013] Identify the target song and extract its sheet music data;

[0014] Multiple music events of the musical notation data are obtained, and each music event is transformed into a binary tuple to obtain the symbol tag sequence;

[0015] The relative pitch sequence is obtained by extracting the relative pitch intervals from the symbolic sequence.

[0016] Obtain the absolute time value sequences corresponding to multiple music events, and convert the absolute time value sequences into the relative time value sequences;

[0017] The relative tone sequence is used to generate a direction sequence, and the direction sequence is classified according to the pattern corresponding to the direction sequence to obtain the contour shape sequence.

[0018] Optionally, in the song style analysis method that integrates heterogeneous information, the step of constructing a simplified musical notation knowledge graph based on multiple simplified musical notation features specifically includes:

[0019] The target song is used as the first central entity, and an initial simplified musical notation knowledge graph is constructed based on the first central entity, the symbol tag sequence, the relative pitch sequence, the relative time value sequence, and the contour morphology sequence.

[0020] Define the relationship types between nodes in the initial simplified spectrum knowledge graph to obtain the simplified spectrum knowledge graph.

[0021] Optionally, the song style analysis method that integrates heterogeneous information, wherein obtaining the multi-level geographical features corresponding to the target song, constructing a geographical context graph based on the multi-level geographical features, and constructing a heterogeneous knowledge graph based on the musical notation knowledge graph and the geographical context graph, specifically includes:

[0022] The geographical region where the target song is located is determined, and the multi-level geographical features corresponding to the geographical region are obtained, wherein the multi-level geographical features include physical geographical features, human geographical features and cultural geographical features.

[0023] The geographical region is used as the second central entity, and an initial geographical map is constructed based on the second central entity, the physical geographical features, the human geographical features, and the cultural geographical features.

[0024] Define the geographic context node type for the initial geographic map to obtain the geographic context map;

[0025] By performing node connection processing on the simplified spectrum knowledge graph and the geographic context graph, a heterogeneous knowledge graph is obtained.

[0026] Optionally, the song style analysis method that integrates heterogeneous information, wherein the step of performing node feature vectorization, heterogeneous graph aggregation, and contrastive learning on the heterogeneous knowledge graph to obtain the target embedding vector specifically includes:

[0027] Obtain music feature nodes and geographic entity nodes from the heterogeneous knowledge graph, and perform node feature vectorization processing on the music feature nodes and the geographic entity nodes to obtain a vectorized heterogeneous knowledge graph.

[0028] A heterogeneous graph converter is determined, and the vectorized heterogeneous knowledge graph is subjected to heterogeneous graph aggregation processing through the heterogeneous graph converter to obtain the target representation vector.

[0029] The target representation vector is subjected to contrastive learning processing using the InfoNCE loss function to obtain the target embedding vector.

[0030] Optionally, the song style analysis method that integrates heterogeneous information, wherein the heterogeneous graph aggregation process performed on the vectorized heterogeneous knowledge graph by the heterogeneous graph converter to obtain the target representation vector specifically includes:

[0031] The target node and source node in the vectorized heterogeneous knowledge graph are obtained through the heterogeneous graph converter, and the query vector is calculated based on the representation vector corresponding to the target node.

[0032] Calculate the message vector and key vector based on the representation vector corresponding to the source node, and calculate the attention score based on the query vector and the key vector;

[0033] The weighted sum is calculated based on the message vector and the attention score, and then the weighted sum is subjected to projection processing, residual connection processing, and layer normalization processing to obtain the target representation vector.

[0034] Optionally, the song style analysis method that integrates heterogeneous information, wherein performing unsupervised style clustering and interpretability analysis on the target embedding vector to obtain the song style analysis result of the target song specifically includes:

[0035] The target clustering method is determined based on the target embedding vector, wherein the target clustering method includes any one of the K-Means algorithm, the density-based DBSCAN algorithm, and the hierarchical clustering algorithm;

[0036] The target embedding vector is subjected to unsupervised style clustering processing according to the target clustering method to obtain the clustering analysis results;

[0037] The interpretability analysis of the clustering analysis results was performed using the GNNExplainer model to obtain the song style analysis results of the target song.

[0038] Furthermore, to achieve the above objectives, the present invention also provides a song style analysis system that integrates heterogeneous information, wherein the song style analysis system that integrates heterogeneous information includes:

[0039] The simplified musical notation knowledge graph construction module is used to acquire the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features;

[0040] The heterogeneous knowledge graph construction module is used to obtain multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph.

[0041] The target embedding vector generation module is used to perform node feature vectorization processing, heterogeneous graph aggregation processing, and contrastive learning processing on the heterogeneous knowledge graph to obtain the target embedding vector.

[0042] The song style analysis result output module is used to perform unsupervised style clustering and interpretability analysis on the target embedding vector to obtain the song style analysis result of the target song.

[0043] In this invention, the simplified musical notation data of a target song is acquired, and features are extracted from the simplified musical notation data to obtain multiple simplified musical notation features. A simplified musical notation knowledge graph is then constructed based on these features. Multi-level geographical features corresponding to the target song are acquired, and a geographical context graph is constructed based on these features. A heterogeneous knowledge graph is then constructed based on the simplified musical notation knowledge graph and the geographical context graph. The heterogeneous knowledge graph undergoes node feature vectorization, heterogeneous graph aggregation, and contrastive learning processing to obtain a target embedding vector. Unsupervised style clustering and interpretability analysis are then performed on the target embedding vector to obtain the song style analysis result of the target song. This invention constructs a simplified musical notation knowledge graph using the song's simplified musical notation data and introduces geographical features corresponding to the song to construct a geographical context graph. The simplified musical notation knowledge graph and the geographical context graph are then fused to obtain a heterogeneous knowledge graph. Embedding learning and interpretability clustering analysis are then performed on the heterogeneous knowledge graph to finally generate the song style analysis result of the target song, effectively improving the accuracy of song style analysis. Attached Figure Description

[0044] Figure 1 This is a flowchart of a preferred embodiment of the song style analysis method that integrates heterogeneous information according to the present invention;

[0045] Figure 2 This is a schematic diagram of the overall structural implementation process of a preferred embodiment of the song style analysis method that integrates heterogeneous information according to the present invention;

[0046] Figure 3 This is a structural diagram of a preferred embodiment of the song style analysis system that integrates heterogeneous information according to the present invention;

[0047] Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] This invention uses Chinese folk songs as an example. Chinese folk songs are an important part of traditional Chinese music culture, carrying rich historical information, regional characteristics, and ethnic customs. With the development of digital humanities, artificial intelligence, and knowledge graph technology, academia and industry are actively exploring digital modeling, style analysis, and cultural visualization methods for folk songs. Currently, knowledge graphs, as an effective tool for structured semantic expression and multimodal entity relationship modeling, are being applied in various fields such as music recommendation, music entity relationship analysis, and cultural semantic mining.

[0050] However, current research on the style of Chinese folk songs still faces multiple challenges, mainly in the following aspects: 1. Insufficient symbolic data modeling capabilities: Most current research focuses on data types such as lyrics, audio spectra, or expert labels, neglecting the clearly structured and information-rich musical notation form. Notation not only possesses good potential for structured modeling but is also suitable for unified expression across languages, yet it has not been fully utilized. 2. Lack of geographical context modeling: Folk song style is highly coupled with geographical space; the ecological environment, historical culture, and ethnic composition of different regions have a significant impact on the formation of musical styles. However, most current methods fail to express multi-level, multi-scale spatial semantic features, lacking systematic modeling of the nested structures between geographical divisions, physical geography, and cultural geography, making it difficult to support style discrimination based on regional commonalities and individual characteristics. 3. Lack of interpretability in style analysis: Existing clustering and classification methods mostly remain at a shallow modeling level based on audio features, often lacking structured style representation and knowledge support. Even when using graph neural networks for relational modeling, the heterogeneous types of nodes and edges are often ignored, resulting in insufficient interpretability and difficulty in meeting the needs of cultural research and academic tracing.

[0051] To address the aforementioned issues, this invention proposes a systematic approach to the analysis of Chinese folk song styles that integrates heterogeneous information, constructing a scalable, well-structured style modeling method with cultural and academic value.

[0052] The preferred embodiment of the song style analysis method that integrates heterogeneous information according to the present invention, such as... Figure 1 As shown, the song style analysis method that integrates heterogeneous information includes the following steps:

[0053] Step S10: Obtain the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features. The simplified musical notation features include symbol marker sequences, relative pitch sequence sequences, relative time value sequences, and contour shape sequences.

[0054] This invention adopts a theoretically independent knowledge graph ontology design for simplified musical notation. Theoretically independent means that it does not use Western music theory (such as C major and 4 / 4 time signature) to explain and analyze Chinese folk song simplified musical notation data in advance, but only learns the most original and objective symbol relationships on the simplified musical notation.

[0055] Furthermore, this invention proposes a modeling method that does not rely on traditional music theory, transforming the simplified musical score into a clearly structured symbolic knowledge graph. This symbolic knowledge graph takes the song as its core and associates elements such as melody fragments, rhythmic patterns, outlines, and symbolic markers. It expresses the structural logic and temporal relationships of musical fragments through relational edges, providing a unified and analyzable symbolic expression framework for subsequent modeling.

[0056] In this context, a relation edge is a directed arrow connecting two nodes. It represents a specific relationship with type and direction. For example, consider a node named "Song" (the song title is "Jasmine Flower") and a node named "MelodicFragment" (the melody fragment is 32 3 5). The relation edge connecting these two nodes is called "hasFragment" (containing). This relation edge indicates that the "song" contains the "melody fragment," meaning the relationship is directional and has a type.

[0057] Specifically, a target song is identified, and its musical notation data is extracted; multiple music events from the musical notation data are obtained, and each music event is transformed into a binary tuple to obtain the symbolic sequence; relative intervals are extracted from the symbolic sequence to obtain the relative pitch sequence; the absolute time value sequences corresponding to the multiple music events are obtained, and the absolute time value sequences are converted into the relative time value sequences; a direction sequence is generated based on the relative pitch sequence, and the direction sequence is categorized according to the pattern corresponding to it to obtain the contour morphology sequence.

[0058] First, the following principles must be followed when performing preprocessing of numbered musical notation data: 1. Maintain theoretical agnosticism: All numbered musical notation symbols should be treated as independent low-level musical language units, and the processing should not map them to high-level music theory concepts such as C major or 4 / 4 time. 2. Preserve the original symbol structure: Every pitch symbol, time value mark, and ornamentation symbol must be preserved in its original structured form, avoiding simplification or classification into standard music theory categories. 3. Extract relative relationships rather than absolute parameters: Pitches need to be converted into relative pitches (such as adjacent intervals or directional changes), and rhythms need to be expressed as relative time values ​​(such as the proportion within a unit of time), avoiding the use of absolute values.

[0059] Symbol System and Definition: Before preprocessing, it is necessary to first define the basic symbol set for the simplified musical notation. Let... This is the set of all basic symbols that may appear in numbered musical notation, and this set must contain at least: 1. Pitch symbols: 2. Rest symbol: 3. Octave Modifiers: (Representing the higher and lower octaves respectively); 4. Duration Modifiers: (representing the diminution line, double diminution line, delay line, and dot respectively); a complete musical event (Note Event) E is a tuple consisting of a basic pitch symbol or rest, and additional modifiers.

[0060] like Figure 2 As shown, the preprocessing procedure for simplified spectrum data is as follows:

[0061] 1. Pitch Symbolization: First, the pitch information in each musical event E (each independent note in the musical score is defined as a "musical event") is converted into a concise mathematical representation. Definition: A sequence of musical events (melody fragments) can be represented as M = (E1, E2, ... E...). i E n For any event E in the sequence. i Its basic pitch symbols are Its octave modification can be quantized into an integer O. i The middle octave is defined as o = 0. For each octave higher, the value of o increases by 1; for each octave lower, the value of o decreases by 1. The pitch p of a rest is 0, and the octave o is also 0. Transition rule: Event E i It is converted into a tuple (p) i ,o i It serves as its unique symbolic token.

[0062] Through this step, a melody is transformed into a pure mathematical symbol sequence (first, each "musical event" is converted into a "binary tuple," for example, the event with pitch 3 becomes (3, 0). Then, all the "binary tuples" in the entire melody are arranged in order to form a list or sequence). This sequence can serve as the basis for capturing N-gram patterns, corresponding to MelodicFragment in knowledge graphs.

[0063] 2. Relative Interval Extraction: In this step, intervals are based on scale degrees, not on the number of semitones in the twelve-tone equal temperament. Definition: Given a sequence of pitch symbols... Only consider pitch events other than rests for calculation (i.e., p) i ≠0). Relative interval I k The formula for calculating (from the (k-1)th note to the kth note) is as follows:

[0064] I k =(p k -p k-1 )+N×(o k -o k -1)(for k = 2, ..., n);

[0065] Where N is the total number of pitches in a scale. For a heptatonic scale, N = 7, and n is the pitch. This formula ensures the accuracy of calculating intervals across octaves.

[0066] The final relative tone sequence I k This sequence faithfully records the leaps in the melody, without relying on any key setting.

[0067] 3. Rhythmic Information Relativization: This step aims to convert the absolute time values ​​of notes (such as quarter notes) into relative time value proportions to accurately describe the complex and free rhythms common in Chinese folk songs.

[0068] Definition: First, a quantization benchmark needs to be established for the representation of time values ​​in simplified musical notation. Let the duration of a quarter note be one "Standard Duration Unit" (SDU), then:

[0069] Quarter note: D abs =1.0 SDU;

[0070] Eighth note (a diminution line): D abs =0.5 SDU;

[0071] Sixteenth note (two diminution lines): D abs =0.25 SDU;

[0072] Half note (with an augmentation line): D abs =2.0 SDU;

[0073] Dotted quarter note: D abs =1.0 + 1.0 × 0.5 = 1.5 SDU;

[0074] Given a segment M = (E1, E2, ..., E...) containing n musical events. n ), where the absolute time value sequence of each event is

[0075] Conversion rule: Represent the rhythmic pattern of this segment as a relative time value ratio sequence. r n Let be the duration of the nth note. If the sequence is normalized based on the duration of the first note in the segment, then we have: Therefore, the relative time value r1 of the first note is always 1. This representation is independent of the specific tempo (BPM) and only retains the rhythmic pattern itself, corresponding to the RhythmicPattern in the knowledge graph.

[0076] 4. Melody outline abstraction: Based on the calculated relative pitch sequence, this step further abstracts the macroscopic shape of the melody line.

[0077] Definition: The sequence of relative tones used Generate a symbolic directional sequence.

[0078]

[0079] This will produce a sequence of Up, Down, Flat (basic direction labels), where Up (ascending): when the pitch of the next note is higher than the pitch of the previous note (i.e., relative interval I). k >0), marked with Up, for example, moving from 1 to 2 is Up. Down: when the pitch of the next note is lower than the pitch of the previous note (i.e., relative interval 1). k <0) is marked with Down. For example, moving from 5 to 3 is Down. Flat: When the pitch of the next note is exactly the same as the previous note (i.e., relative interval I). k =0), use Flat to mark it, for example, walking from 5 to 5 is Flat.

[0080] Furthermore, this invention employs a contour classification algorithm to analyze the direction sequence. The patterns are categorized into predefined contour shapes, such as Ascending, Descending, Convex, Concave, and Flat. A simplified classification logic can be as follows:

[0081] Flat: The proportion of Flat in the sequence exceeds the threshold (90%).

[0082] Ascending: The proportion of Up in the sequence exceeds the threshold (90%).

[0083] Descending: The proportion of Down in the sequence exceeds the threshold (90%).

[0084] Convex: If the first half of the sequence is mainly Up and the second half is mainly Down, this can be identified by detecting the turning point of the direction change.

[0085] Concave: If the first half of the sequence is mainly Down and the second half is mainly Up.

[0086] 5. Structured Output Generation: After the above four steps, the original musical notation fragment is transformed into multi-dimensional, structured, and theoretically independent data. For each input melody fragment, the preprocessing module outputs a set of features used to generate entities and relations in the knowledge graph: Symbolic Token Sequence: A sequence of (p,o) tuples generated by pitch symbolization, such as ((1,0),(2,0),(5,1)), which constitutes the core content of the MelodicFragment node. Rhythmic Pattern: A sequence of relative time values ​​generated by rhythm relativization, such as [1,0.5,0.5,1], which constitutes the RhythmicPattern node. ContourShape: Shape labels generated by the abstraction of the melody contour, such as Convex, which constitute the ContourShape node. These outputs directly correspond to the node types defined in the knowledge graph ontology.

[0087] Furthermore, the target song is taken as the first central entity, and an initial simplified musical notation knowledge graph is constructed based on the first central entity, the symbol tag sequence, the relative pitch sequence, the relative time value sequence, and the contour morphology sequence; the relationship types between nodes are defined in the initial simplified musical notation knowledge graph to obtain the simplified musical notation knowledge graph.

[0088] like Figure 2 As shown, in order to systematically model the multi-dimensional, theoretically unrelated data produced by the aforementioned preprocessing process and effectively reveal its inherent musical style patterns, a dedicated knowledge graph ontology needs to be designed. The specific process is as follows:

[0089] 1. Define the graph node type:

[0090] Song: This is the central entity of the graph, representing a specific folk song. Each Song node is the collection point of all relevant information and contains metadata such as song name and source.

[0091] MelodicFragment: Represents an ordered sequence (i.e., N-gram) of multiple symbolic markers, the core of which is a sequence of symbolic markers generated in pitch symbolization. For example, ((1,0), (2,0), (5,0)) is a MelodicFragment instance.

[0092] RhythmicPattern: Represents a sequence of relative time value proportions. Its core content is the sequence of relative time value proportions generated through the relativization of rhythmic information. For example, [1, 0.5, 0.5] is a RhythmicPattern instance.

[0093] ContourShape: Represents the abstract shape of the melody line. Its core content is the shape tags generated in the abstract extraction of the melody contour. For example, Ascending, Convex, Flat, etc.

[0094] SymbolicToken: Represents the basic unit in musical notation, namely the tuple (p, o) defined in pitch symbolization. For example, (1, 0) represents the pitch 1 in the middle octave, and (0, 0) represents a rest.

[0095] 2. Second step: Define the relationship type between nodes

[0096] Song--(hasFragment)-->MelodicFragment: Indicates that a song contains a melodic fragment.

[0097] Song--(hasRhythm)-->RhythmicPattern: Indicates that a song contains a certain rhythmic pattern.

[0098] MelodicFragment--(hasShape)-->ContourShape: Indicates that a melodic fragment has a certain outline shape.

[0099] MelodicFragment--(hasRhythm)-->RhythmicPattern: Indicates that a melodic fragment has a certain rhythmic pattern.

[0100] MelodicFragment--(startsWith)-->SymbolicToken: Represents the starting note of a melodic fragment, i.e., the first (p, o) tuple in its ordered sequence.

[0101] MelodicFragment --(containsSequence)-->(ordered list): This edge points to a data structure (such as a JSON array or ordered list) that precisely stores the sequence of SymbolicTokens (p, o) that make up the fragment, preserving complete sequence information.

[0102] By constructing such a knowledge graph, not only is the simplified musical notation data structured, but more importantly, a completely new analytical framework is created specifically for discovering the inherent stylistic patterns of Chinese folk songs.

[0103] Step S20: Obtain the multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph.

[0104] like Figure 2 As shown, for the construction of a contextual ontology for a multi-layered geographical environment: In order to supplement the geographical dimension expression in style analysis, this invention constructs a geographical knowledge ontology covering elements such as topography, climate, language, production methods and cultural practices. This ontology takes geographical regions as the main axis and establishes semantic links with song nodes, supporting the explanation of regional differences in music styles from a spatial semantic level.

[0105] Specifically, the geographical region where the target song is located is determined, and the multi-level geographical features corresponding to the geographical region are obtained. These multi-level geographical features include physical geographical features, human geographical features, and cultural geographical features. The geographical region is used as a second central entity, and an initial geographical map is constructed based on the second central entity, the physical geographical features, the human geographical features, and the cultural geographical features. Geographical context node types are defined for the initial geographical map to obtain a geographical context map. Node connection processing is performed on the simplified musical notation knowledge graph and the geographical context map to obtain a heterogeneous knowledge graph.

[0106] The construction process for a multi-level geographic context ontology is as follows:

[0107] like Figure 2 As shown, after theoretically modeling the pitch, rhythm, and contour of folk song styles, a multi-layered geographical context ontology needs to be constructed to further explore the geographical causes of folk song style formation, supporting cross-analysis from multiple dimensions such as physical, human, and cultural dimensions. To this end, this invention designs a geographical context model encompassing three levels: physical, human, and cultural, including:

[0108] 1. Physical Geography: Natural environmental elements related to the formation of musical styles, including topography: such as Plateau, Plain, Basin, and Mountain. For example, the high-pitched and resonant style of Shaanxi folk songs is considered closely related to the Loess Plateau terrain in which they are located. Hydrology: such as River and Lake; the themes and rhythms of songs from cultures living along rivers may differ significantly from those of inland cultures. Climate: such as Monsoon, Arid, and Temperate. Climate influences agricultural production methods and festival activities, which in turn affect the rhythm and content of music.

[0109] 2. Human Geography: Human social elements that shape musical style, including modes of production (Livelihood): such as agriculture, pastoralism, fishing; the rhythm and melody of work songs directly reflect their productive activities. Linguistic Area: The rhythm, cadence, and melodic direction of music are often closely related to the tones and rhythms of the local dialect.

[0110] 3. Cultural Geography: Cultural environmental factors that influence the function and dissemination of musical styles, including cultural corridors: proximity to ancient trade routes (such as the Silk Road), canals, or migration routes of ethnic groups. Local Customs and Rituals: The functionality of songs, such as wedding songs and funeral songs, is an important component of style.

[0111] Based on the multi-level context model described above, the following knowledge graph ontology structure can be constructed to define its core entities (nodes) and relationships (edges):

[0112] 1. Define the geographic context node type:

[0113] Geographic Region: A central node in a geographical location, such as "Northern Shaanxi" or "Yangtze River Basin".

[0114] Topography: Examples include Plateau, Plain, etc.

[0115] Climate Zone: Examples include Temperate Monsoon, Arid Climate, etc.

[0116] LinguisticArea: Examples include Jin_Chinese and Southwestern_Mandarin.

[0117] LivelihoodStyle: Examples include Terraced Farming and Nomadic Herding.

[0118] Cultural Practice: Examples include Nuptial Lament, Field Song, and Singing Market.

[0119] 2. Define geographic context relationship types:

[0120] Song --(originatesIn)-->GeographicRegion: (Bridge Relationship) Connecting the Song node in the music knowledge graph with the GeographicRegion node in the geographic context graph is key to enabling cross-domain analysis. Here, "originatesIn" means "originating from" or "originating from," for example, "Blue Flower" --(originatesIn)-->Shaanxi region means that the song "Blue Flower" originated in the Shaanxi region.

[0121] Song--(associatedWith)-->CulturalPractice: This relationship stems from research in ethnomusicology and folklore. Research data will indicate in what cultural activities a certain type of song (such as Xintianyou) is typically sung (such as folk songs, love songs).

[0122] GeographicRegion--(hasTopography)-->Topography: The topographic features of the region.

[0123] GeographicRegion--(hasClimate)-->ClimateZone: The climate characteristics of this region.

[0124] Geographic Region -- (primary Dialect) -- Linguistic Area: The main dialect of this region.

[0125] GeographicRegion--(dominantLivelihood)-->LivelihoodStyle: The primary mode of production in this region.

[0126] Song--(associatedWith)-->CulturalPractice: A song is associated with a specific cultural activity.

[0127] Understandably, the node connection process works as follows: these node connections are achieved in the Neo4j native graph database by creating structured data called "Relationships." A "Relationship" always establishes a connection with a clear type and direction between a start node and an end node. Taking GeographicRegion--(hasTopography)-->Topography as an example, its connection process in Neo4j is as follows:

[0128] 1. Ensure node existence: First, the system will ensure that there is a starting node in the database with the label: GeographicRegion and the attribute {name: "Shaanbei Region"}, and an ending node with the label: Topography and the attribute {name: "Plateau"}.

[0129] 2. Create a relationship: The system creates a directed relationship between the two nodes. The direction of this relationship is from "Northern Shaanxi" to "Plateau", and its type is explicitly set to: HAS_TOPOGRAPHY.

[0130] Through this process, a directed connection named HAS_TOPOGRAPHY is physically formed in the database, pointing from "Northern Shaanxi" to "Plateau". All other node connections follow this pattern.

[0131] At the same time, the process of node connection is also determined based on objective, real-world data.

[0132] Song -- (originatesIn) --> GeographicRegion: This relationship usually comes from musicological data. For example, records of folk song collections, music classics, or album information will clearly indicate the location of recording or origin of a song (such as "Jasmine Flower") (e.g., Jiangsu).

[0133] Geographic Region -- (has Topography) -- -> Topography: This relationship comes from geographical knowledge. For example, the main topography of "Northern Shaanxi" is the "Loess Plateau".

[0134] Geographic Region -- (primary dialect) -- -> Linguistic Area: This relationship comes from linguistic research. The results of language mapping and dialect studies can indicate the main dialect spoken in a geographical region (such as northern Shaanxi) (e.g., Jin dialect).

[0135] This structured approach enables advanced algorithms such as Graph Neural Networks (GNNs) to reason about complex relationships that span multiple data dimensions within a unified framework.

[0136] The aforementioned "bridge relationship edge" of originatesIn tightly connects these two originally independent graphs, ultimately forming a complete and heterogeneous knowledge graph that includes both musical details and geographical causes.

[0137] Step S30: Perform node feature vectorization, heterogeneous graph aggregation, and contrastive learning on the heterogeneous knowledge graph to obtain the target embedding vector.

[0138] like Figure 2 As shown, for heterogeneous graph modeling and HGT style embedding generation: This invention constructs a heterogeneous knowledge graph that integrates musical notation structure and geographic semantics. The node types include songs, melody fragments, rhythm patterns, and geographic entities. A Heterogeneous Graph Transformer (HGT) is introduced to perform embedding learning on the heterogeneous graph. By learning the attention weights of different types of edges and nodes, cross-modal relationship modeling is achieved, and finally, a song style representation vector that integrates structural and semantic information is generated.

[0139] Specifically, music feature nodes and geographic entity nodes in the heterogeneous knowledge graph are obtained, and node feature vectorization processing is performed on the music feature nodes and the geographic entity nodes to obtain a vectorized heterogeneous knowledge graph.

[0140] The specific process of heterogeneous graph modeling and HGT style embedding generation is as follows:

[0141] Node feature vectorization: Before inputting the heterogeneous knowledge graph into the HGT model, an initial numerical feature vector must be generated for each node in the graph. Let d be the dimension of the model embedding.

[0142] The vectorization process for music feature nodes in heterogeneous knowledge graphs is as follows:

[0143] 1. SymbolicToken: For each unique (p, o) tuple, through an embedding layer Map it to a d-dimensional vector, where |V token | represents the total number of all unique symbol markers. Let V be the set of real numbers, a set of size |V token |*d, and a matrix consisting of real numbers.

[0144] 2. MelodicFragment: A sequence S as a symbolic marker frag = (t1, t2, ..., t k ), whose vector representation is h frag The embedding sequence of its members is encoded using a recurrent neural network (LSTM) to obtain: h frag =LSTM([E token (t1), E token (t2), ..., E token (t k )]);

[0145] 3. Rhythmic Pattern: As a numerical sequence S rhy = [r1, r2, ..., r m ], whose vector representation is h rhy Encoding is performed using a multilayer perceptron (MLP) to obtain h. rhy =MLP([r1, r2, ..., r m ]).

[0146] 4. ContourShape: As a category label (such as Convex), its vector representation h contour Through a separate embedding layer It can be found.

[0147] The vectorization process for geographic entity nodes in heterogeneous knowledge graphs is as follows:

[0148] 1. For categorized geographic nodes, such as Topography, ClimateZone, LinguisticArea, LivelihoodStyle, and CulturalPractice, assign a learnable embedding vector to each instance of each type (e.g., Plateau, AridClimate). For example, for terrain nodes: h topo =E topo (id topo );in, It is an embedding matrix of terrain type, |V topo | represents the number of terrain-related terms, id topoA unique identifier representing a specific terrain instance. In the model, "plateau" will be assigned ID 1, "plain" will be assigned ID 2, and so on.

[0149] 2. For GeographicRegion nodes, their features are composed of their own categorical embeddings and associated numerical features (such as latitude and longitude coordinates, area, etc.), which are then fused through an MLP:

[0150] h region =MLP(CONCAT(E) region (id region ),F numeric ));

[0151] Among them, h region This refers to the numerical vector that fully represents the characteristics of a Geographic Region node after processing. region It is an embedding layer specifically designed for GeographicRegion nodes; id region For "identifier", it refers to a unique ID for a specific geographical area; F numeric F stands for "Features", and numeric refers to "numerical type", which is a vector containing all objective numerical features related to the geographic region, such as the region's latitude and longitude coordinates, area, population density, etc. CONCAT is an abbreviation for "Concatenate", which means to concatenate the two vectors into a longer vector.

[0152] Song Node: The initial features of a Song node are set to zero vectors, and its final rich representation will be learned entirely from its neighboring nodes through the aggregation process of HGT.

[0153] A heterogeneous graph converter is determined. The target node and source node in the vectorized heterogeneous knowledge graph are obtained through the heterogeneous graph converter, and a query vector is calculated based on the representation vector corresponding to the target node. A message vector and key vector are calculated based on the representation vector corresponding to the source node, and an attention score is calculated based on the query vector and the key vector. A weighted sum is calculated based on the message vector and the attention score, and the weighted sum is subjected to projection processing, residual connection processing, and layer normalization processing to obtain the target representation vector. The heterogeneous information aggregation process based on HGT is as follows: HGT propagates and aggregates information by stacking multiple layers. For any target node t in the heterogeneous knowledge graph, its representation H in the (l+1)th layer... (l+1)τ(t) is calculated by aggregating the information of its neighbor nodes at layer l. The core of this process is the heterogeneous mutual attention mechanism. Let τ(t) represent the type of the target node t, and φ(e) represent the type of the edge e. The meta-relation triple is defined as R = <τ s , φ e , τ t >。τ s This refers to the type of the starting node of this relationship. For example, in the relationship Song--(hasFragment)-->MelodicFragment, τ is of type Song.

[0154] Heterogeneous Mutual Attention (HGT) uses a multi-head attention mechanism (i∈[1,...,h]) to compute attention for a target node t and a source node s connected by a meta-relation R.

[0155] The key computation for a query is specific to the meta-relation R and the attention head i, expressed as:

[0156]

[0157] in, It is a learnable projection matrix, where d′ is the dimension of each head. H (l) (i.e., the node representation at layer l): Here, H represents the "hidden state," which is the node's representation vector or "embedding vector" at a certain layer. The superscript l represents the l-th layer of the network. Therefore, H (l) (t) refers to the representation vector of the target node t after information aggregation through layer l. Similarly, H (l) (s) refers to the representation vector of the source node s at layer l.

[0158] Q i (t)(Query Vector): Q is short for "Query", and it is the representation vector H of the target node t. (l) (t) multiplied by a learnable weight matrix What we get is that we are looking for information related to t, which is the query condition for t.

[0159] K i (s)(key vector): K is an abbreviation for "Key". It is the representation vector H of the source node s. (l) (s) multiplied by another learnable weight matrix What we get can be understood as: these are the information tags that s possesses; let's see if t matches them.

[0160] The expression for calculating the Attention Score is as follows:

[0161]

[0162] in, These are all source node neighbors of the target node t under the meta-relation R. s′ itself has no special meaning; it is simply used to refer to each neighbor node during the normalization process. (Subscript) It means: "For a set For each neighbor of node t (i.e., all its neighbors), we temporarily call it s′, and then perform a calculation on it. T is a standard linear algebra notation representing the transpose. K i (s) is a column vector. (K) i (s)) T This involves transposing the column vector into a row vector. This ensures that the dot product operation (K) can be performed correctly. i (s)) T ·Q i (t) is the result of multiplying a row vector by a column vector to obtain a single numerical value (similarity score).

[0163] In heterogeneous message passing, the message from each source node s is also transformed through a matrix specific to the meta-relation:

[0164]

[0165] in It is a learnable projection matrix, V i (s) represents the "message vector" or "value vector". It can be understood as: the truly valuable and detailed information that node S is prepared to pass on to its neighbors.

[0166] Message Aggregation and Node Update: The information aggregated by the target node t from the neighbors of the meta-relation R is a weighted sum of messages from each neighbor.

[0167]

[0168] The information from all attention points is concatenated and projected through a linear layer specific to the target node type:

[0169]

[0170] in, (For intermediate representation vectors): H represents the node's "representation vector". The superscript (l+1) indicates that the representation at level l+1 is being calculated. ~ (tilde) represents a temporary, intermediate value. The complete meaning is: the intermediate representation vector of the target node t at layer l+1. It is a preliminary result obtained by splicing and fusing information passed from all neighbors through a multi-head attention mechanism. This result has not yet undergone the final residual connection and normalization processing. W is a specific type of output weight matrix; W is a learnable "weight matrix"; the subscript O is an abbreviation for "Output", indicating that this matrix is ​​used in the output stage of information aggregation. The subscript τ... t The type of the target node t (e.g., Song type or GeographicRegion type). The full meaning is: a special one for τ t This type of node learns a weight matrix for its output. Its function is to linearly transform (or "compress / project") the concatenated long vector from all attention heads back to the desired standard dimension d. Because different types of nodes (such as songs and terrain) may process information differently, equipping each type of node with a dedicated output matrix allows the model to learn richer semantics.

[0171] Finally, residual connections and layer normalization are applied to obtain the final updated representation:

[0172]

[0173] Among them, H (l+1) (t) represents the final representation vector of the target node t at layer l+1. (l+1) (t) will be used as the input to the next layer of the network (layer l+2) to continue information aggregation, and so on, layer by layer.

[0174] The target representation vector is subjected to contrastive learning processing using the InfoNCE loss function to obtain the target embedding vector.

[0175] like Figure 2 As shown, a contrastive learning-driven embedding optimization is employed: In an unsupervised setting, HGT is trained through a self-supervised task. An effective method is contrastive learning, which aims to make closely related nodes in the graph also close to each other in the embedding space, while unrelated nodes are kept far apart. For a given node u (anchor point), a positive sample v (such as a neighbor directly connected by an edge) and a set of negative samples are used. Optimization can be achieved using the InfoNCE loss function:

[0176]

[0177] Where sim(·,·) is the cosine similarity, θ is the temperature hyperparameter, and H(u) is the representation vector of the anchor node; H represents the final representation vector of the node; u is a selected "anchor" node, which is the center point of the current analysis. H(u) is the final vector representation of the anchor node u after processing by the HGT model. H(v) is the representation vector of the positive sample node; v is a "positive sample" node. In the graph, neighbors that are directly connected to the anchor node u are usually considered semantically similar to u. Therefore, H(v) is the final vector representation of this positive sample neighbor v. H(v′) is the representation vector of any sample node; v′ is a traversal variable that represents all sample nodes in the denominator in turn. This sample set It includes a positive sample v and a set of "negative samples". (That is, nodes randomly selected from the graph that are not directly related to u). Therefore, H(v′) refers to the vector representation of each positive / negative sample that is substituted in sequentially when calculating the total score. Let be the loss function, representing the "gap" or "error" between the model's current prediction and the desired result. The goal of this formula is to adjust the model parameters to achieve this. The smaller the value, the better. When The smaller the similarity, the better. This means that the model has learned to maximize the similarity (numerator) between positive samples H(v) and anchor points H(u), while minimizing the similarity (denominator) between negative samples and anchor points.

[0178] Furthermore, the style-geographic fusion song embedding output is performed: Through the above training process, HGT ultimately generates a high-dimensional, information-dense embedding vector for each Song node in the knowledge graph. Through a multi-layer attention mechanism, it not only encodes the symbolic musical features of the song itself (such as melody and rhythm), but also adaptively integrates the complex geographical context information (such as terrain, dialect, and mode of production).

[0179] Step S40: Perform unsupervised style clustering and interpretability analysis on the target embedding vector to obtain the song style analysis results of the target song.

[0180] like Figure 2As shown, this invention employs an interpretable style clustering and analysis mechanism: after obtaining the song embedding representation, unsupervised clustering algorithms (such as K-Means, DBSCAN, hierarchical clustering) are used to classify the songs by style, and the GNNExplainer mechanism (an interpretation method for graph neural networks (GNNs) designed to provide interpretability for predictions by GNN-based models in graph machine learning tasks) is introduced to extract key feature subgraphs and identify key factors affecting the clustering results, thereby improving the interpretability of style clustering.

[0181] Specifically, a target clustering method is determined based on the target embedding vector, wherein the target clustering method includes any one of the K-Means algorithm, the density-based DBSCAN algorithm, and the hierarchical clustering algorithm; unsupervised style clustering is performed on the target embedding vector according to the target clustering method to obtain the clustering analysis results.

[0182] The specific implementation process of the interpretable style clustering and analysis mechanism is as follows: Unsupervised style clustering: After obtaining the embedding vectors of each "Song" node output by the heterogeneous graph neural network (HGT) model, the embedding vectors are executed. The embedding vectors integrate multi-dimensional information such as the melody, rhythm, outline and geographical culture of the song. By performing unsupervised clustering on the embedding vectors, the automatic style classification of the song nodes is realized.

[0183] For different research objectives and data characteristics, the following are recommended methods for selecting appropriate clustering methods:

[0184] 1. K-Means: Suitable for applications where clear partitioning is desired and the number of clusters is known or estimable. This implementation uses the K-Means algorithm to divide all song nodes into a pre-defined number (K) of style clusters. The number of clusters K can be preset by those skilled in the art based on experience, or its optimal value can be determined before executing the algorithm using techniques such as the silhouette coefficient method or the "elbow rule." After determining the value of K, the K-Means algorithm can be executed, assigning the embedding vector of each song node to one of the K style clusters based on its spatial location, thus completing the style partitioning.

[0185] 2. DBSCAN: Suitable for exploratory style discovery and outlier style identification. This implementation uses the density-based DBSCAN algorithm to automatically discover style clusters without pre-setting the number of styles. By setting two core parameters, the neighborhood radius eps and the minimum number of neighbors min_sample, and executing the algorithm, sufficiently dense embedding vectors in the space can be divided into clusters. At the same time, any isolated song node that does not belong to any cluster is identified as an outlier, thereby achieving exploratory style discovery and unique style identification.

[0186] 3. Hierarchical Clustering: Suitable for constructing style evolution genealogies and hierarchical analysis, this implementation uses a hierarchical clustering algorithm to construct a tree structure that reveals the subordinate and kinship relationships between different styles. By executing this algorithm (e.g., using a bottom-up aggregation strategy), a tree-like hierarchical graph of all song nodes is directly generated. Analyzing this graph can determine the style categories, subcategories, and their subordinate relationships at different levels.

[0187] Evaluation and Assignment of Clustering Results: After executing any of the above clustering implementation methods, the method further includes evaluating the effectiveness of the generated style clusters (e.g., calculating their silhouette coefficients) and assigning a unique cluster ID to each song node. This ID is used to ultimately identify the style genre to which the node belongs, as discovered by the method of this invention, for use in subsequent steps. The GNNExplainer model is used to perform interpretability analysis on the clustering analysis results to obtain the song style analysis results of the target song.

[0188] Furthermore, based on the interpretability analysis of GNNExplainer, this graph neural network post-hoc explanation model is adopted. For a specific prediction given by a GNN, GNNExplainer aims to identify the explanatory subgraph that contributes the most to the decision through optimization methods. The specific implementation process and principle are as follows:

[0189] 1. Optimization objective: GNNExplainer optimizes the computational graph G by maximizing the graph's computational properties. c A subset A of the adjacency matrix sub With node feature subset X sub The explanation is found using mutual information. Its optimization objective can be formally expressed as:

[0190]

[0191] Among them, G s =(A s ,X s Let G(Y) be the explanatory subgraph to be found. To encourage simplicity of explanation, regularization terms for subgraph size and feature number are added to the objective function. Y is the prediction target, representing that a Song node has been assigned to style A by the clustering algorithm. Therefore, Y is a variable representing the final classification result. H(Y) is the information entropy. For example, if it is known beforehand that a certain style cluster is very rare, then the uncertainty of a song belonging to this cluster is high, and the value of H(Y) is high. H(Y|G=G) s) is the conditional entropy, which represents the remaining uncertainty of the Y decision result after GNNExplainer finds the key "explanatory subgraph" Gs. A good explanatory subgraph G s (such as highlighting the plateau topography and arched contour), should make it very certain that this song belongs to the "Xintianyou style cluster". In this case, the remaining uncertainty H(Y|G = G s ) will become very small.

[0192] 2. Explanation Generation and Visualization: Select the target: Select one or more representative Song nodes from an unexplained style cluster (such as the "Xintianyou style cluster"). Run optimization: For the selected Song nodes, run the optimization process of GNNExplainer. The model will learn an edge mask, assigning an importance weight between [0, 1] to each edge in the computational graph. Extract the subgraph: According to the learned weights, filter out the most important edges (for example, the top-K edges with the highest weights). These edges and the nodes they connect together form the final explanatory subgraph. Result interpretation: This subgraph intuitively shows the key factors defining the song style. For example, for a song in the "Xintianyou" style, the explanatory subgraph may highlight the following connections: strong connections between the Song node and specific ContourShape (such as Convex) and RhythmicPattern (such as free tempo pattern). The GeographicRegion node that the Song node connects to through the originatesIn relationship, and this geographic node is closely connected to the Plateau_Topography (plateau topography) and Jin_Chinese_LinguisticArea (Jin dialect area) nodes. This explanatory subgraph provides direct and visual evidence for ethnomusicologists, linking the abstract clustering results with specific musical features and geographical origins.

[0193] Example: Take the northern Shaanxi folk song "Lan Huahua" as an example. "Lan Huahua" is a typical Xintianyou style, and its musical features are closely related to the geographical environment of the Loess Plateau, manifested as high-pitched and open, with a free rhythm.

[0194] Original data:

[0195] Song excerpt (simplified score): Take the classic melody of the line "Blue thread, blue thread" in "Lan Huahua". 3 5 5|6 16 5|3 5 3 2|1--||(where 1, 2, 3 are in the high octave).

[0196] Geographical and cultural context:

[0197] Origin (GeographicRegion): Northern Shaanxi region;

[0198] Topography: Loess Plateau;

[0199] Linguistic Area: Jin Chinese;

[0200] Production method (Livelihood Style): Agriculture;

[0201] Cultural Practice: Mountain songs and love songs.

[0202] Preprocessing of simplified musical notation data:

[0203] Step 1: Pitch Symbolization

[0204] Convert musical phrases into a sequence of (p, o) tuples.

[0205] High 3 → (3, 1);

[0206] High 5 → (5, 1);

[0207] High 6 → (6, 1);

[0208] High 1 → (1, 2) (two octaves higher than middle octave);

[0209] High 2 → (2, 1);

[0210] Output the core content of MelodicFragment: ((3,1),(5,1),(5,1),(6,1),(1,2),(6,1),(5,1),(3,1),(5,1),(3,1),(2,1),(1,1)).

[0211] Step 2: Extracting Relative Intervals

[0212] Calculate the pitch change sequence I k (Taking N=7 as an example.)

[0213] I2 = (5_3) + 7 × (1_1) = +2;

[0214] I3 = (5_5) + 7 × (1_1) = 0;

[0215] I4 = (6_5) + 7 × (1_1) = +1;

[0216] I5=(1_6)+7×(2_1)=-5+7=+2;

[0217] And so on, the output relative tone sequence is: (+2, 0, +1, +2, -2 -1, -2, +2, -2, -1, -1).

[0218] Step 3: Relativity of rhythmic information

[0219] The first note (eighth note, 0.5 SDU) is used as the base 1.

[0220] Musical rhythm pattern: eighth, eighth, eighth | eighth, eighth, eighth, eighth | eighth, eighth, eighth, eighth | eighth, eighth, eighth, eighth | dotted fourth, eighth |

[0221] The absolute time value sequence Dabs is: (0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1.5, 0.5).

[0222] Output the core content of RhythmicPattern R: [1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1].

[0223] Step 4: Abstracting the Melody Outline

[0224] Based on the relative tone sequence, the direction sequence is first generated, and then its macroscopic form is summarized.

[0225] Direction sequence Cdir: (Up, Flat, Up, Up, Down, Down, Down, Up, Down, Down, Down).

[0226] Contour analysis: The sequence contains significant upward jumps, showing an overall trend of first rising and then falling.

[0227] Output the core content of ContourShape: Convex (arch).

[0228] Construction of a simplified musical notation knowledge graph without theoretical ties:

[0229] Step 1: Define the graph node type (instantiation)

[0230] Based on the music ontology definition, the following music feature nodes were created for the excerpt from "Orchid Flower":

[0231] Song node: Creates the central node "Orchid Flower" that represents the song itself.

[0232] MelodicFragment node: Creates a node frag_lanhuahua_1, whose core content is the sequence of symbol markers generated in pitch symbolization ((3,1), (5,1), ...).

[0233] RhythmicPattern node: Create node rhy_lanhuahua_1, whose core content is the relative time value sequence [1, 1, ...] generated in the rhythm information relativization.

[0234] ContourShape node: Creates (or associates with) an existing Convex node.

[0235] Step 2: Define the relationship types between nodes (instantiation). Based on the relationships defined in the music ontology, connect the above nodes to form structured music knowledge:

[0236] Create the relationship "Lanhuahua" --(hasFragment)-->frag_lanhuahua_1, indicating that the song contains this melody fragment.

[0237] Create the relationship frag_lanhuahua_1--(hasRhythm)-->rhy_lanhuahua_1, indicating that the fragment has this rhythmic pattern.

[0238] Create the relationship frag_lanhuahua_1--(hasShape)-->Convex, indicating that the fragment has this contour shape.

[0239] Meanwhile, the frag_lanhuahua_1 node internally stores a complete ordered list of symbol tags through the containsSequence relationship.

[0240] At this point, a subgraph describing the internal musical structure of "Lanhuahua" has been completed.

[0241] Multi-layered geographic context ontology construction:

[0242] Step 1: Define the geographic context node type (instantiation)

[0243] Based on the geographic ontology definition, create the following geographic and cultural entity nodes:

[0244] GeographicRegion node: Create a node for the Shaanxi region.

[0245] Topography node: Creates (or associates with) an existing Plateau node.

[0246] LinguisticArea node: Creates (or associates with) an existing Jin_Chinese node.

[0247] LivelihoodStyle node: Creates (or associates with) an existing Agriculture node.

[0248] CulturalPractice node: Creates (or associates with) existing Mountain Song and Love Song nodes.

[0249] Step 2: Define the geographic context relationship type (instantiation)

[0250] Based on the relationships defined by geographic ontology, connect geographic nodes and establish a bridge with the music graph:

[0251] Constructing geographic internal relationships:

[0252] Create a relationship for the Shaanbei region --(hasTopography)-->Plateau.

[0253] Create a relationship for Northern Shaanxi region --(primaryDialect)-->Jin_Chinese.

[0254] Create a relationship in Northern Shaanxi --(dominantLivelihood)-->Agriculture.

[0255] Establish a connection with the song (key bridge):

[0256] Create the relationship "Orchid Flower" --(originatesIn)-->Shaanxi Region. This relationship is key to connecting the music world and the geographical world, linking the Song node with the GeographicRegion node.

[0257] Create relationship "Orchid Flower" --(associatedWith)-->Mountain Song.

[0258] Through the above steps, the originally independent music structure graph is embedded into a rich, multi-layered geographical and cultural context, forming a complete heterogeneous knowledge graph that can be used for subsequent model analysis.

[0259] For heterogeneous graph modeling and HGT-style embedding generation:

[0260] Step 1: Node Feature Vectorization

[0261] Generate an initial numerical vector for each node in the graph.

[0262] Music Nodes:

[0263] MelodicFragment node: Its core sequence first converts each symbol into a vector through an embedding layer, then inputs it into an LSTM encoder, and outputs a fixed-dimensional vector h_frag.

[0264] The RhythmicPattern node takes an MLP as its core vector input and outputs a vector h_rhy.

[0265] ContourShape node: Convex tags find their corresponding vector h_contour through the embedding layer.

[0266] Geographic nodes:

[0267] Topography node Plateau and LinguisticArea node Jin_Chinese, etc., all find their corresponding vectors h_topo and h_ling through their respective embedding layers.

[0268] Song node: The initial feature of the Song node "Orchid Flower" is set to zero vector.

[0269] Step 2: Heterogeneous information aggregation based on HGT

[0270] The HGT model receives these initialized node vectors as input to layer 0. (0) It propagates and aggregates information by stacking multiple attention layers.

[0271] First-level aggregation: For the Song node "Orchid Flower", its representation H at level 0. (0) (Orchid) is the zero vector.

[0272] In the first-level aggregation, the model calculates the mutual attention between "Lanhuahua" and its direct neighbors (such as the MelodicFragment node frag_lanhuahua_1 and the GeographicRegion node Shaanxi region).

[0273] Information from these neighboring nodes is weighted and aggregated to update the representation of "Orchid Flower", resulting in H. (1) (Orchid Flower). At this point, it has begun to integrate the two facts of "having a specific melodic fragment" and "coming from northern Shaanxi".

[0274] Multilayer polymerization:

[0275] In the second-level aggregation, the node representation of "Orchid Flower" is H. (2) (Orchid Flower) will aggregate information from its two-hop neighbors. For example, information will flow from the Topography: Plateau node to the GeographicRegion: Shaanxi node, and then to the Orchid Flower node.

[0276] Through this layer-by-layer information transmission, the final node representation of "Orchid Flower" is able to capture the complex, long-distance dependencies that span multiple nodes and relationship types in the graph.

[0277] Step 3: Comparative Learning-Driven Embedding Optimization

[0278] To ensure that the attention weights learned by HGT are meaningful, that is, to identify the factors most important for style definition.

[0279] Sample construction: Taking the Song node "Orchid Flower" as an example.

[0280] Anchor: The "Orchid Flower" node.

[0281] Positive Sample: A node closely associated with it, such as the directly connected MelodicFragment node frag_lanhuahua_1, or the Song node of another known song of the same style (such as "Walking to the West").

[0282] Negative Sample: Nodes that are not related to it, such as MelodicFragment nodes randomly sampled from the graph that belong to other styles (such as Jiangnan Folk Song).

[0283] Optimization objective: Optimization is performed using the InfoNCE loss function. The goal is to adjust all learnable parameters of the HGT model (such as the attention matrix, embedding layers, etc.) so that in the final embedding space:

[0284] The cosine similarity between the embedding vector of "Orchid Flower" and its positive sample embedding vector is maximized (i.e., they are close to each other).

[0285] The cosine similarity between the embedding vector of "Orchid Flower" and the embedding vectors of all negative samples is minimized (i.e., they are far apart from each other).

[0286] Through this process, HGT learns how to distinguish and weigh the importance of different pieces of information, thereby generating embeddings with high style distinctiveness.

[0287] Step 4: Style-Geographic Fusion Song Embedding Output

[0288] After the above training process, HGT ultimately generates a high-dimensional, information-dense embedding vector for each Song node (including "Orchid Flower") in the knowledge graph. This vector is one of the core outputs of this technical solution and can be directly used for subsequent style clustering and analysis.

[0289] Interpretable style clustering and analysis mechanism:

[0290] Step 1: Unsupervised Style Clustering

[0291] The HGT output of the "Orchid Flower" embedding vector is placed into a "style space" composed of all song vectors.

[0292] Clustering results: In the final "style space", the embedding vector of "Lanhuahua" will be spatially close to other Shaanxi and Shanxi folk songs with similar high-pitched and open characteristics (such as "Zou Xikou"), and thus will be automatically clustered into the "Xintianyou style cluster".

[0293] Step 2: Interpretability analysis based on GNNExplainer

[0294] Explanation generation: Run GNNExplainer, whose goal is to find the computational path that contributes most to the clustering decision of "Orchid Flower".

[0295] Interpretation of Results: GNNExplainer outputs a highlighted explanatory subgraph. This subgraph clearly demonstrates the extremely high importance weights of the two connecting paths from Song: Orchid Flower to ContourShape: Convex and to Topography: Plateau. This provides data-driven, visual evidence for the musicological conclusion that "the style of Xintianyou (a type of folk song) is shaped by its arched melodic contours and plateau geography."

[0296] In summary, this invention constructs a systematic style research process that extends from spectroscopic symbols and multi-layered geographic semantics to multimodal fusion embedding and interpretation analysis. It breaks through the limitations of traditional symbol modeling and geographic representation, and provides innovative technical support for the digital research and cultural inheritance of Chinese folk song styles.

[0297] Furthermore, such as Figure 3 As shown, based on the above-mentioned method for analyzing song styles by fusing heterogeneous information, the present invention also provides a system for analyzing song styles by fusing heterogeneous information, wherein the system for analyzing song styles by fusing heterogeneous information includes:

[0298] The simplified musical notation knowledge graph construction module 51 is used to acquire the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features.

[0299] The heterogeneous knowledge graph construction module 52 is used to obtain multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph.

[0300] The target embedding vector generation module 53 is used to perform node feature vectorization processing, heterogeneous graph aggregation processing, and contrastive learning processing on the heterogeneous knowledge graph to obtain the target embedding vector.

[0301] The song style analysis result output module 54 is used to perform unsupervised style clustering and interpretability analysis on the target embedding vector to obtain the song style analysis result of the target song.

[0302] Furthermore, such as Figure 4 As shown, based on the above-mentioned song style analysis method and system that integrates heterogeneous information, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0303] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a song style analysis program 40 that integrates heterogeneous information. This song style analysis program 40 can be executed by the processor 10 to implement the song style analysis method integrating heterogeneous information in this application.

[0304] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the song style analysis method that integrates heterogeneous information.

[0305] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface.

[0306] In one embodiment, the steps of the song style analysis method that integrates heterogeneous information are implemented when the processor 10 executes the song style analysis program 40 that integrates heterogeneous information in the memory 20.

[0307] In summary, this invention provides a method, system, and terminal for song style analysis that integrates heterogeneous information. The method includes: acquiring the simplified musical notation data of a target song; extracting features from the simplified musical notation data to obtain multiple simplified musical notation features; and constructing a simplified musical notation knowledge graph based on the multiple simplified musical notation features; acquiring multi-level geographical features corresponding to the target song; constructing a geographical context graph based on the multi-level geographical features; and constructing a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph; performing node feature vectorization processing, heterogeneous graph aggregation processing, and contrastive learning processing on the heterogeneous knowledge graph to obtain a target embedding vector; and performing unsupervised style clustering processing and interpretability analysis on the target embedding vector to obtain the song style analysis result of the target song. This invention constructs a musical notation knowledge graph using the musical notation data of songs, and introduces the geographical features corresponding to the songs to construct a geographical context graph. Then, the musical notation knowledge graph and the geographical context graph are fused to obtain a heterogeneous knowledge graph. Subsequently, embedding learning and interpretability clustering analysis are performed on the heterogeneous knowledge graph to finally generate the song style analysis results of the target song, which can effectively improve the accuracy of song style analysis.

[0308] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0309] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0310] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for analyzing song styles by integrating heterogeneous information, characterized in that, The song style analysis method that integrates heterogeneous information includes: Obtain the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features; Obtain the multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph; The process of obtaining the multi-level geographic features corresponding to the target song, constructing a geographic context graph based on the multi-level geographic features, and constructing a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographic context graph specifically includes: The geographical region where the target song is located is determined, and the multi-level geographical features corresponding to the geographical region are obtained, wherein the multi-level geographical features include physical geographical features, human geographical features and cultural geographical features. The geographical region is used as the second central entity, and an initial geographical map is constructed based on the second central entity, the physical geographical features, the human geographical features, and the cultural geographical features. Define the geographic context node type for the initial geographic map to obtain the geographic context map; By performing node connection processing on the simplified spectrum knowledge graph and the geographic context graph, a heterogeneous knowledge graph is obtained. The heterogeneous knowledge graph is subjected to node feature vectorization, heterogeneous graph aggregation, and contrastive learning to obtain the target embedding vector. Unsupervised style clustering and interpretability analysis are performed on the target embedding vector to obtain the song style analysis results of the target song.

2. The song style analysis method based on the fusion of heterogeneous information according to claim 1, characterized in that, The simplified musical notation features include a symbolic sequence, a relative pitch sequence, a relative time value sequence, and a contour morphology sequence. The process involves acquiring the simplified musical notation data of the target song, extracting features from the simplified musical notation data to obtain multiple simplified musical notation features, specifically including: Identify the target song and extract its sheet music data; Multiple music events of the musical notation data are obtained, and each music event is transformed into a binary tuple to obtain the symbol tag sequence; The relative pitch sequence is obtained by extracting the relative pitch intervals from the symbolic sequence. Obtain the absolute time value sequences corresponding to multiple music events, and convert the absolute time value sequences into the relative time value sequences; The relative tone sequence is used to generate a direction sequence, and the direction sequence is classified according to the pattern corresponding to the direction sequence to obtain the contour shape sequence.

3. The song style analysis method based on the fusion of heterogeneous information according to claim 2, characterized in that, The construction of a simplified musical knowledge graph based on multiple simplified musical features specifically includes: The target song is used as the first central entity, and an initial simplified musical notation knowledge graph is constructed based on the first central entity, the symbol tag sequence, the relative pitch sequence, the relative time value sequence, and the contour morphology sequence. Define the relationship types between nodes in the initial simplified spectrum knowledge graph to obtain the simplified spectrum knowledge graph.

4. The song style analysis method based on the fusion of heterogeneous information according to claim 1, characterized in that, The process of performing node feature vectorization, heterogeneous graph aggregation, and contrastive learning on the heterogeneous knowledge graph to obtain the target embedding vector specifically includes: Obtain music feature nodes and geographic entity nodes from the heterogeneous knowledge graph, and perform node feature vectorization processing on the music feature nodes and the geographic entity nodes to obtain a vectorized heterogeneous knowledge graph. A heterogeneous graph converter is determined, and the vectorized heterogeneous knowledge graph is subjected to heterogeneous graph aggregation processing through the heterogeneous graph converter to obtain the target representation vector. The target representation vector is subjected to contrastive learning processing using the InfoNCE loss function to obtain the target embedding vector.

5. The song style analysis method based on the fusion of heterogeneous information according to claim 4, characterized in that, The step of performing heterogeneous graph aggregation processing on the vectorized heterogeneous knowledge graph through the heterogeneous graph converter to obtain the target representation vector specifically includes: The target node and source node in the vectorized heterogeneous knowledge graph are obtained through the heterogeneous graph converter, and the query vector is calculated based on the representation vector corresponding to the target node. Calculate the message vector and key vector based on the representation vector corresponding to the source node, and calculate the attention score based on the query vector and the key vector; The weighted sum is calculated based on the message vector and the attention score, and then the weighted sum is subjected to projection processing, residual connection processing, and layer normalization processing to obtain the target representation vector.

6. The song style analysis method based on the fusion of heterogeneous information according to claim 1, characterized in that, The unsupervised style clustering and interpretability analysis of the target embedding vector to obtain the song style analysis results of the target song specifically include: The target clustering method is determined based on the target embedding vector, wherein the target clustering method includes K Any one of the following algorithms: Means algorithm, density-based DBSCAN algorithm, and hierarchical clustering algorithm; The target embedding vector is subjected to unsupervised style clustering processing according to the target clustering method to obtain the clustering analysis results; The interpretability analysis of the clustering analysis results was performed using the GNNExplainer model to obtain the song style analysis results of the target song.

7. A song style analysis system that integrates heterogeneous information, characterized in that, The song style analysis system that integrates heterogeneous information is used to implement the song style analysis method that integrates heterogeneous information as described in any one of claims 1-6, wherein the song style analysis system that integrates heterogeneous information includes: The simplified musical notation knowledge graph construction module is used to acquire the simplified musical notation data of the target song, extract features from the simplified musical notation data to obtain multiple simplified musical notation features, and construct a simplified musical notation knowledge graph based on the multiple simplified musical notation features; The heterogeneous knowledge graph construction module is used to obtain multi-level geographical features corresponding to the target song, construct a geographical context graph based on the multi-level geographical features, and construct a heterogeneous knowledge graph based on the simplified musical notation knowledge graph and the geographical context graph. The target embedding vector generation module is used to perform node feature vectorization processing, heterogeneous graph aggregation processing, and contrastive learning processing on the heterogeneous knowledge graph to obtain the target embedding vector. The song style analysis result output module is used to perform unsupervised style clustering and interpretability analysis on the target embedding vector to obtain the song style analysis result of the target song.

8. A terminal, characterized in that, The terminal includes: a memory, a processor, and a song style analysis program that integrates heterogeneous information, stored in the memory and executable on the processor. When the song style analysis program that integrates heterogeneous information is executed by the processor, it implements the steps of the song style analysis method that integrates heterogeneous information as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a song style analysis program that integrates heterogeneous information. When the song style analysis program that integrates heterogeneous information is executed by a processor, it implements the steps of the song style analysis method that integrates heterogeneous information as described in any one of claims 1-6.

Images