A method for generating narrative elements of dance drama music based on a knowledge graph

Abstract

The application provides a dance drama music narrative element generation method fusing a knowledge graph, and belongs to the technical field of artificial intelligence, which can at least partially solve the problem that existing music generation models lack understanding and reasoning ability for plot semantics and are difficult to realize semantic alignment of music and plots in the time dimension. The dance drama music narrative element generation method fusing the knowledge graph comprises the following steps: step S1, constructing a dynamic situation dance drama knowledge graph; step S2, generating a unified multi-modal narrative state sequence; step S3, generating a music control parameter sequence corresponding to the current narrative context; and step S4, generating a specific music event sequence in the decoder. The application realizes intelligent narrative generation of dance drama music through collaborative work of a multi-modal feature extraction module, an emotion dynamic modeling module and a semantic-driven music generation module, has understanding and reasoning ability for plot semantics and can realize semantic alignment of music and plots in the time dimension.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method for generating narrative elements of dance drama music that integrates knowledge graphs. Background Technology

[0002] As a comprehensive stage art, dance drama's narrative system is often achieved through the triple fusion of "dance, music, and plot." As a comprehensive art form, the music in dance drama not only needs to possess independent aesthetic value but also bears the core narrative function of narrating the plot, portraying characters, and creating atmosphere. Currently, dance drama composers typically rely on literary scripts and choreography to manually construct the musical structure, arranging thematic motifs and tonal changes around plot twists. This method has a long creation cycle, strong subjective dependence, and is not conducive to the intelligent generation of large-scale dance drama content. In recent years, with the development of music generation technology, people have attempted to use algorithmic models to generate music suitable for stage productions.

[0003] However, traditional music generation models mostly focus on melodic continuity and harmonic rationality, lacking the ability to understand and reason about plot semantics, making it difficult to achieve semantic alignment between music and plot in the time dimension.

[0004] Therefore, there is an urgent need for an intelligent music generation method that can deeply understand narrative semantics, integrate multimodal contexts, and perform dynamic calculations. Summary of the Invention

[0005] The present invention aims to at least solve one of the technical problems existing in the prior art, and to provide a method for generating narrative elements of dance drama music that integrates knowledge graphs.

[0006] One aspect of this invention provides a method for generating narrative elements in dance drama music that integrates knowledge graphs, comprising the following steps: Step S1: Construct a dynamic contextual dance drama knowledge graph. The dynamic contextual dance drama knowledge graph includes entities that represent the narrative elements of the dance drama, the relationships between entities, and assigns dynamic attributes to entities that change with the time step of the plot. The dynamic attributes include contextual attributes and evolution attributes. Step S2: Receive multimodal input data from the dance drama, perform hierarchical narrative parsing on the multimodal input data, fuse and time-align the results, and generate a unified multimodal narrative state sequence; Step S3: Based on the multimodal narrative state sequence, context-aware graph reasoning and music parameter planning are performed in the dynamic context dance drama knowledge graph to generate a music control parameter sequence corresponding to the current narrative context. Step S4: Using the music control parameter sequence as a high-level semantic condition, and utilizing the conditional music Transformer model, a specific music event sequence is generated in the decoder.

[0007] Optionally, step S1 specifically includes: Step S1.1: Extract entity sets from the dance drama script, character settings, dance movement data, and stage visual information using a natural language processing model. With relation set This forms a preliminary knowledge graph. ; Step S1.2, for each entity e in the entity set E i Constructing context attribute vectors Where t represents the plot time step, d represents the context feature dimension, and the context attribute vector is obtained by fusing and encoding the text features, dance action features, and visual features corresponding to the entity, i.e. ,in These represent the encoding functions for text, dance movements, and visual features, respectively. This represents a vector concatenation operation; Step S1.3, for each entity e in the entity set E i Constructing the evolution property Evol(e) i , t), to represent the time series evolution of the entity's state, , where the function Implemented using gated recurrent units or long short-term memory networks, the evolution attribute is based on the entity's evolution attribute Evol(e) at the previous time step. i (t-1) and the current context attribute C(e) i , t), are obtained through deep neural network calculation; Step S1.4, transfer entity e i The embedding vector, context attribute vector C(ei, t), and evolution attribute vector Evol(ei, t) are concatenated to form a graph node represented as a composite vector. , , For entity embedding vectors, This represents a vector concatenation operation; Step S1.5: The composite vector set vi(t) of all nodes and the relation set R together constitute the dynamic context dance drama knowledge graph. in Let R be the set of all node vectors, and let R be the set of edge relationships. Step S1.6: Perform graph embedding training on the dynamic context dance drama knowledge graph to obtain vectorized representations of nodes and relationships.

[0008] Optionally, the context attribute vector Generate using the following formula: ,in These represent the encoding functions for text, dance movements, and visual features, respectively. This indicates a vector concatenation operation.

[0009] Optionally, step S2 specifically includes: S2.1, features are extracted from the text narrative modality, dance vocabulary modality, and visual scene modality respectively to obtain the corresponding feature sequences T, D, and V; S2.2, using a cross-modal attention mechanism, with the text feature sequence T as the query, it interacts with the dance feature sequence D and the visual feature sequence V respectively to generate a cross-modal fusion feature sequence H; S2.3 performs time alignment on the feature sequences T, D, V, and H, and then concatenates the aligned modal features along the feature dimension. A preliminary fusion sequence was obtained, in which The multimodal narrative state is shown at time step t; S2.4, the preliminary fused sequence is input into the hierarchical Transformer model and subjected to multi-level semantic encoding to output the final multimodal narrative state sequence. .

[0010] Optionally, the time alignment is achieved through a dynamic time warping algorithm.

[0011] Optionally, the text narrative modality uses the Transformer model to extract contextual semantic representations. ; The dance vocabulary modality uses a motion feature extractor based on a graph convolutional network (GCN), with the human skeleton topology as the graph input, i.e. Where A is the adjacency matrix of human joints. The action feature vector of t, output This represents the semantic state of the dance at the corresponding time step.

[0012] Optionally, the visual scene modality uses a visual Transformer model to extract semantic representations of the scene images. , where each vector .

[0013] Optionally, step S3 specifically includes: S3.1, the multimodal narrative state sequence The state vector at the current time step t As a semantic query vector, the contextual semantic similarity is calculated, and the most relevant nodes are activated to form a set of activated nodes. ,in

[0014] , For the nodes in the dynamic context dance drama knowledge graph The dynamic vector representation, The semantic similarity threshold; S3.2, Utilize a graph neural network to process the set of activated nodes. The relationship propagation and implicit semantic reasoning of the associated nodes in the dynamic context dance drama knowledge graph are performed to obtain the updated implicit representation of the nodes; S3.3 After multi-layer propagation, the updated node implicit representation is mapped to the music parameter space to obtain preliminary music control parameters. The music parameter space includes rhythm, mode, orchestration, and emotional tension. S3.4, Based on the semantic contribution weight of the activated nodes, the preliminary music control parameters are weighted and fused to obtain the planning parameters Pt for the current time step; S3.5 introduces a context-aware dynamic programming module to generate the final music control parameters. .

[0015] Optionally, the graph neural network used in step S3.2 is a graph attention network.

[0016] Optionally, step S4 specifically includes: S4.1, the music control parameter sequence Encode and generate music sequences that conform to narrative semantics; S4.2, Encode the historically generated music event sequence into an event embedding sequence; S4.3, The conditional embedding sequence is input into the Transformer encoder for encoding to obtain the conditional encoded output; S4.4, using the event embedding sequence as input and the conditional encoding output as conditions, the Transformer decoder generates the music event for the current time step. ; S4.5, will generate new music events Add the historical sequence and repeat steps S4.2 to S4.4 until a complete music event sequence is generated.

[0017] The beneficial effects of this invention are as follows: This invention proposes a method for generating narrative elements in dance drama music that integrates knowledge graphs. This method constructs a narrative semantic relationship model based on knowledge graphs and achieves intelligent narrative generation of dance drama music through the collaborative work of multimodal feature extraction, emotional dynamic modeling, and a semantically driven music generation module. Compared with existing technologies, this invention has significant differences and advantages in the following aspects: 1. Through layered narrative analysis and multimodal information fusion, this method can capture deeper information such as character arcs, plot tension, dance dynamics and stage atmosphere, so that the basis for music generation is no longer surface keywords, but a three-dimensional and in-depth understanding of the dance drama.

[0018] 2. Dynamic Contextual Knowledge Graph (DC-DDKG) provides an explicit, structured knowledge base that links abstract narrative concepts with specific musical techniques in a refined and contextualized manner. The generation process is highly traceable, facilitating artist understanding and adjustments, and overcoming the drawbacks of traditional AI "black box" creation.

[0019] 3. Through context-aware graph reasoning and conditional Transformer models, this method can generate musical segments with inherent developmental logic. The reproduction, transformation, and transition of musical themes are closely linked to the narrative development, solving the problem of musical fragmentation caused by fragmented generation and significantly enhancing the drama and appeal of the dance drama music.

[0020] 4. Unifying the script, dance, and stage design information within a single framework guides music generation. This allows the generated music to precisely and expressively coordinate with the actors' physical movements and stage lighting changes, achieving a higher level of comprehensive artistic effect and making true "intelligent collaborative creation" possible.

[0021] 5. It ensures the quality and controllability of the generation process by adopting a two-stage framework of "semantic planning → event generation". While giving the system creativity, it ensures the controllability of the generation process and the rationality of the music structure through high-level parameters. Attached Figure Description

[0022] Figure 1 The overall system architecture of the method for generating narrative elements in dance drama music provided in a specific embodiment of the present invention; Figure 2 A schematic diagram illustrating the process of constructing a dynamic contextual dance drama knowledge graph using a method for generating narrative elements of dance drama music, as provided in a specific embodiment of the present invention. Figure 3 A detailed flowchart illustrating the method for generating multimodal narrative state sequences in a dance drama music narrative element generation method provided in a specific embodiment of the present invention; Figure 4 A schematic diagram illustrating the operation of the music control parameter sequence block in the method for generating narrative elements of dance drama music provided in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the conditional music generation model of the method for generating narrative elements in dance drama music provided in a specific embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] like Figure 1 As shown, a method for generating narrative elements of dance drama music by integrating knowledge graphs is characterized by the following steps: This invention relates to a method for generating narrative elements in dance drama music that integrates knowledge graphs, comprising the following steps: S1: Construct a dynamic contextual dance drama knowledge graph (DC-DDKG). This knowledge graph not only contains static entities and relationships, but also introduces "Context" and "Evolution" attributes. S2: Based on multimodal input, hierarchical narrative parsing uses a cross-modal attention mechanism to fuse and align the results of the above three parsing methods—text narrative parsing, dance vocabulary parsing, and visual information parsing—into a unified "multimodal narrative state sequence" containing rich details.

[0025] S3: Context-aware graph reasoning and music parameter planning uses the state vector of the current time step as a query to activate relevant narrative entity nodes in DC-DDKG.

[0026] S4: A conditional music Transformer model is adopted, and the generated music control parameter sequence is used as a high-level condition input into the encoder of the Transformer, and specific music events are generated one by one in the decoder.

[0027] like Figure 2 As shown, the specific implementation method for constructing a dynamic contextual dance drama knowledge graph for this core step S1 is as follows: To achieve intelligent narrative generation of dance drama music, a Dynamic Contextual Dance Drama Knowledge Graph (DC-DDKG) is first constructed. This graph not only includes static entities and relationships in the dance drama but also introduces "Context" and "Evolution" attributes to depict the state changes of entities at different stages of the plot. The specific implementation steps are as follows: First, the dance drama script, character designs, dance movement data, and stage visual information are used as input data. A natural language processing model is then used to perform word segmentation, named entity recognition, and relation extraction on the text, thereby obtaining the entity set from the dance drama. With relation set The entity types include characters, events, scenes, props, and music themes, while the relationship types include "character-participation-event," "event-occurrence-scene," and "character's emotion-emotional state," etc. The preliminary graph composed of entities and relationships can be represented as a set of triples: ; To characterize the semantic state of entities within a specific context, contextual attribute vectors are introduced. ,in Indicates the timeline of the plot. This represents the contextual feature dimension. Contextual vectors can be generated by fusing multimodal features, for example: ; in These represent the encoding functions for text, dance movements, and visual features, respectively. This indicates a vector concatenation operation. This context vector simultaneously depicts the character's emotions, movement rhythm, and stage atmosphere, providing semantic contextual information for subsequent music generation.

[0028] To describe the dynamic changes of entities as the plot unfolds, an evolution attribute is introduced. Used to represent the time series evolution of node states: ; Where the function It can be implemented using gated recurrent units (GRU) or long short-term memory networks (LSTM) to capture the state changes and emotion transfer patterns of nodes over time.

[0029] By combining context vectors and evolutionary attributes, graph nodes can be represented as composite vectors: ; in For entity embedding vectors, This represents a vector concatenation operation. Therefore, a dynamic knowledge graph can be represented as:

[0030] in For the set of all node vectors, Let be the set of edge relations.

[0031] To facilitate interaction with the subsequent music generation module, the graph is vectorized. The vector representation of each edge is calculated using a graph embedding algorithm. And by minimizing the loss function:

[0032] like Figure 3As shown, the specific implementation method of hierarchical narrative parsing based on multimodal input for the core step S2 is as follows: To achieve unified modeling of multimodal information in dance dramas at the narrative level, a hierarchical narrative parsing and fusion method based on multimodal input is proposed. The core idea of ​​this method is to extract features and semantically model three types of inputs from the dance drama: textual narrative, dance vocabulary, and visual scenes. Then, a cross-modal attention mechanism is used to achieve semantic alignment and hierarchical fusion between different modalities, thereby generating a unified "multimodal narrative state sequence" containing rich narrative details, which serves as the semantic driving signal for subsequent graph reasoning and music generation.

[0033] (1) Textual narrative modality , indicates a sequence of script text or plot descriptions.

[0034] (2) Dance vocabulary modality , representing the skeletal points or sequence of movement symbols of a dancer's movements.

[0035] (3) Visual scene modality This indicates visual frame information such as stage lighting, scenery, and actor postures.

[0036] At the narrative analysis level, semantic features are extracted for each of the three modalities: For text modalities, the Transformer language model is used to extract contextual semantic representations: ; For the dance modality, a motion feature extractor based on a graph convolutional network (GCN) is used, with the human skeleton topology as the graph input: ; in This is the adjacency matrix of human joints. For a moment The action feature vector. Output This represents the semantic state of the dance at the corresponding time step.

[0037] For the visual modality, a visual Transformer (ViT) is used to extract semantic representations of the scene images: ; Obtain visual feature sequence , where each vector .

[0038] To achieve deep narrative semantic fusion among the three modalities, a cross-modal attention mechanism is introduced. Let the text modality be the primary narrative channel, with its feature vector serving as the query, and the dance and visual modalities as the key and value, respectively. The fusion representation is then: ; in These are the projection matrices for the query, key, and value, respectively. This mechanism embeds the merged multimodal narrative. It ensures that the semantics of the text are explicitly aligned with the temporal and semantic spatial correspondences of the dance movements and the visual environment.

[0039] Because multimodal information exhibits differences in sampling rate and temporal granularity, a time alignment function is introduced to ensure narrative continuity and consistency. Time matching of different modal features is achieved through dynamic time warping: ; in For time-matched paths, after alignment, the modal features are concatenated into a fused sequence: ; in Indicates at time step The multimodal narrative state.

[0040] To further capture the hierarchical structure and semantic logic of the narrative, a Hierarchical Transformer model is introduced. This model encodes short-term plot dependencies at the lower level and captures the plot evolution structure at the higher level. Its computational form is as follows: ; The final narrative state sequence is as follows: ; This sequence serves as input for subsequent DC-DDKG inference and music generation, providing semantic drive for the dynamic matching of musical themes and plot rhythm.

[0041] like Figure 4 The following is a detailed implementation of the context-aware graph reasoning and music parameter planning of this core step S3: Using the multimodal narrative state sequence obtained in step S2 above Select the current time step Narrative state vector This serves as the semantic query vector for graph retrieval. Through contextual semantic similarity calculation, the set of entity nodes most relevant to this state is activated. ,in: ; For nodes in DC-DDKG The dynamic vector representation, This is the semantic similarity threshold. In this way, the system can dynamically retrieve narrative nodes most closely related to the current plot state based on the plot context, such as "character emotional changes," "plot conflict events," or "scene transitions." This applies to the set of activated nodes. By using graph neural networks (GNNs) for relationship propagation and implicit semantic reasoning, potential plot causality and emotion transmission paths can be discovered.

[0042] Let the initial value of the node feature be... Then after After propagation in the layered graph neural network, the hidden features of the nodes are updated as follows: ; in Represents a node The neighborhood group, For the first The weight matrix of the layer, The activation function. Attention coefficients. The weight used to measure semantic dependency between nodes is defined as: ; in This represents a vector concatenation operation. This represents the attention weight vector. It is the hidden representation of the node obtained after multiple propagations. This allows for a comprehensive reflection of the logical relationships and semantic weights between entities within the context of the plot. After completing the graph reasoning, the nodes are implicitly represented... These are transformed into high-level control parameters that can be used for music generation. The music parameter space is defined as follows: ; in These represent rhythm, tone, instrumentation, and emotion, respectively. Mapping function. Defined as: ; MLP, or Multilayer Perceptron, is used to perform nonlinear mapping from high-dimensional narrative features to the music parameter space. To further achieve comprehensive planning of multi-entity information, a context-weighted mechanism is introduced to weight and fuse the music control parameters of multiple activated nodes. ； Among them, weight , representing a node The semantic contribution to the current plot rhythm. This is a learnable parameter vector. To ensure the continuity and logical consistency of the musical narrative in the time dimension, a context-aware dynamic programming module is introduced. Let the time step be... The music control parameters are The final planning result Smooth updates by incorporating historical information: ; like Figure 5 As shown, the specific implementation of the S4 conditional music Transformer model is as follows: To achieve narrative semantic-driven music generation for dance dramas, a music event generation model based on conditionalized Transformer is proposed. This model uses the context-aware music control parameter sequence obtained in step S3. As a high-level conditional input, specific musical events (including pitch, duration, velocity, instrumentation, and rhythm) are generated progressively based on time steps. The model employs a dual-channel Transformer encoder-decoder framework, where the encoder is responsible for feature modeling of the semantic conditional sequence, and the decoder is used to generate musical sequences that conform to narrative semantics. Let the time step be... The music control parameters are Meanwhile, the existing sequence of historical musical events is represented as follows: ; Each event These represent pitch, duration, velocity, and instrumentation, respectively. To enable the model to exhibit both semantic conditional dependence and musical temporal dependence, the above inputs are mapped to the same embedding space: ; in Embed functions for music events. This is the conditional parameter encoding function. The embedded sequence input is fed into the encoder and decoder modules respectively. The encoder consists of... The encoder consists of multiple Transformer layers used to model the temporal correlation and hierarchical semantics of music control parameters. Each encoder layer includes a multi-head attention mechanism and a feedforward network, calculated as follows: ; in , This is a linear transformation matrix for the query, key, and value. The multi-head attention output is then processed through residual connections and layer normalization. ; ; in .go through After stacking the layers, the conditionally encoded output is obtained: ; This output indicates the time step. Contextual semantics and music control constraints. Given a plot time step. Conditional input Gradually generate music events This is then used as the next input until the sequence terminator (EOS) is reached. To balance creativity and controllability, a Top-k truncation mechanism is introduced:

[0043] The final generated sequence of musical events, after being aligned by rhythm and smoothed by key, forms a complete musical trajectory that corresponds one-to-one with the narrative of the dance drama.

[0044] By introducing a conditional Transformer model, this embodiment achieves a hierarchical mapping from narrative semantics to musical events, enabling music generation to not only reflect the rhythm and emotional dynamics of the plot but also possess long-term structural consistency and multimodal correspondence. Compared with traditional music generation methods based on emotion classification or keyword matching, this method significantly improves semantic accuracy, musical logic, and expressiveness, achieving true "narrative-driven music generation" and providing intelligent and controllable technical support for dance drama creation.

[0045] The method of this invention can be implemented using an embedded chip, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0046] As described above, this invention proposes a method for generating narrative elements in dance drama music that integrates knowledge graphs. This method constructs a narrative semantic relationship model based on knowledge graphs and achieves intelligent narrative generation of dance drama music through the collaborative work of multimodal feature extraction, emotional dynamic modeling, and a semantically driven music generation module. Compared with existing technologies, this invention has significant differences and advantages in the following aspects: 1. Through layered narrative analysis and multimodal information fusion, this method can capture deeper information such as character arcs, plot tension, dance dynamics and stage atmosphere, so that the basis for music generation is no longer surface keywords, but a three-dimensional and in-depth understanding of the dance drama.

[0047] 2. Dynamic Contextual Knowledge Graph (DC-DDKG) provides an explicit, structured knowledge base that links abstract narrative concepts with specific musical techniques in a refined and contextualized manner. The generation process is highly traceable, facilitating artist understanding and adjustments, and overcoming the drawbacks of traditional AI "black box" creation.

[0048] 3. Through context-aware graph reasoning and conditional Transformer models, this method can generate musical segments with inherent developmental logic. The reproduction, transformation, and transition of musical themes are closely linked to the narrative development, solving the problem of musical fragmentation caused by fragmented generation and significantly enhancing the drama and appeal of the dance drama music.

[0049] 4. Unifying the script, dance, and stage design information within a single framework guides music generation. This allows the generated music to precisely and expressively coordinate with the actors' physical movements and stage lighting changes, achieving a higher level of comprehensive artistic effect and making true "intelligent collaborative creation" possible.

[0050] 5. It ensures the quality and controllability of the generation process by adopting a two-stage framework of "semantic planning → event generation". While giving the system creativity, it ensures the controllability of the generation process and the rationality of the music structure through high-level parameters.

[0051] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for generating narrative elements in dance drama music that integrates knowledge graphs, characterized in that, Includes the following steps: Step S1: Construct a dynamic contextual dance drama knowledge graph. The dynamic contextual dance drama knowledge graph includes entities that represent the narrative elements of the dance drama, the relationships between entities, and assigns dynamic attributes to entities that change with the time step of the plot. The dynamic attributes include contextual attributes and evolution attributes. Step S2: Receive multimodal input data from the dance drama, perform hierarchical narrative parsing on the multimodal input data, fuse and time-align the results, and generate a unified multimodal narrative state sequence; Step S3: Based on the multimodal narrative state sequence, context-aware graph reasoning and music parameter planning are performed in the dynamic context dance drama knowledge graph to generate a music control parameter sequence corresponding to the current narrative context. Step S4: Using the music control parameter sequence as a high-level semantic condition, and utilizing the conditional music Transformer model, a specific music event sequence is generated in the decoder.

2. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 1, characterized in that, Step S1 specifically includes: Step S1.1: Extract entity sets from the dance drama script, character settings, dance movement data, and stage visual information using a natural language processing model. With relation set This forms a preliminary knowledge graph. ; Step S1.2, for each entity e in the entity set E i Constructing context attribute vectors Where t represents the plot time step, d represents the context feature dimension, and the context attribute vector is obtained by fusing and encoding the text features, dance action features, and visual features corresponding to the entity, i.e. ,in These represent the encoding functions for text, dance movements, and visual features, respectively. This represents a vector concatenation operation; Step S1.3, for each entity e in the entity set E i Constructing the evolution property Evol(e) i , t), to represent the time series evolution of the entity's state, , where the function Implemented using gated recurrent units or long short-term memory networks, the evolution attribute is based on the entity's evolution attribute Evol(e) at the previous time step. i ,t-1) and the current context attribute C(e i , t), are obtained through deep neural network calculation; Step S1.4, transfer entity e i The embedding vector, context attribute vector C(ei, t), and evolution attribute vector Evol(ei, t) are concatenated to form a graph node represented as a composite vector. , , For entity embedding vectors, This represents a vector concatenation operation; Step S1.5: The composite vector set vi(t) of all nodes and the relation set R together constitute the dynamic context dance drama knowledge graph. in Let R be the set of all node vectors, and let R be the set of edge relationships. Step S1.6: Perform graph embedding training on the dynamic context dance drama knowledge graph to obtain vectorized representations of nodes and relationships.

3. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 2, characterized in that, The context attribute vector Generate using the following formula: ,in These represent the encoding functions for text, dance movements, and visual features, respectively. This indicates a vector concatenation operation.

4. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 1, characterized in that, Step S2 specifically includes: S2.1, features are extracted from the text narrative modality, dance vocabulary modality, and visual scene modality respectively to obtain the corresponding feature sequences T, D, and V; S2.2, using a cross-modal attention mechanism, with the text feature sequence T as the query, it interacts with the dance feature sequence D and the visual feature sequence V respectively to generate a cross-modal fusion feature sequence H; S2.3 performs time alignment on the feature sequences T, D, V, and H, and then concatenates the aligned modal features along the feature dimension. A preliminary fusion sequence was obtained, in which The multimodal narrative state is shown at time step t; S2.4, the preliminary fused sequence is input into the hierarchical Transformer model and subjected to multi-level semantic encoding to output the final multimodal narrative state sequence. .

5. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 4, characterized in that, The time alignment is achieved through a dynamic time warping algorithm.

6. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 4, characterized in that, The text narrative modality uses the Transformer model to extract contextual semantic representations. ; The dance vocabulary modality uses a motion feature extractor based on a graph convolutional network (GCN), with the human skeleton topology as the graph input, i.e. Where A is the adjacency matrix of human joints. The action feature vector of t, output This represents the semantic state of the dance at the corresponding time step.

7. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 4, characterized in that, The visual scene modality uses a visual Transformer model to extract semantic representations of scene images. , where each vector .

8. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 4, characterized in that, Step S3 specifically includes: S3.1, the multimodal narrative state sequence The state vector at the current time step t As a semantic query vector, the contextual semantic similarity is calculated, and the most relevant nodes are activated to form a set of activated nodes. ,in For the nodes in the dynamic context dance drama knowledge graph The dynamic vector representation, The semantic similarity threshold; S3.2, Utilize a graph neural network to process the set of activated nodes. The relationship propagation and implicit semantic reasoning of the associated nodes in the dynamic context dance drama knowledge graph are performed to obtain the updated implicit representation of the nodes; S3.3 After multi-layer propagation, the updated node implicit representation is mapped to the music parameter space to obtain preliminary music control parameters. The music parameter space includes rhythm, mode, orchestration, and emotional tension. S3.4, Based on the semantic contribution weight of the activated nodes, the preliminary music control parameters are weighted and fused to obtain the planning parameters Pt for the current time step; S3.5 introduces a context-aware dynamic programming module to generate the final music control parameters. .

9. The method for generating narrative elements of dance drama music by integrating knowledge graphs according to claim 8, characterized in that, The graph neural network used in step S3.2 is a graph attention network.

10. The method for generating narrative elements of dance drama music based on a fused knowledge graph according to any one of claims 1 to 9, characterized in that, Step S4 specifically includes: S4.1, the music control parameter sequence Encode and generate music sequences that conform to narrative semantics; S4.2, Encode the historically generated music event sequence into an event embedding sequence; S4.3, The conditional embedding sequence is input into the Transformer encoder for encoding to obtain the conditional encoded output; S4.4, using the event embedding sequence as input and the conditional encoding output as conditions, the Transformer decoder generates the music event for the current time step. ; S4.5, will generate new music events Add the historical sequence and repeat steps S4.2 to S4.4 until a complete music event sequence is generated.

Images