An artificial intelligence semantic processing system and method for digital media creation

Abstract

The application provides an artificial intelligence semantic processing system and method for digital media creation, and relates to the technical field of artificial intelligence semantic processing.The predicate argument relationship pairs of language texts are extracted, the object space relationship pairs of sketch images are extracted, and a basic semantic unit set is constructed, so that the integrity and accuracy of cross-modal semantic understanding are ensured.Further, the semantic units are clustered by using a dynamic routing algorithm, semantic concept clusters with clear importance weights are generated, deep mining and structured representation of creative intentions are realized, an initial semantic relationship graph is constructed, a graph attention network is used for dynamic reweighting, and finally a reinforced dynamic semantic graph is generated, so that the complex association and hierarchical structure between semantic concepts are effectively captured, and finally, through hierarchical analysis of the semantic graph, a structured semantic blueprint is output, and a reliable semantic processing technology is provided for the creation of high-quality digital media content.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence semantic processing technology, and in particular to an artificial intelligence semantic processing system and method for digital media creation. Background Technology

[0002] The current digital media creation field heavily relies on a combination of human conceptualization and software operation. Creators need to transform abstract linguistic descriptions or sketches into concrete digital elements and layouts. This process is often tedious and requires a high level of professional expertise. While existing technologies can perform basic analysis of single-modal information, such as extracting entity relationships from text using natural language processing or identifying objects in images using computer vision algorithms, they lack the ability to achieve a unified, in-depth understanding and structured organization of the cross-modal semantics carried by text and sketches. The limitation lies in the failure to establish an intelligent processing mechanism that can automatically integrate linguistic concepts and visual spatial relationships and analyze their deep semantic connections. This results in a significant semantic gap between creative input and digital content generation. The machine-readable expression of the creator's intentions is insufficient, making it difficult to automatically generate a semantic blueprint that is hierarchical, clearly structured, and can directly drive the content generation engine. This, in turn, restricts the improvement of creative efficiency and the advancement of intelligence.

[0003] Therefore, it is necessary to provide an artificial intelligence semantic processing system and method for digital media creation to solve the above-mentioned technical problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an artificial intelligence semantic processing system and method for digital media creation, achieving the beneficial effect of deeply analyzing semantic intent and outputting a structured semantic blueprint.

[0005] This invention provides an artificial intelligence semantic processing method for digital media creation, comprising:

[0006] S1: Perform semantic role labeling on the language text and extract predicate argument pairs. At the same time, perform visual scene graph analysis on the sketch image and extract object space pairs. Encode the predicate argument pairs and object space pairs into feature vectors to form a set of basic semantic units.

[0007] S2: Based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector, and based on the semantic similarity, perform clustering through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights;

[0008] S3: Using the semantic concept clusters as nodes, and the semantic similarity between the central vectors of the semantic concept clusters in the high-dimensional semantic space as edge weights, construct an initial semantic relationship graph;

[0009] S4: Through a graph attention network, the nodes and edges of the initial semantic relationship graph are subjected to multiple rounds of iterative learning and dynamic reweighting to obtain an enhanced dynamic semantic graph;

[0010] S5: Perform a hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, the detailed description layer, and the environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, finally synthesizing a structured semantic blueprint.

[0011] Preferably, in step S1, the extraction step of predicate argument pairs includes:

[0012] Dependency parsing is performed on the input language text to generate a dependency parsing tree for the sentence;

[0013] Based on dependency syntax trees, identify the core predicates in a sentence and the arguments that have grammatical dependencies on the predicates;

[0014] Based on a predefined semantic role framework, arguments are classified into roles to generate predicate-argument relation pairs with a predicate, argument, and semantic role triple structure.

[0015] Preferably, in step S1, the extraction step of object space relation pairs includes:

[0016] An object detection algorithm is used to identify visual objects in the sketch image, and the bounding box coordinates and category labels of the visual objects are output.

[0017] Calculate the relative spatial position between any two bounding boxes of visual objects based on the bounding box coordinates of the visual objects.

[0018] Based on a predefined set of spatial relation predicates, relative spatial positions are mapped to specific spatial relations, generating object spatial relation pairs with a triple structure of subject object, spatial relation, and object object.

[0019] Preferably, in step S1, the step of encoding the object space relation pairs into a vector representation includes:

[0020] Based on the object space relationship pair, the corresponding minimum bounding rectangle region containing the subject object and the object object is located and cropped from the sketch image and defined as the object space relationship image;

[0021] The image encoder of the pre-trained cross-modal model encodes the spatial relationship image of the object to obtain a sketch feature vector.

[0022] Preferably, in step S3, constructing the initial semantic relationship graph includes the following steps:

[0023] Using semantic concept clusters as nodes, each node is represented by the center vector of its corresponding semantic concept cluster;

[0024] Calculate the cosine similarity between the center vectors of each pair of nodes as the initial edge weights, and construct a weighted complete graph;

[0025] Calculate the average and standard deviation of the weights of all edges in the complete graph. Determine the pruning threshold based on the average and standard deviation. Traverse every edge in the complete graph and remove edges with weights lower than the pruning threshold from the complete graph. Keep edges with weights not lower than the pruning threshold to obtain the initial semantic relation graph.

[0026] Preferably, in step S4, the graph attention network adopts a multi-head self-attention mechanism, wherein each attention head independently calculates the attention weights between nodes, and the outputs of all attention heads are fused by vector concatenation.

[0027] Preferably, in step S4, when calculating the attention weights of the graph attention network, the weights of the corresponding edges in the initial semantic relation graph are introduced into the calculation process as structural bias terms.

[0028] Preferably, in step S5, the identification steps for the core narrative layer, the detailed description layer, and the environmental atmosphere layer include:

[0029] Based on the eigenvector centrality algorithm, the influence score of each node in the enhanced dynamic semantic graph is calculated, and nodes with high influence scores are selected according to a preset screening threshold to form the core narrative layer.

[0030] Remove the nodes of the core narrative layer to obtain the remaining structural graph. Use a modularity optimization algorithm to perform community detection on the remaining structural graph and divide it into multiple communities.

[0031] Calculate the internal connection density of each community, and classify communities whose size is greater than a preset threshold for the number of nodes and whose internal connection density is higher than a preset density threshold into the detailed description layer;

[0032] Communities whose size is no greater than a preset threshold for the number of nodes and whose internal connection density is no higher than a preset threshold for connection density, as well as isolated nodes, are classified into the environmental atmosphere layer.

[0033] Preferably, in step S5, generating machine-executable semantic constraints for each layer involves mapping the node and relationship information obtained from the hierarchical analysis to a predefined set of parameterized instructions that can drive the graphics rendering engine.

[0034] This invention provides an artificial intelligence semantic processing system for digital media creation, comprising:

[0035] The semantic unit construction module is used to annotate the semantic roles of the language text and extract predicate argument pairs. At the same time, it performs visual scene graph analysis on the sketch image and extracts object space pairs. The predicate argument pairs and object space pairs are encoded into feature vectors to form a basic semantic unit set.

[0036] The semantic concept clustering module is used to calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, and to perform clustering based on the semantic similarity through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights.

[0037] The semantic relation graph construction module is used to construct an initial semantic relation graph using the semantic concept clusters as nodes and the semantic similarity between the central vectors of the semantic concept clusters in the high-dimensional semantic space as edge weights.

[0038] The graph attention optimization module is used to perform multiple rounds of iterative learning and dynamic reweighting on the nodes and edges of the initial semantic relationship graph through a graph attention network to obtain an enhanced dynamic semantic graph.

[0039] The semantic blueprint generation module is used to perform hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, detailed description layer and environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, and finally synthesize a structured semantic blueprint.

[0040] Compared with related technologies, the artificial intelligence semantic processing system and method for digital media creation provided by this invention has the following beneficial effects:

[0041] This invention deeply integrates semantic information from both text and sketch modalities. It extracts predicate-argument pairs by semantically tagging the text and simultaneously analyzes sketch images to obtain object spatial relationship pairs. Based on this, a set of basic semantic units is constructed, ensuring the completeness and accuracy of cross-modal semantic understanding. Furthermore, a dynamic routing algorithm is used to cluster semantic units, generating semantic concept clusters with clearly defined importance weights, thereby achieving in-depth mining and structured representation of creative intent. By constructing an initial semantic relationship graph and using a graph attention network for dynamic reweighting, a strengthened dynamic semantic graph is generated, effectively capturing the complex relationships and hierarchical structures between semantic concepts. Based on this, hierarchical analysis of the semantic graph automatically identifies the core narrative layer, detailed description layer, and environmental atmosphere layer, generating machine-executable semantic constraints. Finally, a structured semantic blueprint is output, overcoming the limitations of traditional digital media creation tools that rely on manual conception, have superficial semantic understanding, and lack cross-modal information fusion. It also achieves full-process automation and intelligence from creative input to content generation, ensuring a high degree of consistency between the generated content and the creator's intent, providing a reliable semantic processing technology for the creation of high-quality digital media content. Attached Figure Description

[0042] Figure 1 This is a flowchart of an artificial intelligence semantic processing method for digital media creation according to the present invention;

[0043] Figure 2 This is a module structure diagram of an artificial intelligence semantic processing system for digital media creation according to the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0045] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0046] Example 1

[0047] An AI-powered semantic processing method for digital media creation, in its specific implementation, such as... Figure 1 As shown, a flowchart of an artificial intelligence semantic processing method for digital media creation according to the present invention is illustrated, including:

[0048] Step S1: Perform semantic role labeling on the language text and extract predicate argument pairs. At the same time, perform visual scene graph analysis on the sketch image and extract object space pairs. Encode the predicate argument pairs and object space pairs into feature vectors to form a set of basic semantic units.

[0049] Specifically, in step S1, the extraction steps of predicate argument relation pairs include:

[0050] Dependency parsing is performed on the input language text to generate a dependency parsing tree for the sentence;

[0051] Based on dependency syntax trees, identify the core predicates in a sentence and the arguments that have grammatical dependencies on the predicates;

[0052] Based on a predefined semantic role framework, arguments are classified into roles, generating predicate-argument relation pairs with a predicate, argument, and semantic role triple structure.

[0053] Specifically, in step S1, the extraction steps of object space relation pairs include:

[0054] An object detection algorithm is used to identify visual objects in a sketch image and output the bounding box coordinates and category labels of the visual objects.

[0055] Calculate the relative spatial position between any two bounding boxes of visual objects based on the bounding box coordinates of the visual objects.

[0056] Based on a predefined set of spatial relation predicates, relative spatial positions are mapped to specific spatial relations, generating object spatial relation pairs with a triple structure of subject object, spatial relation, and object object.

[0057] Specifically, in step S1, the step of encoding the object space relation pairs into a vector representation includes:

[0058] Based on object space relationship pairs, the minimum bounding rectangle region containing the subject object and the object object is located and cropped from the sketch image and defined as the object space relationship image;

[0059] The image encoder of the object spatial relationship is encoded by a pre-trained cross-modal model to obtain sketch feature vectors.

[0060] In the specific implementation process, a core pre-trained cross-modal model needs to be trained first. This model employs a contrastive learning-based dual-encoder architecture, specifically including a text encoder and an image encoder. The text encoder uses a twelve-layer Transformer architecture with twelve attention heads, processing the input text through a multi-layer self-attention mechanism and a feedforward neural network. The image encoder uses a Vision Transformer architecture, segmenting the input image into 16x16 pixel blocks, which are then processed by the same Transformer architecture after linear projection and positional encoding. The cross-modal model is trained using a large-scale public image-text pairing dataset. A contrastive learning loss function is used to narrow the cosine similarity of paired image-text feature vectors and to keep unpaired image-text feature vectors further apart. Ultimately, the model embeds semantically similar images and texts into similar positions within the same high-dimensional semantic space. For the input language text, dependency parsing is performed first. A rule-based method is used to identify the core predicates of the sentence and arguments with grammatical dependencies from the analysis results. Then, arguments are classified according to a predefined semantic role framework, generating predicate-argument semantic role triples as text relation pairs. For sketch images… The processing involves using a pre-trained object detection neural network to identify all visual objects in the image and output their bounding box coordinates and category labels. Next, the relative coordinate offset and intersection-union ratio (IUR) of the center points of any two visual object bounding boxes are calculated. Based on a predefined set of spatial relation predicates, the numerical values ​​are mapped to specific spatial relations, generating subject-object spatial relation-object triples as image relation pairs. Finally, the text relation pairs are input into the text encoder of the cross-modal model to obtain text feature vectors. The subject-object bounding boxes in the image relation pairs are merged into a minimum bounding rectangle region. Based on the minimum bounding rectangle region, the object spatial relation image cropped from the sketch image is input into the image encoder of the cross-modal model to obtain image feature vectors. All text feature vectors and image feature vectors together constitute a basic semantic unit set, providing a unified feature representation for cross-modal semantic alignment in subsequent steps.

[0061] Step S2: Based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector, and based on the semantic similarity, perform clustering through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights.

[0062] In the specific implementation process, the high-dimensional conceptual semantic space defined by the pre-trained cross-modal model is first calculated. This high-dimensional conceptual semantic space is represented by hundreds of conceptual prototype vectors, which are obtained by clustering text and image feature vectors generated by the cross-modal model on a large corpus. Then, the cosine similarity between each semantic unit and each conceptual prototype vector in the basic semantic unit set is calculated, and this cosine similarity is the semantic similarity between them. Next, clustering is performed using a dynamic routing algorithm. For example, the coupling coefficient between each semantic unit and each conceptual prototype vector is first initialized to zero, and the conceptual prototype vector is used as the initial output vector of the high-level semantic capsule. Then, the iterative process begins. The first step is to cluster each semantic unit with the... The coupling coefficients of high-level semantic capsules are converted into attribution weights using the Softmax function. The second step involves weighted summation of all semantic units for each high-level semantic capsule according to their attribution weights to obtain a composite vector. The third step applies a compression function to each composite vector, maintaining the vector direction while compressing its magnitude to between 0 and 1, generating the high-level semantic capsule output vector for the current iteration. The fourth step calculates the inner product between the feature vector of each semantic unit and the output vectors of each high-level semantic capsule, and adds this inner product value to the original coupling coefficients. This iterative process is repeated a preset number of times until termination. Finally, the stabilized high-level semantic capsule output vector is used as the center vector of the semantic concept cluster, and the magnitude of the center vector of the semantic concept cluster is the importance weight of that semantic concept cluster.

[0063] Step S3: Using semantic concept clusters as nodes and the semantic similarity between the central vectors of semantic concept clusters in the high-dimensional semantic space as edge weights, construct an initial semantic relationship graph.

[0064] Specifically, in step S3, constructing the initial semantic relation graph includes the following steps:

[0065] Using semantic concept clusters as nodes, each node is represented by the center vector of its corresponding semantic concept cluster;

[0066] Calculate the cosine similarity between the center vectors of each pair of nodes as the initial edge weights, and construct a weighted complete graph;

[0067] Calculate the average and standard deviation of the weights of all edges in the complete graph. Determine the pruning threshold based on the average and standard deviation. Traverse every edge in the complete graph and remove edges with weights lower than the pruning threshold from the complete graph. Keep edges with weights not lower than the pruning threshold to obtain the initial semantic relation graph.

[0068] In the specific implementation process, semantic concept clusters are first used as nodes, and each node is represented by the center vector of its corresponding semantic concept cluster. Then, the cosine similarity between the center vectors of each pair of nodes is calculated as the initial edge weight, thereby constructing a weighted complete graph. After the graph is constructed, a pruning step is performed. For example, the average and standard deviation of the weights of all edges in the complete graph are calculated first, and the average minus one standard deviation is set as the pruning threshold. Then, each edge in the complete graph is traversed, and edges with weights lower than the pruning threshold are removed from the initial semantic relation graph, while edges with weights higher than or equal to the pruning threshold are retained, forming a pruned sparse semantic relation graph, i.e., the initial semantic relation graph. The pruning process effectively removes noisy connections and highlights significant semantic associations, providing a clearer and more effective graph structure foundation for subsequent graph attention network processing.

[0069] Step S4: Through a graph attention network, the nodes and edges of the initial semantic relationship graph are subjected to multiple rounds of iterative learning and dynamic reweighting to obtain a reinforced dynamic semantic graph;

[0070] Specifically, in step S4, the graph attention network adopts a multi-head self-attention mechanism, in which each attention head independently calculates the attention weights between nodes, and the outputs of all attention heads are fused by vector concatenation.

[0071] Specifically, in step S4, when calculating the attention weights of the graph attention network, the weights of the corresponding edges in the initial semantic relation graph are introduced into the calculation process as structural bias terms.

[0072] In the specific implementation process, a graph attention network is used to perform multiple rounds of iterative learning and dynamic reweighting on the initial semantic relationship graph to obtain a reinforced dynamic semantic graph. This graph attention network employs a multi-head self-attention mechanism, where the network structure contains multiple independent attention heads, each with independent learnable weight parameters. For each node in the initial semantic relationship graph, each attention head first calculates the attention weights between that node and all its neighboring nodes. The calculation process involves first projecting the node features onto a new feature space through a shared linear transformation layer, then calculating the dot product between the projected features as the attention score. Simultaneously, the weights of the corresponding edges in the initial semantic relationship graph are added to this attention score as structural bias terms. Then... The LeakyReLU activation function is used, and the final attention weights are obtained by normalization using the softmax function. Based on the final attention weights, each attention head performs a weighted summation of the features of its neighboring nodes to generate a new feature representation for that node. The output features of all attention heads are fused by vector concatenation to form the final updated features of that node. The above process is executed in parallel on all nodes in the initial semantic relationship graph to complete one round of graph attention learning. The multi-round iterative learning process repeats the preset number of learning rounds. After each round of iteration, the node features are dynamically updated, and the edge weights are indirectly reweighted through the changes in the attention weights. Finally, after multiple rounds of iteration, a reinforced dynamic semantic graph with optimized node features and edge relationships is obtained.

[0073] Step S5: Perform a hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, detailed description layer and environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, finally synthesizing a structured semantic blueprint.

[0074] Specifically, in step S5, the identification steps for the core narrative layer, the detailed description layer, and the environmental atmosphere layer include:

[0075] Based on the eigenvector centrality algorithm, the influence score of each node in the enhanced dynamic semantic graph is calculated, and nodes with high influence scores are selected according to a preset screening threshold to form the core narrative layer.

[0076] Remove the nodes of the core narrative layer to obtain the remaining structural graph. Use a modularity optimization algorithm to perform community detection on the remaining structural graph and divide it into multiple communities.

[0077] Calculate the internal connection density of each community, and classify communities whose size is greater than a preset threshold for the number of nodes and whose internal connection density is higher than a preset density threshold into the detailed description layer;

[0078] Communities whose size is no greater than a preset threshold for the number of nodes and whose internal connection density is no higher than a preset threshold for connection density, as well as isolated nodes, are classified into the environmental atmosphere layer.

[0079] Specifically, in step S5, generating machine-executable semantic constraints for each layer involves mapping the node and relationship information obtained from the layered analysis into a predefined set of parameterized instructions that can drive the graphics rendering engine.

[0080] In the specific implementation process, firstly, the reinforced dynamic semantic graph is analyzed based on graph centrality, and the influence score of each node in the reinforced dynamic semantic graph is calculated using the eigenvector centrality algorithm. The eigenvector centrality algorithm assigns an influence score to each node through an iterative process, where the influence score of each node is determined by the influence scores of all its neighboring nodes and the weights of the connecting edges. This process is repeated multiple times until the influence scores of all nodes stabilize and converge. Then, based on a preset screening threshold, the node scores are sorted from high to low, and the top 10% of nodes with the highest scores are selected. These selected nodes are the high-influence nodes, which constitute the core narrative layer. Next, these core narrative layer nodes are removed from the reinforced dynamic semantic graph to obtain the remaining structure graph. The remaining structure graph is then used for community discovery using a modularity optimization algorithm. For example, the Louvain algorithm is used, which continuously adjusts the community affiliation of nodes to maximize the modularity value. The modularity value is used to measure the gain of the connection density within a community relative to a random network. The iterative process continues until the modularity no longer increases, thereby dividing the nodes into multiple communities. Then, the calculation is performed. The internal connection density of each community is calculated as the ratio of the sum of the actual weights of all edges within the community to the theoretical maximum sum of connection weights formed by the community nodes. The theoretical maximum sum of connection weights is calculated by assuming that all nodes within the community are interconnected pairwise and that the weight of each edge is the maximum value preset by the system. Then, based on preset node number thresholds and preset connection density thresholds, communities with a size greater than the preset node number threshold and an internal connection density higher than the preset connection density threshold are classified as the detail description layer. Communities with a size less than or equal to the preset node number threshold, an internal connection density not higher than the density threshold, and isolated nodes not included in any community in the graph are classified as the atmosphere layer. Finally, machine-executable semantic constraints are generated for each layer, mapping the node and relationship information obtained from the layered analysis to a predefined set of parameterized instructions. For example, instructions are generated for controlling the layout of main objects and key action sequences for nodes in the core narrative layer, instructions are generated for managing local object grouping and detail rendering for communities in the detail description layer, and instructions are generated for adjusting global illumination and background effects for nodes and communities in the atmosphere layer. Finally, a structured semantic blueprint is synthesized.

[0081] The working principle of the artificial intelligence semantic processing method for digital media creation provided by this invention is as follows:

[0082] This invention automatically transforms abstract creative ideas into executable digital media blueprints through cross-modal semantic alignment and hierarchical graph structure analysis. First, semantic role parsing is performed on the linguistic text to extract predicate argument relations. Simultaneously, visual scene parsing is performed on the sketch image to extract object spatial relations. A pre-trained cross-modal model is used to encode both types of relations into a single high-dimensional semantic space, forming basic semantic units. Next, a dynamic routing algorithm adaptively clusters semantic units based on their similarity to conceptual prototypes, forming semantic concept clusters with clearly defined importance weights. Subsequently, an initial semantic relation graph is constructed using concept clusters as nodes and semantic similarity as edges. Multiple rounds of iterative learning are performed using a graph attention network, dynamically optimizing node features and edge weights during message passing to form a reinforced semantic graph. Finally, core narrative nodes are identified based on graph centrality. Modularity optimization is used to divide the remaining graph into communities, distinguishing between a detail description layer and an environmental atmosphere layer based on community size and connection density. The semantic constraints of each layer are mapped to parameterized instructions driving the graphics engine, achieving end-to-end automatic generation from multimodal input to a structured semantic blueprint, providing a reliable semantic processing technology for the creation of high-quality digital media content.

[0083] Example 2

[0084] An AI-powered semantic processing system for digital media creation, in its specific implementation, such as... Figure 2 As shown, it illustrates a modular structure diagram of an artificial intelligence semantic processing system for digital media creation according to the present invention, including:

[0085] The semantic unit construction module 100 is used to perform semantic role annotation on language text and extract predicate argument pairs. At the same time, it performs visual scene graph analysis on sketch images and extracts object space pairs. The predicate argument pairs and object space pairs are encoded into feature vectors to form a basic semantic unit set.

[0086] The semantic concept clustering module 200 is used to calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, and to perform clustering based on the semantic similarity through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights.

[0087] The semantic relation graph construction module 300 is used to construct an initial semantic relation graph with semantic concept clusters as nodes and the semantic similarity between the central vectors of semantic concept clusters in the high-dimensional semantic space as edge weights.

[0088] The graph attention optimization module 400 is used to perform multiple rounds of iterative learning and dynamic reweighting on the nodes and edges of the initial semantic relationship graph through a graph attention network to obtain a reinforced dynamic semantic graph.

[0089] The semantic blueprint generation module 500 is used to perform hierarchical analysis on reinforced dynamic semantic graphs based on graph theory centrality and community discovery. It automatically identifies the core narrative layer, the detailed description layer, and the environmental atmosphere layer, and generates machine-executable semantic constraints for each layer, ultimately synthesizing a structured semantic blueprint.

[0090] The working principle of the artificial intelligence semantic processing system for digital media creation provided by this invention is as follows:

[0091] This invention first uses a semantic unit construction module 100 to perform parallel parsing of the input language text and sketch image, extracting predicate argument pairs and object space pairs respectively, and using a pre-trained cross-modal model to encode the two types of heterogeneous data into a unified high-dimensional semantic space to form a basic semantic unit set. Next, a semantic concept clustering module 200 uses a dynamic routing algorithm to adaptively cluster the semantic units, generating important semantic concept clusters with clear weights. Subsequently, a semantic relationship graph construction module 300 uses these concept clusters as nodes, constructs an initial relationship graph based on their semantic similarity, and a graph attention optimization module 400 dynamically strengthens the node features and edge relationships in the graph through multiple rounds of iterative learning. Finally, a semantic blueprint generation module 500 performs hierarchical analysis on the optimized semantic graph, identifies core narrative elements based on graph centrality, distinguishes between detail description units and environmental atmosphere units through a community detection algorithm, and maps the analysis results into parameterized instructions that can drive the graphics engine, thereby forming a structured semantic blueprint. This achieves end-to-end automatic generation from multimodal input to a structured semantic blueprint, providing a reliable semantic processing technology for the creation of high-quality digital media content.

[0092] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0094] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. An artificial intelligence semantic processing method for digital media creation, characterized in that, The semantic processing method includes: S1: Perform semantic role labeling on the language text and extract predicate argument pairs. At the same time, perform visual scene graph analysis on the sketch image and extract object space pairs. Encode the predicate argument pairs and object space pairs into feature vectors to form a set of basic semantic units. S2: Based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector, and based on the semantic similarity, perform clustering through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights; S3: Using the semantic concept clusters as nodes, and the semantic similarity between the central vectors of the semantic concept clusters in the high-dimensional semantic space as edge weights, construct an initial semantic relationship graph; S4: Through a graph attention network, the nodes and edges of the initial semantic relationship graph are subjected to multiple rounds of iterative learning and dynamic reweighting to obtain an enhanced dynamic semantic graph; S5: Perform a hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, the detailed description layer, and the environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, finally synthesizing a structured semantic blueprint.

2. The artificial intelligence semantic processing method for digital media creation according to claim 1, characterized in that, In step S1, the extraction steps for predicate argument pairs include: Dependency parsing is performed on the input language text to generate a dependency parsing tree for the sentence; Based on dependency syntax trees, identify the core predicates in a sentence and the arguments that have grammatical dependencies on the predicates; Based on a predefined semantic role framework, arguments are classified into roles to generate predicate-argument relation pairs with a predicate, argument, and semantic role triple structure.

3. The artificial intelligence semantic processing method for digital media creation according to claim 2, characterized in that, In step S1, the extraction steps of object space relation pairs include: An object detection algorithm is used to identify visual objects in the sketch image, and the bounding box coordinates and category labels of the visual objects are output. Calculate the relative spatial position between any two bounding boxes of visual objects based on the bounding box coordinates of the visual objects. Based on a predefined set of spatial relation predicates, relative spatial positions are mapped to specific spatial relations, generating object spatial relation pairs with a triple structure of subject object, spatial relation, and object object.

4. The artificial intelligence semantic processing method for digital media creation according to claim 3, characterized in that, In step S1, the step of encoding the object space relation pairs into a vector representation includes: Based on the object space relationship pair, the corresponding minimum bounding rectangle region containing the subject object and the object object is located and cropped from the sketch image and defined as the object space relationship image; The image encoder of the pre-trained cross-modal model encodes the spatial relationship image of the object to obtain a sketch feature vector.

5. The artificial intelligence semantic processing method for digital media creation according to claim 4, characterized in that, In step S3, constructing the initial semantic relationship graph includes the following steps: Using semantic concept clusters as nodes, each node is represented by the center vector of its corresponding semantic concept cluster; Calculate the cosine similarity between the center vectors of each pair of nodes as the initial edge weights, and construct a weighted complete graph; Calculate the average and standard deviation of the weights of all edges in the complete graph. Determine the pruning threshold based on the average and standard deviation. Traverse every edge in the complete graph and remove edges with weights lower than the pruning threshold from the complete graph. Keep edges with weights not lower than the pruning threshold to obtain the initial semantic relation graph.

6. The artificial intelligence semantic processing method for digital media creation according to claim 5, characterized in that, In step S4, the graph attention network adopts a multi-head self-attention mechanism, in which each attention head independently calculates the attention weights between nodes, and the outputs of all attention heads are fused by vector concatenation.

7. The artificial intelligence semantic processing method for digital media creation according to claim 6, characterized in that, In step S4, when calculating the attention weights of the graph attention network, the weights of the corresponding edges in the initial semantic relation graph are introduced into the calculation process as structural bias terms.

8. The artificial intelligence semantic processing method for digital media creation according to claim 7, characterized in that, In step S5, the identification steps for the core narrative layer, the detailed description layer, and the environmental atmosphere layer include: Based on the eigenvector centrality algorithm, the influence score of each node in the enhanced dynamic semantic graph is calculated, and nodes with high influence scores are selected according to a preset screening threshold to form the core narrative layer. Remove the nodes of the core narrative layer to obtain the remaining structural graph. Use a modularity optimization algorithm to perform community detection on the remaining structural graph and divide it into multiple communities. Calculate the internal connection density of each community, and classify communities whose size is greater than a preset threshold for the number of nodes and whose internal connection density is higher than a preset density threshold into the detailed description layer; Communities whose size is no greater than a preset threshold for the number of nodes and whose internal connection density is no higher than a preset threshold for connection density, as well as isolated nodes, are classified into the environmental atmosphere layer.

9. The artificial intelligence semantic processing method for digital media creation according to claim 8, characterized in that, In step S5, generating machine-executable semantic constraints for each layer involves mapping the node and relationship information obtained from the layered analysis into a predefined set of parameterized instructions that can drive the graphics rendering engine.

10. An artificial intelligence semantic processing system for digital media creation, characterized in that, An artificial intelligence semantic processing method for digital media creation, as described in any one of claims 1 to 9, wherein the semantic processing system comprises: The semantic unit construction module is used to annotate the semantic roles of the language text and extract predicate argument pairs. At the same time, it performs visual scene graph analysis on the sketch image and extracts object space pairs. The predicate argument pairs and object space pairs are encoded into feature vectors to form a basic semantic unit set. The semantic concept clustering module is used to calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, and to perform clustering based on the semantic similarity through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights. The semantic relation graph construction module is used to construct an initial semantic relation graph using the semantic concept clusters as nodes and the semantic similarity between the central vectors of the semantic concept clusters in the high-dimensional semantic space as edge weights. The graph attention optimization module is used to perform multiple rounds of iterative learning and dynamic reweighting on the nodes and edges of the initial semantic relationship graph through a graph attention network to obtain an enhanced dynamic semantic graph. The semantic blueprint generation module is used to perform hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, detailed description layer and environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, and finally synthesize a structured semantic blueprint.

Images