Artificial intelligence semantic processing system and method for digital media creation

Abstract

The invention provides an artificial intelligence semantic processing system and method oriented to digital media creation, and relates to the technical field of artificial intelligence semantic process.The artificial intelligence semantic processing method comprises the steps that predicate argument relation pairs of language texts are extracted, object space relation pairs of sketch images are extracted at the same time, and a basic semantic unit set is constructed; the integrity and accuracy of cross-modal semantic understanding are ensured, further, semantic units are clustered by using a dynamic routing algorithm, a semantic concept cluster with a clear importance weight is generated, deep mining and structured representation of creation intentions are realized, and the creation intentions are quickly and accurately understood. An initial semantic relation graph is constructed, a graph attention network is used for dynamic reweighting, finally, an enhanced dynamic semantic graph is generated, complex association and a hierarchical structure between semantic concepts are effectively captured, finally, hierarchical analysis is carried out on the semantic graph, and a structured semantic blueprint is output, so that the dynamic semantic graph is obtained. And a reliable semantic processing technology is provided for creation of high-quality digital media contents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence semantic processing, and particularly relates to an artificial intelligence semantic processing system and method for digital media creation. BACKGROUND

[0002] The current digital media creation field highly depends on the combination of artificial thinking and software operation. Creators need to convert abstract language text description or sketch thinking into specific digital elements and layout. This process is often tedious and requires high professional accomplishment. Although the existing technology can perform basic analysis on single modal information, for example, using natural language processing technology to extract entity relationships in the text or using computer vision algorithm to identify objects in the image, it lacks the ability to uniformly and deeply understand and structurally organize the cross-modal semantics carried by the text and sketch. Its limitation lies in the inability to establish an intelligent processing mechanism that can automatically fuse language concepts and visual spatial relationships and analyze their deep semantic associations, resulting in a significant semantic gap between creative input and digital content generation. The machine-readable expression of the creator's intention is insufficient, and it is difficult to automatically generate a semantic blueprint with clear hierarchy, clear structure and directly driving the content generation engine, thereby restricting the improvement of creation efficiency and the progress of intelligent level.

[0003] Therefore, it is necessary to provide an artificial intelligence semantic processing system and method for digital media creation to solve the above technical problems. SUMMARY

[0004] To solve the above technical problems, the present application provides an artificial intelligence semantic processing system and method for digital media creation, which achieves the beneficial effect of in-depth analysis of semantic intention and output of structured semantic blueprint.

[0005] The present application provides an artificial intelligence semantic processing method for digital media creation, comprising:

[0006] S1: performing semantic role labeling on the language text and extracting predicate argument relationship pairs, at the same time, performing visual scene graph analysis on the sketch image and extracting object spatial relationship pairs, encoding the predicate argument relationship pairs and the object spatial relationship pairs into feature vectors to form a basic semantic unit set;

[0007] S2: calculating the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, and based on the semantic similarity, clustering through a dynamic routing algorithm to obtain a plurality of semantic concept clusters and their importance weights;

[0008] S3: constructing an initial semantic relation graph with the semantic concept clusters as nodes and the semantic similarity between the center vectors of the semantic concept clusters in the high-dimensional concept semantic space as edge weights;

[0009] S4: obtaining a reinforced dynamic semantic graph through multi-round iteration learning and dynamic reweighting of the nodes and edges of the initial semantic relation graph by a graph attention network;

[0010] S5: performing hierarchical analysis based on graph theory centrality and community discovery on the reinforced dynamic semantic graph, automatically identifying the core narrative layer, the detailed description layer and the environmental atmosphere layer, and generating machine executable semantic constraints of each layer, and finally synthesizing a structured semantic blueprint.

[0011] Preferably, in step S1, the extraction step of the predicate argument pair includes:

[0012] performing dependency syntax analysis on the input language text to generate a dependency syntax tree of the sentence;

[0013] based on the dependency syntax tree, identifying the core predicate in the sentence and the arguments that have a grammatical dependency relationship with the predicate;

[0014] based on a predefined semantic role framework, performing role classification on the arguments to generate the predicate argument pair of the predicate, argument and semantic role triple structure.

[0015] Preferably, in step S1, the object space relation pair extraction step includes:

[0016] using a target detection algorithm to identify visual objects in the sketch image and output the bounding box coordinates and class labels of the visual objects;

[0017] based on the bounding box coordinates of the visual objects, calculating the relative spatial position between any two visual object bounding boxes;

[0018] based on a predefined set of spatial relation predicates, mapping the relative spatial position to a specific spatial relation to generate the object space relation pair of the subject object, spatial relation and object object triple structure.

[0019] Preferably, in step S1, the step of encoding the object space relation pair into a vector representation includes:

[0020] based on the object space relation pair, locating and cropping a minimum bounding rectangle region containing the subject object and the object object from the sketch image, defined as an object space relation image;

[0021] encoding the object space relation image through the image encoder of the pre-trained cross-modal model to obtain a sketch feature vector.

[0022] Preferably, in step S3, constructing the initial semantic relation graph comprises the following steps:

[0023] Taking the semantic concept cluster as a node, each node is represented by the center vector of its corresponding semantic concept cluster;

[0024] Calculate the cosine similarity between the center vectors of each pair of nodes as the initial edge weight, and construct a complete graph with weights;

[0025] Calculate the mean and standard deviation of all edge weights in the complete graph, determine the pruning threshold based on the mean and standard deviation, traverse each edge in the complete graph, remove edges with weights below the pruning threshold from the complete graph, and retain edges with weights not below the pruning threshold, to obtain the initial semantic relation graph.

[0026] Preferably, in step S4, the graph attention network adopts a multi-head self-attention mechanism, wherein each attention head independently calculates the attention weight between nodes, and the outputs of all attention heads are fused by vector splicing.

[0027] Preferably, in step S4, when calculating the attention weight of the graph attention network, the weight of the corresponding edge in the initial semantic relation graph is introduced as a structural bias term into the calculation process.

[0028] Preferably, in step S5, the identification steps of the core narrative layer, the detail description layer and the environmental atmosphere layer comprise:

[0029] Based on the feature vector centrality algorithm, calculate the influence score of each node in the reinforced dynamic semantic graph, and filter out nodes with high influence scores according to a preset filtering threshold to form the core narrative layer;

[0030] Remove the nodes of the core narrative layer to obtain a remaining structure graph, and use a modularity optimization algorithm to perform community discovery on the remaining structure graph to divide it into multiple communities;

[0031] Calculate the internal connection density of each community, and classify the communities with a community size greater than a preset node number threshold and an internal connection density higher than a preset density threshold as the detail description layer;

[0032] Classify the communities with a community size not greater than a preset node number threshold and an internal connection density not higher than a preset connection density threshold, and the isolated nodes, as the environmental atmosphere layer.

[0033] Preferably, in step S5, generating machine-executable semantic constraints for each layer is to map the node and relation information obtained by hierarchical analysis into a set of pre-defined parameterized instructions that can drive a graphical rendering engine.

[0034] The application provides an artificial intelligence semantic processing system for digital media creation, comprising:

[0035] A semantic unit construction module is configured to perform semantic role labeling on language text, extract predicate argument relation pairs, perform visual scene graph analysis on sketch images, extract object space relation pairs, encode the predicate argument relation pairs and the object space relation pairs into feature vectors, and construct a basic semantic unit set;

[0036] A semantic concept clustering module is configured to calculate semantic similarity between each semantic unit in the basic semantic unit set and a concept prototype vector based on a high-dimensional concept semantic space defined by a pre-trained cross-modal model, and perform clustering based on the semantic similarity through a dynamic routing algorithm to obtain a plurality of semantic concept clusters and importance weights thereof;

[0037] A semantic relation graph construction module is configured to construct an initial semantic relation graph by taking the semantic concept clusters as nodes and taking semantic similarity between center vectors of the semantic concept clusters in the high-dimensional concept semantic space as edge weights;

[0038] A graph attention optimization module is configured to perform multi-round iterative learning and dynamic reweighting on nodes and edges of the initial semantic relation graph through a graph attention network to obtain a reinforced dynamic semantic graph;

[0039] A semantic blueprint generation module is configured to perform hierarchical analysis based on graph theory centrality and community discovery on the reinforced dynamic semantic graph, automatically identify a core narrative layer, a detail description layer and an environment atmosphere layer, and generate machine-executable semantic constraints of each layer to finally synthesize a structured semantic blueprint.

[0040] Compared with the related art, the artificial intelligence semantic processing system and method for digital media creation provided by the application have the following beneficial effects:

[0041] This invention deeply integrates semantic information from both text and sketch modalities. It extracts predicate-argument pairs by semantically tagging the text and simultaneously analyzes sketch images to obtain object spatial relationship pairs. Based on this, a set of basic semantic units is constructed, ensuring the completeness and accuracy of cross-modal semantic understanding. Furthermore, a dynamic routing algorithm is used to cluster semantic units, generating semantic concept clusters with clearly defined importance weights, thereby achieving in-depth mining and structured representation of creative intent. By constructing an initial semantic relationship graph and using a graph attention network for dynamic reweighting, a strengthened dynamic semantic graph is generated, effectively capturing the complex relationships and hierarchical structures between semantic concepts. Based on this, hierarchical analysis of the semantic graph automatically identifies the core narrative layer, detailed description layer, and environmental atmosphere layer, generating machine-executable semantic constraints. Finally, a structured semantic blueprint is output, overcoming the limitations of traditional digital media creation tools that rely on manual conception, have superficial semantic understanding, and lack cross-modal information fusion. It also achieves full-process automation and intelligence from creative input to content generation, ensuring a high degree of consistency between the generated content and the creator's intent, providing a reliable semantic processing technology for the creation of high-quality digital media content. Attached Figure Description

[0042] Figure 1 This is a flowchart of an artificial intelligence semantic processing method for digital media creation according to the present invention;

[0043] Figure 2 This is a module structure diagram of an artificial intelligence semantic processing system for digital media creation according to the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0045] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0046] Example 1

[0047] An AI-powered semantic processing method for digital media creation, in its specific implementation, such as... Figure 1 As shown, a flowchart of an artificial intelligence semantic processing method for digital media creation according to the present invention is illustrated, including:

[0048] Step S1: Perform semantic role labeling on the language text and extract predicate argument pairs. At the same time, perform visual scene graph analysis on the sketch image and extract object space pairs. Encode the predicate argument pairs and object space pairs into feature vectors to form a set of basic semantic units.

[0049] Specifically, in step S1, the extraction steps of predicate argument relation pairs include:

[0050] Dependency parsing is performed on the input language text to generate a dependency parsing tree for the sentence;

[0051] Based on dependency syntax trees, identify the core predicates in a sentence and the arguments that have grammatical dependencies on the predicates;

[0052] Based on a predefined semantic role framework, arguments are classified into roles, generating predicate-argument relation pairs with a predicate, argument, and semantic role triple structure.

[0053] Specifically, in step S1, the extraction steps of object space relation pairs include:

[0054] An object detection algorithm is used to identify visual objects in a sketch image and output the bounding box coordinates and category labels of the visual objects.

[0055] Calculate the relative spatial position between any two visual object bounding boxes based on the bounding box coordinates of the visual objects.

[0056] Based on a predefined set of spatial relation predicates, relative spatial positions are mapped to specific spatial relations, generating object spatial relation pairs with a triple structure of subject object, spatial relation, and object object.

[0057] Specifically, in step S1, the step of encoding the object space relation pairs into a vector representation includes:

[0058] Based on object space relationship pairs, the minimum bounding rectangle region containing the subject object and the object object is located and cropped from the sketch image and defined as the object space relationship image;

[0059] The image encoder of the object spatial relationship is encoded by a pre-trained cross-modal model to obtain sketch feature vectors.

[0060] In the specific implementation process, a core pre-trained cross-modal model needs to be trained first. This model employs a contrastive learning-based dual-encoder architecture, specifically including a text encoder and an image encoder. The text encoder uses a twelve-layer Transformer architecture with twelve attention heads, processing the input text through a multi-layer self-attention mechanism and a feedforward neural network. The image encoder uses a Vision Transformer architecture, segmenting the input image into 16x16 pixel blocks, which are then processed by the same Transformer architecture after linear projection and positional encoding. The cross-modal model is trained using a large-scale public image-text pairing dataset. A contrastive learning loss function is used to narrow the cosine similarity of paired image-text feature vectors and to keep unpaired image-text feature vectors further apart. Ultimately, the model embeds semantically similar images and texts into similar positions within the same high-dimensional semantic space. For the input language text, dependency parsing is performed first. A rule-based method is used to identify the core predicates of the sentence and arguments with grammatical dependencies from the analysis results. Then, arguments are classified according to a predefined semantic role framework, generating predicate-argument semantic role triples as text relation pairs. For sketch images… The processing involves using a pre-trained object detection neural network to identify all visual objects in the image and output their bounding box coordinates and category labels. Next, the relative coordinate offset and intersection-union ratio (IUR) of the center points of any two visual object bounding boxes are calculated. Based on a predefined set of spatial relation predicates, the numerical values ​​are mapped to specific spatial relations, generating subject-object spatial relation-object triples as image relation pairs. Finally, the text relation pairs are input into the text encoder of the cross-modal model to obtain text feature vectors. The subject-object bounding boxes in the image relation pairs are merged into a minimum bounding rectangle region. Based on the minimum bounding rectangle region, the object spatial relation image cropped from the sketch image is input into the image encoder of the cross-modal model to obtain image feature vectors. All text feature vectors and image feature vectors together constitute a basic semantic unit set, providing a unified feature representation for cross-modal semantic alignment in subsequent steps.

[0061] Step S2: Based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector, and based on the semantic similarity, perform clustering through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights.

[0062] In the specific implementation process, the high-dimensional conceptual semantic space defined by the pre-trained cross-modal model is first calculated. This high-dimensional conceptual semantic space is represented by hundreds of conceptual prototype vectors, which are obtained by clustering text and image feature vectors generated by the cross-modal model on a large corpus. Then, the cosine similarity between each semantic unit and each conceptual prototype vector in the basic semantic unit set is calculated, and this cosine similarity is the semantic similarity between them. Next, clustering is performed using a dynamic routing algorithm. For example, the coupling coefficient between each semantic unit and each conceptual prototype vector is first initialized to zero, and the conceptual prototype vector is used as the initial output vector of the high-level semantic capsule. Then, the iterative process begins. The first step is to cluster each semantic unit with the... The coupling coefficients of high-level semantic capsules are converted into attribution weights using the Softmax function. The second step involves weighted summation of all semantic units for each high-level semantic capsule according to their attribution weights to obtain a composite vector. The third step applies a compression function to each composite vector, maintaining the vector direction while compressing its magnitude to between 0 and 1, generating the high-level semantic capsule output vector for the current iteration. The fourth step calculates the inner product between the feature vector of each semantic unit and the output vectors of each high-level semantic capsule, and adds this inner product value to the original coupling coefficients. This iterative process is repeated a preset number of times until termination. Finally, the stabilized high-level semantic capsule output vector is used as the center vector of the semantic concept cluster, and the magnitude of the center vector of the semantic concept cluster is the importance weight of that semantic concept cluster.

[0063] Step S3: Using semantic concept clusters as nodes and the semantic similarity between the central vectors of semantic concept clusters in the high-dimensional semantic space as edge weights, construct an initial semantic relationship graph.

[0064] Specifically, in step S3, constructing the initial semantic relation graph includes the following steps:

[0065] Using semantic concept clusters as nodes, each node is represented by the center vector of its corresponding semantic concept cluster;

[0066] Calculate the cosine similarity between the center vectors of each pair of nodes as the initial edge weights, and construct a weighted complete graph;

[0067] Calculate the average and standard deviation of the weights of all edges in the complete graph. Determine the pruning threshold based on the average and standard deviation. Traverse every edge in the complete graph and remove edges with weights lower than the pruning threshold from the complete graph. Keep edges with weights not lower than the pruning threshold to obtain the initial semantic relation graph.

[0068] In the specific implementation process, semantic concept clusters are first used as nodes, and each node is represented by the center vector of its corresponding semantic concept cluster. Then, the cosine similarity between the center vectors of each pair of nodes is calculated as the initial edge weight, thereby constructing a weighted complete graph. After the graph is constructed, a pruning step is performed. For example, the average and standard deviation of the weights of all edges in the complete graph are calculated first, and the average minus one standard deviation is set as the pruning threshold. Then, each edge in the complete graph is traversed, and edges with weights lower than the pruning threshold are removed from the initial semantic relation graph, while edges with weights higher than or equal to the pruning threshold are retained, forming a pruned sparse semantic relation graph, i.e., the initial semantic relation graph. The pruning process effectively removes noisy connections and highlights significant semantic associations, providing a clearer and more effective graph structure foundation for subsequent graph attention network processing.

[0069] Step S4: Through a graph attention network, the nodes and edges of the initial semantic relationship graph are subjected to multiple rounds of iterative learning and dynamic reweighting to obtain a reinforced dynamic semantic graph;

[0070] Specifically, in step S4, the graph attention network adopts a multi-head self-attention mechanism, in which each attention head independently calculates the attention weights between nodes, and the outputs of all attention heads are fused by vector concatenation.

[0071] Specifically, in step S4, when calculating the attention weights of the graph attention network, the weights of the corresponding edges in the initial semantic relation graph are introduced into the calculation process as structural bias terms.

[0072] In the specific implementation process, a graph attention network is used to perform multiple rounds of iterative learning and dynamic reweighting on the initial semantic relationship graph to obtain a reinforced dynamic semantic graph. This graph attention network employs a multi-head self-attention mechanism, where the network structure contains multiple independent attention heads, each with independent learnable weight parameters. For each node in the initial semantic relationship graph, each attention head first calculates the attention weights between that node and all its neighboring nodes. The calculation process involves first projecting the node features onto a new feature space through a shared linear transformation layer, then calculating the dot product between the projected features as the attention score. Simultaneously, the weights of the corresponding edges in the initial semantic relationship graph are added to this attention score as structural bias terms. Then... The LeakyReLU activation function is used, and the final attention weights are obtained by normalization using the softmax function. Based on the final attention weights, each attention head performs a weighted summation of the features of its neighboring nodes to generate a new feature representation for that node. The output features of all attention heads are fused by vector concatenation to form the final updated features of that node. The above process is executed in parallel on all nodes in the initial semantic relationship graph to complete one round of graph attention learning. The multi-round iterative learning process repeats the preset number of learning rounds. After each round of iteration, the node features are dynamically updated, and the edge weights are indirectly reweighted through the changes in the attention weights. Finally, after multiple rounds of iteration, a reinforced dynamic semantic graph with optimized node features and edge relationships is obtained.

[0073] Step S5: Perform a hierarchical analysis on the enhanced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify the core narrative layer, the detailed description layer, and the environmental atmosphere layer, and generate machine-executable semantic constraints for each layer, finally synthesizing a structured semantic blueprint.

[0074] Specifically, in step S5, the identification steps for the core narrative layer, the detailed description layer, and the environmental atmosphere layer include:

[0075] Based on the eigenvector centrality algorithm, the influence score of each node in the enhanced dynamic semantic graph is calculated, and nodes with high influence scores are selected according to a preset screening threshold to form the core narrative layer.

[0076] Remove the nodes of the core narrative layer to obtain the remaining structural graph. Use a modularity optimization algorithm to perform community detection on the remaining structural graph and divide it into multiple communities.

[0077] Calculate the internal connection density of each community, and classify communities whose size is greater than a preset threshold for the number of nodes and whose internal connection density is higher than a preset density threshold into the detailed description layer;

[0078] Communities whose size is no greater than a preset threshold for the number of nodes and whose internal connection density is no higher than a preset threshold for connection density, as well as isolated nodes, are classified into the environmental atmosphere layer.

[0079] Specifically, in step S5, generating machine-executable semantic constraints for each layer involves mapping the node and relationship information obtained from the layered analysis into a predefined set of parameterized instructions that can drive the graphics rendering engine.

[0080] In the specific implementation process, firstly, the reinforced dynamic semantic graph is analyzed based on graph centrality, and the influence score of each node in the reinforced dynamic semantic graph is calculated using the eigenvector centrality algorithm. The eigenvector centrality algorithm assigns an influence score to each node through an iterative process, where the influence score of each node is determined by the influence scores of all its neighboring nodes and the weights of the connecting edges. This process is repeated multiple times until the influence scores of all nodes stabilize and converge. Then, based on a preset screening threshold, the node scores are sorted from high to low, and the top 10% of nodes with the highest scores are selected. These selected nodes are the high-influence nodes, which constitute the core narrative layer. Next, these core narrative layer nodes are removed from the reinforced dynamic semantic graph to obtain the remaining structure graph. The remaining structure graph is then used for community discovery using a modularity optimization algorithm. For example, the Louvain algorithm is used, which continuously adjusts the community affiliation of nodes to maximize the modularity value. The modularity value is used to measure the gain of the connection density within a community relative to a random network. The iterative process continues until the modularity no longer increases, thereby dividing the nodes into multiple communities. Then, the calculation is performed. The internal connection density of each community is calculated as the ratio of the sum of the actual weights of all edges within the community to the theoretical maximum sum of connection weights formed by the community nodes. The theoretical maximum sum of connection weights is calculated by assuming that all nodes within the community are interconnected pairwise and that the weight of each edge is the maximum value preset by the system. Then, based on preset node number thresholds and preset connection density thresholds, communities with a size greater than the preset node number threshold and an internal connection density higher than the preset connection density threshold are classified as the detail description layer. Communities with a size less than or equal to the preset node number threshold, an internal connection density not higher than the density threshold, and isolated nodes not included in any community in the graph are classified as the atmosphere layer. Finally, machine-executable semantic constraints are generated for each layer, mapping the node and relationship information obtained from the layered analysis to a predefined set of parameterized instructions. For example, instructions are generated for controlling the layout of main objects and key action sequences for nodes in the core narrative layer, instructions are generated for managing local object grouping and detail rendering for communities in the detail description layer, and instructions are generated for adjusting global illumination and background effects for nodes and communities in the atmosphere layer. Finally, a structured semantic blueprint is synthesized.

[0081] The working principle of the artificial intelligence semantic processing method for digital media creation provided by this invention is as follows:

[0082] This invention automatically transforms abstract creative ideas into executable digital media blueprints through cross-modal semantic alignment and hierarchical graph structure analysis. First, semantic role parsing is performed on the linguistic text to extract predicate argument relations. Simultaneously, visual scene parsing is performed on the sketch image to extract object spatial relations. A pre-trained cross-modal model is used to encode both types of relations into a single high-dimensional semantic space, forming basic semantic units. Next, a dynamic routing algorithm adaptively clusters semantic units based on their similarity to conceptual prototypes, forming semantic concept clusters with clearly defined importance weights. Subsequently, an initial semantic relation graph is constructed using concept clusters as nodes and semantic similarity as edges. Multiple rounds of iterative learning are performed using a graph attention network, dynamically optimizing node features and edge weights during message passing to form a reinforced semantic graph. Finally, core narrative nodes are identified based on graph centrality. Modularity optimization is used to divide the remaining graph into communities, distinguishing between a detail description layer and an environmental atmosphere layer based on community size and connection density. The semantic constraints of each layer are mapped to parameterized instructions driving the graphics engine, achieving end-to-end automatic generation from multimodal input to a structured semantic blueprint, providing a reliable semantic processing technology for the creation of high-quality digital media content.

[0083] Example 2

[0084] An AI-powered semantic processing system for digital media creation, in its specific implementation, such as... Figure 2 As shown, it illustrates a modular structure diagram of an artificial intelligence semantic processing system for digital media creation according to the present invention, including:

[0085] The semantic unit construction module 100 is used to perform semantic role annotation on language text and extract predicate argument pairs. At the same time, it performs visual scene graph analysis on sketch images and extracts object space pairs. The predicate argument pairs and object space pairs are encoded into feature vectors to form a basic semantic unit set.

[0086] The semantic concept clustering module 200 is used to calculate the semantic similarity between each semantic unit in the basic semantic unit set and the concept prototype vector based on the high-dimensional concept semantic space defined by the pre-trained cross-modal model, and to perform clustering based on the semantic similarity through a dynamic routing algorithm to obtain multiple semantic concept clusters and their importance weights.

[0087] The semantic relation graph construction module 300 is used to construct an initial semantic relation graph with semantic concept clusters as nodes and the semantic similarity between the central vectors of semantic concept clusters in the high-dimensional semantic space as edge weights.

[0088] The graph attention optimization module 400 is used to perform multiple rounds of iterative learning and dynamic reweighting on the nodes and edges of the initial semantic relationship graph through a graph attention network to obtain a reinforced dynamic semantic graph.

[0089] The semantic blueprint generation module 500 is used to perform hierarchical analysis on reinforced dynamic semantic graphs based on graph theory centrality and community discovery. It automatically identifies the core narrative layer, the detailed description layer, and the environmental atmosphere layer, and generates machine-executable semantic constraints for each layer, ultimately synthesizing a structured semantic blueprint.

[0090] The working principle of the artificial intelligence semantic processing system for digital media creation provided by this invention is as follows:

[0091] This invention first uses a semantic unit construction module 100 to perform parallel parsing of the input language text and sketch image, extracting predicate argument pairs and object space pairs respectively, and using a pre-trained cross-modal model to encode the two types of heterogeneous data into a unified high-dimensional semantic space to form a basic semantic unit set. Next, a semantic concept clustering module 200 uses a dynamic routing algorithm to adaptively cluster the semantic units, generating important semantic concept clusters with clear weights. Subsequently, a semantic relationship graph construction module 300 uses these concept clusters as nodes, constructs an initial relationship graph based on their semantic similarity, and a graph attention optimization module 400 dynamically strengthens the node features and edge relationships in the graph through multiple rounds of iterative learning. Finally, a semantic blueprint generation module 500 performs hierarchical analysis on the optimized semantic graph, identifies core narrative elements based on graph centrality, distinguishes between detail description units and environmental atmosphere units through a community detection algorithm, and maps the analysis results into parameterized instructions that can drive the graphics engine, thereby forming a structured semantic blueprint. This achieves end-to-end automatic generation from multimodal input to a structured semantic blueprint, providing a reliable semantic processing technology for the creation of high-quality digital media content.

[0092] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0094] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. An artificial intelligence semantic processing method for digital media creation, characterized in that, The semantic processing method comprises: S1: performing semantic role labeling on the language text, and extracting a predicate argument relationship pair, and simultaneously, performing visual scene graph analysis on the sketch image, and extracting an object space relationship pair, and encoding the predicate argument relationship pair and the object space relationship pair into a feature vector to form a basic semantic unit set; S2: calculating semantic similarity of each semantic unit in the basic semantic unit set and a concept prototype vector based on a high-dimensional concept semantic space defined by a pre-trained cross-modal model, and clustering through a dynamic routing algorithm based on the semantic similarity to obtain a plurality of semantic concept clusters and importance weights thereof; S3: constructing an initial semantic relation graph by taking the semantic concept clusters as nodes and taking semantic similarity between center vectors of the semantic concept clusters in the high-dimensional concept semantic space as edge weights; S4: obtaining a reinforced dynamic semantic graph by performing multi-round iteration learning and dynamic reweighting on nodes and edges of the initial semantic relation graph through a graph attention network; S5: performing hierarchical analysis based on graph theory centrality and community discovery on the reinforced dynamic semantic graph to automatically identify a core narrative layer, a detail description layer and an environment atmosphere layer, and generating machine executable semantic constraints of each layer to finally synthesize a structured semantic blueprint.

2. The artificial intelligence semantic processing method for digital media creation according to claim 1, wherein, In step S1, the extraction step of the predicate argument relationship pair comprises: performing dependency syntax analysis on the input language text to generate a dependency syntax tree of the sentence; identifying a core predicate in the sentence and an argument having a grammatical dependency relationship with the predicate based on the dependency syntax tree; performing role classification on the argument based on a predefined semantic role framework to generate the predicate argument relationship pair in a predicate, argument and semantic role triple structure.

3. The artificial intelligence semantic processing method for digital media creation of claim 2, wherein, In step S1, the extraction step of the object space relationship pair comprises: identifying visual objects in the sketch image using a target detection algorithm and outputting boundary box coordinates and class labels of the visual objects; calculating relative spatial positions between boundary boxes of any two visual objects based on the boundary box coordinates of the visual objects; mapping the relative spatial positions to specific spatial relationships based on a predefined set of spatial relationship predicates to generate the object space relationship pair in a subject object, spatial relationship and object object triple structure.

4. The artificial intelligence semantic processing method for digital media creation according to claim 3, wherein, In step S1, the step of encoding the object space relationship pair into a vector representation comprises: based on the object space relationship pair, locating and cropping a minimum circumscribed rectangular region containing the subject object and the object object from the sketch image, defined as an object space relationship image; encoding the object space relationship image through an image encoder of the pre-trained cross-modal model to obtain a sketch feature vector.

5. The artificial intelligence semantic processing method for digital media creation of claim 4, wherein, In step S3, constructing the initial semantic relation graph comprises the following steps: taking the semantic concept clusters as nodes, each node using a center vector of the corresponding semantic concept cluster to represent; calculating cosine similarity between center vectors of each pair of nodes as initial edge weights to construct a complete graph with weights; The average value and the standard deviation of the weight of all edges in the complete graph are calculated, a pruning threshold is determined based on the average value and the standard deviation, each edge in the complete graph is traversed, edges with a weight lower than the pruning threshold are removed from the complete graph, and edges with a weight not lower than the pruning threshold are retained, to obtain an initial semantic relation graph.

6. The artificial intelligence semantic processing method for digital media creation according to claim 5, wherein, In step S4, the graph attention network adopts a multi-head self-attention mechanism, wherein each attention head independently calculates the attention weight between nodes, and the outputs of all attention heads are fused by vector splicing.

7. The artificial intelligence semantic processing method for digital media creation of claim 6, wherein, In step S4, when calculating the attention weight of the graph attention network, the weight of the corresponding edge in the initial semantic relation graph is introduced as a structural bias term into the calculation process.

8. The artificial intelligence semantic processing method for digital media creation of claim 7, wherein, In step S5, the identification steps of the core narrative layer, the detail description layer and the environment atmosphere layer include: Based on the feature vector centrality algorithm, the influence score of each node in the reinforced dynamic semantic graph is calculated, and nodes with high influence scores are selected according to a preset filtering threshold to form a core narrative layer. The nodes of the core narrative layer are removed to obtain a remaining structure graph, and a modularity optimization algorithm is used to perform community discovery on the remaining structure graph to divide a plurality of communities. The internal connection density of each community is calculated, and a community with a community size greater than a preset node number threshold and an internal connection density higher than a preset density threshold is classified as a detail description layer. A community with a community size not greater than the preset node number threshold and an internal connection density not higher than the preset connection density threshold, and an isolated node, are classified as an environment atmosphere layer.

9. The artificial intelligence semantic processing method for digital media creation of claim 8, wherein, In step S5, the machine executable semantic constraints of each layer are generated by mapping the node and relation information obtained by the hierarchical analysis into a set of parameterized instructions predefined and drivable for a graphic rendering engine.

10. An artificial intelligence semantic processing system for digital media authoring, characterized by, The semantic processing system is applied to the artificial intelligence semantic processing method for digital media creation according to any one of claims 1 to 9, and the semantic processing system comprises: A semantic unit construction module is configured to perform semantic role labeling on language texts, extract predicate argument relation pairs, perform visual scene graph analysis on sketch images, extract object spatial relation pairs, encode the predicate argument relation pairs and the object spatial relation pairs into feature vectors, and construct a basic semantic unit set. A semantic concept clustering module is configured to calculate semantic similarity between each semantic unit in the basic semantic unit set and a concept prototype vector based on a high-dimensional concept semantic space defined by a pre-trained cross-modal model, and obtain a plurality of semantic concept clusters and importance weights thereof by dynamic routing algorithm based on the semantic similarity. A semantic relation graph construction module is configured to construct an initial semantic relation graph by taking the semantic concept clusters as nodes and taking semantic similarity between center vectors of the semantic concept clusters in the high-dimensional concept semantic space as edge weights. A graph attention optimization module is configured to obtain a reinforced dynamic semantic graph by performing multi-round iteration learning and dynamic reweighting on nodes and edges of the initial semantic relation graph through a graph attention network. A semantic blueprint generation module is configured to perform hierarchical analysis on the reinforced dynamic semantic graph based on graph theory centrality and community discovery, automatically identify a core narrative layer, a detailed description layer and an environmental atmosphere layer, and generate machine-executable semantic constraints of each layer, and finally synthesize a structured semantic blueprint.

Images