Real-time data driven marketing copy generation method and system

Abstract

The present application relates to the technical field of big data processing, in particular to a marketing copy generation method and system driven by real-time data. The method comprises: receiving a multi-modal real-time data stream, performing entity relationship joint extraction according to time slicing to generate a time sequence subgraph; performing edge weight time attenuation and conflict resolution on overlapping nodes of adjacent time sequence subgraphs to generate a dynamic knowledge graph; performing graph convolution aggregation on the dynamic knowledge graph to obtain an entity topological structure vector representation; and superimposing the entity topological structure vector representation as an attention bias matrix to a self-attention layer of a generation model transformer architecture to constrain the generation probability distribution of entity tokens in the decoding process to generate a marketing copy. The present application eliminates the interference of obsolete and contradictory information through dynamic graph evolution, uses the graph topological features as an attention bias to constrain the decoding process, suppresses the generation of content entity hallucinations, keeps the text logic and real-time entity association state consistent, and overcomes semantic drift under data flow.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, specifically to a method and system for generating marketing copy driven by real-time data. Background Technology

[0002] In the field of marketing copy generation technology based on big data processing, leveraging real-time data streams to drive content generation has become a crucial technological direction in information distribution. Existing conventional solutions typically employ a data truncation and concatenation architecture with end-to-end generation. In this architecture, the big data processing platform receives heterogeneous real-time data streams from different data sources, extracts data segments according to fixed time windows, performs text-flattening operations on these segments, and concatenates structured data, image tags, and text information into a continuous string sequence. Subsequently, the concatenated string sequence is directly input as contextual cue words into a pre-trained generative model. The generative model encodes the input string sequence based on a self-attention mechanism and predicts each word individually through autoregression, outputting marketing copy corresponding to the input sequence. This end-to-end generation method relies on the feature extraction capabilities within the generative model to capture relevant information in the real-time data.

[0003] To enhance the accuracy of entity relationships in generated content, some existing technologies introduce static knowledge graphs as an external knowledge base. These approaches pre-build a knowledge graph containing entities and their relationships offline. Upon receiving real-time data streams, they extract key entity terms from the real-time data and retrieve adjacent nodes and edges from the static knowledge graph. The retrieved subgraph structure is converted into a linearized text sequence, concatenated with real-time data fragments, and then input into the generative model. However, the topology of the static knowledge graph is fixed after construction and cannot change with the real-time data stream. When new entity nodes or new relationships between entities appear in the real-time data stream that were not included in the offline graph, the static knowledge graph cannot provide corresponding topological support. The generative model can only rely on its own parameterized knowledge to complete the structure, resulting in the output text structure being constrained by the fixed knowledge graph.

[0004] The aforementioned existing technical solutions suffer from severe semantic and logical conflicts and entity illusion problems when processing high-concurrency real-time data streams. Directly inputting flattened and spliced ​​real-time data sequences into the generative model disrupts the inherent temporal and causal topological structure between multi-source heterogeneous data. When processing flattened sequences lacking explicit structural constraints, the generative model's self-attention mechanism is prone to establishing erroneous long-distance dependencies, generating entity associations that contradict the current real-time context, leading to semantic and logical conflicts. While introducing static knowledge graphs provides external topological constraints, their fixed structure cannot reflect the transient evolution of entity relationships in the real-time data stream. Outdated and contradictory entity associations are forcibly injected into the generation process. When data distribution changes abruptly, the generative model, lacking proper structural guidance, tends to generate false entities or erroneous relationships that do not exist in the current real-time data based on local statistical probabilities, resulting in entity illusion. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time data-driven marketing copy generation method and system, which can solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A real-time data-driven marketing copy generation method includes: receiving multimodal real-time data streams based on a big data processing platform, and performing joint entity relationship extraction on the multimodal real-time data streams by time slices to generate time-series subgraphs; Edge weight time decay and conflict resolution are performed on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph that reflects the entity association state at the current moment. The dynamic knowledge graph is subjected to graph convolution aggregation to obtain a vectorized representation of the entity topology; The entity topology is vectorized and represented as an attention bias matrix, which is then superimposed on the self-attention layer of the Transformer architecture of the generative model to constrain the generation probability distribution of entity tokens during the decoding process in order to generate marketing copy.

[0007] Preferably, the step of performing joint entity relation extraction on the multimodal real-time data stream by time slice to generate a time-series subgraph includes: dividing the multimodal real-time data stream into multiple data segments according to a fixed time window, and extracting the text semantic features of text data and the visual region features of image data in each data segment; Cross-modal feature alignment is performed between the text semantic features and the visual region features to identify shared entity nodes in the data fragment; Based on the co-occurrence frequency and dependency syntax relationship of the shared entity nodes in the multimodal real-time data stream, edge connections are constructed between the shared entity nodes to form the time-series subgraph corresponding to the time slice.

[0008] Preferably, the step of performing edge weight time decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph reflecting the current entity association state includes: identifying overlapping entity nodes that exist simultaneously in the preceding and following time slices during the superposition of adjacent temporal subgraphs. For the historical edges associated with the overlapping entity nodes, perform edge weight time decay operation according to the time difference between the generation timestamp of the historical edge and the current timestamp; When the overlapping entity node has conflicting edges with opposite attribute values ​​in the adjacent temporal subgraph, the conflicting edge whose generation timestamp is closest to the current timestamp is retained and the remaining conflicting edges are deleted. The edges retained after decay are merged with the newly added edges to generate the dynamic knowledge graph.

[0009] Preferably, performing graph convolution aggregation on the dynamic knowledge graph to obtain a vectorized representation of the entity topology includes: in the dynamic knowledge graph, for a target node, aggregating the initial feature vectors of neighboring nodes that have edge connections with the target node, and combining the edge type encoding between the target node and the neighboring nodes to calculate the message passing weight of the neighboring nodes to the target node; The initial feature vectors of the neighboring nodes are weighted and summed according to the message passing weights. The weighted summation result is then residually connected and nonlinearly mapped to the original feature vector of the target node to generate the updated feature vector of the target node. Traverse all nodes in the dynamic knowledge graph to obtain a vectorized representation of the entity topology.

[0010] Preferably, the vectorized representation of the entity topology is superimposed as an attention bias matrix onto the self-attention layer of the Transformer architecture of the generative model, and the probability distribution of entity word generation during the decoding process is constrained to generate marketing copy, including: mapping the topological correlation between entity node pairs in the vectorized representation of the entity topology to a bias scalar, and constructing the attention bias matrix that matches the dimension of the self-attention matrix; When calculating the dot product attention score of the query vector and the key vector in the self-attention layer of the generative model Transformer architecture, the bias scalar in the attention bias matrix is ​​superimposed on the corresponding dot product attention score. Based on the attention score after superimposing the bias scalar, a normalization operation and weighted summation are performed to change the probability distribution weight of the lexical units associated with entities in the dynamic knowledge graph on the decoding vocabulary, and the marketing copy is output.

[0011] Preferably, during the conflict resolution process, when there are implicit conflict edges in the adjacent temporal subgraphs where the overlapping entity nodes have opposite semantic directions but not directly opposing literal attributes, the context vector of the neighboring nodes connected by the implicit conflict edge is extracted. Calculate the cosine similarity between the context vector and the current base semantic vector of the overlapping entity node, and determine the implicit conflict edge with the cosine similarity lower than a preset judgment threshold as a semantic contradiction edge; Set the edge weight of the semantically contradictory edge to zero and sever the connection between the semantically contradictory edge and the overlapping entity node. At the same time, reinitialize the local topology structure adjacent to the overlapping entity node based on the current reference semantic vector of the overlapping entity node.

[0012] Preferably, when aggregating the initial feature vectors of the neighboring nodes, an adaptive feature selection mechanism based on temporal freshness is introduced; Obtain the latest update timestamp of the edge connected to the neighbor node in the dynamic knowledge graph, and calculate the time interval between the latest update timestamp and the current graph evolution cycle; A feature retention coefficient is assigned to the neighbor node according to the time interval, and the feature retention coefficient is inversely proportional to the time interval. The initial feature vectors of neighboring nodes whose feature retention coefficient is lower than the activity threshold are filtered out, and only the retained initial feature vectors of neighboring nodes are included in the weighted summation calculation to eliminate the interference of outdated entity relationships on the updated feature vectors of the target node.

[0013] Preferably, after generating the marketing copy, a copy entity backtracking and verification mechanism is introduced; Extract the target entity keywords contained in the generated marketing copy and the generation association relationships between the target entity keywords; The target entity lexical units and the generated association relationship are mapped back to the dynamic knowledge graph for subgraph matching and verification. If the generated association does not have a corresponding connected path in the dynamic knowledge graph, then the illusion fragment in the marketing copy is located, and the bias scalar of the query vector corresponding to the illusion fragment in the self-attention layer is reverse-amplified and corrected. Based on the corrected attention bias matrix, the illusion fragment is re-executed with autoregressive decoding until the entity association in the generated marketing copy is completely consistent with the topology of the dynamic knowledge graph.

[0014] Preferably, when extracting the text semantic features of text data and the visual region features of image data in each data segment, if the image data in the current time slice is detected to be missing, a cross-modal feature compensation mechanism is triggered. Based on the text data existing in the current time slice, extract text entity description information, and retrieve the historical image feature vector with the highest semantic similarity to the text entity description information in the historical time series sub-image library; The historical image feature vector is used as a visual region feature proxy for the current time slice, and cross-modal feature alignment is performed with the text semantic features to maintain the structural integrity of the temporal subgraph in the modality-deficient state.

[0015] A real-time data-driven marketing copy generation system includes: an apparatus for receiving multimodal real-time data streams based on a big data processing platform and performing joint entity relationship extraction on the multimodal real-time data streams by time slices to generate time-series subgraphs; An apparatus for performing edge weight time decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph reflecting the entity association state at the current moment. A device for performing graph convolution aggregation on the dynamic knowledge graph to obtain a vectorized representation of the entity topology; An apparatus for vectorizing the entity topology as an attention bias matrix and superimposing it onto the self-attention layer of the Transformer architecture of the generative model, constraining the generation probability distribution of entity lexical units during the decoding process to generate marketing copy.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By extracting entity relationships from real-time multimodal data streams by time slices to generate temporal subgraphs, and applying edge weight temporal decay and conflict resolution to overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph, the entity topology obtained by graph convolution aggregation is vectorized and superimposed as an attention bias matrix onto the self-attention layer of the generative model transformer architecture. This solves the technical problems of semantic logic conflicts and entity illusions caused by direct input of real-time data streams. The dynamic knowledge graph reduces the impact of outdated entity relationships on the current graph topology through the edge weight temporal decay mechanism, and eliminates contradictory entity associations in adjacent temporal subgraphs through the conflict resolution mechanism, enabling the dynamic knowledge graph to accurately reflect the transient association state between entities at the current moment. The entity topology, which reflects the transient association state, is vectorized and superimposed on the self-attention layer as an attention bias matrix. This introduces explicit structural constraints when calculating the attention score of the dot product between the query vector and the key vector, thereby changing the probability distribution weight of entity words on the decoding vocabulary. This avoids the generative model from establishing incorrect long-distance dependencies based on local statistical probabilities, suppresses the generation of false entities that contradict the current real-time context, and overcomes the semantic drift phenomenon under high-concurrency data streams.

[0017] 2. To address implicit conflict edges, cosine similarity calculations using contextual vectors are introduced to identify semantically contradictory edges and sever connections. This eliminates interference from entity associations with indirect but opposite semantic directions in terms of literal attributes on the graph topology, maintaining the consistency of the internal logic of the dynamic knowledge graph. When aggregating neighbor node features, time intervals are calculated based on the latest update timestamp, and feature retention coefficients are assigned. Neighbor node features below the activity threshold are filtered out, eliminating interference from outdated entity relationships on the target node's updated feature vectors and reducing redundant computational resource consumption. After generating marketing copy, target entity words and generated associations are mapped back to the dynamic knowledge graph for subgraph matching and verification. Illusory fragments are located, and the bias scalar is amplified and corrected in reverse, forming a closed-loop error correction mechanism that ensures that the entity associations in the final output text are completely consistent with the topology of the dynamic knowledge graph. When image data is missing, historical image feature vectors retrieved based on text entity description information are used as proxies for cross-modal feature alignment, maintaining the structural integrity of the temporal subgraph in modal missing states and avoiding breaks in the feature extraction chain. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the entire process from multimodal data to marketing copy. Figure 2 Flowchart for multimodal feature extraction, cross-modal alignment, and temporal subgraph construction; Figure 3 A flowchart for generating dynamic graphs through temporal subgraph fusion, edge weight decay, and explicit conflict resolution; Figure 4 This is a flowchart of implicit conflict edge semantic discrimination and dynamic graph local topology reconstruction processing. Figure 5 Flowchart for generating entity topology vectors using graph attention convolution aggregation and temporal adaptive filtering; Figure 6 A flowchart illustrating the closed-loop processing of attention bias construction, copy generation, and backtracking error correction. Detailed Implementation

[0019] The technical solution of the present invention will be described in detail through the following embodiments. Those skilled in the art can implement the technical solution of the present invention based on the content disclosed in this specification.

[0020] Appendix Figure 1In one embodiment, the sources of the multimodal real-time data stream include user behavior logs from e-commerce platforms, product review data, product main images and detail images, topic content from social media platforms, and bullet screen data and real-time screenshots from live streaming platforms. The big data processing platform adopts a stream processing architecture to receive the aforementioned multimodal data with low latency. The data formats include structured JSON format behavioral data, semi-structured XML format product information, unstructured text data, and image data. The multimodal real-time data stream is processed in time slices. A fixed time window is used for each time slice, with the window size dynamically adjusted based on the data concurrency. A 30-second window is used when the data concurrency is below 1000 data entries per second, and a 10-second window is used when it is above 1000 data entries per second. At the end of each time window, all the multimodal data within that window is integrated into a single data segment.

[0021] For each data segment, entity relationships are jointly extracted to generate a time-series subgraph. The entity relationship joint extraction employs an end-to-end joint extraction model. The input consists of text and image data from the data segment, and the output is a set of entity nodes and a set of entity relationship edges. Entity nodes include types such as product entities, user entities, brand entities, attribute entities, and behavior entities. Entity relationship edges include types such as user-purchase-product, product-belong-to-brand, product-has-attribute, and user-review-product. Each time-series subgraph corresponds to a time slice, containing all identified entity nodes and relationship edges within that time slice. Each node and edge carries a generation timestamp with millisecond precision.

[0022] Adjacent temporal subgraphs are processed to generate a dynamic knowledge graph. The evolution cycle of the dynamic knowledge graph is consistent with the window size of the time slice. At the end of each evolution cycle, the temporal subgraph generated in the current time slice is superimposed with the dynamic knowledge graph generated in the previous evolution cycle. During the superposition process, overlapping entity nodes that exist simultaneously in the current temporal subgraph and the dynamic knowledge graph of the previous cycle are identified. For historical edges associated with overlapping entity nodes, a time decay operation is performed on the edge weights. The degree of decay is positively correlated with the time difference between the generation timestamp of the historical edge and the current timestamp. For newly added entity nodes and relational edges in the current temporal subgraph, they are directly added to the dynamic knowledge graph, with the initial edge weight set to 1.0. When overlapping entity nodes have conflicting edges with opposite attribute values ​​in adjacent temporal subgraphs, the conflicting edge with the generation timestamp closest to the current timestamp is retained, and the remaining conflicting edges are deleted. The decayed historical edges, retained conflicting edges, and newly added edges are merged to generate a dynamic knowledge graph reflecting the entity association state at the current moment.

[0023] Graph convolutional aggregation is performed on the dynamic knowledge graph to obtain a vectorized representation of the entity topology. The graph convolutional aggregation uses a multi-layer graph convolutional network with two layers. The first layer aggregates the features of the first-order neighbors of the target node, and the second layer aggregates the features of the second-order neighbors. In each layer, for each target node, the initial feature vectors of all neighboring nodes connected by edges are first obtained. Then, the message passing weight of each neighbor node to the target node is calculated by combining the edge type encoding between the target node and its neighbors. The initial feature vectors of the neighboring nodes are weighted and summed according to the message passing weights to obtain the aggregated feature vector of the neighboring nodes. The aggregated feature vector is residually concatenated with the original feature vector of the target node, and then mapped using a non-linear activation function to generate the updated feature vector of the target node. All nodes in the dynamic knowledge graph are traversed to obtain the updated feature vectors of all entity nodes. These updated feature vectors constitute the vectorized representation of the entity topology.

[0024] The entity topology structure is vectorized and used as an attention bias matrix, which is then superimposed onto the self-attention layer of the Transformer generative model to generate marketing copy. The generative model employs a pre-trained Transformer decoder architecture, with pre-training datasets including large-scale e-commerce marketing copy, product description text, and user review text. First, the topological association between entity node pairs in the entity topology structure vectorization is mapped to a bias scalar, constructing an attention bias matrix that matches the dimension of the self-attention matrix. When calculating the dot product attention score between the query vector and the key vector in the Transformer self-attention layer, the corresponding bias scalar from the attention bias matrix is ​​superimposed onto the dot product attention score. Based on the superimposed attention score, a softmax normalization operation is performed, followed by a weighted sum with the value vector to obtain the output of the self-attention layer. Through processing by multiple Transformer decoders, the probability distribution weights of terms associated with entities in the dynamic knowledge graph on the decoding vocabulary are changed, generating terms one by one in an autoregressive manner, ultimately outputting the marketing copy.

[0025] This embodiment fully implements the technical solution of transforming multimodal real-time data streams into dynamic knowledge graphs and introducing the topological constraints of dynamic knowledge graphs into the decoding process of generative models. Through explicit structural constraints, it avoids the generative model from establishing incorrect long-distance dependencies and suppresses the generation of false entities and incorrect relationships.

[0026] Appendix Figure 2In one embodiment, the process of jointly extracting entity relationships from a multimodal real-time data stream by time slice to generate a temporal subgraph includes the following specific steps: After dividing the multimodal real-time data stream into multiple data segments according to a fixed time window, the textual semantic features of the text data and the visual region features of the image data in each data segment are extracted. For the text data, a pre-trained language model is used to extract textual semantic features. The language model adopts a 12-layer Transformer encoder architecture with a hidden layer dimension of 768. The text data is segmented into words, converted into a word sequence, and then input into the language model after adding special tags. The language model outputs a 768-dimensional feature vector corresponding to each word. For sentence-level textual semantic features, the average pooling method is used to obtain the features; for entity-level textual semantic features, the feature vector of the corresponding entity word position is extracted as the textual semantic feature of that entity.

[0027] For image data, a pre-trained convolutional neural network (CNN) is used to extract visual region features. The CNN adopts the ResNet50 architecture, and the input image size is adjusted to 224×224 pixels. After the image data is input into the CNN, the feature map output by the last convolutional layer is extracted, and the feature map size is 7×7×2048. An object detection algorithm is used to process the feature map to identify object regions in the image. The object detection algorithm adopts the Faster R-CNN architecture, and outputs the bounding box coordinates and class probability of each detected object region. For each detected object region, the ROIAlign operation is used to extract the corresponding region's feature vector from the feature map. The feature vector output by ROIAlign has a dimension of 2048, and this feature vector is the visual region feature of that visual region.

[0028] Cross-modal feature alignment is performed on the extracted text semantic features and visual region features to identify shared entity nodes in the data fragments. Cross-modal feature alignment is achieved using a cross-modal encoder, which consists of two linear projection layers and a similarity calculation layer. Text semantic features and visual region features are input to their respective linear projection layers, mapping them to the same 512-dimensional common feature space. The weights of the linear projection layers are obtained through contrastive learning pre-training. The pre-training task is to match text descriptions with corresponding images, making matched text-image pairs closer in the common feature space and mismatched text-image pairs farther apart. In the common feature space, the cosine similarity between each text entity semantic feature and each visual region feature is calculated. When the cosine similarity is higher than a preset alignment threshold, the text entity and the visual region are determined to correspond to the same shared entity node. The preset alignment threshold is set to 0.7.

[0029] Based on the co-occurrence frequency and dependency syntax relations of shared entity nodes in multimodal real-time data streams, edge connections are constructed between shared entity nodes to form a temporal subgraph corresponding to a time slice. For co-occurrence frequency, the number of times two shared entity nodes appear simultaneously in the same text sentence or image within the current time slice is counted, and this co-occurrence frequency is used as part of the initial edge weight. For dependency syntax relations, a dependency parser is used to process the text data and extract dependency syntax relations, including subject-verb, verb-object, attributive-head, and adverbial-head relations. When a direct dependency syntax relation exists between two shared entity nodes, a corresponding relation edge is established, with the edge type determined by the type of dependency syntax relation. For example, when there is a verb-object relation of "user buys goods," a user-buy-goods relation edge is established; when there is an attributive-head relation of "red dress," a dress-have-red relation edge is established. All edges constructed based on co-occurrence frequency and dependency syntax relations are combined with shared entity nodes to form the temporal subgraph corresponding to that time slice.

[0030] This embodiment details the specific implementation methods of multimodal feature extraction, cross-modal feature alignment, and temporal subgraph construction. Through cross-modal feature alignment, a unified representation of entities in text and image data is achieved. The edge connections constructed based on co-occurrence frequency and dependency syntax relations accurately reflect the association relationships between entities.

[0031] Appendix Figure 3 In one embodiment, the process of generating a dynamic knowledge graph reflecting the current entity association state by performing edge weight time decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs includes the following specific steps. During the overlay process of adjacent temporal subgraphs, overlapping entity nodes that exist simultaneously in preceding and following time slices are first identified. The unique identifier of an entity node consists of a hash value of the entity type and entity name. By comparing the unique identifiers of entity nodes in the two temporal subgraphs, overlapping entity nodes are identified. For entity nodes that exist only in the preceding time slice, they are retained in the dynamic knowledge graph, but edge weight time decay is performed on all edges associated with them; for entity nodes that exist only in the following time slice, they are added to the dynamic knowledge graph as new nodes.

[0032] For historical edges associated with overlapping entity nodes, an edge weight time decay operation is performed based on the time difference between the generation timestamp of the historical edge and the current timestamp. The edge weight time decay uses an exponential decay function, calculated as follows:

[0033] in, The edge weights after decay. Let be the initial edge weight. The attenuation coefficient is... This represents the time difference between the generation timestamp of the historical edge and the current timestamp, in seconds. Decay coefficient. The value is determined based on the timeliness of the entity relationship. For entity relationships with high timeliness, such as commodity-price-XX yuan, The value is 0.01; for entity relationships with weak time sensitivity, such as product-belongs-brand, The value is 0.001. When the weakened edge weight is lower than the edge weight threshold of 0.1, the edge is deleted from the dynamic knowledge graph.

[0034] When overlapping entity nodes have conflicting edges with opposite attribute values ​​in adjacent time-series subgraphs, retain the conflicting edge whose generation timestamp is closest to the current timestamp and delete the remaining conflicting edges. Conflicting edges with opposite attribute values ​​refer to edges connecting the same entity node and the same attribute entity node, but with completely opposite attribute values. For example, two edges connecting product A and the price attribute entity node, one with an attribute value of 100 yuan and the other with an attribute value of 200 yuan, are explicit conflicting edges. To facilitate the handling of different types of explicit conflicting edges, the explicit conflicting edge handling rules are defined as shown in Table 1 below: Table 1. Comparison of Explicit Conflict Edge Types and Handling Methods

[0035] The historical edges retained after decay, the conflict edges retained after conflict resolution, and the newly added edges in the current temporal subgraph are merged to generate the dynamic knowledge graph at the current moment. Each node and edge in the dynamic knowledge graph has a latest update timestamp. The latest update timestamp of a node is the timestamp when the node was last added or modified, and the latest update timestamp of an edge is the timestamp when the edge was last added or modified.

[0036] This embodiment details the specific implementation of edge weight time decay and explicit conflict resolution. It reduces the impact of outdated entity relationships through an exponential decay function and eliminates contradictory entity associations through an explicit conflict resolution mechanism, ensuring that the dynamic knowledge graph can accurately reflect the entity association status at the current moment.

[0037] Appendix Figure 4 In one embodiment, during conflict resolution, when overlapping entity nodes have implicit conflict edges in adjacent temporal subgraphs with opposite semantic directions but not directly opposing literal attributes, the following processing steps are performed. First, the context vectors of the neighboring nodes connected by the implicit conflict edge are extracted. The context vectors are obtained by extracting semantic features from the text sentence containing the implicit conflict edge. The same pre-trained language model as the text semantic feature extraction is used. The input is the complete text sentence containing the implicit conflict edge, and the output is the average pooled feature vector of the sentence. This feature vector is the context vector, with a dimension of 768.

[0038] Then, the cosine similarity between the context vector and the current baseline semantic vector of the overlapping entity node is calculated. The current baseline semantic vector of the overlapping entity node is the feature vector of that node in the current dynamic knowledge graph, obtained by graph convolution aggregation from the previous evolution cycle. The formula for calculating the cosine similarity is:

[0039] in, For context vectors, This is the current baseline semantic vector for overlapping entity nodes. and These are the L2 norms of the two vectors. The preset judgment threshold is set to 0.3. When the cosine similarity is lower than the preset judgment threshold, the implicit conflict edge is judged as a semantic contradiction edge.

[0040] The weight of the semantically contradictory edge is set to zero, and the connection between the semantically contradictory edge and the overlapping entity node is severed. Simultaneously, the local topology adjacent to the overlapping entity node is reinitialized based on the current baseline semantic vector of the overlapping entity node. The reinitialization process includes: deleting all edges directly or indirectly associated with the semantically contradictory edge that have a weight lower than 0.5; retrieving entity nodes with a semantic similarity higher than 0.6 in the current temporal subgraph based on the current baseline semantic vector of the overlapping entity node, and establishing new candidate edges; and initializing the weights of the candidate edges, setting the initial weight to 0.5.

[0041] This embodiment provides a method for identifying and processing implicit conflict edges. By calculating the cosine similarity between the context vector and the baseline semantic vector, it can identify entity associations whose literal attributes are not directly opposed but whose semantic directions are opposite. This eliminates the interference of semantically contradictory edges on the topological structure of the dynamic knowledge graph and maintains the consistency of the internal logic of the graph.

[0042] Appendix Figure 5 In one embodiment, the process of performing graph convolution aggregation on a dynamic knowledge graph to obtain a vectorized representation of the entity topology includes the following specific steps: In the dynamic knowledge graph, for each target node, the initial feature vectors of all neighboring nodes that have edge connections with the target node are first obtained. The initial feature vectors are the original feature vectors of the entity nodes, obtained from the entity relation joint extraction stage, with a dimension of 768. Then, the message passing weights of the neighboring nodes to the target node are calculated by combining the edge type encoding between the target node and its neighboring nodes.

[0043] Edge type encoding employs a combination of one-hot encoding and linear projection. First, all possible edge types are one-hot encoded, with the dimension of the one-hot encoding equal to the total number of edge types. Then, the one-hot encoding is input to the linear projection layer, mapping to a 128-dimensional edge type encoding vector. Examples of common edge types and their corresponding encoding vectors are shown in Table 2 below. Table 2 Example Table of Edge Type Encoding

[0044] The message passing weight is calculated using a graph attention mechanism, and the formula is as follows:

[0045] in, Let J be the message passing weight of neighbor node J to target node i. Let be the initial feature vector of the target node i. Let be the initial feature vector of neighbor node j. Let W be the edge type encoding vector for edge (i,j), and let W be a trainable weight matrix with dimensions 768×512. The attention vector has dimensions of (512+512+128)×1=1152×1. This represents a vector concatenation operation, with the negative slope of LeakyReLU set to 0.2. The softmax operation is performed on all neighboring nodes of the target node i, making the sum of the message passing weights of all neighboring nodes equal to 1.

[0046] The initial feature vectors of neighboring nodes are weighted and summed according to the message passing weights to obtain the aggregated feature vector of neighboring nodes. The calculation formula is as follows:

[0047] in, This is the aggregated feature vector of the neighboring nodes. Let i be the set of all neighboring nodes of the target node i.

[0048] The aggregated feature vector is residually concatenated with the original feature vector of the target node, and then mapped using a non-linear activation function to generate the updated feature vector of the target node. The calculation formula is as follows:

[0049] in, Let be the updated feature vector of target node i, and ReLU be the linear rectified activation function.

[0050] By traversing all nodes in the dynamic knowledge graph, updated feature vectors for all entity nodes are obtained. These updated feature vectors constitute a vectorized representation of the entity topology. The dimension of the vectorized representation of the entity topology is consistent with the dimension of the updated feature vector of each entity node, which is 768.

[0051] This embodiment details the graph convolution aggregation process based on the graph attention mechanism. By introducing edge type encoding, it is possible to distinguish the influence of different types of entity relationships on the features of the target node. The original feature information of the target node is preserved through residual connections. The generated entity topology vectorized representation accurately reflects the topological position and association of the entity in the dynamic knowledge graph.

[0052] In one embodiment, an adaptive feature selection mechanism based on temporal freshness is introduced when aggregating the initial feature vectors of neighboring nodes. First, the latest update timestamp of the edge connecting each neighboring node to the dynamic knowledge graph is obtained, and the time interval between this latest update timestamp and the current graph evolution cycle is calculated. The timestamp of the current graph evolution cycle is the end timestamp of the current temporal subgraph.

[0053] Each neighbor node is assigned a feature retention coefficient based on a time interval. The feature retention coefficient is inversely proportional to the time interval, and the calculation formula is as follows:

[0054] in, Here, k is the feature retention coefficient for neighbor node j, and k is the scaling factor with a value of 0.005. The time interval between the latest update timestamp of the edge connection corresponding to neighbor node j and the current graph evolution cycle is expressed in seconds.

[0055] The activity threshold is set to 0.2. Initial feature vectors of neighboring nodes with feature retention coefficients lower than the activity threshold are filtered out, and only the initial feature vectors of the retained neighboring nodes are included in the weighted summation calculation. For example, when the latest update timestamp of the edge connected to a neighboring node is 300 seconds from the current time, the feature retention coefficient is 1 / (1+0.005×300)=1 / (1+1.5)=0.4, which is higher than the activity threshold of 0.2, and the feature of this neighboring node is retained. When the time interval is 1000 seconds, the feature retention coefficient is 1 / (1+0.005×1000)=1 / (1+5)=0.167, which is lower than the activity threshold of 0.2, and the feature of this neighboring node is filtered out.

[0056] This embodiment introduces an adaptive feature selection mechanism based on temporal freshness. It quantifies the timeliness of neighbor node features through feature retention coefficients, filters out outdated neighbor node features, eliminates the interference of outdated entity relationships on the target node's updated feature vector, and reduces the computational load in the graph convolution aggregation process.

[0057] Appendix Figure 6 In one embodiment, the vectorized representation of entity topology is superimposed as an attention bias matrix onto the self-attention layer of the Transformer architecture of the generative model. The process of constraining the generation probability distribution of entity lexical units during decoding to generate marketing copy includes the following specific steps. First, the topological correlation between entity node pairs in the vectorized representation of entity topology is mapped as a bias scalar, and an attention bias matrix matching the dimension of the self-attention matrix is ​​constructed.

[0058] The topological affinity between entity node pairs is represented by the reciprocal of the shortest path length between the nodes, calculated using the following formula:

[0059] in, Let c be the bias scalar between entity node i and entity node j, and c be a constant coefficient with a value of 1.0. Let be the shortest path length between entity node i and entity node j in the dynamic knowledge graph. This is defined when there is no connecting path between the two entity nodes. The value is infinity, corresponding to the bias scalar. The value is 0. The shortest path length is calculated using Dijkstra's algorithm, with edge weights serving as the weights for the path length.

[0060] The input to the self-attention layer of the Transformer architecture is the word feature vector of the decoded sequence. The length of the decoded sequence is n, and the dimension of the self-attention matrix is ​​n×n. The dimension of the attention bias matrix is ​​also set to n×n. For each word in the decoded sequence, it is first determined whether the word corresponds to an entity node in the dynamic knowledge graph. If it corresponds, the row and column of the word in the attention bias matrix are associated with the corresponding entity node; if they do not correspond, the bias scalars of the corresponding row and column of the word are both set to 0. For two words that both correspond to entity nodes, the bias scalars at their corresponding positions in the attention bias matrix are calculated as described above. .

[0061] When calculating the dot product attention score of the query vector and key vector in the self-attention layer of the Transformer generative model architecture, the corresponding bias scalars in the attention bias matrix are superimposed on the dot product attention score. The formula for calculating the self-attention score is:

[0062] Where Q is the query vector matrix with dimensions n×d_k, K is the key vector matrix with dimensions n×d_k, V is the value vector matrix with dimensions n×d_v, d_k is the dimension of the query vector and the key vector, d_v is the dimension of the value vector, and B is the attention bias matrix with dimensions n×n. This is a scaling factor used to prevent the softmax function from entering the gradient saturation region due to excessively large dot product results.

[0063] The attention scores, after being superimposed with bias scalars, are subjected to softmax normalization to obtain the attention weight matrix. The attention weight matrix and the value vector matrix are then weighted and summed to obtain the output of the self-attention layer. Through processing by a multi-layer Transformer decoder, the probability distribution weights of terms associated with entities in the dynamic knowledge graph on the decoding vocabulary are altered. For terms strongly associated with entities in the dynamic knowledge graph, their generation probability is increased; for terms with no or weak association with entities in the dynamic knowledge graph, their generation probability is decreased. Terms are generated one by one in an autoregressive manner until a generation end marker is reached, ultimately outputting the marketing copy.

[0064] This embodiment details the construction and superposition process of the attention bias matrix. By transforming the topological correlation of the dynamic knowledge graph into a bias scalar of the self-attention layer, explicit structural constraints are introduced during the decoding process, adjusting the generation probability distribution of entity words, and avoiding the generation model from generating false entities and incorrect relationships based on local statistical probabilities.

[0065] In one embodiment, after generating marketing copy, a copy entity backtracking verification mechanism is introduced. First, the target entity words and their generational relationships within the generated marketing copy are extracted. The extraction of target entity words employs a named entity recognition model, which combines a pre-trained BERT model with a Conditional Random Field (CRF) layer. This model can identify entity words of types such as products, brands, users, attributes, and behaviors. The generational relationships between target entity words are extracted through dependency parsing and semantic role labeling, identifying subject-verb, verb-object, modifier-head, and adverbial relationships between entities. These relationships are then mapped to generational relationships consistent with the edge types in the dynamic knowledge graph.

[0066] The extracted target entity lexical units and generated associations are mapped back to the dynamic knowledge graph for subgraph matching verification. Subgraph matching employs the VF2 subgraph isomorphism algorithm, which matches the generated subgraph (composed of target entity lexical units and generated associations) with the dynamic knowledge graph, checking whether each generated association in the generated subgraph has a corresponding connected path in the dynamic knowledge graph. A connected path refers to a path in the dynamic knowledge graph where two entity nodes are connected by an edge, and the edge type on the path matches the type of the generated association.

[0067] If a generated association does not have a corresponding connected path in the dynamic knowledge graph, the marketing copy fragment corresponding to that association is identified as an illusory fragment. Illusory fragments include false entity fragments and incorrect relationship fragments. False entity fragments refer to generated entity words that do not exist in the dynamic knowledge graph, while incorrect relationship fragments refer to generated entity words that exist in the dynamic knowledge graph, but the association between the entities does not exist. Different correction strategies are adopted for different types of illusory fragments, as shown in Table 3 below.

[0068] Table 3. Comparison of Hallucination Fragment Types and Correction Strategies

[0069] After inversely amplifying and correcting the bias scalar of the query vector corresponding to the hallucination fragment in the self-attention layer, the hallucination fragment is re-decoded based on the corrected attention bias matrix. During the decoding process, only the content of the hallucination fragment and the two words before and after it are regenerated; the rest of the generated content remains unchanged. The subgraph matching verification and correction decoding process is repeated until all entity relationships in the generated marketing copy are completely consistent with the topology of the dynamic knowledge graph.

[0070] This embodiment introduces a copywriting entity backtracking verification mechanism. Through subgraph matching verification, it can accurately locate the illusion fragments in the generated copywriting. By reverse amplifying and correcting the attention bias scalar, a closed-loop error correction mechanism is formed, ensuring the accuracy of entity relationships in the final output marketing copywriting.

[0071] In one embodiment, when extracting textual semantic features from textual data and visual region features from image data in each data segment, if image data is detected to be missing within the current time slice, a cross-modal feature compensation mechanism is triggered. Missing image data includes situations where no image data is received within the current time slice, or where the received image data is corrupted and cannot be used for feature extraction.

[0072] First, text entity description information is extracted based on the text data existing within the current time slice. This text entity description information is obtained through entity recognition and attribute extraction of the text data, including information such as the entity's name, category, and attributes. For example, from the text data "new red dress, slim-fitting and suitable for summer wear," the extracted text entity description information would be: dress, red, new, slim-fitting, slimming, summer.

[0073] Then, the historical image feature vector with the highest semantic similarity to the text entity description information is retrieved from the historical time-series sub-graph library. The historical time-series sub-graph library stores all time-series sub-graphs generated in the past 30 days and their corresponding image feature vectors. The retrieval process uses a combination of inverted index and semantic similarity calculation. First, based on the entity name and category in the text entity description information, the historical time-series sub-graphs containing the same or similar entities are quickly filtered out using the inverted index. Then, the cosine similarity between the semantic feature vector of the text entity description information and the text description semantic feature vector of the image feature vector corresponding to the filtered historical time-series sub-graphs is calculated. The formula for calculating the cosine similarity is the same as formula (2), where A semantic feature vector that describes the information of a text entity. This is the semantic feature vector of the text description corresponding to the feature vector of the historical image.

[0074] The historical image feature vector with the highest cosine similarity is selected as the visual region feature proxy for the current time slice. This visual region feature proxy is then aligned with the textual semantic features of the current time slice across modalities. The alignment process is consistent with that described in the second embodiment. Through this alignment, shared entity nodes in the current time slice are identified, and a temporal subgraph is constructed based on co-occurrence frequency and dependency syntax relations, maintaining the structural integrity of the temporal subgraph even in modality-deficient states.

[0075] This embodiment provides a cross-modal feature compensation mechanism. When image data is missing, cross-modal feature alignment is performed by retrieving historical image feature vectors as proxies, which avoids the break in the feature extraction link and ensures the continuity and structural integrity of the time-series subgraph generation process.

[0076] In one embodiment, a real-time data-driven marketing copy generation system includes the following apparatus: an apparatus for receiving multimodal real-time data streams based on a big data processing platform, and performing joint entity relation extraction on the multimodal real-time data streams by time slices to generate a time-series subgraph. This apparatus includes a multimodal data receiving module, a time-slicing module, a text feature extraction module, an image feature extraction module, a cross-modal feature alignment module, and a time-series subgraph construction module. The multimodal data receiving module uses the Apache Flink stream processing framework to achieve low-latency reception of multi-source heterogeneous real-time data streams. The time-slicing module divides the multimodal real-time data stream into multiple data segments according to a fixed time window. The text feature extraction module and the image feature extraction module extract text semantic features and visual region features from the data segments, respectively. The cross-modal feature alignment module achieves cross-modal alignment of text and visual features, identifying shared entity nodes. The time-series subgraph construction module constructs a time-series subgraph based on co-occurrence frequency and dependency syntax relations.

[0077] This device is used to perform edge weight temporal decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph reflecting the current entity association state. The device includes an overlapping node identification module, an edge weight decay module, an explicit conflict resolution module, an implicit conflict resolution module, and a dynamic graph update module. The overlapping node identification module identifies overlapping entity nodes in adjacent temporal subgraphs. The edge weight decay module decays the edge weights of historical edges according to an exponential decay function. The explicit conflict resolution module handles explicit conflict edges with opposite attribute values. The implicit conflict resolution module handles implicit conflict edges with opposite semantic directions. The dynamic graph update module merges the decayed edges, retained conflict edges, and newly added edges to generate the dynamic knowledge graph for the current time step.

[0078] A device for performing graph convolution aggregation on dynamic knowledge graphs to obtain a vectorized representation of entity topology. This device includes a neighbor node acquisition module, an edge type encoding module, a message passing weight calculation module, a feature aggregation module, and a feature update module. The neighbor node acquisition module acquires the initial feature vectors of all neighbor nodes of the target node. The edge type encoding module encodes the entity relationship edges. The message passing weight calculation module calculates the message passing weights of neighbor nodes based on a graph attention mechanism. The feature aggregation module performs a weighted summation of the neighbor node features according to the message passing weights. The feature update module generates the updated feature vector of the target node through residual connections and nonlinear mappings.

[0079] This device is used to vectorize the entity topology as an attention bias matrix and superimpose it onto the self-attention layer of the Transformer generative model architecture to constrain the generation probability distribution of entity lexical units during the decoding process to generate marketing copy. The device includes an attention bias construction module, a Transformer decoding module, an entity backtracking verification module, and a copy output module. The attention bias construction module maps entity topology relevance to a bias scalar to construct the attention bias matrix. The Transformer decoding module superimposes the attention bias matrix onto the self-attention layer to generate marketing copy in an autoregressive manner. The entity backtracking verification module verifies the entity relevance of the generated copy, locating and correcting any illusory fragments. The copy output module outputs the final marketing copy.

[0080] This embodiment details the components and functions of the real-time data-driven marketing copy generation system. The components work together to realize a complete process from receiving multimodal real-time data streams, generating dynamic knowledge graphs, vectorizing entity topology, to generating and validating marketing copy.

Claims

1. A real-time data-driven marketing copy generation method, characterized in that, include: Based on the big data processing platform, multimodal real-time data streams are received, and entity relationships are jointly extracted from the multimodal real-time data streams according to time slices to generate time-series subgraphs. Edge weight time decay and conflict resolution are performed on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph that reflects the entity association state at the current moment. The dynamic knowledge graph is subjected to graph convolution aggregation to obtain a vectorized representation of the entity topology; The entity topology is vectorized and represented as an attention bias matrix, which is then superimposed on the self-attention layer of the Transformer architecture of the generative model to constrain the generation probability distribution of entity tokens during the decoding process in order to generate marketing copy.

2. The real-time data-driven marketing copy generation method according to claim 1, characterized in that, The step of performing joint entity relation extraction on the multimodal real-time data stream by time slice to generate a time-series subgraph includes: dividing the multimodal real-time data stream into multiple data segments according to a fixed time window, and extracting the text semantic features of text data and the visual region features of image data in each data segment; Cross-modal feature alignment is performed between the text semantic features and the visual region features to identify shared entity nodes in the data fragment; Based on the co-occurrence frequency and dependency syntax relationship of the shared entity nodes in the multimodal real-time data stream, edge connections are constructed between the shared entity nodes to form the time-series subgraph corresponding to the time slice.

3. The real-time data-driven marketing copy generation method according to claim 1, characterized in that, The step of performing edge weight time decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph reflecting the current entity association state includes: identifying overlapping entity nodes that exist simultaneously in the preceding and following time slices during the superposition of adjacent temporal subgraphs. For the historical edges associated with the overlapping entity nodes, perform edge weight time decay operation according to the time difference between the generation timestamp of the historical edge and the current timestamp; When the overlapping entity node has conflicting edges with opposite attribute values ​​in the adjacent temporal subgraph, the conflicting edge whose generation timestamp is closest to the current timestamp is retained and the remaining conflicting edges are deleted. The edges retained after decay are merged with the newly added edges to generate the dynamic knowledge graph.

4. The real-time data-driven marketing copy generation method according to claim 1, characterized in that, The process of performing graph convolution aggregation on the dynamic knowledge graph to obtain a vectorized representation of the entity topology includes: in the dynamic knowledge graph, for a target node, aggregating the initial feature vectors of neighboring nodes that have edge connections with the target node, and combining the edge type encoding between the target node and the neighboring nodes to calculate the message passing weight of the neighboring nodes to the target node; The initial feature vectors of the neighboring nodes are weighted and summed according to the message passing weights. The weighted summation result is then residually connected and nonlinearly mapped to the original feature vector of the target node to generate the updated feature vector of the target node. Traverse all nodes in the dynamic knowledge graph to obtain a vectorized representation of the entity topology.

5. The real-time data-driven marketing copy generation method according to claim 1, characterized in that, The entity topology structure is vectorized and superimposed as an attention bias matrix onto the self-attention layer of the Transformer architecture of the generative model. The probability distribution of entity word generation during the decoding process is constrained to generate marketing copy. This includes mapping the topological correlation between entity node pairs in the entity topology structure vectorized representation to a bias scalar and constructing the attention bias matrix that matches the dimension of the self-attention matrix. When calculating the dot product attention score of the query vector and the key vector in the self-attention layer of the generative model Transformer architecture, the bias scalar in the attention bias matrix is ​​superimposed on the corresponding dot product attention score. Based on the attention score after superimposing the bias scalar, a normalization operation and weighted summation are performed to change the probability distribution weight of the lexical units associated with entities in the dynamic knowledge graph on the decoding vocabulary, and the marketing copy is output.

6. The real-time data-driven marketing copy generation method according to claim 3, characterized in that, During the conflict resolution process, when there are implicit conflict edges in the adjacent temporal subgraphs where the overlapping entity nodes have opposite semantic directions but not directly opposing literal attributes, the context vector of the neighboring nodes connected by the implicit conflict edge is extracted. Calculate the cosine similarity between the context vector and the current base semantic vector of the overlapping entity node, and determine the implicit conflict edge with the cosine similarity lower than a preset judgment threshold as a semantic contradiction edge; Set the edge weight of the semantically contradictory edge to zero and sever the connection between the semantically contradictory edge and the overlapping entity node. At the same time, reinitialize the local topology structure adjacent to the overlapping entity node based on the current reference semantic vector of the overlapping entity node.

7. The real-time data-driven marketing copy generation method according to claim 4, characterized in that, When aggregating the initial feature vectors of the neighboring nodes, an adaptive feature filtering mechanism based on temporal freshness is introduced; Obtain the latest update timestamp of the edge connected to the neighbor node in the dynamic knowledge graph, and calculate the time interval between the latest update timestamp and the current graph evolution cycle; A feature retention coefficient is assigned to the neighbor node according to the time interval, and the feature retention coefficient is inversely proportional to the time interval. The initial feature vectors of neighboring nodes whose feature retention coefficient is lower than the activity threshold are filtered out, and only the retained initial feature vectors of neighboring nodes are included in the weighted summation calculation to eliminate the interference of outdated entity relationships on the updated feature vectors of the target node.

8. The real-time data-driven marketing copy generation method according to claim 1, characterized in that, After generating the marketing copy, a copy entity backtracking and verification mechanism is introduced; Extract the target entity keywords contained in the generated marketing copy and the generation association relationships between the target entity keywords; The target entity lexical units and the generated association relationship are mapped back to the dynamic knowledge graph for subgraph matching and verification. If the generated association does not have a corresponding connected path in the dynamic knowledge graph, then the illusion fragment in the marketing copy is located, and the bias scalar of the query vector corresponding to the illusion fragment in the self-attention layer is reverse-amplified and corrected. Based on the corrected attention bias matrix, the illusion fragment is re-executed with autoregressive decoding until the entity association in the generated marketing copy is completely consistent with the topology of the dynamic knowledge graph.

9. The real-time data-driven marketing copy generation method according to claim 2, characterized in that, When extracting the text semantic features of text data and the visual region features of image data in each data segment, if the image data in the current time slice is detected to be missing, a cross-modal feature compensation mechanism is triggered. Based on the text data existing in the current time slice, extract text entity description information, and retrieve the historical image feature vector with the highest semantic similarity to the text entity description information in the historical time series sub-image library; The historical image feature vector is used as a visual region feature proxy for the current time slice, and cross-modal feature alignment is performed with the text semantic features to maintain the structural integrity of the temporal subgraph in the modality-deficient state.

10. A real-time data-driven marketing copy generation system, characterized in that, include: An apparatus for receiving multimodal real-time data streams based on a big data processing platform, and performing joint entity relation extraction on the multimodal real-time data streams by time slices to generate time-series subgraphs; An apparatus for performing edge weight time decay and conflict resolution on overlapping nodes in adjacent temporal subgraphs to generate a dynamic knowledge graph reflecting the entity association state at the current moment. A device for performing graph convolution aggregation on the dynamic knowledge graph to obtain a vectorized representation of the entity topology; An apparatus for vectorizing the entity topology as an attention bias matrix and superimposing it onto the self-attention layer of the Transformer architecture of the generative model, constraining the generation probability distribution of entity lexical units during the decoding process to generate marketing copy.

Images