Content generation method based on memory trace backtracking

Abstract

The invention discloses a memory trace backtracking-based content generation method, and particularly relates to the technical field of big data, and the method comprises the following steps: S01, obtaining to-be-analyzed multi-source heterogeneous original data; s02, carrying out data preprocessing on the multi-source heterogeneous original data; s03, constructing a dynamic evolution memory mark graph based on the multi-dimensional memory mark feature vector; s04, performing backtracking path planning and reasoning on the memory mark graph; and S05, generating target content based on the backtracking path and the multi-dimensional memory mark feature vector. The content generation method based on memory trace backtracking can effectively fuse multi-source heterogeneous data, dynamically model memory trace features and intelligently optimize a backtracking path, and shows high applicability and superiority in various complex application scenes such as accident analysis, fault diagnosis, data archiving and event restoration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, in particular to a content generation method based on trace back. BACKGROUND

[0002] With the rapid development of artificial intelligence and big data technology, content generation methods based on trace back analysis have gradually become a research hotspot. By extracting key information from historical data and conducting reverse deduction, it can effectively assist content generation tasks such as accident analysis, fault diagnosis, data archiving, etc. However, existing trace back methods still have deficiencies in multi-dimensional data correlation mining, dynamic scene adaptability, and trace feature extraction, affecting the efficiency and accuracy of content generation.

[0003] In the prior art, although the accident trace back method and device disclosed in the patent with the publication number CN110996066B can achieve fast accident analysis through video marking, it mainly relies on pre-marked video data and lacks the ability to process unmarked data or unstructured data, making it difficult to meet the dynamic data trace back demand in complex scenarios. In addition, this method relies heavily on video data and fails to fully consider the fusion analysis of multi-source heterogeneous data (such as text, images, sensor data, etc.), limiting its application range in diversified scenarios. On the other hand, the method for archiving and tracing back data disclosed in the patent with the publication number CN116738009B uses data knowledge graph and self-attention network to improve archiving efficiency, but it mainly focuses on the archiving and tracing back of static data and lacks adaptability to real-time data changes in dynamic scenarios. At the same time, the construction of data knowledge graph in this method relies on explicit classification rules and correlation models, and the extraction and modeling ability of fuzzy trace features is weak, making it difficult to meet the demand for deep trace and correlation analysis in complex content generation tasks.

[0004] The above problems show that existing trace back methods still have certain deficiencies in multi-source heterogeneous data fusion, dynamic scene adaptability, and trace feature extraction. Therefore, the present application provides a content generation method based on trace back, aiming to improve the efficiency and accuracy of content generation by fusing multi-source data, dynamically modeling trace features, and optimizing trace back paths, so as to meet the demand of diversified application scenarios. SUMMARY

[0005] The present application aims to provide a content generation method based on trace back to solve the deficiencies of existing trace back methods in multi-dimensional data correlation mining, dynamic scene adaptability, and trace feature extraction as mentioned in the background.

[0006] In a first aspect, the present application provides a content generation method based on trace back, characterized in that it comprises:

[0007] acquiring multi-source heterogeneous raw data to be analyzed, the multi-source heterogeneous raw data including text data, image data, and time series sensor data;

[0008] performing data preprocessing on the multi-source heterogeneous raw data to generate standardized data in a unified format, the data preprocessing including text cleaning on the text data, image denoising on the image data, and timestamp synchronization processing on the time series sensor data;

[0009] performing engram feature extraction and representation on the standardized data to generate a multi-dimensional engram feature vector, the engram feature extraction and representation including single-modal encoding of different modal data by a multi-modal encoder and feature fusion by a cross-modal fusion network;

[0010] constructing a dynamic evolution engram graph based on the multi-dimensional engram feature vector, the engram graph including nodes and edges, the nodes representing entities or events, and the edges representing relationships between entities or events, the dynamic evolution including real-time updating of the structure and node attributes of the engram graph based on new engram features;

[0011] performing retroactive path planning and reasoning on the engram graph to determine an event sequence or a causal chain from a starting point to a target point, the retroactive path planning and reasoning including path search and causal chain reasoning on the event sequence;

[0012] generating target content based on the retroactive path and the multi-dimensional engram feature vector, the content generation including converting the engram feature vector and the causal reasoning result into readable text content and structurally organizing the text content.

[0013] Preferably, the acquiring multi-source heterogeneous raw data to be analyzed includes:

[0014] constructing a dynamic evolution engram graph based on a multi-dimensional engram feature vector acquiring text data such as log files, report documents, or user comments through a text acquisition interface;

[0015] constructing a dynamic evolution engram graph based on a multi-dimensional engram feature vector acquiring image data such as monitoring video frames, photos, or schematic diagrams through an image acquisition interface;

[0016] and acquiring real-time or historical time series data from various sensors through a sensor data interface, the sensors including temperature sensors, pressure sensors, vibration sensors, position sensors, or network flow sensors.

[0017] Preferably, the performing data preprocessing on the multi-source heterogeneous raw data to generate standardized data in a unified format specifically includes:

[0018] text cleaning, word segmentation, part-of-speech tagging, stop word removal, and text length normalization on the text data;

[0019] image denoising, scale normalization, color correction, and region cropping on the image data;

[0020] timestamp synchronization, missing value filling, outlier detection, and data smoothing on the time-series sensor data;

[0021] uniform data format conversion on all pre-processed data to generate standardized data streams for the episodic feature extraction module.

[0022] Preferably, the episodic feature extraction and representation on the standardized data to generate a multi-dimensional episodic feature vector specifically includes:

[0023] inputting the standardized data stream into a multi-modal encoder for feature extraction, the multi-modal encoder including a text encoder, an image encoder, and a time-series encoder;

[0024] the text encoder generates a text feature vector by performing semantic encoding on the text data based on a pre-trained language model with a transformer architecture;

[0025] the image encoder generates an image feature vector by performing visual encoding on the image data based on a deep learning model with a convolutional neural network or visual transformer architecture;

[0026] the time-series encoder generates a time-series feature vector by performing dynamic encoding on the time-series sensor data based on a recurrent neural network with a long short-term memory network, a gated recurrent unit, or a temporal convolutional network architecture.

[0027] Preferably, the episodic feature extraction and representation on the standardized data further includes:

[0028] inputting the text feature vector, the image feature vector, and the time-series feature vector into a cross-modal fusion network to perform feature fusion through a multi-head self-attention mechanism or a cross-attention mechanism to capture implicit associations between different modal data, generating a unified, high-dimensional episodic feature vector;

[0029] the cross-modal fusion network further quantifies the uncertainty of the fused episodic feature vector through a probability embedding layer or a Gaussian mixture model, representing the episodic feature vector as a probability distribution or a fuzzy set to capture the uncertainty information of fuzzy episodic features.

[0030] Preferably, the construction of a dynamic evolutionary episodic map based on the multi-dimensional episodic feature vector specifically includes:

[0031] initializing a memory graph, the memory graph being composed of an initial node set and an initial edge set constructed based on a domain ontology and initial memory features;

[0032] when a new memory feature vector arrives, inputting the new memory feature vector into a graph neural network for node embedding and edge prediction to update the structure and node attributes of the memory graph;

[0033] the graph neural network learns the topological structure and node features of the memory graph through a graph convolution network or a graph attention network, and automatically discovers the potential association and evolution law between nodes;

[0034] storing the memory feature vector and its corresponding event time, event location and related entity information in the memory graph.

[0035] Preferably, the construction and evolution of the memory graph further comprises:

[0036] adopting knowledge graph embedding technology, such as TransE or RotatE algorithm, to predict and complete the entity relationship in the memory graph, enhancing the integrity and reasoning ability of the graph;

[0037] based on defining the association strength index between nodes and edges, such as based on feature similarity, semantic distance or graph structure connection degree, to quantify the association degree between memories.

[0038] Preferably, the backtracking path planning and reasoning on the memory graph to determine the event sequence or causal chain from the starting point to the target point, specifically comprises:

[0039] parsing the user input query or system preset target into a structured query vector or graph query statement;

[0040] inputting the query vector or graph query statement into a path search module, the path search module executing a breadth-first search, a depth-first search, an A-star algorithm or a graph walking strategy based on reinforcement learning on the memory graph to find a backtracking path that meets the query conditions;

[0041] the backtracking path is a series of mutually associated memory events or entity sequences arranged in chronological order;

[0042] causal chain reasoning on the event sequence on the backtracking path, the causal chain reasoning analyzing the causal relationship between events by combining a causal graph model such as a Bayesian network.

[0043] Secondly, the present invention provides a content generation system based on memory trace back, including a multi-source heterogeneous raw data acquisition module for acquiring multi-source heterogeneous raw data to be analyzed, wherein the multi-source heterogeneous raw data includes text data, image data and time-series sensor data;

[0044] The output of the multi-source heterogeneous raw data acquisition module is electrically connected to the input of the data preprocessing module 2. The data preprocessing module is used to preprocess the multi-source heterogeneous raw data to generate standardized data in a unified format.

[0045] The output of the data preprocessing module is electrically connected to the input of the memory trace feature extraction and representation module 3. The memory trace feature extraction and representation module is used to extract and represent memory trace features from the standardized data to generate a multidimensional memory trace feature vector.

[0046] The output of the memory trace feature extraction and representation module is electrically connected to the input of the memory trace map construction and evolution module 4. The memory trace map construction and evolution module is used to construct a dynamically evolving memory trace map based on the multidimensional memory trace feature vector. The memory trace map includes nodes and edges. The nodes represent entities or events, and the edges represent the relationships between entities or events.

[0047] The output of the memory map construction and evolution module is electrically connected to the input of the backtracking path planning and reasoning module. The backtracking path planning and reasoning module is used to perform backtracking path planning and reasoning on the memory map to determine the event sequence or causal chain from the starting point to the target point.

[0048] The output of the backtracking path planning and reasoning module is electrically connected to the input of the content generation module; the content generation module is used to generate target content based on the backtracking path and the multidimensional memory trace feature vector.

[0049] Preferably, the multi-source heterogeneous raw data acquisition module includes:

[0050] The text acquisition interface is used to obtain text data such as log files, report documents, or user comments;

[0051] Image acquisition interface, used to acquire image data such as surveillance video frames, photos or diagrams;

[0052] A sensor data interface is used to acquire real-time or historical time-series data from various sensors, including temperature sensors, pressure sensors, vibration sensors, position sensors, or network traffic sensors.

[0053] Compared with the prior art, the beneficial effects of the present invention are: by constructing a multi-source heterogeneous data acquisition and preprocessing module, it is possible to effectively acquire and standardize raw data from multiple modalities such as text, images, and time-series sensors, overcoming the dependence of the prior art on a single data source or pre-labeled data, and greatly expanding the breadth and depth of data processing.

[0054] By designing a memory trace feature extraction and representation module, a deep learning model is used to independently encode data from different modalities, and a cross-modal fusion network is employed to capture deep implicit relationships between data, generating a unified multidimensional memory trace feature vector. In particular, uncertainty quantification of memory trace features is performed through probabilistic embedding or Gaussian mixture models, effectively handling fuzzy memory trace features and improving the accuracy and robustness of feature representation.

[0055] A dynamic memory trace map construction and evolution module was established, which constructs and continuously updates the memory trace map based on memory trace feature vectors. The application of graph neural networks enables the map to automatically discover potential relationships and evolutionary patterns between nodes, achieving adaptation to real-time data changes in dynamic scenarios and overcoming the shortcomings of existing technologies in adaptability to dynamic scenarios.

[0056] By combining intelligent search algorithms and causal chain reasoning with the backtracking path planning and reasoning module on the dynamic memory trace map, key events and causal relationships can be intelligently identified, and efficient and accurate backtracking paths can be planned. This avoids the inefficiency and subjectivity of traditional manual backtracking and significantly improves the accuracy of backtracking.

[0057] The content generation module integrates advanced generation models, which can quickly and accurately generate structured or unstructured target content, such as incident reports, fault diagnosis descriptions, or event summaries, based on precise backtracking paths and memory trace features. This significantly improves the efficiency and quality of content generation and meets the needs of diverse application scenarios.

[0058] It can effectively integrate multi-source heterogeneous data, dynamically model memory trace features, intelligently optimize backtracking paths, and demonstrate high applicability and superiority in various complex application scenarios such as accident analysis, fault diagnosis, data archiving, and event reconstruction. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the architecture of the present invention; Figure 2 This is a flowchart illustrating the logical process of multi-source heterogeneous data memory trace feature extraction and cross-modal fusion in this invention. Figure 3 This is a flowchart illustrating the construction and dynamic evolution of the memory map in this invention. Figure 4 This is a flowchart illustrating the backtracking path planning and reasoning process of this invention. Figure 5 This is a flowchart of the backtracking path planning and reasoning result generation content in this invention; Figure 6 This is a system flowchart of the present invention.

[0065] The diagram shows: 1. Multi-source heterogeneous raw data acquisition module; 2. Data preprocessing module; 3. Memory trace feature extraction and representation module; 4. Memory trace map construction and evolution module; 5. Backtracking path planning and reasoning module; 6. Content generation module. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1:

[0068] refer to Figure 1 This invention proposes a content generation method based on memory trace back, comprising the following steps:

[0069] S01: Obtain the multi-source heterogeneous raw data to be analyzed. The multi-source heterogeneous raw data includes text data, image data, and time-series sensor data.

[0070] S02: Perform data preprocessing on multi-source heterogeneous raw data to generate standardized data in a unified format. Data preprocessing includes text cleaning of text data, image denoising of image data, and timestamp synchronization of time-series sensor data.

[0071] Memory trace features are extracted and represented from standardized data to generate multidimensional memory trace feature vectors. The memory trace feature extraction and representation includes single-modal encoding of different modal data through a multimodal encoder and feature fusion through a cross-modal fusion network.

[0072] S03: Construct a dynamically evolving memory trace graph based on multidimensional memory trace feature vectors. The memory trace graph contains nodes and edges. Nodes represent entities or events, and edges represent the relationships between entities or events. Dynamic evolution includes real-time updating of the structure and node attributes of the memory trace graph based on new memory trace features.

[0073] S04: Perform backtracking path planning and reasoning on the memory map to determine the event sequence or causal chain from the starting point to the target point. Backtracking path planning and reasoning includes path search and causal chain reasoning of the event sequence.

[0074] S05: Based on the backtracking path and multidimensional memory trace feature vector, generate target content. Content generation includes converting the memory trace feature vector and causal inference results into readable text content and organizing it in a structured manner.

[0075] Specifically, step S01 involves acquiring multi-source heterogeneous input data, which aims to provide a comprehensive and real-time information foundation for memory traceback. The multi-source heterogeneous input data specifically includes text data, image data, video data, sensor data, and various types of structured or semi-structured log data. These data types are widely used in practical applications.

[0076] In specific routine operations, such as industrial fault diagnosis, text data can include equipment operation manuals and maintenance logs; image data can include screenshots from on-site monitoring and photos of equipment components; video data can be recorded operation processes; sensor data can be real-time monitoring data such as temperature, pressure, and vibration; and log data can be system error logs and user behavior logs. The acquisition of this data is multi-channel and parallel, achieved through pre-configured interfaces and data acquisition agents.

[0077] Specifically, text data can be obtained from web pages via web crawlers or by querying database log systems; image and video data can be read through webcam interfaces or local file systems; and sensor data can be subscribed to in real time from sensor networks or data acquisition units via IoT communication protocols (e.g., message queue telemetry transmission, Advanced Message Queuing Protocol). Each type of data is accompanied by a precise timestamp and data source identifier upon acquisition for subsequent time-series alignment and source tracing. To ensure data quality, the data acquisition layer also integrates preliminary verification mechanisms, including data integrity verification, data type verification, and data source availability checks. When anomalies such as unreachable data sources, mismatched data formats, or acquisition timeouts are detected, the system will trigger an alarm mechanism and attempt alternative data sources or log the anomaly to ensure the continuity and robustness of data input. For text data, the parameter source can be set to a specific document database path or webpage URL, requiring corresponding access permissions. Its typical value range is a string set of arbitrary length. For sensor data, the parameter source is set to a specific sensor identifier and data stream port, with typical values ​​ranging from floating-point numbers or integers, and usually including predefined maximum and minimum values.

[0078] refer to Figure 2In specific implementation: In step S02, memory trace feature extraction and cross-modal fusion are performed on the multi-source heterogeneous input data to transform the heterogeneous raw data into a unified and highly semantically dense memory trace feature representation, laying the foundation for subsequent map construction. Specifically, this includes:

[0079] a: Extract single-modal memory trace features from multi-source heterogeneous input data.

[0080] b: Perform cross-modal collaborative fusion of single-modal memory trace features.

[0081] Specifically, in step a, single-modal memory trace feature extraction is performed on multi-source heterogeneous input data, employing dedicated deep learning models for different data types. For text data, a pre-trained large-scale language model, such as a multi-layer transformer network, is used for semantic entity recognition, event extraction, and relationship classification, identifying key figures, locations, organizations, times, and specific events mentioned in the text, as well as their semantic relationships. For example, in the sentence "equipment overheating caused shutdown," the entities "equipment," "overheating," and "shutdown" can be extracted, and the causal relationship of "overheating caused shutdown" can be identified. For image data, convolutional neural networks are used to extract visual features, including but not limited to object detection, scene classification, key region recognition, and visual saliency features. Abnormal wear or smoke from a component can be identified from equipment monitoring images. For video data, convolutional neural networks and recurrent neural networks or temporal transformer networks are combined to extract temporal dynamic features from video frame sequences, including behavior recognition, the spatiotemporal location of events, and motion trajectories. Operator errors or abnormal vibrations of equipment components can be identified. For sensor data, temporal convolutional networks or long short-term memory networks are used to extract dynamic patterns such as trends, periodic changes, and sudden anomalies from continuous time-series signals. Specifically, abnormal temperature spikes or drops are detected from temperature curves. Each single-modal feature extraction model outputs a high-dimensional vector representation that captures the deep semantic information of that modality's data, along with precise timestamps and source information.

[0082] In step b, cross-modal collaborative fusion is performed on single-modal memory features. The multi-head self-attention mechanism is used for cross-modal fusion, which involves unified representation learning of memory features from different modalities to capture the implicit correlations between modalities.

[0083] Specifically, the feature vectors extracted from all single modalities (e.g., text feature vectors, image feature vectors, sensor feature vectors) are concatenated to form an initial joint feature matrix. Subsequently, learnable modality embedding vectors are introduced to provide a unique identifier for each modality. The modality embedding vectors and the joint feature matrix are then fed into a multi-head self-attention network. In the self-attention mechanism, each modality's feature vector can act as a query, key, and value, interacting with feature vectors from other modalities to calculate the correlation weights between features from different modalities. This mechanism allows the model to dynamically learn which modal features are more important to the fused representation in the current context and capture the non-linear relationships between them. When the text mentions "overheating" and the image shows "smoke," the self-attention mechanism assigns higher co-weights to the features of these two modalities, thus highlighting the composite memory trace of "fire risk" in the fused features. The fusion process can be represented by the following formula:

[0084] Where Q represents the query matrix, K represents the key matrix, and V is the multidimensional memory trace feature vector constructing the dynamically evolving memory trace map representation matrix. These are all generated from single-modal feature vectors through linear transformation. It is a scaling factor used to prevent the inner product from becoming too large. This represents the fusion features after attention weighting.

[0085] This fused feature vector is the final memory trace feature, which represents the comprehensive information of multi-source heterogeneous data in a unified latent space. Its dimension is typically set to several hundred dimensions to balance information capacity and computational efficiency. In this way, even ambiguous potential correlations in the original data, such as a weak connection between a specific operational behavior and subsequent sensor data anomalies, can be effectively captured through cross-modal fusion, thereby significantly improving the comprehensiveness and accuracy of memory trace representation.

[0086] refer to Figure 3 Specifically, in step S03, the memory trace map is constructed and dynamically evolved, aiming to organize discrete memory trace features into a structured knowledge network and enable it to be updated in real time to reflect the evolution of the system state. This includes the following steps:

[0087] (1): Initialize the nodes and edges of the memory trace graph.

[0088] (2): Extract the relationships between nodes and add them to the graph.

[0089] (3): Perform graph temporal reasoning and consistency verification.

[0090] (4): Dynamically update the map based on the characteristics of new memory traces.

[0091] Specifically, in step (1), the nodes and edges of the memory trace graph are initialized based on the memory trace feature vectors generated in step S02. Each memory trace feature vector typically corresponds to a node in the graph, which represents an entity, event, or concept. A memory trace feature about "device A failing at time T" will be resolved into the node "device A", the node "failure event", and the node "time T". Node attributes include its type, timestamp, spatial coordinates, and the memory trace feature vector itself. The initial edges are constructed based on the explicit relational information in the memory trace features, such as causal relationships, temporal sequences, and part-whole relationships. The weights of these edges can be initialized based on the confidence or relevance of the memory trace features. In device failure analysis, there may be a "occurred at" relational edge between "device A" and "failure event".

[0092] In step (2), extracting relationships between nodes and adding them to the graph is to enrich the graph's connectivity and uncover potential implicit relationships. This step utilizes a relationship extraction model to infer new relationships based on the semantic and contextual information of nodes and add them as edges to the graph. If "Device A" and "Operator B" both appear in the memory traces of multiple fault events within a certain time period, the system may infer the relationship that "Operator B is the main operator of Device A". The types of relationships can be complex, such as causal, conditional, spatial, and temporal. Relationship extraction can employ rule-based methods, machine learning-based methods, or deep learning-based relationship extraction models. For complex relationships that are difficult to observe directly, knowledge graph embedding technology can be used to map nodes and edges into a low-dimensional vector space and predict potential relationships through vector operations. Each edge in the graph contains the relationship type, weight, and its discovery timestamp.

[0093] In step (3), graph temporal reasoning and consistency verification are performed to ensure the logical integrity and temporal accuracy of the memory trace graph. This step involves traversing the graph to detect and correct any potential logical conflicts or temporal paradoxes. If there are two contradictory edges in the graph, namely "Event A occurred before Event B" and "Event B occurred before Event A," the system will resolve the conflict based on the confidence level of the memory trace, the accuracy of the timestamp, and preset priority rules, selecting the more reliable relationship. Temporal reasoning also includes perfecting the event chain by filling in missing intermediate events through logical rules, or deriving unmentioned events forward or backward based on known causal chains. This step also includes periodic consistency verification of the graph, checking whether node attributes and edge relationships conform to preset ontological constraints. Inconsistent entities or relationships will be marked or corrected.

[0094] In step (4), dynamically updating the graph based on new memory trace features is crucial for maintaining the real-time nature and adaptability of the memory trace graph. When new memory trace features are input, the graph management module first uses entity linking technology to determine whether the new memory trace represents the same entity or event as a node in the existing graph. If they match, the attributes of the existing node are updated or new relation edges are added; if they do not match, new nodes and edges are created. Dynamic updates also include graph pruning, i.e., removing outdated, low-confidence, or inactive nodes and edges to control the size of the graph and improve query efficiency. Pruning strategies can be based on time thresholds (e.g., retaining memories from the most recent month), confidence thresholds (e.g., removing relations with confidence below a certain value), or influence assessment. This dynamic evolution mechanism enables the memory trace graph to continuously reflect changes in the system state, providing the latest and most comprehensive knowledge background for real-time backtracking and content generation.

[0095] The construction and dynamic evolution of this memory trace map can be assisted by graph neural networks to learn the embeddings of the map structure, thereby improving the accuracy of relation extraction and consistency verification. Its map evolution strategy can be expressed as:

[0096] in, This represents the memory trace at time t. This represents the set of memory trace features newly added between time t and t+1. This represents the strategy function for graph updates, pruning, and consistency checks.

[0097] refer to Figure 4 Specifically, in step S04, backtracking path planning and reasoning are performed based on the memory map. The aim is to efficiently find the most explanatory causal or temporal path within the vast memory map, based on a given backtracking target, thereby revealing the event's occurrence process. This specifically includes:

[0098] Set the starting point and target of the backtracking; generate candidate backtracking paths on the memory trace map; evaluate and screen the candidate backtracking paths; construct the memory trace evidence chain.

[0099] Specifically, setting the starting point and the target for backtracking is a crucial initial step in the backtracking process. The starting point is typically a known event, entity state, or point in time specified by the user. For example, in a production accident, the starting point could be "equipment A stopped on a certain day and time." The target is the aspect to be investigated, such as "the direct cause of equipment A's stoppage," "what happened five minutes before equipment A stopped," or "all personnel and operations related to equipment A's stoppage." The starting point and target can be input using natural language descriptions, which the system will map to specific nodes and relationships in the memory map using entity recognition and intent understanding modules. Setting clear starting points and targets helps narrow the search scope and improves backtracking efficiency and relevance.

[0100] Generating candidate backtracking paths on a memory trace graph involves searching the graph for all possible paths connecting nodes based on a given starting point and target. This invention employs a strategy combining breadth-first search and depth-first search, supplemented by heuristic pruning. During the search, paths with higher edge weights (representing higher relationship confidence) or those closer to the backtracking starting point in time are prioritized. For specific types of backtracking tasks, such as causal backtracking, the search algorithm pays particular attention to edges with causal semantics. To avoid path explosion, maximum path length, maximum number of nodes, or pruning thresholds can be set. If the backtracking objective is to find the direct cause, the path length typically does not exceed three hops. Each generated candidate path is a sequence of nodes and edges, representing a possible event evolution process or relationship chain.

[0101] Evaluating and selecting candidate backtracking paths is crucial to ensuring the selected paths have high interpretability and accuracy. For each generated candidate path, the system evaluates it based on multiple dimensions: the confidence of each memory trace on the path, the overall temporal consistency of the path, the relevance of the path to the backtracking target, and the plausibility of the causal logic revealed by the path. If the timestamps on a path are reversed or the causal relationship is invalid, the path's score will be significantly reduced. The comprehensive scoring function below is used to quantify the merits of each path.

[0102]

[0103] Where L is the length of the path. It is the confidence score of the i-th trace or edge in the path. It is the path's time consistency score. It is the relevance score between the path and the backtracking target. , , These are the corresponding weights. By setting a scoring threshold, the highest-scoring paths are selected as the final backtracking results.

[0104] Constructing a chain of evidence involves presenting the selected optimal backtracking path in a structured form. Each chain of evidence is an ordered sequence of memory traces, detailing all key events, entities, and their relationships from the starting point to the target, including their time, location, and original data source. The chain of evidence not only provides "what" of the event but also explains "why" and "how" it occurred. For an equipment downtime event, the chain of evidence might include: "Sensor data anomaly (time T1)"—a dynamic evolutionary memory trace graph constructed from multi-dimensional memory trace feature vectors; "System alarm (time T2)"—a dynamic evolutionary memory trace graph constructed from multi-dimensional memory trace feature vectors; "Operator error (time T3)"—a dynamic evolutionary memory trace graph constructed from multi-dimensional memory trace feature vectors; "Equipment overheating (time T4)"—a dynamic evolutionary memory trace graph constructed from multi-dimensional memory trace feature vectors; and "Equipment downtime (time T5)". Constructing the chain of evidence provides clear, logical, and traceable input for content generation.

[0105] refer to Figure 5 Specifically, in step S05, generating content based on the backtracking path planning and reasoning results is the final output stage of this invention, aiming to transform the abstract memory trace evidence chain into actual content that users can understand and utilize. This includes: structuring the backtracking results; extracting and sorting content elements; calling the content generation model to generate a draft; and post-processing and verifying the generated content.

[0106] Specifically, the structured processing of the backtracking results involves converting the memory trace evidence chain output by the backtracking path planning and reasoning module 5 into a standardized data structure. This includes parsing each node and edge in the path into a data record with clearly defined fields, such as event type, time, involved entities, key attributes, and causal relationship strength, which can be represented using structured query language or data frames. This structured processing enables the subsequent content generation module to access and utilize the backtracking results in a unified and efficient manner, avoiding the difficulties of directly processing complex graph structures, and transforming the equipment failure evidence chain into a list containing fields such as "event sequence," "timestamp," "event description," and "related entities."

[0107] Extracting and sorting content elements involves filtering core information from structured backtesting results based on specific content generation needs, and organizing it according to importance, chronological order, or logical relationships. For example, if an accident report needs to be generated, key elements might include the accident time, location, involved personnel, direct cause, root cause, and scope of impact. Sorting rules can be preset, describing the earliest event first, followed by its subsequent impact; or describing the primary cause first, followed by secondary causes. This extraction and sorting ensures that the core information of the generated content is complete and logically clear, conforming to user reading habits. For example, for equipment failure, the "final failure phenomenon," "direct cause," "occurrence time," and "scope of impact" are extracted and sorted first.

[0108] The content generation model generates a draft by using a pre-trained natural language generation model (such as a sequence-to-sequence model based on a transformer architecture) to produce a text draft of the content based on extracted and sorted content elements. For report-type content, template filling can be used to populate structured data into a predefined report template; for summary-type content, extractive or generative summarization models can be used to automatically extract and restate key information. The generation model organizes discrete content elements into fluent, grammatically correct sentences and paragraphs based on context and semantic coherence. The model parameters can be fine-tuned according to the specific generation task to adapt to different writing styles and terminology requirements.

[0109] Post-processing and verification of the generated content are crucial steps in ensuring the quality of the final content. Post-processing includes grammar checking, spelling correction, terminology consistency checks, and removal of redundant information from the generated text. Verification involves comparing the generated content with the original backtracking results to ensure that the generated content accurately reflects the facts revealed by the memory trace evidence chain and does not introduce errors or false information. Furthermore, manual review mechanisms or user feedback loops can be introduced to evaluate and optimize the logical coherence, readability, and usability of the generated content. Only through rigorous post-processing and verification can the final content be considered reliable and qualified output. For example, after generating a device failure report, verify whether the failure time and component names mentioned in the report are completely consistent with the original information in the memory trace map.

[0110] Example 2:

[0111] This invention provides a content generation system based on memory trace back, including a multi-source heterogeneous raw data acquisition module 1, used to acquire multi-source heterogeneous raw data to be analyzed, including text data, image data and time-series sensor data;

[0112] Data preprocessing module 2 is used to preprocess multi-source heterogeneous raw data to generate standardized data in a unified format;

[0113] The memory trace feature extraction and representation module 3 is used to extract and represent memory trace features from standardized data to generate multidimensional memory trace feature vectors.

[0114] The memory trace graph construction and evolution module 4 is used to construct a dynamically evolving memory trace graph based on multidimensional memory trace feature vectors. The memory trace graph contains nodes and edges. Nodes represent entities or events, and edges represent the relationships between entities or events.

[0115] The backtracking path planning and reasoning module 5 is used to perform backtracking path planning and reasoning on the memory map to determine the event sequence or causal chain from the starting point to the target point.

[0116] Content generation module 6 is used to generate target content based on backtracking paths and multidimensional memory trace feature vectors.

[0117] The multi-source heterogeneous raw data acquisition module 1 is designed to acquire raw, multi-source heterogeneous data from various data sources, including but not limited to text data, image data, video data, sensor data, and various structured or semi-structured log data. This module can collect data in real-time or in batches through preset interface protocols, such as application programming interface calls, database connection queries, message queue subscriptions, and file system reads. The data acquisition module has multiple data source adapters, capable of handling data of different formats and encodings, and initially standardizes the acquired raw data for subsequent processing. When the system needs to trace back a device failure, the data acquisition module will acquire device operation logs, operator maintenance records, on-site monitoring images, and relevant sensor time-series data.

[0118] Data preprocessing module 2 is used to preprocess the multi-source heterogeneous raw data to generate standardized data in a unified format.

[0119] The memory trace feature extraction and representation module 3 receives the raw data output from the data acquisition module and performs in-depth analysis and processing on this data. This module first performs single-modal feature extraction on various types of raw data. It uses natural language processing to extract entities, events, and relationships from text; computer vision to extract targets, scenes, and behavioral features from images or videos; and time-series analysis to extract abnormal patterns from sensor data. Subsequently, this module performs cross-modal fusion of features from different modalities. By designing specific fusion strategies, such as multimodal attention mechanisms, collaborative embedding, or gating mechanisms, it maps heterogeneous features into a unified semantic space, forming memory trace feature vectors with higher dimensions and richer semantic information. These memory trace feature vectors not only contain deep semantics within a single modality but also capture complex correlations and collaborative information between different modalities. When receiving equipment fault logs and on-site images, this module not only identifies error codes in the logs and faulty components in the images but also correlates them to form a composite memory trace feature of "a certain error code and a specific component failure."

[0120] The Memory Trace Graph Construction and Evolution Module 4 is designed to construct a dynamically evolving memory trace graph based on the memory trace feature vectors output by the memory trace feature extraction and fusion module. This graph uses events, entities, and concepts as nodes, and various relationships between them (such as temporal order, causal relationships, spatial proximity, and semantic association) as edges. This module possesses the capabilities for graph initialization, incremental updates, consistency checks, conflict resolution, and graph pruning. When new memory trace features are input, the module can automatically identify them and incorporate them into the existing graph, updating the graph structure and the attributes of nodes and edges. Simultaneously, the module can perform temporal reasoning within the graph, maintaining the temporal logical consistency of events and optimizing potentially redundant or conflicting information to ensure the accuracy and real-time performance of the graph. For example, when a device repair event is detected, the module will update the fault status node of the device and create a causal relationship edge between the "repair" event and the "fault" event.

[0121] The backtracking path planning and reasoning module 5 receives backtracking targets set by the user or system (e.g., a specific event, the state of a specific entity, or a situation during a certain period) and performs path planning and reasoning on the memory trace graph provided by the memory trace graph construction and evolution module 4. This module supports various backtracking strategies, such as graph traversal algorithms based on depth-first search, breadth-first search, reinforcement learning, or attention mechanisms. During the search process, the module comprehensively considers the relevance, confidence, temporal consistency, and causal logic of memory traces along the path, evaluating the rationality of each candidate path. Through reasoning, the module can identify a series of memory traces highly relevant to the backtracking target and their interrelationships, thereby constructing one or more explanatory memory trace evidence chains, forming the backtracking path planning and reasoning results. When a user queries "the direct cause of equipment downtime," the module searches the graph for paths directly causally related to the "downtime" event and infers the upstream failure event sequence.

[0122] Content generation module 6 receives the memory trace evidence chain output by the backtracking path planning and reasoning module 5 and generates the final content based on this information. This module can output content in the form of text reports, summaries, timelines, multimedia combinations, or structured data, depending on different application scenarios and user needs. Content generation module 6 integrates a natural language generation model, a summary generation model, or a template filling engine. It first performs structured processing on the backtracking results, extracting and sorting key content elements; then, it calls the corresponding generation model to transform the structured information into fluent, accurate, and logically coherent readable content; finally, it performs post-processing and verification on the generated content to ensure its quality and compliance with preset specifications. Based on the backtracking evidence chain of the equipment failure, it generates a detailed report including the failure time, cause, scope of impact, and repair recommendations.

[0123] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A content generation method based on memory trace tracing, characterized in that, Includes the following steps: S01: Obtain the multi-source heterogeneous raw data to be analyzed, which includes text data, image data, and time-series sensor data; S02: Perform data preprocessing on the multi-source heterogeneous raw data to generate standardized data in a unified format. The data preprocessing includes text cleaning of the text data, image denoising of the image data, and timestamp synchronization processing of the time-series sensor data. The standardized data is subjected to memory trace feature extraction and representation to generate a multidimensional memory trace feature vector. The memory trace feature extraction and representation includes using a multimodal encoder to perform single-modal encoding on data of different modalities, and then performing feature fusion through a cross-modal fusion network. S03: Construct a dynamically evolving memory trace graph based on the multidimensional memory trace feature vector. The memory trace graph contains nodes and edges. The nodes represent entities or events, and the edges represent the relationships between entities or events. The dynamic evolution includes updating the structure and node attributes of the memory trace graph in real time based on new memory trace features. S04: Perform backtracking path planning and reasoning on the memory trace map to determine the event sequence or causal chain from the starting point to the target point. The backtracking path planning and reasoning includes path search and causal chain reasoning on the event sequence. S05: Based on the backtracking path and the multidimensional memory trace feature vector, generate target content. The content generation includes converting the memory trace feature vector and the causal inference result into readable text content and organizing it in a structured manner.

2. The content generation method based on memory trace tracing according to claim 1, characterized in that, The acquisition of multi-source heterogeneous raw data to be analyzed includes: constructing a dynamic evolutionary memory map based on multidimensional memory trace feature vectors; and acquiring log files, report documents, or user comment text data through a text acquisition interface. Multidimensional memory trace feature vectors are used to construct a dynamic evolution memory trace map, which acquires image data such as monitoring video frames, photos, or diagrams through an image acquisition interface; It also uses sensor data interfaces to acquire real-time or historical time-series data from various sensors, including temperature sensors, pressure sensors, vibration sensors, position sensors, or network traffic sensors.

3. The content generation method based on memory trace tracing according to claim 2, characterized in that, The step of preprocessing the multi-source heterogeneous raw data to generate standardized data in a unified format specifically includes: The text data is subjected to text cleaning, word segmentation, part-of-speech tagging, stop word removal, and text length normalization. The image data is subjected to image denoising, scale normalization, color correction, and region cropping. The time-series sensor data is subjected to timestamp synchronization, missing value filling, outlier detection, and data smoothing. All preprocessed data undergoes a unified data format conversion to generate a standardized data stream that can be processed by the memory trace feature extraction module.

4. The content generation method based on memory trace tracing according to claim 3, characterized in that, The step of extracting and representing memory trace features from the standardized data to generate a multidimensional memory trace feature vector specifically includes: The standardized data stream is input into a multimodal encoder for feature extraction. The multimodal encoder includes a text encoder, an image encoder, and a temporal encoder. The text encoder performs semantic encoding on the text data using a pre-trained language model based on a transformer architecture, generating a text feature vector. The image encoder performs visual encoding on the image data using a deep learning model based on a convolutional neural network or visual transformer architecture to generate image feature vectors. The timing encoder dynamically encodes the timing sensor data using a recurrent neural network based on a long short-term memory network, a gated recurrent unit, or a temporal convolutional network architecture, generating a timing feature vector.

5. The content generation method based on memory trace tracing according to claim 4, characterized in that, The process of extracting and representing trace features from the standardized data further includes: The text feature vector, the image feature vector, and the temporal feature vector are input into a cross-modal fusion network. Feature fusion is performed through a multi-head self-attention mechanism or a cross-attention mechanism to capture the implicit correlation between different modal data and generate a unified, high-dimensional memory trace feature vector. The cross-modal fusion network further quantifies the uncertainty of the fused memory trace feature vector through a probabilistic embedding layer or a Gaussian mixture model, representing the memory trace feature vector as a probability distribution or a fuzzy set to capture the uncertainty information of fuzzy memory trace features.

6. The content generation method based on memory trace tracing according to claim 5, characterized in that, The construction of a dynamically evolving memory trace map based on the multidimensional memory trace feature vector specifically includes: Initialize the memory trace graph, which is constructed from an initial node set and edge set based on the domain ontology and initial memory trace features; When a new memory trace feature vector arrives, the new memory trace feature vector is input into the graph neural network for node embedding and edge prediction to update the structure and node attributes of the memory trace graph. The graph neural network learns the topology and node features of the memory trace map through graph convolutional networks or graph attention networks, and automatically discovers the potential associations and evolutionary patterns between nodes; The memory trace feature vectors, along with their corresponding event time, event location, and related entity information, are stored in the memory trace map.

7. The content generation method based on memory trace tracing according to claim 6, characterized in that, The construction and evolution of the memory trace map also includes: By employing knowledge graph embedding technology, entity relationships in memory graphs are predicted and completed, enhancing the graph's integrity and reasoning capabilities. Based on the defined correlation strength index between nodes and edges, and based on feature similarity, semantic distance, or graph structure connectivity, the degree of correlation between memory traces is quantified.

8. The content generation method based on memory trace tracing according to claim 7, characterized in that, The process of planning and reasoning back along the memory map to determine the event sequence or causal chain from the starting point to the target point specifically includes: Parse user-input queries or system-preset targets into structured query vectors or graph query statements; The query vector or graph query statement is input into the path search module, which performs a graph walking strategy based on breadth-first search, depth-first search, A* algorithm or reinforcement learning on the memory trace graph to find a backtracking path that meets the query conditions. The backtracking path is a series of interconnected memory trace events or entity sequences arranged in chronological order; Causal chain reasoning is performed on the event sequence along the backtracking path, and the causal chain reasoning analyzes the causal relationships between events by combining a causal graph model.

9. The system comprising the content generation method based on memory trace tracing according to any one of claims 1-8, characterized in that, It includes a multi-source heterogeneous raw data acquisition module (1), used to acquire multi-source heterogeneous raw data to be analyzed, wherein the multi-source heterogeneous raw data includes text data, image data and time-series sensor data; The output end of the multi-source heterogeneous raw data acquisition module (1) is electrically connected to the input end of the data preprocessing module (2). The data preprocessing module (2) is used to preprocess the multi-source heterogeneous raw data to generate standardized data in a unified format. The output of the data preprocessing module (2) is electrically connected to the input of the memory trace feature extraction and representation module (3). The memory trace feature extraction and representation module (3) is used to extract and represent memory trace features from the standardized data to generate a multidimensional memory trace feature vector. The output of the memory trace feature extraction and representation module (3) is electrically connected to the input of the memory trace map construction and evolution module (4). The memory trace map construction and evolution module (4) is used to construct a dynamically evolving memory trace map based on the multidimensional memory trace feature vector. The memory trace map contains nodes and edges. The nodes represent entities or events, and the edges represent the relationships between entities or events. The output of the memory map construction and evolution module (4) is electrically connected to the input of the backtracking path planning and reasoning module (5). The backtracking path planning and reasoning module (5) is used to perform backtracking path planning and reasoning on the memory map to determine the event sequence or causal chain from the starting point to the target point. The output of the backtracking path planning and reasoning module (5) is electrically connected to the input of the content generation module (6); the content generation module (6) is used to generate target content based on the backtracking path and the multidimensional memory trace feature vector.

10. The system according to claim 9, characterized in that, The multi-source heterogeneous raw data acquisition module (1) includes: The text acquisition interface is used to obtain text data from log files, report documents, or user comments; Image acquisition interface, used to acquire image data of surveillance video frames, photos or diagrams; A sensor data interface is used to acquire real-time or historical time-series data from various sensors, including temperature sensors, pressure sensors, vibration sensors, position sensors, or network traffic sensors.

Images