An online time series event analysis method and system based on heterogeneous memory fusion

Abstract

The application belongs to the technical field of time series data analysis, and discloses an online time series event analysis method and system based on heterogeneous memory fusion, which comprises the following steps: acquiring streaming time series data and preprocessing, extracting features from the preprocessed data to obtain time series feature sequences; long-term feature memory, event state probability memory and event-level semantic memory are constructed and fused to form a fusion memory; an event query representation is initialized, and the event query representation is enhanced through a short-term memory and a fusion memory in turn to generate a final event query representation; based on the final event query representation, an instantaneous event state and a candidate event instance at the current time are generated, the candidate event instance is de-redundant and cross-time deduplication processed, and a final event set is output. By constructing and fusing heterogeneous historical memories, the application realizes effective utilization of multi-granularity time series knowledge, thereby significantly improving the accuracy, stability and real-time performance of online event analysis under strict causal constraints.

Description

Technical Field

[0001] This invention relates to the field of time series data analysis technology, and in particular to an online time series event analysis method and system based on heterogeneous memory fusion. Background Technology

[0002] In applications such as real-time monitoring, intelligent interaction, and robot perception, systems need to perform instantaneous analysis of continuously input time-series data streams to simultaneously identify instantaneous state changes and structured events with durations. This type of task falls under the key research direction of online sequence analysis. Because the input is a continuous data stream, the system can only rely on current and historical information when making inferences at any given time, and cannot use future fragments. Therefore, it must strictly satisfy temporal causality to meet the requirements of low latency and rapid response.

[0003] Existing online time series analysis methods face two major challenges. First, constructing stable and reliable event-level time series representations under streaming conditions is extremely difficult. Traditional offline methods typically rely on complete event fragments for feature aggregation and modeling, but in online scenarios, only partial event sequences are often observed in the early stages, making it difficult to fully represent the event structure in the initial stages. Second, existing methods have limited effectiveness in utilizing historical information to compensate for missing future information. Most current mainstream methods use a single feature stream (such as the hidden states of RNNs or Transformers) to represent historical information. This approach has a simplistic structure and ignores the inherent multi-granularity and heterogeneity of historical information. In fact, historical information contains knowledge at different time scales and semantic levels, such as the temporal evolution patterns of low-level features, the dynamic probability distribution of high-level event states, and the semantic structure priors of events that have already occurred. These information differ significantly in time span, abstraction level, and structural form, and are complementary. The single historical representation model of existing methods cannot effectively capture and fuse this heterogeneous information, resulting in insufficient understanding of the global temporal structure of the system, which in turn affects the stability of event boundary detection, the accuracy of category discrimination, and the temporal consistency of the entire output sequence.

[0004] Therefore, there is an urgent need in this field for an online time series analysis method that can construct and integrate historical information from multiple granular and multi-level perspectives to overcome the limitations of existing technologies and improve analysis performance in complex streaming data scenarios. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an online temporal event analysis method and system based on heterogeneous memory fusion. By constructing and fusing heterogeneous historical memories, it enables the effective utilization of multi-granularity temporal knowledge, thereby significantly improving the accuracy, stability, and real-time performance of online event analysis under strict causal constraints.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an online time-series event analysis method based on heterogeneous memory fusion, comprising the following steps:

[0008] Acquire streaming time-series data and preprocess it; extract features from the preprocessed data to obtain time-series feature sequences.

[0009] Construct and integrate long-term feature memory, event state probabilistic memory, and event-level semantic memory to form a unified fused memory;

[0010] The event query representation is initialized and then enhanced sequentially through short-term memory and fused memory to generate the final event query representation.

[0011] Based on the final event query representation, generate the instantaneous event state and candidate event instances at the current moment;

[0012] The candidate event instances are deduplicated and de-duplicated across time points to output the final event set and update the event-level semantic memory.

[0013] As an alternative implementation, features within a long-term window are extracted from historical time-series feature sequences, and compressed using a learnable query vector through a cross-attention mechanism to generate a long-term memory representation; distributions within a probability window are extracted from historical event state probability sequences, and encoded using a one-dimensional convolutional network to generate event state probability memory representations; event features are extracted from a historical event feature library, and compressed using a learnable event query vector through a cross-attention mechanism to generate event-level semantic memory representations.

[0014] As an alternative implementation, the final event query representation is generated as follows:

[0015] Initialize an event status query flag and multiple event anchor queries, and concatenate them to form the initial query;

[0016] Extract features within a short window from the feature sequence at the current time, perform self-attention enhancement, and then perform cross-attention operation with the initial query to generate an intermediate query representation;

[0017] The intermediate query representation and the fused memory representation are subjected to cross-attention operation to generate the final event query representation.

[0018] As an alternative implementation method, candidate event instances for the current moment are generated, specifically as follows:

[0019] The features corresponding to the event anchor queries are extracted from the final event query representation. The prediction head network is then used to obtain the event category probability, time boundary, and feature representation corresponding to each anchor, forming a candidate event instance set.

[0020] As an alternative implementation method, the instantaneous event state at the current moment is generated, specifically as follows:

[0021] The features corresponding to the event state query tags are extracted from the final event query representation, and the event state probability distribution and label at the current moment are obtained through a classifier.

[0022] As an alternative implementation, the final event set includes the instantaneous event status label at the current moment and one or more candidate persistent event instances;

[0023] Candidate event instances include the predicted start time, end time, event category, and confidence level.

[0024] Secondly, the present invention provides an online time-series event analysis system based on heterogeneous memory fusion, comprising:

[0025] The data acquisition and encoding module is configured to: acquire streaming time-series data and preprocess it, extract features from the preprocessed data, and obtain a time-series feature sequence;

[0026] The heterogeneous memory bank construction and fusion module is configured to: construct and fuse long-term feature memory, event state probabilistic memory and event-level semantic memory to form a unified fused memory;

[0027] The event query building module is configured to: initialize the event query representation, and then enhance the event query representation sequentially through short-term memory and fusion memory to generate the final event query representation;

[0028] The time-series event prediction module is configured to generate the instantaneous event state and candidate event instances at the current moment based on the final event query representation.

[0029] The online event post-processing module is configured to: perform redundancy removal and cross-time deduplication on candidate event instances, output the final event set, and update the event-level semantic memory.

[0030] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0031] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0032] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] 1. Superior Capability of Utilizing Multi-Granularity Historical Information: This invention overcomes the limitations of traditional single-feature flow representation by constructing three types of heterogeneous memory: long-term feature memory, event state probabilistic memory, and event-level semantic memory. It enables structured modeling of historical information at multiple granularities, from low-level feature dynamics and mid-level state evolution to high-level event semantics. This multi-layered representation system allows the model to more comprehensively and deeply understand complex temporal patterns, providing a solid information foundation for accurate event recognition and boundary localization.

[0035] 2. Efficient and Compact Information Fusion: Employing a learnable query vector-driven attention compression mechanism, key information can be adaptively extracted from large-scale historical data and compressed into a fixed-length compact representation. This method effectively avoids the computational overhead and feature redundancy problems caused by directly processing long sequences, achieving efficient and intensive utilization of historical information and ensuring the real-time performance of the system.

[0036] 3. Effective unification of local and global temporal dependencies: Through a two-stage event query enhancement strategy, firstly, short-term context is used to capture fine-grained local dynamics, and then long-term structured priors are introduced by fusing memory. This ensures that the final query representation used for event prediction possesses both local sensitivity and global consistency. This significantly improves the robustness of event analysis, especially in scenarios with ambiguous event boundaries or long-range dependencies.

[0037] 4. Achieving high-precision inference under causal constraints: The entire process of this invention strictly follows temporal causality, requires no future information, and is fully applicable to online streaming processing. The rich historical priors provided by the heterogeneous memory module effectively compensate for the uncertainty caused by the lack of future information, enabling the system to maintain near-offline batch processing high analytical accuracy even in unidirectional data streams, meeting the dual requirements of real-time applications for performance and latency.

[0038] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a data processing flowchart of the online time-series event analysis method based on heterogeneous memory fusion according to the present invention;

[0041] Figure 2 This is a flowchart illustrating the framework of the online time-series event analysis method based on heterogeneous memory fusion according to the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0046] Example 1

[0047] like Figures 1 to 2 As shown, this embodiment provides an online time-series event analysis method based on heterogeneous memory fusion, including the following steps:

[0048] Acquire streaming time-series data and preprocess it; extract features from the preprocessed data to obtain time-series feature sequences.

[0049] Construct and integrate long-term feature memory, event state probabilistic memory, and event-level semantic memory to form a unified fused memory;

[0050] The event query representation is initialized and then enhanced sequentially through short-term memory and fused memory to generate the final event query representation.

[0051] Based on the final event query representation, generate the instantaneous event state and candidate event instances at the current moment;

[0052] The candidate event instances are deduplicated and de-duplicated across time points to output the final event set and update the event-level semantic memory.

[0053] The specific solution of the present invention is as follows:

[0054] This invention proposes an online temporal event analysis method based on heterogeneous memory fusion. By fusing sequence-level features (i.e., long-term feature memory), event state probability information (i.e., event state probability memory), and event-level semantic priors (i.e., event-level semantic memory), a structured historical memory bank with stronger expressive power is constructed. Compared with traditional methods that rely on only a single form of memory, this invention can more comprehensively model the spatiotemporal evolution of sequences at multiple semantic levels, thereby significantly improving the accuracy and stability of streaming temporal event analysis.

[0055] Step S1: Acquire streaming time series data and preprocess it. Extract features from the preprocessed data to obtain time series feature sequences.

[0056] The method can be applied to various types of streaming time-series data (video, sensor data, robot multimodal sequences, etc.), with video sequences being a typical input format. The method is illustrated below using a video sequence as an example. Given a video sequence... Divide it into sequential video clip sequences ,in This represents the total number of video segments, and each segment... From continuous Frame composition, i.e. ,in Indicates the video's first Frames. The system processes video segment by segment in chronological order. While processing the current segment... At that time, only the current and previous segments can be accessed. And future fragments Invisible to satisfy the causal constraints of streaming processing. A pre-trained spatiotemporal feature encoder (e.g., I3D) is used to extract features from each segment, resulting in... 3D feature vector:

[0057] ;

[0058] At the present moment The extracted feature sequences are This serves as input for subsequent steps.

[0059] Step S2: Construct and integrate long-term feature memory, event state probability memory, and event-level semantic memory to form a unified fused memory.

[0060] This step involves constructing three types of heterogeneous memory modules (long-term feature memory, event state probabilistic memory, and event-level memory) and encoding and fusing them to form a unified representation of historical knowledge. Specifically, it includes the following four sub-steps:

[0061] (1) Construction and encoding of long-term feature memory.

[0062] The system maintains a feature buffer to record historical semantic information over a relatively long time period. The feature sequence output from step S1... Extracting time intervals Fragment features within the memory constitute a long-term memory feature sequence:

[0063] ;

[0064] when When the sequence is zero-padded to the length, ,get ,in This represents the length of the long-term memory window. This long-term memory encompasses temporal evolution patterns and contextual dependencies over a significant time span, providing a global temporal reference and semantic support for event analysis of the current segment. To reduce computational complexity while preserving key historical information, a learnable query vector is introduced. (in (This refers to the compressed memory length), and adaptive compression of long-term memory is achieved through a cross-attention mechanism:

[0065] ;

[0066] The attention operation is defined as follows:

[0067] ;

[0068] Cross-attention is a special form of attention mechanism where the key and value come from the same input:

[0069] ;

[0070] Here As a query It serves as both a key and a value. Compressed long-term memory. It uses a fixed length to adaptively aggregate key historical information over a long period of time.

[0071] (2) Encoding of event state probability memory.

[0072] The system maintains an event state probability memory buffer to record the event state probability distribution at past time points. This buffer stores events from time point onwards. arrive of total One historical prediction result (generated from previous time points in step S4(1) and accumulated time-by-time):

[0073] ;

[0074] when When the sequence is zero-padded, its length is increased to the specified value. ,get ,in The time window length for probabilistic memory. Indicates the system at the 1st Output at each moment The probability distribution of event states. Event state probabilistic memory characterizes the evolution trend of an event's state over time. To effectively encode this temporal structure, [the following is used:] Transpose The format is such that each row corresponds to the probability change of a certain event state on the time axis; then a one-dimensional temporal convolutional network (Conv1D) is used to encode it:

[0075] ;

[0076] in This is the transposed probability matrix. Conv1D extracts the dynamic patterns of event states over time through multi-layer convolution and mapping operations, and maps them to... Dimensional representation space, ultimately obtained Event state 3D temporal representation. This event state probabilistic memory serves as input to the high-level temporal context, providing the model with information on the persistence, transition patterns, and dynamic trends of event states, which helps improve the stability and prediction accuracy of subsequent temporal event analysis.

[0077] (3) Encoding of event-level memories.

[0078] The system maintains an event memory. Used to store the closest time to the current time. The feature representations of each output event (accumulated and maintained time-by-time by step S5(3)) form a set of historical event features. ,in This refers to the capacity of the event memory. When the number of historical events is insufficient... At this time, zero vectors are used for padding. These historical event features contain semantic information, temporal evolution patterns, and structured knowledge of previous events, providing explicit event-level prior constraints and temporal structure guidance for understanding the event at the current moment. To achieve efficient encoding while preserving key historical information, learnable event query vectors are introduced. (in (This refers to the compressed memory length), and event memories are dynamically compressed using a cross-attention mechanism:

[0079] ;

[0080] in Used for querying It serves as both a key and a value. Compressed event memory. Represented with a fixed length, it adaptively aggregates the core semantic features and structural priors of historical events, providing high-level semantic support for subsequent event inference and decoding processes.

[0081] (4) Memory fusion.

[0082] To fully utilize different types of historical information, the three types of memory encoding mentioned above are integrated to form a unified historical memory representation (i.e., a unified integrated memory). Because... , and Maintain consistency in feature dimensions (all are consistent) (Dimensional), therefore, a concatenation method along the sequence dimension is used to fuse the three types of memories:

[0083] ;

[0084] in This represents the splicing operation on the first dimension. The merged memory representation. It integrates three complementary types of historical information: long-term temporal context, evolutionary patterns of event state probabilities, and event-level semantic priors. This unified fused memory provides comprehensive and structured historical constraints for subsequent temporal event analysis.

[0085] Step S3: Initialize the event query representation, and then enhance the event query representation sequentially through short-term memory and fusion memory to generate the final event query representation.

[0086] This step involves fusing short-term memory information with multi-source fusion memory. (i.e., unified fused memory) enhances the initial event query features in stages to generate a unified query representation for subsequent time-series event analysis. The initial query features consist of an event state query token ([CLS] token) and a set of event anchor queries for modeling the temporal structure of candidate events. The specific process includes the following sub-steps:

[0087] (1) Query initialization.

[0088] First, initialize the event status query token ([CLS] token). This is used to represent the event state at the current moment. Then, initialization... Event anchor query features ,in This is the preset number of queries. For feature dimensions. Each event anchor query corresponds to a learnable timescale template. ( This represents the typical duration of a candidate event over time. The event state query markers and event anchor queries are concatenated along the sequence dimension to form the initial query representation:

[0089] ;

[0090] in This indicates a splicing operation. It serves as input for subsequent query enhancement steps, used to generate the final event query representation after fusing multi-source historical information.

[0091] (2) Enhanced event query based on short-term memory.

[0092] The feature sequence obtained from step S1 Extract the nearest These fragments constitute the characteristics of short-term memory at the current moment:

[0093] ;

[0094] when When the sequence is zero-padded, its length is increased to the specified value. ,get ,in This represents the length of the short-term memory window. This short-term memory is used to characterize the local temporal context near the current moment. To capture fragment dependencies within a short timeframe, [the following is used:] ... Applying self-attention:

[0095] ;

[0096] Self-attention is a special case of the attention mechanism where the query, key, and value all come from the same input. For context-enhanced short-term memory features, the temporal association patterns between segments near the current moment are characterized. Subsequently, the initial event query features are... As a query, enhanced short-term memory features As keys and values, the first-stage query enhancement is performed through a cross-attention mechanism:

[0097] ;

[0098] Obtain an intermediate event query representation that incorporates short-term context information. .

[0099] (3) Enhanced event query based on fusion memory.

[0100] To integrate the multi-source fusion memory constructed in step S2 Effectively integrated into event query representation, further enhancing... Apply cross-attention enhancement based on heterogeneous memory. Specifically, with As a query, As keys and values, memory fusion is performed:

[0101] ;

[0102] Obtain the final event query representation The two-stage enhancement process first extracts local dynamic information from the short-term context (step S3(2)), and then integrates multi-source historical information such as long-term feature patterns, event state probabilities, and event-level semantic priors, thereby capturing both local and global temporal structures and providing rich and structured feature support for subsequent temporal event prediction in step S4.

[0103] Step S4: Based on the final event query representation, generate the instantaneous event state and candidate event instances at the current moment (i.e., perform time-series event prediction based on the final event query representation).

[0104] This step uses the fused event query representation (i.e., the final event query representation) to generate the current time-series event output, including the identification of instantaneous event states and the generation of candidate event instances. The specific process includes the following two parts.

[0105] (1) Instantaneous event state recognition.

[0106] The event query representation output from step S3 Extract the event status query token ([CLS] token) from row 0, and denote it as... .Will The input is fed into the event state classifier to obtain the current moment (fragment). The instantaneous event state probability distribution:

[0107] ;

[0108] in The weight matrix of the classifier. Indicates that the current moment belongs to The probability distribution of each event state category. This is to obtain the instantaneous event state label at the current moment. Select the category with the highest probability from the probability distribution:

[0109] ;

[0110] The probability of the event state at the current moment. Appended to the event state probability memory buffer, and when the length exceeds The earliest prediction is removed to maintain a fixed-length sliding window. The updated probability sequence will be used in step S2(2) at the next time step, at which point the historical probability matrix will contain the predictions from time step [time]. arrive Probability sequence:

[0111] ;

[0112] (2) Generation of candidate event instances.

[0113] from Extract the feature representation corresponding to the event anchor query to obtain , Will Input the event prediction head network to obtain event category prediction and event time boundary prediction, respectively.

[0114] Event category prediction:

[0115] ;

[0116] in .here, express An event anchor query in The score matrix for each event category is used to describe the response strength of each anchor point under different event categories.

[0117] Event time boundary prediction:

[0118] ;

[0119] in , , representing the logarithmic scale of the predicted event end time offset and the event duration, respectively. Based on the . Time template for each event anchor point This allows us to obtain the predicted event time range for the anchor point:

[0120] ;

[0121] right The Perform Softmax normalization on the rows to obtain the class probability distribution of the event anchor point:

[0122] ;

[0123] Based on this, the predicted event category is: The corresponding confidence level is At the same time, from Feature representation of event instances extracted from:

[0124] ;

[0125] Ultimately, we can obtain the set of candidate event instances for the current moment:

[0126] ;

[0127] The candidate event set will undergo further online post-processing in step S5.

[0128] Step S5: Perform redundancy removal and cross-time deduplication on candidate event instances, output the final event set, and update the event-level semantic memory (i.e., the online event post-processing step).

[0129] This step performs online post-processing on the candidate event instances generated in step S4. By removing redundancy within the current time frame and checking for duplicates across time frames, it ensures that the event sequence output by the system has temporal consistency and no repetition, and updates the event-level historical memory based on this. The entire process includes the following three sub-steps.

[0130] (1) Remove redundancy from the event instance at the current moment.

[0131] For the current moment The generated collection of candidate event instances Non-maximum suppression (NMS) is used to remove redundant predictions within the same event category. For each event category... Extract candidate instances belonging to this category:

[0132] ;

[0133] By confidence level Sort in descending order, select the instances with the highest scores as the retained objects, and calculate the time intersection-union ratio (tIoU) with other instances:

[0134] ;

[0135] when At that time, suppress instances Finally, we obtain the set of event instances after redundancy removal at the current moment. .

[0136] (2) Deduplication of events across time periods to ensure consistency.

[0137] The system maintains a set of historical output events. At any moment Initialize to To avoid repeatedly outputting the same event instance in adjacent time steps, it is necessary to optimize the event set at the current time step. Collection of historical events Perform cross-time consistency checks. For each event instance. Calculate its relationship with the historical set Maximum temporal intersection-union ratio among events of the same category:

[0138] ;

[0139] like If the event is identified in the historical output, it is considered to have been identified and will not be output again to maintain the uniqueness of the output sequence; otherwise, it will be retained as a new event instance. This yields the final set of events output at the current moment. After processing the current event, its final output event is added to the historical event set.

[0140] ;

[0141] The updated set of historical events will continue to be used in cross-time consistency deduplication in subsequent time-to-time moments.

[0142] (3) Update of event-level feature memory.

[0143] To support the construction and use of the event-level memory module in step S2(3), the system maintains an event feature memory. To obtain the final output at the current moment. Then, the characteristics of each event instance are represented. Add to memory:

[0144] ;

[0145] in This indicates a queue append operation. The memory uses a fixed-capacity strategy, retaining only the most recently appended entries. The event characteristics are as follows: when the capacity exceeds the limit, the oldest entry is removed according to the first-in-first-out (FIFO) principle to ensure the controllability and timeliness of the memory.

[0146] Example 2

[0147] This embodiment provides an online time-series event analysis system based on heterogeneous memory fusion, including:

[0148] The data acquisition and encoding module is configured to: acquire streaming time-series data and preprocess it, extract features from the preprocessed data, and obtain a time-series feature sequence;

[0149] The heterogeneous memory bank construction and fusion module is configured to: construct and fuse long-term feature memory, event state probabilistic memory and event-level semantic memory to form a unified fused memory;

[0150] The event query building module is configured to: initialize the event query representation, and then enhance the event query representation sequentially through short-term memory and fusion memory to generate the final event query representation;

[0151] The time-series event prediction module is configured to generate the instantaneous event state and candidate event instances at the current moment based on the final event query representation.

[0152] The online event post-processing module is configured to: perform redundancy removal and cross-time deduplication on candidate event instances, output the final event set, and update the event-level semantic memory.

[0153] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0154] In further embodiments, the following is also provided:

[0155] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0156] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0157] A computer-readable storage medium for storing computer instructions that, when executed by a processor, perform the method of Embodiment 1.

[0158] The method in Example 1 can be directly executed by a hardware processor, or it can be executed by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0159] A computer program product includes a computer program that, when executed by a processor, implements the method in Embodiment 1.

[0160] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0161] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0162] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0163] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for online temporal event analysis based on heterogeneous memory fusion, characterized in that, Includes the following steps: Acquire streaming time-series data and preprocess it; extract features from the preprocessed data to obtain time-series feature sequences. The streaming time-series data is a video sequence; We construct and fuse long-term feature memory, event state probabilistic memory, and event-level semantic memory to form a unified fused memory. We extract features within a long-term window from historical time-series feature sequences, compress them using learnable query vectors through a cross-attention mechanism, and generate long-term memory representations. We extract distributions within probability windows from historical event state probabilistic sequences, encode them using a one-dimensional convolutional network, and generate event state probabilistic memory representations. We extract event features from a historical event feature library, compress them using learnable event query vectors through a cross-attention mechanism, and generate event-level semantic memory representations. The event query representation is initialized and then enhanced sequentially through short-term memory and fused memory to generate the final event query representation. Specifically, the final event query representation is generated as follows: Initialize an event state query flag and multiple event anchor queries, and concatenate them to form an initial query; extract features within a short window from the feature sequence at the current time, perform self-attention enhancement, and then perform cross-attention operation with the initial query to generate an intermediate query representation; perform cross-attention operation with the intermediate query representation and the fused memory representation to generate the final event query representation; Based on the final event query representation, generate the instantaneous event state and candidate event instances at the current moment; The candidate event instances are deduplicated and de-duplicated across time points to output the final event set and update the event-level semantic memory.

2. The online temporal event analysis method based on heterogeneous memory fusion according to claim 1, characterized in that, Generate candidate event instances for the current moment, specifically as follows: The features corresponding to the event anchor queries are extracted from the final event query representation. The prediction head network is then used to obtain the event category probability, time boundary, and feature representation corresponding to each anchor, forming a candidate event instance set.

3. The online temporal event analysis method based on heterogeneous memory fusion of claim 1, wherein, Generate the instantaneous event state at the current moment, specifically as follows: The features corresponding to the event state query tags are extracted from the final event query representation, and the event state probability distribution and label at the current moment are obtained through a classifier.

4. The online temporal event analysis method based on heterogeneous memory fusion of claim 1, wherein, The final event set includes the instantaneous event status label at the current moment and one or more candidate persistent event instances; Candidate event instances include the predicted start time, end time, event category, and confidence level.

5. An online temporal event analysis system based on heterogeneous memory fusion, characterized in that, include: The data acquisition and encoding module is configured to: acquire streaming time-series data and preprocess it, extract features from the preprocessed data, and obtain a time-series feature sequence; The streaming time-series data is a video sequence; The heterogeneous memory bank construction and fusion module is configured to: construct and fuse long-term feature memory, event state probabilistic memory, and event-level semantic memory to form a unified fused memory; extract features within a long-term window from historical time-series feature sequences, compress them using learnable query vectors through a cross-attention mechanism to generate long-term memory representations; extract distributions within probability windows from historical event state probabilistic sequences, encode them using a one-dimensional convolutional network to generate event state probabilistic memory representations; and extract event features from a historical event feature bank, compress them using learnable event query vectors through a cross-attention mechanism to generate event-level semantic memory representations. The event query building module is configured to: initialize the event query representation, and then enhance the event query representation sequentially through short-term memory and fusion memory to generate the final event query representation; Specifically, the final event query representation is generated as follows: Initialize an event state query flag and multiple event anchor queries, and concatenate them to form an initial query; extract features within a short window from the feature sequence at the current time, perform self-attention enhancement, and then perform cross-attention operation with the initial query to generate an intermediate query representation; perform cross-attention operation with the intermediate query representation and the fused memory representation to generate the final event query representation; The time-series event prediction module is configured to generate the instantaneous event state and candidate event instances at the current moment based on the final event query representation. The online event post-processing module is configured to: perform redundancy removal and cross-time deduplication on candidate event instances, output the final event set, and update the event-level semantic memory.

6. An electronic device, comprising: It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-4.

7. A computer readable storage medium characterized by Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-4.

8. A computer program product, characterised in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-4.

Images