Real-time data filtering method and system based on multi-dimensional feature fusion

Abstract

The present invention relates to the field of real-time data filtering technology, and in particular to a real-time data filtering method and system with multi-dimensional feature fusion. The method comprises the following steps: obtaining data transmission logs and extracting original transmission data features, parsing data frame field structure information; identifying multi-dimensional abnormal coupling of data transmission based on field time gradient change rate and dynamic offset status; further detecting abnormal dilution of transmission data, predicting the data transmission complexity exceeding limit state, and detecting the overload condition of data filtering chip structure; then evaluating the attenuation trend of data real-time filtering efficiency, and optimizing the data real-time filtering path accordingly, completing real-time filtering processing of data transmission complexity exceeding limit condition, and outputting filtering results; the present invention achieves higher credibility of data filtering by filtering data in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time data filtering, and in particular to a real-time data filtering method and system using multi-dimensional feature fusion. Background Art

[0002] The ability to process data in real time is directly related to business response efficiency and system stability. This is especially true in scenarios involving complex heterogeneous networks and frequent dynamic data interactions. Traditional data filtering methods are difficult to adapt to high-speed, high-concurrency, and multi-protocol data flow environments. Existing data filtering mechanisms generally rely on static rule matching, single-dimensional anomaly identification, or monitoring methods based on fixed thresholds. They lack the ability to dynamically perceive the correlation between multi-dimensional data features. As a result, when faced with complex abnormal coupling situations such as sudden protocol anomalies, transmission compression disturbances, link layer congestion, and protocol synchronization imbalances, they are often unable to accurately identify potential data dilution phenomena and filtering path degradation risks, which in turn leads to problems such as reduced filtering efficiency, chip structure overload, or system response delays. However, traditional real-time data filtering has the problem of inaccurate detection of multi-dimensional abnormal coupling of data transmission, as well as inaccurate detection of overload conditions in the data filtering chip structure. Summary of the Invention

[0003] Based on this, it is necessary to provide a real-time data filtering method and system for multi-dimensional feature fusion to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a real-time data filtering method based on multi-dimensional feature fusion includes the following steps:

[0005] Step S1: Acquire a data transmission log; extract original transmission data features based on the data transmission log to obtain original transmission data features; perform field structure analysis on the data transmission log and original transmission data features to obtain data frame field analysis structure information;

[0006] Step S2: Calculate the transmission field time gradient change rate based on the data frame field analysis structure information; determine the transmission field dynamic offset status based on the transmission field time gradient change rate data; determine the multi-dimensional abnormal coupling of data transmission based on the transmission field dynamic offset status and the transmission field time gradient change rate;

[0007] Step S3: detecting abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predicting a data transmission complexity exceeding limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining a data transmission complexity exceeding limit; performing data filter chip structure overload detection based on the data transmission complexity exceeding limit, thereby obtaining a data filter chip structure overload condition;

[0008] Step S4: Detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization condition of the real-time data filtering path; use the optimization condition of the real-time data filtering path to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

[0009] By performing structured processing on data transmission logs, this method can accurately extract the time gradient variation patterns during the transmission process and improve the ability to identify the micro-evolution trend of transmission anomalies. By linking the time gradient with the dynamic offset analysis of the field, it can achieve accurate judgment of the multi-dimensional anomaly coupling relationship and effectively enhance the response sensitivity to complex transmission behaviors. In the scenario where data anomaly dilution and protocol anomaly behavior co-occur, the forward-looking prediction of the hidden dangers of excessive data transmission complexity can be made, and the system's ability to assess structural stability during high-load operation can be enhanced. Combining the structural overload diagnostic information with the transmission link abnormal topology mapping relationship, it can accurately capture the failure evolution path of the key areas of the chip and improve the efficiency of blocking the spread of anomalies. On this basis, the filtering path is reconstructed in real time, which helps to improve the data processing throughput in high-concurrency scenarios, strengthen the reliability and adaptability of the data processing link, and enhance the overall stable operation capability of the data filtering system in a complex transmission environment. The present invention is an optimization of traditional real-time data filtering, which solves the problem of inaccurate detection of multi-dimensional abnormal coupling of data transmission and inaccurate detection of overload conditions of data filtering chip structure in traditional real-time data filtering, and improves the accuracy of detection of multi-dimensional abnormal coupling of data transmission and the accuracy of detection of multi-dimensional abnormal coupling of data transmission.

[0010] The present invention further provides a real-time data filtering system for multi-dimensional feature fusion, which is used to execute the real-time data filtering method for multi-dimensional feature fusion as described above. The real-time data filtering system for multi-dimensional feature fusion includes:

[0011] The data frame field parsing structure processing module is used to obtain data transmission logs; extract original transmission data features based on the data transmission logs to obtain original transmission data features; and perform field structure parsing on the data transmission logs and original transmission data features to obtain data frame field parsing structure information.

[0012] A multi-dimensional abnormal coupling determination module is used to calculate the transmission field time gradient change rate based on the data frame field analysis structure information; determine the transmission field dynamic offset status based on the transmission field time gradient change rate data; and determine the multi-dimensional abnormal coupling of data transmission based on the transmission field dynamic offset status and the transmission field time gradient change rate;

[0013] The chip structure overload detection module is used to detect abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predict the data transmission complexity exceeding the limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining the data transmission complexity exceeding the limit; and perform data filtering chip structure overload detection based on the data transmission complexity exceeding the limit to obtain the data filtering chip structure overload status;

[0014] The real-time data filtering processing module is used to detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization status of the real-time data filtering path; use the real-time data filtering path optimization status to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

[0015] The real-time data filtering system of the present invention is capable of implementing the real-time data filtering method of the present invention that integrates any multi-dimensional features. It is used to combine the operations between various modules and the medium of signal transmission to complete the real-time data filtering method that integrates multi-dimensional features. The internal modules of the system cooperate with each other, and through multi-dimensional feature fusion and dynamic analysis, it realizes accurate detection and optimization of problems such as abnormal coupling, transmission dilution, complexity exceeding the limit, and chip structure overload in the data transmission process, thereby improving the efficiency of real-time data filtering and system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of the steps of a real-time data filtering method using multi-dimensional feature fusion;

[0017] Figure 2 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0018] Figure 3 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0019] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0020] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. It is obvious that the embodiments described are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present invention.

[0021] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller approaches.

[0022] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0023] To achieve this, please refer to Figures 1 to 3 , a real-time filtering method for data with multi-dimensional feature fusion, comprising the following steps:

[0024] Step S1: Acquire a data transmission log; extract original transmission data features based on the data transmission log to obtain original transmission data features; perform field structure analysis on the data transmission log and original transmission data features to obtain data frame field analysis structure information;

[0025] In an embodiment of the present invention, edge data acquisition equipment deployed at data transmission nodes extracts continuous, time-synchronized data transmission logs from high-speed serial bus interfaces or Ethernet physical layer interfaces. The data transmission logs are based on a frame structure and contain fields such as the timestamp, frame header, frame length, payload field, check bit, and frame trailer for each data frame. An FPGA-based high-speed data frame analysis module reads the raw transmission frame information in the logs and extracts characteristic parameters such as the payload field length, inter-frame time interval, and field value variation frequency within each frame to form a set of raw transmission data features. Subsequently, a field analysis engine based on a structured parsing algorithm performs field structure parsing on the raw data transmission logs and the extracted raw transmission data features. By mapping the log data to known data frame structures in a preset field template library, the engine identifies information such as field order, field type, and its position and length within the frame structure. This information then extracts and forms data frame field parsing structure information, including parameters such as field name, field index, field start bit, field length, and field association relationships.

[0026] Step S2: Calculate the transmission field time gradient change rate based on the data frame field analysis structure information; determine the transmission field dynamic offset status based on the transmission field time gradient change rate data; determine the multi-dimensional abnormal coupling of data transmission based on the transmission field dynamic offset status and the transmission field time gradient change rate;

[0027] In this embodiment of the present invention, the data frame field parsing structure information obtained in step S1 is used as input to construct a transmission field record matrix based on field time series. A fixed time window is set to calculate the time series of the position, time interval, and field length of each field in consecutive data frames. The temporal gradient change rate of each field within the time period is calculated based on the differential changes within the sliding window. The temporal gradient change rate is calculated using a first-order difference formula, normalized by combining the frame timestamp and the relative position of the field, thereby eliminating the effects of different frame lengths or sampling rates. The transmission field temporal gradient change rate is structured and stored, and a field change rate distribution map is constructed to determine the continuity of field fluctuations in the temporal dimension. Furthermore, by setting a deviation threshold (e.g., the gradient change rate within three consecutive sampling windows exceeds a set range), dynamic deviation conditions of each field are detected, including field mutations, field order misalignment, and field drift. Based on a joint analysis of field deviation conditions and temporal gradient change rates, a multi-dimensional anomaly factor cross matrix is ​​constructed to extract coupling indicators between fields, such as the probability of field linkage deviation, coupling strength, and common deviation direction, thereby determining the multi-dimensional abnormal coupling of data transmission.

[0028] Step S3: detecting abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predicting a data transmission complexity exceeding limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining a data transmission complexity exceeding limit; performing data filter chip structure overload detection based on the data transmission complexity exceeding limit, thereby obtaining a data filter chip structure overload condition;

[0029] In an embodiment of the present invention, based on the multidimensional abnormal coupling of data transmission obtained in step S2, its distribution density and interval consistency in the transmission data sequence are further analyzed. A dilution detection method based on the field coupling distribution diagram is used to calculate the concentration, duration, and dispersion of abnormal coupling events per unit time, and extract their abnormal dilution coefficient. If the distribution of abnormal coupling events within a unit window exhibits a jump-like or rapidly shrinking distribution, it is marked as a high dilution region. This generates a transmission data abnormal dilution result table containing the start time, duration, field participation, and dilution coefficient of each abnormal dilution region. On this basis, combined with the multidimensional abnormal coupling of data transmission and the dilution situation, by calculating the overlap probability of coupling events within the abnormal dilution region, field overlap, and transmission density changes, data transmission complexity indicators are derived, including the number of dynamic field combinations, instantaneous conflict frequency, and timing interference index. Based on this, a data transmission complexity threshold model is constructed, and the currently calculated complexity index is compared with the threshold to identify areas where data transmission complexity exceeds the limit, and the data transmission complexity exceeds the limit is obtained. Next, the data frame information in the overloaded area is retroactively extracted and loaded into the data filter chip structure simulation module within the hardware simulation platform. The chip's buffering delay, packet processing rate, and internal cache accumulation are compared under standard and overloaded data loads. Based on the throughput capacity of the chip's internal structure pin data paths and cache scheduling timing, the system detects whether there are abnormal transmission delays, cache overflows, or structural blockages. Based on these assessments, a data filter chip structure overload status report is generated, including the chip overload level, the affected module identifiers, and the overload trigger period.

[0030] Step S4: Detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization condition of the real-time data filtering path; use the optimization condition of the real-time data filtering path to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

[0031] In this embodiment of the present invention, based on the data filtering chip structure overload status report generated in step S3, data packets during periods of high chip overload are selected for filtering path tracing. Using a bus timing replay method, the transit time, latency, and number of processing hops of the data packets along the filtering path are analyzed frame by frame. The queue lengths and processing times of each cache node in different paths are recorded to construct a time series sequence of real-time data filtering efficiency. The filtering efficiency decay rate is calculated from this sequence to determine the degree of filtering rate degradation at each node and identify the critical path nodes causing the efficiency degradation. Based on this information, a node priority reordering strategy is applied to prioritize high-frequency transmission fields and high-coupling fields to nodes with low processing path loads. By adjusting the priority routing index in the field path forwarding table, the real-time data filtering path is dynamically optimized. A real-time data filtering path optimization status file is generated, recording the processing time, average number of hops, and cache delay ratio of each path before and after optimization. Finally, this optimized path is applied to data streams in the data transmission complexity-exceeding region detected in step S3 to execute the real-time filtering process. Using the optimized path configuration, data filtering is re-performed to collect the actual processing time, cache occupancy, and field filtering accuracy of each frame of data under the optimized path. Real-time data filtering information is output, including key indicators such as filter field retention rate, error recognition rate, and processing cycle, which serves as system feedback input to support subsequent filtering strategy updates.

[0032] Preferably, step S1 includes the following steps:

[0033] Step S11: Obtain data transmission log;

[0034] In an embodiment of the present invention, data is automatically collected through the monitoring tools of the transmission network or system. The data transmission log is a detailed record of the system transmission process, covering information such as the transmission status, timestamp, data packet size, and transmission protocol type of each data transmission. The log is directly obtained through the network interface or device monitoring system connected to the transmission system. The network data acquisition module used by the system can seamlessly interact with the data stream in the communication link and capture various types of data packets in the transmission process in real time. The collected data will be stored in a structured format in a local or cloud server database to facilitate subsequent data extraction and analysis. The acquisition accuracy of log data can cover the transmission delay at the millisecond level, ensuring that the transmission details of each data packet can be accurately recorded. The key technology of this step is the real-time acquisition of data logs. The tools used include efficient data acquisition software and hardware device interfaces to ensure that all key transmission data are fully recorded under multiple transmission protocol environments. Through the adaptive hardware interface and acquisition engine, the data is arranged in chronological order to facilitate subsequent log processing.

[0035] Step S12: performing data preprocessing on the data transmission log to obtain transmission log preprocessing data;

[0036] In an embodiment of the present invention, the process of cleaning, filtering and formatting the original data transmission log obtained in step S11 is used to deduplicate the obtained log data and delete duplicate log entries to avoid interference with subsequent analysis. Secondly, invalid data in the log, such as transmission timeouts, log records in the wrong format, etc., are identified and eliminated. In addition, the system will standardize the timestamps of the transmission logs and convert timestamps in different time formats into a standard format for subsequent time series analysis. Missing values ​​in the log data are processed. For missing records caused by system errors or network fluctuations, interpolation or inference methods based on contextual information are used to supplement them. On this basis, all log data will be normalized, and all numerical data will be standardized to a unified magnitude and unit. The preprocessed data is called transmission log preprocessing data, which is the result of optimizing and integrating the original log data, and is convenient for subsequent feature extraction and analysis. The data cleaning algorithms used in this step include outlier detection algorithms, missing value filling algorithms and data normalization tools.

[0037] Step S13: extracting original transmission data features based on transmission log preprocessing data;

[0038] In this embodiment of the present invention, multi-dimensional features of data packets are extracted based on the pre-processed transmission log data obtained in step S12. These features include, but are not limited to, packet size, transmission delay, transmission frequency, destination address, source address, and protocol type. Through detailed analysis of each packet, the system extracts representative features, forming a set of raw transmission data features. Each transmission log entry is analyzed using a dedicated feature extraction algorithm to extract metrics such as transmission time, delay, and flow rate for each packet. To ensure efficient and accurate feature extraction, the system utilizes a sliding window technique to ensure that features within each time period are updated promptly and compared with the data for that time period to obtain dynamic features. During this process, the data feature extraction module combines key information from the pre-processed transmission log data (such as packet start and end timestamps, size, and transmission path) with a customized algorithm to extract corresponding features. These feature extraction relies on time series analysis and statistical analysis of data streams, accurately reflecting the status and characteristics of each data transmission, providing data support for subsequent field structure analysis.

[0039] Step S14: performing field structure parsing processing on the transmission log pre-processing data and the original transmission data features to obtain data frame field parsing structure information.

[0040] In this embodiment of the present invention, field structure parsing is performed on the transmission log preprocessed data in step S12 and the raw transmission data features extracted in step S13 to obtain data frame field parsing structure information. This process analyzes the format of each data frame during data transmission, gradually breaking down the meaning and function of each field within the data frame. The data frame structure is parsed according to the data transmission protocol, identifying the various fields contained in each data packet (such as the header, data, and trailer). For each field, the system parses its specific information type, length, encoding method, and other information. For example, for fields such as the IP address field, port number field, and data content field, the system uses regular expressions and specific protocol format rules to identify each field in the data frame and annotate its location and function. The parsed data frame field information is stored as structured data, forming standard data frame field parsing structure information. This structure information includes not only the basic attributes of the fields, but also the relationships between fields and the flow of field data. This data is used in subsequent steps for dynamic offset detection and anomaly analysis. The entire field structure parsing process relies on protocol parsing algorithms, data matching algorithms, and a regular expression engine, enabling compatibility and processing with a variety of data transmission protocols.

[0041] Preferably, step S14 includes the following steps:

[0042] Step S141: performing field boundary reconstruction processing on the original transmission data features, thereby obtaining transmission field boundary reconstructed data;

[0043] In this embodiment of the present invention, the features of the original transmitted data are processed for field boundary reconstruction. For each data packet, the system analyzes the features of the original transmitted data according to the protocol definition and identifies the boundaries of each field in the data frame. Each field in the data frame has a fixed start and end position during transmission, which is usually marked by a protocol header, identifier, or specific delimiter. The system then extracts the start and end positions of each data field and, in accordance with protocol conventions, reconstructs the boundaries. This boundary reconstruction process includes checking field lengths, correcting field alignment, and verifying field demarcation positions. The binary stream of the transmitted data is read and, based on the protocol format or known data frame identifiers, a byte-based sliding window mechanism is used to match field boundaries in the data stream one by one. The start and end bytes of each field are clearly marked to ensure that the field is not truncated or damaged during transmission. The techniques used in this step include packet decomposition algorithms, boundary identification algorithms, and a byte-stream-based sliding window technique to ensure that each field in the transmitted data is accurately identified and defined. After the field boundary reconstruction process, the system will output a complete transmission field boundary reconstruction data, which contains the accurate boundary information of all fields and stores this boundary information in a standardized format.

[0044] Step S142: constructing a transmission field logical boundary map according to the transmission field boundary reconstruction data;

[0045] In this embodiment of the present invention, data is reconstructed based on the transmission field boundaries obtained in step S141 to construct a logical boundary map of the transmission fields. This map aims to form a mapping and association structure between fields by analyzing the relationships and dependencies between them. Specifically, the system performs a logical analysis of each field's boundaries, identifies the relationships between them, and represents them in a graph. When constructing the logical boundary map, an algorithm automatically identifies the field types in the transmitted data and determines which fields are required core fields and which are auxiliary fields. Based on the characteristics of these fields, the system determines the hierarchical relationships and data flow between them. Next, the system constructs a data flow graph, in which each node represents a data field, and the edges between nodes represent the logical relationships and data flow between fields. This graph is arranged according to the fields' functions and transmission order, ensuring that the logical position of each field and the data flow path are accurately represented. Relying on directed graph algorithms from graph theory, the logical boundaries of each field are accurately represented by establishing and optimizing nodes and edges. The construction process of this graph ensures that the relationship between transmission fields is clearly presented, and subsequent protocol mode regularization can also be analyzed more accurately based on this graph.

[0046] Step S143: performing protocol mode regularization processing according to the transmission field logical boundary map, thereby obtaining field protocol mode characteristics;

[0047] In this embodiment of the present invention, protocol pattern regularization is performed based on the transmission field logical boundary map obtained in step S142. The purpose of protocol pattern regularization is to unify and optimize the usage patterns and change rules of each field during data transmission to obtain structured field protocol pattern features. Protocol pattern features in data transmission are identified based on the field relationships in the logical boundary map. For example, some fields exhibit a fixed order or pattern across multiple data packets, while others change frequently. The system generates protocol pattern features based on these patterns. The protocol pattern features for each field include information such as the field's frequency of occurrence, data value range, transmission latency, and field dependencies. By scanning multiple transmission logs and data packets, the variation pattern of each field is extracted. For example, some fields remain constant over a period of time, while others change frequently as data volume increases. The system automatically identifies and regularizes field protocol pattern features from the transmitted data using frequency statistics, pattern recognition, and data flow analysis techniques. This process utilizes statistical analysis based on pattern recognition algorithms and data flow identification techniques to ensure that the extracted field protocol pattern features reflect the actual patterns of data transmission.

[0048] Step S144: Count the frequency of field historical value changes based on the field protocol pattern characteristics; mark the field change activity level when the field historical value change frequency is high;

[0049] In this embodiment of the present invention, based on the field protocol pattern characteristics obtained in step S143, the frequency of historical value changes for each field is counted. Based on this frequency, the field's change activity is marked. For each field, the system analyzes its historical changes during transmission and calculates its change frequency. This frequency is calculated based on the field values ​​in the transmission log and is determined by comparing whether the field values ​​in adjacent data packets have changed. The field's change activity is defined based on the frequency of historical value changes. If a field changes frequently across multiple data packets, its change activity is marked as "high"; if the field value remains largely unchanged during transmission, its change activity is marked as "low." This activity marking is crucial for subsequent field anomaly detection and data stream filtering, helping the system focus on frequently changing fields and prioritize them. Using data statistics and change detection algorithms, the change frequency is calculated by comparing field history records. Field activity is assessed using a standardized frequency distribution algorithm. The system generates a corresponding change activity mark for each field.

[0050] Step S145: collecting field value range distribution according to field protocol pattern characteristics;

[0051] In this embodiment of the present invention, based on the field protocol pattern characteristics in step S143, the value range distribution of each field is collected. A field range refers to the range of all possible values ​​a field value can take within a period of time. During this process, the system analyzes all records for each field in the historical transmission log to extract the field's maximum and minimum values, as well as its overall distribution. To collect the field range distribution, the system scans the entire time window of the transmitted data and identifies all values ​​for each field. Next, by statistically analyzing these values, the system generates a value range distribution graph for the field, showing the range of variation and distribution trends of the field in the historical data. For example, some fields fluctuate only within a very small range, while others experience larger fluctuations. By analyzing this data, a value range distribution model is established for each field. This process utilizes data statistics and distribution analysis tools, which analyze the field's value range using precise statistical methods to ensure that the value range distribution of each field truly reflects its changing characteristics during data transmission.

[0052] Step S146: Mapping the field value range distribution and the field change activity to obtain field value range mapping data;

[0053] In an embodiment of the present invention, the field value range distribution obtained in step S145 and the field change activity in step S144 are mapped to obtain field value range mapping data, which is then combined with the field's change activity based on the value range distribution of each field. The field's value range distribution range and change activity are two important characteristics that can reflect the stability and frequency of the field. By establishing a mapping relationship, the field's value range is combined with its change activity. If a field has a high change activity, the field's value range will be affected by greater changes; conversely, if the field has a low change activity, its value range is relatively stable. Through this mapping, the system can generate a comprehensive value range mapping data for each field, reflecting the field's fluctuation characteristics and its activity level. The data processing algorithms used in the mapping process include a frequency-based weighted mapping algorithm and a dynamic adjustment algorithm to ensure the completion of the mapping relationship construction.

[0054] Step S147: Perform field cross-indexing based on the transmission log preprocessing data and the field value range mapping data to obtain transmission field location information;

[0055] In this embodiment of the present invention, field cross-indexing is performed based on the transmission log preprocessing data in step S12 and the field value range mapping data obtained in step S146, aiming to accurately locate the position of each field in the transmitted data. This cross-indexing process is based on two primary data sources: field feature information in the transmission log and field value range mapping data. The field value range mapping data provides the value range and change activity of each field, while the transmission log preprocessing data contains the specific values ​​of each field at different time points. Field location information is generated by matching the field features in the preprocessed data with the value range mapping information. The system then matches the field value range mapping data with the corresponding transmission log based on the field's timestamp in the transmitted data and auxiliary information (such as packet identifiers and field order). Through this matching, the system can determine the specific location of each field in the transmitted data, including the field's starting and ending positions, as well as its relative order within the data frame. This cross-indexing process relies on an efficient indexing algorithm that uses timestamps, field identifiers, and mapping information for multi-dimensional matching to ensure that each field in the transmitted data can be accurately located.

[0056] Step S148: Perform field structure parsing processing according to the transmission field positioning information and the field value range mapping data to obtain data frame field parsing structure information.

[0057] In this embodiment of the present invention, field structure parsing is performed based on the transmitted field location information obtained in step S147 and the field value range mapping data in step S146. The goal of this step is to combine the field location information with the field value range characteristics to parse each field in the data frame, thereby obtaining clear field structure information. The transmitted field location information is used to determine the exact location of each field in the data frame. Combined with this location information, the system then parses the field's specific value and its function within the data frame. For example, using the field value range mapping data, the system can identify the value ranges, common variation patterns, and dependencies of certain fields, thereby further decoding the field's content. This parsing process is primarily divided into two phases. The first phase extracts the raw data of each field based on the field location information, including the field's value and how it changes during data transmission. The second phase decodes and structures the extracted field data in conjunction with the field value range mapping data. This process ensures that the content of each field is accurately identified and processed, and the field's function and meaning are accurately restored. The technologies used in field structure parsing include algorithms based on data parsing and decoding technology based on field dependencies to ensure the efficiency and accuracy of field parsing. The system outputs a data frame field parsing structure information containing all field parsing results for subsequent anomaly detection, data analysis, and protocol mode optimization.

[0058] Preferably, the calculation of the transmission field time gradient change rate in step S2 includes:

[0059] According to the data frame field parsing structure information, the transmission field timing timestamp is collected;

[0060] In an embodiment of the present invention, the sequential timestamps of all data fields are extracted from the data frame field parsing structure information. The sequential timestamp is the data transmission time point recorded during the data transmission process, reflecting the time sequence of each data field during the transmission process. Specifically, by traversing each field in the data frame, the timestamp information of each field is extracted from the parsing structure. The data frame field parsing structure includes information such as the type, length, and position of the field. The system identifies the field through this information and extracts its corresponding timestamp data. The sequential timestamp of each field records the specific time point of the field during the transmission process. These time points are obtained from the time synchronization protocol or automatically generated by the device.

[0061] Collect the transmission field timing timestamps exceeding 60ms during the transmission field abnormal missing period;

[0062] In an embodiment of the present invention, the time difference between adjacent field timestamps is calculated based on the transmission field timing timestamp obtained in the previous step. When the time difference between the timestamps exceeds 60 milliseconds, the system marks this period of time as an "abnormal missing period". The timing anomaly of the transmission field is caused by network delay, packet loss or transmission anomaly, so a time difference exceeding 60 milliseconds indicates that there is an abnormality in data transmission. The system will mark the start and end time of these missing periods and provide a basis for subsequent field timing reconstruction. For example, when the difference between timestamps T1 and T2 |T2-T1|>60ms, this period of time is identified as an abnormal missing period of the transmission field, and corresponding supplementation and repair are required afterwards.

[0063] Perform field timing reconstruction based on the abnormal missing period of the transmission field and the transmission field timing timestamp to obtain the transmission field time trajectory sequence data;

[0064] In an embodiment of the present invention, the system performs time series reconstruction for time periods identified as abnormally missing. The purpose of time series reconstruction is to recover the field time data that is missing due to transmission anomalies. The reconstruction method uses interpolation techniques (such as linear interpolation or spline interpolation) to fill in the missing field data. For example, assume that the time series timestamps of transmission fields A and B are T1 and T2 respectively, and the time stamp difference between them exceeds 60 milliseconds. Through the interpolation algorithm, the system generates a new time point T_new based on the changes in the field values ​​between T1 and T2, interpolates the data value of this time point, and thus fills in the missing part. The time series data of all transmission fields will form a complete time trajectory sequence to ensure the continuity of time.

[0065] Measure the difference amplitude of adjacent field values ​​based on the transmission field time trajectory series data;

[0066] In an embodiment of the present invention, the value difference amplitude of adjacent fields is measured using time trajectory sequence data. The value difference amplitude refers to the absolute amplitude of the numerical change between adjacent fields in the time series. If the field value is V1 at time T1 and the field value is V2 at time T2, the value difference amplitude between them is |V2-V1|. The system calculates the difference amplitude of all adjacent fields in sequence and records it in the data structure. During the calculation, all pairs of fields are compared, and it is ensured that the value difference of each pair of adjacent fields is counted and processed. These difference amplitudes provide the basis for the subsequent calculation of jump intensity and rhythm fluctuation characteristics.

[0067] When the difference between adjacent field values ​​exceeds 0.1, the transition strength of adjacent field values ​​is counted;

[0068] In an embodiment of the present invention, the amplitude of the difference in the values ​​of adjacent fields is further analyzed. When the amplitude exceeds 0.1, the system considers that a significant jump has occurred in the field value. Jump intensity refers to the intensity of the change in the value of adjacent fields over a period of time. Specifically, the system counts all cases where the amplitude of the difference between adjacent fields exceeds 0.1, and calculates the number and intensity of these jumps. The jump intensity includes not only the size of the amplitude, but also takes into account the speed at which the field value changes. For example, if two fields change by a large value (more than 0.1) in a short period of time, the jump intensity during this period will be marked as high. The statistics of jump intensity will help in the subsequent analysis of fluctuation characteristics.

[0069] Detect the fluctuation characteristics of the transmission field change rhythm based on the jump intensity of the adjacent field values ​​and the difference amplitude of the adjacent field values;

[0070] In an embodiment of the present invention, the fluctuation characteristics of the transmission field are analyzed in combination with the previously obtained jump strength and field value difference amplitude. The fluctuation characteristics of the transmission field generally reflect the rhythm and fluctuation pattern of the field changes. By statistically analyzing the jump strength and field difference amplitude, it is detected which fields change more frequently and which fields change more steadily. For example, if the difference amplitude between adjacent fields is large and changes frequently within a certain time period, it is determined that the change rhythm of the transmission field within this period has strong fluctuations. At this time, a frequency analysis method (such as fast Fourier transform (FFT)) is used to extract the fluctuation characteristics to determine the rhythm characteristics of the field changes, thereby providing a basis for subsequent gradient calculation.

[0071] The transmission field time gradient change rate is calculated based on the transmission field change rhythm fluctuation characteristics and the transmission field time trajectory series data.

[0072] In this embodiment of the present invention, the fluctuation characteristics of the field's change rhythm are extracted based on the jump intensity and difference amplitude of the transmitted field. This fluctuation characteristic generally represents the intensity or frequency of field changes within a certain period of time. When calculating the time gradient change rate, the system uses this fluctuation characteristic to analyze the data time trajectory series and identify time periods with frequent changes. By determining the field's change trends in different time periods, the system can identify which fields have experienced significant fluctuations in a short period of time, thereby reflecting abnormalities or instability in the transmission process. The time points of all fields are sorted according to the time trajectory series to ensure the consistency of the time series. The system then compares the field values ​​at each time point and analyzes the change amplitude between adjacent time points. The system records the field changes between each pair of adjacent time points, ensuring accurate tracking of field changes at different time points. In this way, the system can analyze the time change rate of each field in detail. Based on the time series data of the transmitted field, the changes in adjacent field values ​​within each time period are statistically analyzed. For each field's change between different time points, the system calculates its change rate (i.e., gradient change rate). If a field experiences a significant change between adjacent time points, the system will deem its time gradient change to be large. In other words, the system determines how quickly a field changes within a period based on the difference in value between adjacent field time points and their duration. Each time period is marked based on the magnitude of the rate of change, and the intensity of each field change is statistically analyzed. A high intensity of change indicates that the field changes frequently and dramatically within that period, indicating the presence of instability or abnormal fluctuations in the data transmission process. By comprehensively analyzing the intensity of change across multiple time periods, the system can accurately grasp the stability of the data transmission process. Based on the above analysis steps, a sequence of data containing the time gradient change rate is generated. This data sequence reflects the rate of field change in each time period throughout the entire data transmission process and assists in subsequent real-time filtering decisions. Based on the magnitude of the change rate, the data transmission strategy can be further adjusted to optimize data flow control or identify potential anomalies.

[0073] Preferably, the determination of the dynamic offset status of the transmission field in step S2 includes:

[0074] Perform 100 Hz multi-scale sliding synchronous slicing processing on the transmission field time gradient change rate data to obtain field change structure slice data;

[0075] In an embodiment of the present invention, the acquired transmission field time gradient change rate data is sliced. Specifically, a sampling frequency of 100 Hz is used to slice the time gradient change rate data in units of fixed time length. This slicing process helps to observe the dynamic changes of the transmission field in different time periods from a finer time dimension. During the slicing process, the data length of each segment will correspond to the time window at a sampling frequency of 100 Hz, ensuring that each time window contains relatively independent time point information. After slicing, the system will obtain a series of structural data representing field changes in different time windows, which will become the basis for subsequent analysis. Each data slice records the time gradient change rate of the transmission field during the period, presenting the change pattern and regularity of the data during the period. This method helps to more accurately analyze and identify the fluctuation of the transmission field in the time domain, thereby improving the accuracy of data processing.

[0076] Dividing the transmission field time gradient change rate data into multiple period field structure data;

[0077] In an embodiment of the present invention, based on the sliced ​​transmission field time gradient change rate data, the system will further divide it into time periods. According to different time scales, the data will be divided into multiple time periods, and the field changes in each time period will be analyzed separately. The basis for division is usually the periodic characteristics of field changes and the frequency of data changes. Within each divided time period, the system will analyze the duration and characteristics of the field changes to ensure that the field changes within each time period are coherent. This division process can ensure comprehensive monitoring of the dynamic changes of the transmission field from different time granularities, especially the understanding of field change trends over a long time span. At the same time, time period division can help the system capture subtle fluctuations in the data transmission process at a fine granularity and identify potential change points.

[0078] Identify the location of abnormal mutation amplitude of multi-period fields when the multi-period field structure data exceeds ±0.2 / ms;

[0079] In an embodiment of the present invention, once the division is completed, the system will identify abnormal mutation amplitudes based on multi-time period field structure data. The system will traverse the field data in each time period to check whether there is any data change with an amplitude change exceeding ±0.2 / ms. This amplitude value is used as a threshold to identify abnormal mutations. If the change in the field within a certain time period exceeds this amplitude value, the system will consider this change to be an abnormal mutation. Through this step, the system can automatically identify mutations that occur in certain time periods and mark the locations of these abnormal fluctuations. The detection of abnormal mutations usually relies on strict threshold settings to ensure that major fluctuations in data transmission can be captured efficiently and accurately, thereby providing a basis for further analysis.

[0080] Extract the starting point position of the field abnormal mutation amplitude based on the position of the field abnormal mutation amplitude in multiple time periods;

[0081] In this embodiment of the present invention, after identifying the location of abnormal mutation amplitudes, the system further analyzes the starting point of each abnormal mutation. Specifically, the system scans the data points before and after each abnormal mutation location to identify the specific starting point of the mutation. This process ensures that the system can accurately locate the specific starting moment of the field change, not just the instant of the mutation. Extracting the starting point of abnormal mutation amplitudes helps to more accurately identify and demarcate key data events in real-time monitoring, especially at key nodes in the transmission process, facilitating subsequent diagnosis and analysis.

[0082] Mark the starting point of the field abnormal mutation amplitude to obtain the field abnormal mutation point marking data;

[0083] In an embodiment of the present invention, after extracting the starting points of abnormal mutations, the system will mark these starting points. The marking method is usually to assign a unique identifier to each abnormal mutation point and record it together with the corresponding timestamp, amplitude value and other information. The marked data will become an important basis for subsequent data analysis and can provide an important basis for the system's real-time detection, data filtering, and abnormal alarms. Through this marking process, the system can clearly record the starting point of each mutation in the entire data stream, facilitating subsequent processing and data tracing. The marked data will help to monitor any abnormal fluctuations in the transmission process in real time and can be correlated with the transmission feature data for analysis.

[0084] According to the field change structure slice data and field abnormal mutation point marking data, the field structure trajectory change situation is collected;

[0085] In this embodiment of the present invention, the system combines field change structure slice data with marked abnormal mutation point data to analyze changes in field structure trajectories. By comparing field changes in the slice data with mutation points in the marked data, the system tracks field change trends and determines whether changes occur around mutation points. By comparing field structure changes in different time slices, it is possible to determine data stability and whether there is persistent drift. The core of this process lies in further identifying the dynamic drift of fields through detailed analysis of field trajectories around abnormal mutation points.

[0086] The dynamic offset status of the transmission field is determined according to the changes in the field structure trajectory.

[0087] In an embodiment of the present invention, the dynamic offset status of the transmission field is comprehensively judged based on the collected field trajectory offset strength and field structure trajectory changes. The field trajectory offset strength reflects the change amplitude and stability of the field within a certain time period, while the field structure trajectory change reflects the fluctuation trend of the field in time series. By comprehensively considering the data of both, the system can clearly determine whether there is a large offset or abnormal fluctuation during the data transmission process. If the system detects abnormal offset, it will further mark these time periods and promptly process them or take corresponding abnormal response measures. This process ensures dynamic monitoring of the transmitted data and can effectively avoid transmission errors or system instability caused by data offset. Through the processing of this step, the system can clearly understand the dynamic offset status of the transmission field in different time periods, thereby providing an accurate basis for subsequent data filtering, error detection, or optimization strategies.

[0088] Preferably, determining the multi-dimensional abnormal coupling condition of data transmission in step S2 includes:

[0089] Detecting the parallel offset of multiple fields in transmission according to the dynamic offset of the transmission field;

[0090] In an embodiment of the present invention, based on the dynamic offset status of the transmission field determined in the previous step, the parallel offset of multiple fields during the transmission process is checked. The offset status of each transmission field is based on the dynamic performance of its time gradient change rate. By comparing the offset patterns of different transmission fields, the system can detect whether there is a synchronous offset phenomenon in multiple fields. Specifically, the system will analyze the offset intensity and time changes of each field, paying special attention to whether there are consistent change trends or asynchronous fluctuations in multiple fields. Using a multi-dimensional data analysis tool, the system converts the offset data of each field into a comparable time series, and detects the offset of each field through parallel processing. The parallel situation of these field offsets reflects the correlation between the fields during the data transmission process and its impact on the stability of the system. The purpose of this step is to preliminarily identify the offset coupling relationship between the fields, and provide a basis for the subsequent analysis of the disorder of the data flow protocol structure.

[0091] Determine the degree of disorder in the data stream protocol structure based on the parallel offset of multiple transmitted fields;

[0092] In an embodiment of the present invention, the degree of disorder of the data stream protocol structure will be further analyzed based on the parallel offset of multiple fields in transmission. When the offset phenomenon of multiple fields is highly correlated or fluctuates violently, the protocol structure of the data stream will be affected and disorder will occur. Specifically, the system will determine whether there is a synchronization error between multiple fields in the data stream, or whether an asynchronous offset pattern occurs that causes the protocol structure to be unable to execute normally through analysis of the parallel offset situation. By comparing the parallel offset data, the system can calculate the degree of disorder index, such as the disorder amplitude, the offset rate difference, etc., to evaluate the stability of the protocol structure. When the degree of disorder is high, it means that the normal transmission of the data stream protocol structure has been seriously disturbed, and the system needs to take corresponding measures to adjust the transmission protocol to ensure the integrity and consistency of the data.

[0093] Determine the data transmission protocol drift based on the disorder of the data stream protocol structure and the drift of multiple fields transmitted in parallel;

[0094] In this embodiment of the present invention, after analyzing the degree of disorder in the data stream protocol structure, the system further determines the drift of the data transmission protocol based on the degree of disorder and the parallel drift of multiple transmission fields. Data transmission protocol drift refers to the gradual deviation of the protocol from its expected state during data transmission, resulting in errors or distortion in data transmission. By combining the degree of disorder with the parallel drift, the system can determine whether the transmission protocol is drifting. Time series analysis is used to compare data stream protocols between different fields to determine whether they exhibit gradual drift over time. This analysis dynamically monitors the protocol synchronization status between fields and utilizes a drift detection algorithm (such as a smoothed difference method) to identify any tendency for the protocol to gradually lose synchronization during transmission. The system outputs the data transmission protocol drift status as a basis for assessing data stability.

[0095] Detecting the severity of data transmission rate changes based on the time gradient change rate of the transmission field;

[0096] In an embodiment of the present invention, the time gradient change rate data of the transmission field is used to detect the severity of changes in the data transmission rate. Specifically, the system analyzes the time gradient change rate of each transmission field and uses this data to infer the fluctuation amplitude of the transmission rate. Through high-frequency analysis of the time gradient change rate, the system identifies periods of drastic changes in the transmission rate, especially sudden rate fluctuations. When detecting drastic changes in the data transmission rate, the system performs a fine-grained analysis of the time gradient of the field and uses sliding window technology to capture drastic fluctuations in the transmission rate. In particular, when certain key points or sudden events occur, drastic fluctuations in the data rate will be clearly marked. Through this analysis, the system can determine whether there is a sudden bandwidth bottleneck or overload phenomenon in the data transmission, thereby providing key information for the statistics of synchronization desynchronization.

[0097] Statistical transmission synchronization mismatch based on the severity of data transmission rate changes;

[0098] In an embodiment of the present invention, data transmission desynchronization is further statistically analyzed and evaluated based on the severity of previously detected data transmission rate changes. Desynchronization refers to a loss of time synchronization between different data streams or multi-field data streams, resulting in data transmission errors or system instability. The system compares the transmission rate changes of different fields to identify those with drastic rate changes that affect synchronization. The amplitude of the rate fluctuation is used as a parameter to calculate the degree of desynchronization. When the transmission rates of multiple data streams or fields change dramatically, the system determines whether this will lead to desynchronization. If the transmission rate change of certain fields exceeds a certain threshold and the fields fail to adjust synchronously, the system will mark the period of time when desynchronization occurred and record its specific severity.

[0099] The multi-dimensional abnormal coupling of data transmission is determined based on the data transmission synchronization imbalance and data transmission protocol drift.

[0100] In an embodiment of the present invention, based on the data transmission synchronization misalignment and data transmission protocol drift obtained in the previous steps, a comprehensive analysis of the multi-dimensional abnormal coupling of data transmission is performed. Multi-dimensional abnormal coupling refers to the interaction of multiple abnormal factors during the data transmission process, resulting in a decrease in overall transmission performance or data transmission errors. By combining synchronization misalignment with protocol drift, the system identifies abnormal coupling points in different transmission dimensions, analyzes the relationship between synchronization misalignment and protocol drift, and detects whether there is a phenomenon in which the misalignment of certain data streams interacts with the drift of the protocol structure. These coupling phenomena lead to serious deviations in data transmission, affecting the integrity and accuracy of the data. By detecting these abnormal coupling phenomena, the system can effectively assess the potential risks in the data transmission process, make timely adjustments and optimizations, and ensure the stability of data transmission. Through this comprehensive analysis, the system can output the multi-dimensional abnormal coupling of data transmission, providing a basis for subsequent real-time data filtering and optimization processing.

[0101] Preferably, step S3 includes the following steps:

[0102] Step S31: detecting abnormal dilution of transmission data according to abnormal multi-dimensional coupling of data transmission;

[0103] In this embodiment of the present invention, based on the previously analyzed abnormal multidimensional coupling of data transmission, the system detects whether the transmitted data is experiencing abnormal dilution. Abnormal data dilution refers to the dilution or loss of critical information in a data stream due to the effects of abnormal multidimensional coupling, resulting in a decrease in data transmission quality. Specifically, the system identifies the diluted portions of the data stream by analyzing the signal variation patterns caused by abnormal multidimensional coupling. The transmitted data of each field is monitored individually and compared with normal transmission patterns. In the presence of abnormal multidimensional coupling, the data stream will experience signal attenuation or distortion. By calculating changes in signal strength, data transmission rate, and inter-field synchronization, the system can assess whether information dilution exists in the data stream. To this end, the system incorporates a threshold detection algorithm based on time series changes. When the degree of signal attenuation exceeds a preset threshold, it determines that abnormal dilution has occurred. Using time domain analysis tools, a sliding window is used to perform fine-grained temporal analysis of the data to accurately identify the time window where the dilution occurs and record the relevant data features. Through this process, the system can determine the degree of dilution and its impact range in each transmitted data stream, providing basic data for subsequent steps.

[0104] Step S32: determining the transmission data interference amplification degree based on the abnormal dilution of the transmission data;

[0105] In an embodiment of the present invention, after identifying abnormal data dilution, the system further analyzes and determines the degree of interference amplification. The degree of interference amplification refers to whether the impact of external or internal interference on the data stream is amplified under the influence of multi-dimensional abnormal coupling, thereby affecting the stability and accuracy of the entire transmission process. Specifically, based on abnormal dilution, the system detects whether interference amplification has occurred in the data stream and uses this information to assess the transmission impact of the interference. By combining interference source monitoring and data stream synchronization analysis techniques with the noise and error characteristics of the transmitted data, the system analyzes the amplification effect of external interference during transmission. For identified abnormal dilution, the system further compares noise variations across different data streams to detect the amplification effect of the interference source during transmission. For example, external noise can cause signal amplitude amplification or increased latency, leading to increased data transmission errors. This effectively isolates the impact of noise interference on data streams. Using differentiated analysis tools, the system can clearly identify data streams experiencing interference amplification and quantify the extent of its impact. Based on the detection results, the system outputs an interference amplification indicator, providing reference data for subsequent complexity prediction and chip overload detection.

[0106] Step S33: predicting a data transmission complexity exceeding limit based on the degree of interference amplification of the transmitted data and the abnormal dilution of the transmitted data;

[0107] In this embodiment of the present invention, after analyzing the interference amplification level of the transmitted data, the system, combined with the anomaly dilution, predicts the probability of exceeding the data transmission complexity limit. Exceeding the data transmission complexity limit refers to the situation where, due to the combined effects of anomaly dilution and interference amplification, the system's computing power exceeds its designed processing capacity during data transmission, resulting in inefficient data processing or transmission. This typically manifests as increased latency during data stream processing and slower system response. Combined with the anomaly dilution characteristics, the system identifies which data streams are affected by the anomaly, resulting in a significant increase in the amount of information within the data stream. Next, based on the interference amplification results and incorporating metrics such as the amount of transmitted data and data transmission speed, the system predicts whether the processing complexity of the data stream will exceed a predetermined value. Using a complexity analysis model based on data stream characteristics, combined with the anomaly dilution and interference amplification data obtained in the previous steps, the system estimates whether the computing resources required for data processing exceed a predetermined load limit. Specifically, the system sets multiple complexity metrics, such as the computational difficulty of the data packet and the processing latency of the data stream, and combines these metrics to predict the limit. If the processing complexity limit is exceeded during data transmission, the system will issue an alarm and prepare to proceed to the next step, overload detection.

[0108] Step S34: detecting the overload condition of the data filtering chip structure according to the data transmission complexity exceeding the limit.

[0109] In an embodiment of the present invention, when the data transmission complexity prediction results indicate an overload condition, the system will detect whether the data filtering chip's structure is overloaded. Data filtering chip overload occurs when the chip processes excessive data or overly complex data streams, exceeding its design capabilities, resulting in a decrease in processing power or even a crash. Specifically, based on the complexity overload prediction results, the system will check the data filtering chip's load and determine whether resource scheduling or increased computing power is necessary. During this step, the system monitors the chip's operating status in real time, recording metrics such as processing time, power consumption, and temperature. Combined with previously predicted complexity data, the system assesses the chip's load. If the chip's load is detected to exceed a predetermined safety range, the system will automatically adjust resource allocation, implementing strategies such as alleviating computing pressure, reducing data processing priority, and adjusting data flow transmission modes to avoid chip overload. Dynamic load monitoring technology is employed to collect and analyze real-time chip data to determine whether the chip's computing power is sufficient to handle the upcoming data processing task. If the chip shows signs of overload, the system will promptly issue an alarm and initiate an emergency response plan to ensure the stability and accuracy of data filtering processing. Through these specific steps, the system can effectively monitor and adjust problems such as abnormal dilution, interference amplification, and complexity exceeding limits in data transmission, ensuring that the data filtering chip will not crash or experience performance degradation under high load conditions, thereby ensuring the stable operation of the entire data transmission system.

[0110] It is particularly important that step S31 includes the following steps:

[0111] Step S311: determining the transmission link layer congestion situation based on the multi-dimensional abnormal coupling situation of data transmission;

[0112] In this embodiment of the present invention, at least five parameters are synchronously extracted from the physical link monitoring unit: average packet delay, instantaneous bandwidth utilization, transmission window retransmission rate, link layer error correction count, and node cache queue length. These parameters are collectively transmitted to the data anomaly analysis module via a high-speed cache bus. The module compares the deviation amplitude and synchronous change rate of each parameter within a fixed period (e.g., every 100ms) to establish an exception coupling vector matrix (EAM). This matrix represents parameter channels in rows and records the trend of change within the period in columns. If three or more parameters in the matrix deviate from their steady-state operating range in the same direction within the period (e.g., delay increases by more than 10%, bandwidth utilization increases by 15%, or retransmission rate exceeds 2%), a multi-dimensional abnormal coupling condition is identified. Coupling factor data in the EAM matrix with an impact level greater than a preset threshold (e.g., 0.7) is then mapped to the link congestion identification module. The link congestion identification module combines the abnormal coupling factor data, cache queue length, and link error correction frequency for attribution analysis. It ultimately outputs a link congestion condition tag (LCCT), which serves as input for data transmission jitter detection in the next step.

[0113] Step S312: Detecting the degree of data transmission jitter based on the congestion condition of the transmission link layer;

[0114] In this embodiment of the present invention, the LCCT tag obtained in step S311 is introduced as an input parameter into the transmission jitter detection module, and periodically generated timestamp data at the network layer is synchronously collected. This timestamp data is the system clock recorded when the packet is sent and received, with a time resolution of microseconds. By comparing the delay fluctuations between multiple packets, namely calculating the delay difference Δt between adjacent packets, a Jitter Time Series (JTS) is constructed. The congestion level in the LCCT is then jointly mapped with the fluctuation rate in the JTS. A sliding window method (with a fixed window size of 20 packets) is used to extract the maximum, minimum, and average Δt values ​​within each window. This is used to construct a Jitter Magnitude Index (JMI). If the JMI exceeds a set threshold (e.g., 20ms), the link is considered to be in a significant jitter state, and the JMI output serves as input for the next step of transmission anomaly compression identification.

[0115] Step S313: Identify abnormal transmission compression based on the transmission link layer congestion and data transmission jitter.

[0116] In this embodiment of the present invention, the LCCT link congestion label obtained in step S311 and the JMI jitter index obtained in step S312 are read and synchronized by a data fusion module to construct a compression identification trigger condition set (CCT). The CCT describes the concurrent state combination of the link congestion level and the jitter intensity level in a logical matrix format. For example, when the LCCT level is 2 (moderate congestion) and the JMI exceeds 20ms, the corresponding CCT state flag is 1 (compression identification is triggered). When the CCT state flag is triggered, the compression identification module is invoked to analyze the payload content variation between packets in the data stream. Specifically, the rate of change of the payload data volume of consecutive packets is compared. If the rate of change is consistently below 10% within a sliding window (10 packets) and the rate of change of the compression coefficient is less than 2%, abnormal compression is determined to exist on the current link. Finally, the abnormal transmission compression condition status flag (TCCS) is output as input for the next step.

[0117] Step S314: Detecting the synchronization imbalance of the transmission protocol layer based on the multi-dimensional abnormal coupling of data transmission;

[0118] In this embodiment of the present invention, the abnormal coupling vector matrix (EAM) generated in step S311 is called and combined with the handshake frame interval data (such as the TCP three-way handshake SYN and ACK frame delays), protocol retransmission counts, and protocol state transition frequency recorded by the protocol control module to construct a protocol layer synchronization fluctuation vector (PSV). The covariance of each coupling factor in the EAM and each type of protocol timing indicator in the PSV is calculated. If the covariance between a coupling factor and multiple protocol indicators in the PSV exceeds a set threshold (for example, 0.8), it is identified as an initial sign of protocol layer synchronization imbalance. Furthermore, the standard deviation of the delay between each state transition is extracted from the protocol stack state transition records. If this standard deviation gradually increases and exceeds 20ms in consecutive cycles, a synchronization imbalance is determined. A synchronization imbalance flag (PDS) is output and transmitted to the next step.

[0119] Step S315: Identify the transmission abnormality protocol reconstruction concealment status based on the transmission protocol layer synchronization imbalance;

[0120] In this embodiment of the present invention, the synchronization imbalance marker (PDS) generated in step S314 is read, and the protocol reconstruction monitoring module is accessed to obtain the insertion frequency of reconstructed protocol frames (such as TCP Option frames), the magnitude of changes in the packet header segment, and the sequence number misalignment ratio during the reconstruction process. This reconstruction feature data is time-windowed using the time period marked in the PDS to extract the differences in protocol frame behavior before and after the synchronization imbalance, and a protocol reconstruction exception vector (PREV) is constructed. If the protocol frame insertion frequency in the PREV jumps by more than three times the normal value in a short period of time, or if the header segment structure undergoes changes in more than three types of fields, and the sequence number misalignment ratio exceeds 0.15, a protocol reconstruction masking state is considered to have occurred. This state is ultimately encapsulated as a protocol reconstruction masking state label (PRCM) and output for use in the next step.

[0121] Step S316: Detecting abnormal dilution of the transmission data according to the transmission abnormality protocol reconstruction concealment condition and the transmission abnormality compression condition.

[0122] In this embodiment of the present invention, the PRCM protocol masking tag output from step S315 and the TCCS compression status flag output from step S313 are read as input to the data dilution detection module. This module first synchronizes the time axis of the two input signals to extract their combined time window. Within this time window, it analyzes the field fill rate (i.e., the proportion of non-zero valid data) and the control field change rate of the packet payload. The field entropy change difference ΔH (calculated by the magnitude of the entropy change of the fields within the packet) is used to assess the level of data dilution. If ΔH is consistently less than a specified threshold (e.g., 0.05) and the data fill rate decreases by more than 30%, data dilution is considered to have occurred. The module also detects whether consecutive packets have had valid fields replaced by protocol reconstruction fields (e.g., ACK replacing the data field) as a factor in determining protocol masking. Finally, the module outputs a DAS (Data Attenuation Status) tag as the final output parameter of the technical process chain.

[0123] Preferably, step S34 includes the following steps:

[0124] Step S341: Identify the growth of data parsing fine-grainedness based on the data transmission complexity exceeding the limit;

[0125] In embodiments of the present invention, based on the previously predicted data transmission complexity exceeding a limit, the system further identifies instances of increasing granularity during data parsing. This refers to the gradual finer granularity of data parsing due to increased data complexity during data processing, resulting in the system requiring more computing resources to perform more detailed analysis. The data stream is monitored, with particular attention paid to the increase in data fields and changes in packet size. By continuously comparing the parsing depth of each packet in the data stream, the system can identify whether data parsing is becoming more granular. For example, if each field in the data stream contains an increasing number of parameters or requires more complex rules for data parsing, the system determines that data parsing granularity is increasing. The system uses a timed analysis algorithm to compare the real-time parsing of data transmissions against a preset complexity threshold. When the data parsing granularity exceeds the expected range, the system identifies an increase in granularity. Through real-time monitoring and data feature extraction, the system can clearly identify the trend of increasing granularity during data parsing and use this as a basis for predicting subsequent resource utilization and chip overload conditions.

[0126] Step S342: Detecting the growth trend of data filtering chip resource usage based on the fine-grained growth of data analysis;

[0127] In this embodiment of the present invention, based on the growth in parsing granularity obtained in the preceding steps, the system further monitors the resource usage growth trend of the data filtering chip. Increased granularity means increasing processing requirements for each data packet, leading to a gradual increase in the resource usage of the filtering chip. To this end, the system monitors chip resource usage in real time, including memory, computing unit, and bandwidth usage. By comparing chip resource usage over different time periods, the system determines the resource usage trend resulting from the increase in data parsing granularity. The system monitors the processing time and computing resources consumed by each data packet, and calculates resource consumption within each time period. Subsequently, based on the growth trend in data parsing granularity, the system calculates the change in resource consumption of the filtering chip as data complexity increases. During this monitoring process, the system compares the resource usage growth trend with the preset maximum resource capacity. If the resource consumption rate exceeds the expected growth range, the system issues a resource usage alert. This information provides the basis for subsequent chip load warnings and resource scheduling.

[0128] Step S343: Identify the growth trend of data filtering requests based on the data transmission complexity exceeding the limit;

[0129] In an embodiment of the present invention, based on the aforementioned data transmission complexity exceeding limit, the growth trend of data filtering requests is further identified. The growth trend of data filtering requests refers to the fact that due to the increase in data transmission complexity, the number of filtering requests received by the system continues to increase, resulting in an increase in the load on the filtering chip. Based on the prediction results of the aforementioned complexity exceeding limit, the number of data filtering requests received per unit time is monitored and recorded. The system will classify these requests by type, with particular attention to requests from high-complexity data streams. The system uses a time series-based analysis method to model the growth trend of data filtering requests and predict future request surges. Through real-time monitoring and trend analysis of data requests, the system can respond in advance when data filtering requests begin to surge, avoiding overloading of chip processing capabilities due to excessive requests.

[0130] Step S344: determining the degree of waiting data accumulation based on the growth trend of data filtering requests and the growth trend of data filtering chip resource usage;

[0131] In an embodiment of the present invention, the degree of accumulation of waiting data is determined based on the growth trend of data filtering requests and the growth trend of resource occupancy obtained in the aforementioned steps. Data accumulation refers to the gradual accumulation of unprocessed data when the amount of data filtering requests is greater than the processing capacity of the chip, resulting in delays in the transmission process and increasing the risk of chip overload. According to the aforementioned request growth trend, the amount of data that needs to be processed in the current time window is calculated, and then the processing capacity and resource occupancy of the chip are compared. If the data request growth trend exceeds the processing capacity of the chip, the system will perform an accumulation assessment on the current unprocessed data. The degree of accumulation is calculated by comparing the amount of data accumulated per unit time with the processing capacity of the chip per unit time. In addition, the system also monitors the delay of the data to ensure that the occurrence of accumulation can be identified in a timely manner. In this way, the system can accurately identify the degree of accumulation of waiting data and prepare for subsequent thermal stress accumulation predictions.

[0132] Step S345: predicting the chip operation thermal stress accumulation according to the waiting data accumulation level and the data filtering request growth trend;

[0133] In an embodiment of the present invention, the thermal stress accumulation of the chip operation is predicted based on the aforementioned obtained waiting data accumulation level and the data filtering request growth trend. Thermal stress accumulation is the internal stress accumulation phenomenon caused by the increase in operating temperature when the chip is running under long-term high load. As the number of data requests and the amount of data accumulation increase, the load of the chip will continue to increase, causing the chip temperature to rise. The temperature changes of the chip will be monitored in real time and analyzed in combination with the request growth trend and data accumulation. When the data accumulation is more serious and the data filtering request growth trend is faster, the system will calculate the temperature increase that the chip will encounter within a certain period of time in the future. Through thermal model calculation, the thermal stress accumulation of the chip is predicted, and the risk of chip overheating is warned in advance.

[0134] Step S346: predicting the thermal fatigue of the chip operation structure based on the chip operation thermal stress accumulation;

[0135] In an embodiment of the present invention, based on the thermal stress accumulation of the chip, the system will further predict the structural thermal fatigue of the chip during operation. Structural thermal fatigue refers to the phenomenon that the internal structural materials of the chip are gradually affected by thermal stress in a continuous high-temperature environment, resulting in performance degradation or even damage. As the chip operation time increases, thermal stress gradually accumulates, causing chip failure. By analyzing the thermal stress changes of the chip, a thermal fatigue prediction model is used to evaluate whether the structure of the chip will be affected by thermal fatigue within a certain period of time. Through in-depth analysis of the physical properties of the chip materials and the effects of thermal stress, the system can predict the durability of the chip in a high-temperature environment and the time when thermal fatigue occurs.

[0136] Step S347: predicting the degree of chip operation electrical shock aggravation based on the chip operation structure thermal fatigue;

[0137] In an embodiment of the present invention, based on the thermal fatigue of the chip, the system will further predict the degree of aggravation of the electrical shock when the chip is running. Electrical shock refers to the decrease in the stability of the internal circuit of the chip under the influence of high temperature and thermal fatigue, which leads to interference in the transmission of electrical signals, resulting in problems such as short circuits or overcurrent. By monitoring the electrical parameters inside the chip, such as voltage, current and circuit stability, combined with the aforementioned thermal fatigue prediction, the degree of aggravation of the electrical shock faced by the chip after long-term operation can be calculated. By using electrical simulation tools and monitoring of actual electrical parameters, the system can analyze the electrical shock risk of the chip in real time and provide maintenance warnings for the subsequent use of the chip.

[0138] Step S348: filtering the chip structure overload condition according to the chip operation electrical shock aggravation degree and the chip operation structure thermal fatigue condition detection data.

[0139] In an embodiment of the present invention, the data filtering chip is detected to see if a structural overload has occurred based on the degree of electrical shock aggravation and thermal fatigue of the chip. Chip overload is usually manifested as a decrease in computing power, delayed response time, and damage to electrical components. By integrating electrical shock and thermal fatigue data, the system can determine whether the chip has exceeded its designed load capacity. The various performance indicators of the chip are monitored in real time, and a multi-dimensional monitoring model is used in combination with thermal stress and electrical shock data to assess whether the chip has reached or exceeded its structural bearing capacity. If an overload is detected, the system will automatically reduce the load, adjust the load, or shut down some processing units to ensure that the chip does not crash or be damaged, and to maintain system stability. These steps ensure that through real-time monitoring and prediction, the system can effectively manage chip resources, avoid overload, and extend the service life of the chip.

[0140] It is particularly important that step S348 includes the following steps:

[0141] Detect the imbalance of charge distribution inside the chip based on the degree of electrical shock during chip operation;

[0142] In an embodiment of the present invention, while the chip is continuously powered on and running, a high-frequency voltage sampling module is set near the power supply pin of the chip, and a coupling capacitor decoupling circuit is used to collect voltage fluctuation curves at different voltage transition moments during operation. At the same time, a micro-electrostatic electric field probe array is arranged on different structural layers on the chip surface, and a high-frequency shielding filter device is used to filter out stray signals to obtain stable chip surface electric field change data. Based on the collected multi-point potential information, a charge distribution change diagram of different regions of the chip per unit time is constructed through a two-dimensional potential map. The potential gradient change rate is differentially calculated with the voltage drop data in the same time slice to extract the change trend of the charge density imbalance in the chip area. The target data finally obtained is the chip charge imbalance degree data.

[0143] Predicting the growth trend of high-intensity transient current based on the uneven charge distribution inside the chip;

[0144] In an embodiment of the present invention, the charge distribution imbalance Q_imbalance is used as input data, combined with the voltage fluctuation threshold during chip operation, and a current detection head with a nanosecond response time is used to perform synchronous sampling between the main power supply path and the ground pin. Through sampling, the amplification trend of transient current under different charge distribution imbalances is obtained, and the rising speed of the current mutation value and the steady-state recovery time are extracted. The maximum current rising speed within each sampling period is further compared and analyzed with the electric field offset vector in the charge imbalance area to form a current growth trend map. Using the transient current rapid growth rate as a measurement indicator, the generated data result is high-intensity transient current growth trend data.

[0145] Detect chip dielectric layer breakdown based on the growth trend of high-intensity transient current;

[0146] In this embodiment of the present invention, the current growth trend, I_rising, is used as input data to extract the spatial coordinates of the current surge region. Combined with chip dielectric structural parameters (such as insulation layer thickness and material dielectric constant), an AC impedance spectrum analyzer performs an AC impedance scan to detect the dielectric state of the target region. The local phase angle of the dielectric layer is measured to determine the inflection point of the phase angle change at the initial stage of dielectric failure. Combined with the scan results, a two-dimensional dielectric stability map is constructed to quantitatively label the degree of dielectric failure within the overlapping region of current surges. Ultimately, data on the breakdown of the chip's dielectric layer is obtained.

[0147] Predict the abnormal damage degree of the chip based on the breakdown of the chip electrolyte layer;

[0148] In this embodiment of the present invention, based on the dielectric breakdown indicator B_Breakdown, dielectric damage points are mapped to their corresponding functional modules using chip layout information. Within the functional modules, a conductivity detection tool is further used to detect the resistance drift of the interconnected metal wire mesh, which is then compared with the designed impedance to assess signal transmission integrity. Combining the structural redundancy and logical criticality of each functional module in the chip process layout, the breakdown-affected areas are functionally weighted to form a module-level damage weighting matrix. Finally, the overall chip functional degradation level is summarized to form an assessment of the degree of abnormal chip damage.

[0149] Detect thermal runaway inside the chip based on the thermal fatigue of the chip's operating structure;

[0150] In an embodiment of the present invention, a patch thermocouple array is used to sample the temperature distribution on the chip surface, and a high-precision infrared thermal imaging device is used to periodically obtain a thermal distribution map, and the temperature fluctuation rate and peak amplitude of each functional area are recorded. The sampled temperature response data is coupled with the thermal expansion coefficient of different materials in the chip structure to obtain a thermal stress concentration index. The temperature rise rate curve on the chip is further collected using a thermistor device. If the temperature rise rate exceeds the set threshold and there is a sudden increase in temperature in multiple regions, it is determined whether there is thermal coupling feedback based on the degree of regional thermal distribution imbalance and the difference in thermal diffusion delay time. After comprehensive judgment, the internal thermal runaway status data of the chip is output.

[0151] The chip structure overload condition is filtered based on the chip internal thermal runaway condition and the abnormal damage degree of the chip.

[0152] In an embodiment of the present invention, T_abnormal and D_damage are used as input variables, and according to the mapping diagram of the functional links and physical transmission paths arranged in the data filtering chip, the transmission nodes in the functional path that pass through the high thermal runaway risk area and the high damage level area are checked one by one. The equivalent resistance mapping calculation module is used to extract the structural reliability factor of the path, and a comprehensive structural degradation diagram is generated by combining the thermal resistance, temperature difference distribution and path voltage drop ratio. The structural degradation diagram is compared with the timing response jitter data recorded during the chip data filtering operation to determine whether the structural stability bottleneck is triggered under the pressure of data transmission. If the critical path is severely degraded in terms of thermodynamics, functional integrity and voltage stability, the final chip structure overload diagnosis result is generated.

[0153] Preferably, step S4 includes the following steps:

[0154] Step S41: filtering abnormal conditions in real time based on data filtering chip structure overload condition detection data;

[0155] In an embodiment of the present invention, the structure of the data filtering chip is monitored to identify whether it is overloaded. This process is achieved by real-time monitoring of the data filtering chip's workload and resource usage. Specifically, hardware performance monitoring tools or specialized on-chip sensors are used to collect real-time data on key chip indicators such as internal temperature, power consumption, processing power, and data processing speed. Based on this data, a chip load analysis is performed, comparing the real-time data with preset performance thresholds to determine whether the chip is overloaded. If the chip load exceeds the set maximum load value, it indicates that the chip is overloaded. Next, based on the chip overload identification result, its impact on the data filtering function is further analyzed to determine whether a data filtering anomaly has occurred. For example, if chip overload causes filtering operation delays or a decrease in data throughput, this indicates a real-time data filtering anomaly. At this point, the data acquisition module continuously tracks the chip status and outputs a real-time anomaly report to determine whether the system currently needs to adjust its data filtering strategy.

[0156] Step S42: detecting the attenuation of the real-time data filtering efficiency based on the abnormal real-time data filtering and the overload condition of the data filtering chip structure;

[0157] In an embodiment of the present invention, once the abnormal situation of real-time data filtering and the chip overload status are determined, the attenuation of data filtering efficiency is further analyzed. By monitoring indicators such as response time, processing speed and data loss rate in the data filtering process, the filtering efficiency of the data filtering chip under overload conditions is evaluated. A detection tool with a delay timer is used to record the filtering time of each cycle, as well as the number of successfully processed data packets and the number of unprocessed data packets in each cycle. If the filtering time increases significantly and the data processing volume decreases significantly within a certain time period, it means that the data filtering efficiency has attenuated. On this basis, the cause of the attenuation is further analyzed, such as whether it is due to insufficient resource allocation caused by chip overload, or whether the data traffic cannot be processed in time due to system design problems. By calculating the processing capacity of the chip under different loads, a specific attenuation trend is obtained, and data support is provided for subsequent optimization based on this trend.

[0158] Step S43: Optimizing the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization status of the real-time data filtering path;

[0159] In an embodiment of the present invention, after confirming that the efficiency of real-time data filtering has declined, in order to improve the overall performance of the data filtering system, it is necessary to optimize the real-time data filtering path, analyze the existing filtering path, evaluate the processing efficiency of each filtering link, and determine which links become bottlenecks. By analyzing the data traffic and delay of each path node, the paths with slower processing speed or overload are identified as the priority targets for optimization. Then, data flow analysis tools, such as bandwidth measurement tools and traffic monitoring tools, are used to design more reasonable data filtering paths to guide data traffic to paths with lighter loads or higher processing capabilities. The core of path optimization is to adjust the way data flows through the path, including dynamically adjusting the path selection of the data flow through a load balancing algorithm to avoid overloading a certain path. Finally, the optimized path is tested through simulation experiments to ensure that it can improve data filtering efficiency and reduce system burden, thereby improving the real-time performance of data processing in the entire system.

[0160] Step S44: using the real-time data filtering path optimization situation to perform real-time data filtering processing on the data transmission complexity exceeding limit situation to obtain real-time data filtering information.

[0161] In an embodiment of the present invention, after completing the optimization of the real-time data filtering path, the next step is to apply the optimized path to handle situations where the complexity of data transmission exceeds the limit. In order to cope with complex data transmission requirements, it is necessary to analyze the entire data transmission network, especially the analysis of multi-dimensional characteristics such as bandwidth, delay, error rate, etc. during data transmission. The optimized data filtering path can share the pressure of the overloaded path in the system, thereby effectively reducing the impact of excessive complexity. The specific operation is to apply the optimized path to actual data transmission, retransmit the data in the environment after the path optimization, and detect the real-time performance of the system when processing data. Based on the actual test results, the data transmission and filtering strategies are readjusted to ensure that the data filtering system can process efficiently under any data complexity. Through real-time data filtering processing, the optimized data real-time filtering information is obtained, including key indicators such as data transmission time, packet loss rate, and processing efficiency, providing data support for subsequent system tuning and performance improvement.

[0162] The present invention further provides a real-time data filtering system for multi-dimensional feature fusion, which is used to execute the real-time data filtering method for multi-dimensional feature fusion as described above. The real-time data filtering system for multi-dimensional feature fusion includes:

[0163] The data frame field parsing structure processing module is used to obtain data transmission logs; extract original transmission data features based on the data transmission logs to obtain original transmission data features; and perform field structure parsing on the data transmission logs and original transmission data features to obtain data frame field parsing structure information.

[0164] A multi-dimensional abnormal coupling determination module is used to calculate the transmission field time gradient change rate based on the data frame field analysis structure information; determine the transmission field dynamic offset status based on the transmission field time gradient change rate data; and determine the multi-dimensional abnormal coupling of data transmission based on the transmission field dynamic offset status and the transmission field time gradient change rate;

[0165] The chip structure overload detection module is used to detect abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predict the data transmission complexity exceeding the limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining the data transmission complexity exceeding the limit; and perform data filtering chip structure overload detection based on the data transmission complexity exceeding the limit to obtain the data filtering chip structure overload status;

[0166] The real-time data filtering processing module is used to detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization status of the real-time data filtering path; use the real-time data filtering path optimization status to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

[0167] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. The present invention is not limited to the embodiments shown herein, but is to be embodied in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A real-time data filtering method based on multi-dimensional feature fusion, characterized in that: The following steps are involved: Step S1: Obtaining data transmission logs; extracting features of original transmission data based on the data transmission logs, thereby obtaining features of the original transmission data; Perform field structure analysis on data transmission logs and original transmission data features to obtain data frame field analysis structure information; Step S2: Calculate the transmission field time gradient change rate based on the data frame field parsing structure information, wherein the transmission field time gradient change rate calculation includes: According to the data frame field parsing structure information, the transmission field timing timestamp is collected; Collect the transmission field timing timestamps exceeding 60ms during the transmission field abnormal missing period; Perform field timing reconstruction based on the abnormal missing period of the transmission field and the transmission field timing timestamp to obtain the transmission field time trajectory sequence data; Measure the difference amplitude of adjacent field values ​​based on the transmission field time trajectory series data; When the difference between adjacent field values ​​exceeds 0.1, the transition strength of adjacent field values ​​is counted; Detect the fluctuation characteristics of the transmission field change rhythm based on the jump intensity of the adjacent field values ​​and the difference amplitude of the adjacent field values; The time gradient change rate of the transmission field is calculated according to the fluctuation characteristics of the transmission field change rhythm and the transmission field time trajectory sequence data; Determining a transmission field dynamic offset condition according to the transmission field time gradient change rate data, wherein determining the transmission field dynamic offset condition includes: Perform 100 Hz multi-scale sliding synchronous slicing processing on the transmission field time gradient change rate data to obtain field change structure slice data; Dividing the transmission field time gradient change rate data into multiple period field structure data; Identify the location of abnormal mutation amplitude of multi-period fields when the multi-period field structure data exceeds ±0.2 / ms; Extract the starting point position of the field abnormal mutation amplitude based on the position of the field abnormal mutation amplitude in multiple time periods; Mark the starting point of the field abnormal mutation amplitude to obtain the field abnormal mutation point marking data; According to the field change structure slice data and field abnormal mutation point marking data, the field structure trajectory change situation is collected; Determine the dynamic offset status of the transmission field according to the change of the field structure trajectory; Determining the multi-dimensional abnormal coupling of data transmission according to the dynamic offset status of the transmission field and the time gradient change rate of the transmission field, wherein determining the multi-dimensional abnormal coupling of data transmission includes: Detecting the parallel offset of multiple fields in transmission according to the dynamic offset of the transmission field; Determine the degree of disorder in the data stream protocol structure based on the parallel offset of multiple transmitted fields; Determine the data transmission protocol drift based on the disorder of the data stream protocol structure and the drift of multiple fields transmitted in parallel; Detecting the severity of data transmission rate changes based on the time gradient change rate of the transmission field; Statistical transmission synchronization mismatch based on the severity of data transmission rate changes; Determine the multi-dimensional abnormal coupling of data transmission based on the data transmission synchronization imbalance and data transmission protocol drift; Step S3: detecting abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predicting a data transmission complexity exceeding limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining a data transmission complexity exceeding limit; performing data filter chip structure overload detection based on the data transmission complexity exceeding limit, thereby obtaining a data filter chip structure overload condition; Step S4: Detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization condition of the real-time data filtering path; use the optimization condition of the real-time data filtering path to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

2. The real-time filtering method for multi-dimensional feature fusion data according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Obtain data transmission log; Step S12: performing data preprocessing on the data transmission log to obtain transmission log preprocessing data; Step S13: extracting original transmission data features based on transmission log preprocessing data; Step S14: performing field structure parsing processing on the transmission log pre-processing data and the original transmission data features to obtain data frame field parsing structure information.

3. The real-time filtering method for multi-dimensional feature fusion data according to claim 2 is characterized in that: Step S14 includes the following steps: Step S141: performing field boundary reconstruction processing on the original transmission data features, thereby obtaining transmission field boundary reconstructed data; Step S142: constructing a transmission field logical boundary map according to the transmission field boundary reconstruction data; Step S143: performing protocol mode regularization processing according to the transmission field logical boundary map, thereby obtaining field protocol mode characteristics; Step S144: Count the frequency of field historical value changes based on the field protocol pattern characteristics; mark the field change activity level when the field historical value change frequency is high; Step S145: collecting field value range distribution according to field protocol pattern characteristics; Step S146: Mapping the field value range distribution and the field change activity to obtain field value range mapping data; Step S147: Perform field cross-indexing based on the transmission log preprocessing data and the field value range mapping data to obtain transmission field location information; Step S148: Perform field structure parsing processing according to the transmission field positioning information and the field value range mapping data to obtain data frame field parsing structure information.

4. The real-time filtering method for multi-dimensional feature fusion data according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: detecting abnormal dilution of transmission data according to abnormal multi-dimensional coupling of data transmission; Step S32: determining the transmission data interference amplification degree based on the abnormal dilution of the transmission data; Step S33: predicting a data transmission complexity exceeding limit based on the degree of interference amplification of the transmitted data and the abnormal dilution of the transmitted data; Step S34: detecting the overload condition of the data filtering chip structure according to the data transmission complexity exceeding the limit.

5. The real-time filtering method for multi-dimensional feature fusion data according to claim 4 is characterized in that: Step S34 includes the following steps: Step S341: Identify the growth of data parsing fine-grainedness based on the data transmission complexity exceeding the limit; Step S342: Detecting the growth trend of data filtering chip resource usage based on the fine-grained growth of data analysis; Step S343: Identify the growth trend of data filtering requests based on the data transmission complexity exceeding the limit; Step S344: determining the degree of waiting data accumulation based on the growth trend of data filtering requests and the growth trend of data filtering chip resource usage; Step S345: predicting the chip operation thermal stress accumulation according to the waiting data accumulation level and the data filtering request growth trend; Step S346: predicting the thermal fatigue of the chip operation structure based on the chip operation thermal stress accumulation; Step S347: predicting the degree of chip operation electrical shock aggravation based on the chip operation structure thermal fatigue; Step S348: filtering the chip structure overload condition according to the chip operation electrical shock aggravation degree and the chip operation structure thermal fatigue condition detection data.

6. The real-time data filtering method based on multi-dimensional feature fusion according to claim 1 is characterized in that: Step S4 includes the following steps: Step S41: filtering abnormal conditions in real time based on data filtering chip structure overload condition detection data; Step S42: detecting the attenuation of the real-time data filtering efficiency based on the abnormal real-time data filtering and the overload condition of the data filtering chip structure; Step S43: Optimizing the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization status of the real-time data filtering path; Step S44: using the real-time data filtering path optimization situation to perform real-time data filtering processing on the data transmission complexity exceeding limit situation to obtain real-time data filtering information.

7. A real-time data filtering system based on multi-dimensional feature fusion, characterized in that: The method for real-time filtering of data by multi-dimensional feature fusion according to claim 1 is used, and the real-time filtering system for real-time filtering of data by multi-dimensional feature fusion comprises: The data frame field parsing structure processing module is used to obtain data transmission logs; extract original transmission data features based on the data transmission logs to obtain original transmission data features; and perform field structure parsing on the data transmission logs and original transmission data features to obtain data frame field parsing structure information. A multi-dimensional abnormal coupling determination module is used to calculate the transmission field time gradient change rate based on the data frame field analysis structure information; determine the transmission field dynamic offset status based on the transmission field time gradient change rate data; and determine the multi-dimensional abnormal coupling of data transmission based on the transmission field dynamic offset status and the transmission field time gradient change rate; The chip structure overload detection module is used to detect abnormal dilution of transmission data based on the abnormal multi-dimensional coupling of data transmission; predict the data transmission complexity exceeding the limit based on the abnormal dilution of transmission data and the abnormal multi-dimensional coupling of data transmission, thereby obtaining the data transmission complexity exceeding the limit; and perform data filtering chip structure overload detection based on the data transmission complexity exceeding the limit to obtain the data filtering chip structure overload status; The real-time data filtering processing module is used to detect the attenuation of the real-time data filtering efficiency according to the overload condition of the data filtering chip structure; optimize the real-time data filtering path according to the attenuation of the real-time data filtering efficiency to obtain the optimization status of the real-time data filtering path; use the real-time data filtering path optimization status to perform real-time data filtering processing on the data transmission complexity exceeding the limit to obtain real-time data filtering information.

Images