A column storage method for simplified sequential pattern computing unit data based on tree structure

Abstract

The present invention discloses a tree-structured simplified sequence pattern calculation unit data column storage method, which relates to the technical field of data storage, including: standardizing calculation unit data to form source data and inputting it into a column storage engine, wherein each source data corresponds to data of a sampling time point in a calculation unit, generating metadata for identifying the storage location and pattern type of the source data for each source data, and forming a plurality of column structures based on a plurality of source data and corresponding metadata; forming an original pattern tree structure according to all metadata, storing each found potential maximum sequence pattern in an improved prefix tree original pattern tree, and finally obtaining a prefix tree simplified pattern tree composed of maximum sequence patterns by pruning the original pattern tree; the method improves the time efficiency of calculation unit storage.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a columnar storage method for simplified sequence pattern computing unit data based on a tree structure. Background Art

[0002] In modern computer systems, computing units are the most basic units for allocating computing resources. Usually, the performance of computing units depends on many factors, such as hardware configuration, operating load, and task type. Among them, hardware configuration information comes from the physical parameters of the computing unit, such as CPU model, memory size, storage space, etc., which is used to describe the processing power of the computing unit. Hardware configuration information is usually used as a reference when purchasing and adjusting computing resources. Operating load information refers to the real-time operating data of the computing unit, including CPU utilization, memory occupancy, I / O, etc. within a certain period of time. Task type information mainly relies on analyzing the applications and services running on the computing unit, such as scientific computing, data analysis, Web services, etc., as the basis for judging the usage pattern of computing resources. The computing unit operation data contains the value of mining the usage rules of computing units and predicting future trends.

[0003] However, current cloud computing platforms manage tens of thousands of computing units. Each computing unit also includes a large amount of monitoring data at different time granularities, and the scale of computing unit operation data is very large. The cloud environment changes rapidly, and resource allocation decisions need to be adjusted in real time based on the current computing unit operation status and historical data, which requires efficient real-time reading and analysis of computing unit data.

[0004] In actual cloud platform resource management, it is difficult to store all TBs of computing unit operation data in memory. Usually, it is necessary to select a database system that meets one's business needs as the data storage engine. Efficient data storage and reading capabilities determine the performance of the resource management system. Currently, commonly used database systems include MySQL, TiDB, PostgreSQL, Oracle, KDB, ClickHouse, etc., but it is difficult to achieve ideal performance requirements when storing and reading ultra-large-scale computing unit data.

[0005] Therefore, in response to the above problems, the inventor believes that the existing general data storage engine is difficult to meet the needs of computing unit operation data in terms of massive scale, real-time analysis, correlation screening, etc. It is urgent to design a storage optimization method specifically for the characteristics of computing unit operation data, so as to accelerate the process of resource management and scheduling decision-making. Summary of the invention

[0006] In order to improve the memory access efficiency of computing unit data, the present application provides a column storage method for computing unit data in a simplified sequence mode based on a tree structure.

[0007] In a first aspect, the present invention discloses a method for storing data in a column format of a simplified sequence pattern calculation unit based on a tree structure, comprising:

[0008] S10: Obtain computing unit operation information of the computing unit, perform data normalization, and obtain source data of the computing unit;

[0009] S20: inputting the source data into a column storage engine, so that the source data forms a plurality of column structures, and determining a storage location of the source data;

[0010] S30: Divide the numerical indicators of the source data into intervals, determine the corresponding mode information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding mode information, wherein the source data includes the numerical indicators and the categorical indicators;

[0011] S40: Obtain metadata of multiple sampling time points and construct an original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information;

[0012] S50: According to the current computing unit operation information, traverse the original pattern tree and prune it to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

[0013] Preferably, traversing the original pattern tree and pruning branches according to the current computing unit operation information includes:

[0014] Get each pattern node in the original pattern tree, calculate the occurrence frequency f of each pattern node, the pattern length L, the time span t of the occurrence of a single pattern node, and the similarity s between the current pattern node and its ancestor node;

[0015] Constructing a pattern importance scoring function through pattern nodes;

[0016] The importance score of each pattern node is calculated by the pattern importance scoring function to obtain a pattern node score list;

[0017] The lowest threshold of the pattern function is set, the pattern node score list is filtered, and the pattern nodes whose scores are smaller than the lowest threshold of the pattern function are removed to obtain the first pattern tree.

[0018] Preferably, the traversing and pruning of the original pattern tree according to the current computing unit operation information also includes introducing a recursive pruning algorithm to prune the first pattern tree from top to bottom to obtain the second pattern tree through importance scores of pattern nodes, hierarchical relationships between pattern nodes, and similarities between pattern nodes and their ancestor nodes.

[0019] Preferably, the pattern importance scoring function is as follows:

[0020] PIS=w1*log(1+f)+w2*(1-e (-L / L0) )+w3*(1-e (-t / T) )+w4*s;

[0021] Among them, w1 to w4 are the weight coefficients of the four factors, T is the total time span covered by all pattern nodes, L0 is a length threshold parameter, L represents the pattern length, log(1+f) represents the logarithm of the frequency, f represents the occurrence frequency of the pattern node, (1-e (-L / L0) ) represents the exponential saturation function of length, s represents the similarity between a pattern node and its ancestor node, and PIS represents the importance score of the node.

[0022] Preferably, the steps for determining the length threshold parameter L0 are as follows:

[0023] Statistics of the length distribution of all pattern nodes: traverse the original pattern tree, calculate the length L of each pattern node, and obtain the set of all pattern node lengths {L1, L2, ..., L n}; where n represents the number of pattern nodes, L n Indicates the length of the nth pattern node and calculates the average length of the pattern node

[0024]

[0025] Calculate the standard deviation of the pattern node lengths σ L :

[0026]

[0027] Where, L i Indicates the length of the i-th pattern node;

[0028] Get L0, where the L0 acquisition function is as follows:

[0029]

[0030] Among them, k is a tuning parameter.

[0031] Preferably, the steps for determining the k value are as follows:

[0032] Get the set of pattern node lengths {L1, L2, ..., L n};

[0033] Calculate the average length of pattern nodes Get standardized mode node data

[0034] Perform K-means clustering on the standardized pattern node data, select the appropriate number of clusters m, and obtain the cluster label of each pattern node length;

[0035] Calculate the skewness γ of the pattern node length L and kurtosis κ L ;

[0036] Construct the feature matrix X, which contains the cluster labels, skewness and kurtosis of each pattern node length;

[0037] The regression model is trained by historical data to predict the k value, wherein the historical data includes the historical feature matrix X history And the k value y corresponding to the history history ;

[0038] Use the trained regression model, input the feature matrix X of the current mode node, get the predicted k value, and take the average of the predicted k values ​​as the final target k value.

[0039] Preferably, the second pattern tree determination process is as follows:

[0040] Starting from the root node of the first pattern tree, set it as the current node;

[0041] Make the following judgments on the current node:

[0042] If the importance score PIS of the current node is greater than or equal to the preset score threshold PIS_min, then keep the current node and set its first child node as the new current node, and recursively execute this step. If the current node has no child nodes, then backtrack to its parent node and set the next child node of the parent node as the new current node, and recursively execute this step;

[0043] If the PIS of the current node is less than the preset scoring threshold PIS_min, check whether its similarity with any ancestor node exceeds the threshold sim_min;

[0044] If the similarity is greater than or equal to the preset similarity threshold sim_min, the current node is retained, its child nodes are no longer recursively processed, and the node is directly traced back to its parent node, and the next child node of the parent node is set as the new current node, and the judgment of the current node is recursively executed;

[0045] If the similarity is less than the preset similarity threshold sim_min, the current node and all its subtrees are pruned; directly backtrack to its parent node, and set the next child node of the parent node as the new current node, and recursively perform the judgment on the current node.

[0046] In a second aspect, the present invention discloses a tree-structured simplified sequence pattern calculation unit data column storage system, wherein the system executes the steps of the tree-structured simplified sequence pattern calculation unit data column storage method, including:

[0047] A source data acquisition module, used to acquire computing unit operation information of a computing unit, perform data normalization, and obtain source data of the computing unit;

[0048] A column structure generation module, used for inputting the source data into a column storage engine, so that the source data forms a plurality of column structures, and determining a storage location of the source data;

[0049] The mode information acquisition module is used to divide the numerical indicators of the source data into intervals, determine the corresponding mode information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding mode information, wherein the source data includes numerical indicators and categorical indicators:

[0050] The original pattern tree construction module is used to obtain metadata of multiple sampling time points and construct the original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information;

[0051] The simplified pattern tree acquisition module traverses the original pattern tree and prunes it according to the current computing unit operation information to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

[0052] In a third aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein:

[0053] The memory stores a computer program that can be called by the processor;

[0054] The processor executes the steps of executing the tree-structure-based reduced sequence pattern calculation unit data column storage method in the background by calling the computer program stored in the memory.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium having a rewritable computer program stored thereon;

[0056] When the computer program is executed on a computer device, the computer device executes the steps of the tree-structure-based reduced sequence pattern calculation unit data column storage method in the background.

[0057] By adopting the above technical solution, at least the following effects are achieved:

[0058] This method can better meet the needs of data query and analysis by organizing the source data of the computing unit in a columnar storage manner. Columnar storage can avoid reading unnecessary data columns and reduce I / O operations, thereby improving data access efficiency. At the same time, by extracting and streamlining the source data, the storage space of the data can be further compressed, saving storage resources.

[0059] Based on column storage, by building a pattern tree to index the pattern information of the data, the required data area can be quickly located, avoiding the inefficient operation of full table scanning. At the same time, by pruning the pattern tree, the most representative and distinguishing pattern nodes are retained, which can further accelerate the data query and matching process.

[0060] The sequence pattern information extracted by this method can reflect the characteristic changes of data at different time nodes, which provides useful information for data trend analysis and prediction. By dynamically maintaining and updating the pattern tree, it can adapt to the changes in data characteristics, discover new patterns and laws in time, and improve the adaptability and flexibility of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of the steps of the tree-structure-based simplified sequence pattern calculation unit data column storage method provided by the present invention;

[0062] Figure 2 A flowchart of the steps of constructing the first pattern tree in the tree-structured simplified sequence pattern calculation unit data column storage method provided by the present invention;

[0063] Figure 3 A flowchart of the steps of constructing the second pattern tree in the tree-structured simplified sequence pattern calculation unit data column storage method provided by the present invention;

[0064] Figure 4 A schematic diagram of the structure of a pattern node in the tree-structured simplified sequence pattern calculation unit data column storage method provided by the present invention;

[0065] Figure 5 A schematic diagram of the structure of the tree-structured simplified sequence pattern calculation unit data column storage system provided by the present invention;

[0066] Figure 6 A schematic diagram of an electronic device provided by the present invention;

[0067] Figure 7 A schematic diagram of a storage medium provided by the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0070] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0071] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0072] Example 1

[0073] like Figure 1 As shown, the present application discloses a tree-structured simplified sequence pattern calculation unit data column storage method, which specifically includes the following steps:

[0074] S10: Obtain computing unit operation information of the computing unit, perform data normalization, and obtain source data of the computing unit; the computing unit operation information represents various performance indicator data collected during the operation of the computing unit.

[0075] The computing unit operation information is the information obtained from the original computing unit transaction website, including various real-time performance indicator information such as the computing unit name, CPU utilization, memory occupancy, disk I / O rate, network bandwidth, etc., among which the real-time performance indicator refers to the indicator value obtained by the computing unit at the previous data collection time point. The time interval between each data collection time point can be one minute or five seconds, which can be determined according to the capacity of the database, data acquisition and processing capabilities.

[0076] The computing unit operation information has different formats. In order to better store it, data normalization is required to process the computing unit operation information into normalized source data.

[0077] Specifically, when the computing unit data to be stored is "Compute-01", the computing unit code is VM-001 at 2023-05-15 10:00:00, "CPU_USAGE", "MEM_USAGE", DISK_IO_RATE", "NET_BANDWIDTH" and other information, which represents the computing unit performance indicator information field. Extract the corresponding data information, and decompose the computing unit data into multiple source data such as "Compute-01", "VM-001", "2023-05-15 10:00:00", "CPU_USAGE", "MEM_USAGE", "DISK_IO_RATE", "NET_BANDWIDTH". Among them, Compute-01: represents the identifier or name of a specific computing resource or server. This label helps to distinguish this unit from other units in a network or data center.

[0078] VM-001: refers to a specific virtual machine running on compute resource "Compute-01". Each virtual machine is usually assigned a unique identifier so that it can be managed and monitored individually.

[0079] 2023-05-1510:00:00: represents the exact date and time of a data snapshot or monitoring record. This timestamp is critical for tracking changes and analyzing trends over time.

[0080] CPU_USAGE: Measures the percentage of CPU capacity currently in use. High CPU usage may indicate that the compute unit is processing a large number of tasks, which may affect performance if it continues to reach its limit. MEM_USAGE (memory usage): Indicates the percentage of memory capacity currently in use. Information about memory usage helps understand how the system consumes resources when handling the current load.

[0081] DISK_IO_RATE (disk I / O rate): indicates the rate at which the disk completes input and output operations per unit time. This rate is an important indicator for evaluating storage system performance and is related to data processing and access speed.

[0082] NET_BANDWIDTH (network bandwidth): indicates the maximum amount of data that a network interface can transmit per unit time. Network bandwidth is a key indicator of network performance and directly affects data transmission and communication efficiency.

[0083] S20: Input the source data into the column storage engine, so that the source data forms a plurality of column structures, and determine the storage location of the source data.

[0084] The column storage engine is used to limit data storage to column storage; the column structure refers to the storage structure of data in column storage form; the source data refers to the data obtained by decomposing the original computing unit data.

[0085] S30: Divide the numerical indicators of the source data into intervals, determine the corresponding pattern information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding pattern information, wherein the source data includes numerical indicators and categorical indicators.

[0086] Specifically, the computing unit mode includes high load, low load, and stable operation modes. High load mode: the most recent high point is higher than the previous high point, and the most recent low point is higher than the previous low point. Low load mode: the most recent high point is lower than the previous high point, and the most recent low point is lower than the previous low point. Stable operation mode: the most recent high point is higher than the previous high point, and the most recent low point is lower than the previous low point; or the most recent high point is lower than the previous high point, and the most recent low point is higher than the previous low point.

[0087] According to the above predetermined rules, it is determined which mode the computing unit is in, and the mode information of the computing unit is determined.

[0088] Metadata is set for the database, wherein the metadata includes location information and mode information of source data corresponding to a sampling time point, etc. The sampling time point represents a time node for data collection for a computing unit.

[0089] S40: Obtain metadata of multiple sampling time points and construct an original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information.

[0090] S50: According to the current computing unit operation information, traverse the original pattern tree and prune it to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

[0091] The reduced sequence pattern is a subset of the largest data, but it contains the most important data information of all the data, and the scale of the reduced sequence pattern is several orders of magnitude smaller than the largest data.

[0092] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0093] Example 2

[0094] Based on Example 1, this embodiment further improves the step of pruning the original pattern tree, such as Figure 2-Figure 4 As shown, including:

[0095] S501: Obtain each pattern node in the original pattern tree, calculate the occurrence frequency f of each pattern node, the pattern length L, the time span t of the occurrence of a single pattern node, and the similarity s between the current pattern node and its ancestor node. The similarity can be expressed by the cosine similarity between the current load feature vector and the pattern feature vector. Specifically, the cosine similarity can be calculated using the following formula:

[0096] s=cos(θ)=(A·B) / (||A||×||B||);

[0097] Among them, A and B represent the current load characteristic vector and mode characteristic vector respectively, and ||A|| and ||B|| represent the Euclidean norm of the vector.

[0098] For example, four indicators, namely CPU usage CPU_usage_t, memory usage MEM_usage_t, disk I / O rate DISK_IO_rate_t, and network bandwidth NET_bandwidth_t, are selected as load features at time t. Then, a four-dimensional vector is constructed using the load features at time point t:

[0099] A_t=[CPU_usage_t, MEM_usage_t, DISK_IO_rate_t, NET_bandwidth_t].

[0100] Wherein, A_t represents the load characteristic vector at the current time point t.

[0101] For pattern node p, the average value of the load feature vectors of all sampling time points it covers is taken as the feature vector of the pattern node p, where pattern node p covers multiple sampling time points (i.e., a set of time points), and its feature vector is the average value of the load feature vectors at these time points.

[0102] B_p=[avg(CPU_usage_p), avg(MEM_usage_p), avg(DISK_IO_rate_p), avg(NET_bandwidth_p)];

[0103] Among them, avg(CPU_usage_p) represents the average CPU usage of all sampling time points covered by pattern node p, avg(MEM_usage_p) represents the average memory usage of all sampling time points covered by pattern node p, avg(DISK_IO_rate_p) represents the average disk I / O rate of all sampling time points covered by pattern node p, NET_bandwidth_p represents the average network bandwidth of all sampling time points covered by pattern node p, and B_p represents the feature vector of pattern node p.

[0104] Substitute A_t and B_p into the cosine similarity formula (A·B) / (||A||×||B||) to calculate the similarity s.

[0105] S502: Based on S501, a pattern importance score function PIS (Pattern Importance Score) is constructed through the pattern node, and the importance score of the pattern node is obtained by combining the above factors:

[0106] PIS=w1*log(1+f)+w2*(1-e (-L / L0) )+w3*(1-e (-t / T) )+w4*s;

[0107] Among them, w1 to w4 are weight coefficients of the four factors, which can be set based on experience or data analysis. T is the total time span covered by all pattern nodes. The total time span T is obtained by finding the earliest time when all pattern nodes start to cover and the latest time when they end to cover, and then calculating the difference between the two.

[0108] L0 is a length threshold parameter, which can be set according to the characteristics of the data to control the saturation of the length factor. L represents the pattern length, which is the number of sampling points in the time window covered by the pattern node. For example, if we have system load data, sampling is performed once a minute. Assume that the time window covered by a pattern node p is from 10:00 to 10:10 (a total of 10 minutes). Within these 10 minutes, sampling is performed once a minute. Therefore, L of the pattern node p is 11 sampling points (including the sampling points of the start and end times). Therefore, the number of sampling points L in the time window from 10:00 to 10:10 is 11.

[0109] t represents the time span of the pattern node, that is, the time range in which the pattern node appears in the data set

[0110] log(1+f) represents the logarithm of frequency. The higher the frequency, the higher the score, but the marginal effect of increasing the frequency decreases. The process of calculating f is as follows:

[0111] f represents the frequency of occurrence of the pattern node, which can be calculated by the following steps:

[0112] Traverse the original pattern tree and for each pattern node p, count the number of times it appears count_p.

[0113] Calculate the total number of nodes total_count in the original pattern tree.

[0114] Calculate the frequency of each pattern node: f_p = count_p / total_count.

[0115] (1-e (-L / L0) ) represents an exponential saturation function of length. The longer the pattern, the higher the score, but the growth slows down after reaching L0.

[0116] Specifically, suppose a pattern node p covers from time t start to end The time window is , and the sampling interval is Δt, then the pattern length L can be expressed as:

[0117]

[0118] L0 is a length threshold parameter used to control the saturation of the length factor. In practical applications, L0 can be set according to data characteristics and specific requirements, or it can be calculated.

[0119] More specifically, the steps to determine L0 are as follows: Count the length distribution of all pattern nodes: traverse the original pattern tree, calculate the length L of each pattern node, and obtain the set of all pattern node lengths {L1, L2, ..., L n}; Calculate the average length of pattern nodes

[0120]

[0121] Calculate the standard deviation of the pattern node lengths σ L :

[0122]

[0123] In order to make L0 reasonably reflect the distribution characteristics of pattern length, it can be set to the mean plus the standard deviation:

[0124]

[0125] Among them, k is a constant that can be adjusted according to specific circumstances. Usually, the value depends on the distribution characteristics of the pattern length in the data set. The usual choice of k = 1.5 is because this value can better balance the central tendency and dispersion of the data, and is suitable for data distribution under general circumstances. At the same time, 1.5 times the standard deviation can effectively avoid the excessive influence of extreme values ​​on the L0 calculation, while not ignoring the role of dispersion. In addition, the k value can also be calculated.

[0126] In a more preferred embodiment, the process of obtaining the k value is as follows:

[0127] Get the set of pattern node lengths {L1, L2, ..., L n}.

[0128] Calculate the average length of pattern nodes Get standardized mode node data

[0129] Perform K-means clustering on the standardized pattern node data, select the appropriate number of clusters m, and obtain the cluster label of each pattern node length.

[0130] Calculate the skewness γ of the pattern node length L and kurtosis κ L .

[0131] Construct the feature matrix X containing the cluster labels, skewness, and kurtosis for each pattern node length.

[0132] The regression model is trained through historical data to predict the k value, where the historical data includes the historical feature moment X history and the corresponding k value y history .

[0133] Use the trained regression model, input the feature matrix X of the current mode node, get the predicted k value, and take the average of the predicted k values ​​as the final target k value.

[0134] Combining the specific calculation process of skewness, kurtosis and regression model prediction value can bring more accurate clustering results, more efficient model performance, lower overfitting and underfitting risks, and a wider range of application scenarios. This method dynamically adjusts clustering parameters in a data-driven way, which can better adapt to different data sets and application requirements.

[0135] (1-e (-t / T) ) represents an exponential saturation function of the time span. The longer the coverage time, the higher the score, but the growth slows down after reaching T.

[0136] s represents the similarity. The higher the similarity, the higher the score.

[0137] The PIS formula introduces nonlinear transformations in the three factors of frequency, length, and time span, which can better balance their influences. At the same time, it simplifies the similarity factor to avoid excessive influence of similarity.

[0138] S503: Calculate the importance score of each pattern node using the pattern importance scoring function to obtain a pattern node score list.

[0139] S504: Set the minimum threshold of the pattern function, filter the score list, remove all pattern nodes whose importance scores are less than the minimum threshold of the pattern function, and obtain the first pattern tree. The purpose of this processing is to quickly remove those obviously unimportant pattern nodes through a simple threshold comparison, so as to obtain a preliminary concise pattern tree representation. This step can significantly reduce the amount of calculation and complexity of subsequent processing.

[0140] S505: A recursive pruning algorithm is introduced to prune the pattern tree from top to bottom by comprehensively considering the importance scores of pattern nodes, the hierarchical relationships between pattern nodes, and the similarities between pattern nodes and their ancestor nodes, in order to obtain a refined, comprehensive and well-structured second pattern tree representation, thereby improving the efficiency and accuracy of subsequent pattern matching and querying.

[0141] Further considering the correlation between patterns, a recursive pruning algorithm is designed. Starting from the root node, each node is judged. The specific steps are as follows:

[0142] S5051, starting from the root node of the first pattern tree, set it as the current node.

[0143] S5052, make the following judgment on the current node:

[0144] If the importance score PIS of the current node is greater than or equal to the preset score threshold PIS_min, the current node is retained and its first child node is set as the new current node, and this step is recursively executed. If the current node has no child nodes, it is backtracked to its parent node and the next child node of the parent node is set as the new current node, and this step is recursively executed.

[0145] If the PIS of the current node is less than the preset scoring threshold PIS_min, it is checked whether its similarity with an important ancestor node (ie, a node whose PIS is not less than PIS_min) exceeds the threshold sim_min.

[0146] If yes, keep the current node, but do not recursively process its child nodes. Go back to its parent node directly, set the next child node of the parent node as the new current node, and recursively execute the judgment on the current node.

[0147] If the answer is no, prune the current node and all its subtrees. Directly backtrack to its parent node, set the next child node of the parent node as the new current node, and recursively execute the judgment on the current node.

[0148] Exemplarily, as Figure 4 shown, assume PIS_min = 0.6, sim_min = 0.8, PIS_A = 0.8, PIS_B = 0.6, PIS_C = 0.7, PIS_D = 0.5. After the filtering in S504, node D will be directly pruned because its PIS = 0.5 is less than PIS_min. However, if the similarity between node D and its parent node C is very high (e.g., sim(C, D) = 0.9), it means that although D's own score is low, it is highly related to the important pattern C and may contain some valuable information. PIS_A represents the pattern importance score at pattern node A, PIS_B represents the pattern importance score at pattern node B, PIS_C represents the pattern importance score at pattern node C, PIS_D represents the pattern importance score at pattern node D, sim(C, D) represents the similarity between pattern node C and pattern node D, and A, B, C, D respectively represent specific pattern nodes.

[0149] For node A, since PIS ≥ PIS_min, it is retained, and its child nodes B and C are recursively processed.

[0150] For node B, PIS < PIS_min, and the similarity with the ancestor node A does not meet the threshold condition, so it is pruned.

[0151] For node C, since PIS ≥ PIS_min, it is retained, and its child node D is recursively processed.

[0152] For node D, although PIS < PIS_min, the similarity with the parent node C meets the threshold condition (sim(C, D) = 0.9 > sim_min), so it is still retained, but its child nodes are no longer recursively processed.

[0153] Through such a recursive pruning process, S505 can, on the basis of S504, further consider the hierarchical relationship and similarity between pattern nodes, selectively retain some important sub-patterns, and obtain a more refined and comprehensive pattern tree representation.

[0154] S5053, when traversing the entire pattern tree, that is, when backtracking to the root node and all child nodes of the root node have been processed, the recursive pruning process ends.

[0155] S5054, a recursively pruned pattern tree is obtained, in which important nodes and sub-patterns that are highly relevant to important patterns are retained, and nodes and sub-trees that are neither important nor have low relevance to important patterns are pruned.

[0156] Through this top-down, depth-first recursive pruning process, S505 can effectively traverse and prune the entire pattern tree. When judging whether to retain each node, by introducing the similarity comparison of ancestor nodes, the hierarchical relationship and semantic relevance between pattern nodes can be more comprehensively considered. In this way, while removing redundant information, the key patterns and their contextual information can be retained to the maximum extent, resulting in a more refined, comprehensive, and well-structured pattern tree representation.

[0157] It should be noted that the similarity calculation method of the ancestor node is consistent with the similarity calculation in S501. Sim_min is a pre-set threshold used to determine whether the similarity between the current node and the important ancestor node is high enough. It determines that during the pruning process, for nodes with lower importance scores,

[0158] At the same time, through recursion, S505 can naturally adapt to pattern trees of arbitrary depth and width, and has good versatility and scalability. Compared with the simple threshold pruning of S504, the recursive pruning of S505 can optimize the structure and content of the pattern tree more finely and comprehensively, and improve the efficiency and accuracy of subsequent pattern matching and query.

[0159] S506: re-encode and optimize storage of the second pattern tree, for example, use a more compact node structure, delete redundant indexes, etc., to obtain a final simplified pattern tree.

[0160] In addition, this improved pruning method can more specifically identify key patterns, extract more refined and practical feature sets, and reduce irrelevant or redundant information. The second pattern tree can accelerate the subsequent matching and query process, improving the real-time performance and resource utilization of the system.

[0161] Example 3

[0162] like Figure 5 As shown, a tree-structured simplified sequence pattern calculation unit data column storage system is provided, including a source data acquisition module, a column structure generation module, a pattern information acquisition module, an original pattern tree construction module, and a simplified pattern tree acquisition module. The tree-structured simplified sequence pattern calculation unit data column storage system corresponds to the tree-structured simplified sequence pattern calculation unit data column storage method in the above embodiment. The detailed description of each functional module is as follows:

[0163] The source data acquisition module is used to acquire the computing unit operation information of the computing unit, perform data normalization, and obtain the source data of the computing unit.

[0164] The column structure generation module is used to input the source data into a column storage engine, so that the source data forms a plurality of column structures and determines the storage location of the source data.

[0165] The pattern information acquisition module is used to divide the numerical indicators of the source data into intervals, determine the corresponding pattern information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding pattern information, wherein the source data includes numerical indicators and categorical indicators.

[0166] The original pattern tree construction module is used to obtain metadata of multiple sampling time points and construct the original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information.

[0167] The simplified pattern tree acquisition module traverses the original pattern tree and prunes it according to the current computing unit operation information to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

[0168] Example 4

[0169] like Figure 6 As shown, the present application also provides an electronic device 500. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the tree-structured simplified sequence pattern calculation unit data column storage method can be executed.

[0170] The method or system according to the embodiment of the present application can also be used by Figure 6 The electronic device architecture shown in FIG. Figure 6As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, and the like. The storage device in the electronic device 500, such as ROM503 or hard disk 507, can store the computing unit operation information of the computing unit provided by the present application, perform data normalization, and obtain the source data of the computing unit; input the source data into a column storage engine, so that the source data forms a plurality of column structures, and determine the storage location of the source data; divide the numerical indicators of the source data into intervals, and determine the corresponding pattern information according to the interval in which the numerical values ​​are located; and directly map the category indicators of the source data to the corresponding pattern information, and the metadata of the source data includes the storage location and pattern information of the source data; set the original pattern tree to be initialized as an empty tree, and construct the original pattern tree structure with the metadata of multiple sampling time points; according to the current computing unit operation information, traverse the original pattern tree and prune it to obtain and save a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 6 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 6 One or more components of an electronic device are shown.

[0171] Example 5

[0172] like Figure 7 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the tree-structured simplified sequence pattern calculation unit data column storage method according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0173] In addition, according to the implementation mode of the present application, the process described in the above reference flow chart can be implemented as a computer software program. For example, the present application provides a non-temporary machine-readable storage medium, the non-temporary machine-readable storage medium stores machine-readable instructions, the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present application, for example: obtaining the computing unit operation information of the computing unit, performing data normalization, and obtaining the source data of the computing unit; inputting the source data into a column storage engine so that the source data forms a plurality of column structures, and determining the storage location of the source data; dividing the numerical index of the source data into intervals, and determining the corresponding mode information according to the interval in which the numerical value is located; and directly mapping the category index of the source data to the corresponding mode information, the metadata of the source data including the storage location and mode information of the source data; setting the original mode tree to be initialized as an empty tree, and constructing the original mode tree structure with the metadata of multiple sampling time points; according to the current computing unit operation information, traversing the original mode tree and pruning, obtaining and saving a simplified mode tree composed of simplified sequence modes with the highest similarity to the current computing unit operation information. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are performed.

[0174] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0175] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0176] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A column storage method for simplified sequence pattern calculation unit data based on a tree structure, comprising: S10: Obtain computing unit operation information of the computing unit, perform data normalization, and obtain source data of the computing unit; S20: inputting the source data into a column storage engine, so that the source data forms a plurality of column structures, and determining a storage location of the source data; S30: Divide the numerical indicators of the source data into intervals, determine the corresponding mode information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding mode information, wherein the source data includes the numerical indicators and the categorical indicators; S40: Obtain metadata of multiple sampling time points and construct an original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information; S50: According to the current computing unit operation information, traverse the original pattern tree and prune it to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

2. According to the tree-structured simplified sequential pattern computing unit data column storage method of claim 1, traversing the original pattern tree and pruning according to the current computing unit operation information comprises: Get each pattern node in the original pattern tree, calculate the occurrence frequency f of each pattern node, the pattern length L, the time span t of the occurrence of a single pattern node, and the similarity s between the current pattern node and its ancestor node; Constructing a pattern importance scoring function through pattern nodes; The importance score of each pattern node is calculated by the pattern importance scoring function to obtain a pattern node score list; The lowest threshold of the pattern function is set, the pattern node score list is filtered, and the pattern nodes whose scores are smaller than the lowest threshold of the pattern function are removed to obtain the first pattern tree.

3. The method for storing data in a column format based on a tree structure for simplified sequential pattern calculation units according to claim 2, characterized in that: The method of traversing the original pattern tree and pruning it according to the current computing unit operation information also includes introducing a recursive pruning algorithm to prune the first pattern tree from top to bottom to obtain a second pattern tree through importance scores of pattern nodes, hierarchical relationships between pattern nodes, and similarities between pattern nodes and their ancestor nodes.

4. The method for storing data in a column format based on a tree structure for simplified sequential pattern computing units according to claim 2, characterized in that: The pattern importance scoring function is as follows: PIS=w1*log(1+f)+w2*(1-e (-L / L0) )+w3*(1-e (-t / T) )+w4*s; Among them, w1 to w4 are the weight coefficients of the four factors, T is the total time span covered by all pattern nodes, L0 is a length threshold parameter, L represents the pattern length, log(1+f) represents the logarithm of the frequency, f represents the occurrence frequency of the pattern node, (1-e (-L / L0) ) represents the exponential saturation function of length, s represents the similarity between the pattern node and its ancestor node, and PIS represents the importance score of the node.

5. The method for storing data in a column format based on a tree structure for simplified sequential pattern computing units according to claim 4, characterized in that: The steps for determining the length threshold parameter L0 are as follows: Statistics of the length distribution of all pattern nodes: traverse the original pattern tree, calculate the length L of each pattern node, and obtain the set of all pattern node lengths {L1,L2,…,L n }; where n represents the number of pattern nodes, L n Indicates the length of the nth pattern node and calculates the average length of the pattern node Among them, i∈n; L i Indicates the length of the i-th pattern node; Calculate the standard deviation σ of the pattern node lengths: Get L0, where the L0 acquisition function is as follows: Among them, k is a tuning parameter.

6. The method for storing data in a column format based on a tree structure for simplified sequential pattern computing units according to claim 5, characterized in that: The steps to determine the k value are as follows: Get the set of pattern node lengths {L1,L2,…,L n }; Calculate the average length of pattern nodes Get standardized mode node data Perform K-means clustering on the standardized pattern node data, select the appropriate number of clusters m, and obtain the cluster label of each pattern node length; Calculate the skewness γ of the pattern node length L and kurtosis κ L ; Construct the feature matrix X, which contains the cluster labels, skewness and kurtosis of each pattern node length; The regression model is trained by historical data to predict the k value, wherein the historical data includes the historical feature matrix X history And the k value y corresponding to the history history ; Use the trained regression model, input the feature matrix X of the current mode node, get the predicted k value, and take the average of the predicted k values ​​as the final target k value.

7. The method for storing data in a column format based on a tree structure for simplified sequential pattern calculation units according to claim 3, characterized in that: The second mode tree determination process is as follows: Starting from the root node of the first pattern tree, set it as the current node; Make the following judgments on the current node: If the importance score PIS of the current node is greater than or equal to the preset score threshold PIS_min, then keep the current node and set its first child node as the new current node, and recursively execute this step; if the current node has no child node, then backtrack to its parent node and set the next child node of the parent node as the new current node, and recursively execute this step; If the PIS of the current node is less than the preset score threshold PIS_min, check whether its similarity with any ancestor node exceeds the preset similarity threshold sim_min; If the similarity is greater than or equal to the preset similarity threshold sim_min, the current node is retained, its child nodes are no longer recursively processed, and the node is directly traced back to its parent node, and the next child node of the parent node is set as the new current node, and the judgment of the current node is recursively executed; If the similarity is less than the preset similarity threshold sim_min, the current node and all its subtrees are pruned; directly backtrack to its parent node, and set the next child node of the parent node as the new current node, and recursively perform the judgment on the current node.

8. A tree-structured simplified sequence pattern calculation unit data column storage system, the system executing the steps of the tree-structured simplified sequence pattern calculation unit data column storage method according to any one of claims 1 to 7, characterized in that: include: A source data acquisition module, used to acquire computing unit operation information of a computing unit, perform data normalization, and obtain source data of the computing unit; A column structure generation module, used for inputting the source data into a column storage engine, so that the source data forms a plurality of column structures, and determining a storage location of the source data; A pattern information acquisition module is used to divide the numerical indicators of the source data into intervals, determine the corresponding pattern information according to the intervals where the numerical indicators are located; and directly map the categorical indicators of the source data to the corresponding pattern information, wherein the source data includes numerical indicators and categorical indicators; The original pattern tree construction module is used to obtain metadata of multiple sampling time points and construct the original pattern tree; the metadata includes the storage location of the source data and the corresponding pattern information; The simplified pattern tree acquisition module traverses the original pattern tree and prunes it according to the current computing unit operation information to obtain a simplified pattern tree composed of simplified sequence patterns with the highest similarity to the current computing unit operation information.

9. An electronic device, characterized in that: include: processor and memory, wherein The memory stores a computer program that can be called by the processor; The processor executes the steps of the tree-structured column storage method for reduced sequence pattern calculation unit data as described in any one of claims 1 to 7 in the background by calling the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: A rewritable computer program is stored thereon; When the computer program is executed on a computer device, the computer device executes the steps of the tree-structured column storage method for reduced sequence pattern calculation unit data as described in any one of claims 1 to 7 in the background.

Images