A multi-cloud storage autonomous intention deployment method and system

Abstract

The present invention relates to the field of cloud computing technology, and in particular to a multi-cloud storage autonomous intention deployment method and system. The method comprises the following steps: obtaining comprehensive operation and maintenance data of multi-cloud storage, preprocessing the comprehensive operation and maintenance data, and obtaining a unified data format; constructing a policy tree by using a directed acyclic graph, inputting the policy tree data into a control layer to parse the policy tree structure and divide it into a plurality of subtrees, initializing a control layer state set to enter an initialization state; transferring the initialization state to a subtree execution state, and executing subtree node tasks hierarchically; performing feature matching based on error information, generating a policy candidate set, sorting the candidate policies, and selecting an optimal policy to perform an error recovery operation. The policy layer of the present invention stores a response policy set, dynamically adjusts the matching relationship between the policy and the error information, and the policy management mechanism can continuously optimize the policy selection according to the actual situation, thereby improving the success rate and efficiency of fault handling.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a multi-cloud storage autonomous intention deployment method and system. Background Art

[0002] In the existing technology, cloud-network integration has become a key trend in the development of the industry. Multi-cloud platforms have emerged to meet the urgent needs of future service customization networks for cloud-network integration. Among the various cloud resources in the multi-cloud platform, storage resources occupy a pivotal position. Their efficient management and utilization are crucial to the stable operation of the entire platform and user experience.

[0003] As a key link in end-to-end management and control, intention deployment plays a decisive role in the reliability and integrity of the intelligent management and control system. In a multi-cloud environment, intention deployment needs to interact with many application programming interfaces (APIs) of different cloud vendors. However, there are complex logical relationships between different APIs, and their automatic execution must strictly follow a specific order. Errors in any link may lead to deployment failures. What is more serious is that once the API call fails, the entire intention deployment process will be interrupted. If the intelligent management and control system lacks effective self-repair capabilities, it will not only waste a lot of time and resources, but may also cause business interruptions and bring serious losses to the enterprise. Most of the existing intention deployment solutions have limitations. Some solutions lack an effective processing mechanism for complex API logic and cannot guarantee the smooth progress of the deployment process; some solutions cannot quickly and accurately recover from faults when faced with errors, which seriously affects the reliability and stability of the system. At this stage, a multi-cloud storage autonomous intention deployment method and system is needed. Summary of the invention

[0004] In order to solve the problem that the intelligent management and control system lacks effective self-repair capabilities, the present invention provides a multi-cloud storage autonomous intention deployment method and system. The present invention transplants deterministic policies to the multi-cloud platform and automatically executes incremental execution, monitors the status of multi-cloud storage resources in real time, and promptly discovers situations that do not comply with policy rules. Once an anomaly is detected, the breakpoint correction mechanism is triggered, and the fault processing database is used to automatically complete breakpoint error recovery.

[0005] In a first aspect, the present invention provides a multi-cloud storage autonomous intention deployment method, which adopts the following technical solution:

[0006] A multi-cloud storage autonomous intention deployment method, comprising:

[0007] Obtain comprehensive operation and maintenance data of multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format;

[0008] Constructing a strategy tree using a directed acyclic graph, including determining a set of nodes and a set of directed edges based on comprehensive operation and maintenance data;

[0009] Input the strategy tree data into the control layer to parse the strategy tree structure and divide it into several subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state;

[0010] Transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically;

[0011] Construct a fault handling database model based on the error data in the node task, including constructing the error information layer, strategy layer and mapping relationship layer;

[0012] Perform feature matching based on error information to generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operations.

[0013] Furthermore, the preprocessing of the comprehensive operation and maintenance data includes generating a hash value of a fixed length according to the acquired comprehensive operation and maintenance data, processing the hash value using a plurality of different hash functions to obtain a plurality of index values ​​for setting Bloom filter bits, removing duplicate data by comparing the hash values, and then filling in the missing data values ​​using a filling method based on association rule mining and weighted average, converting the cleaned data from the original format to the target format according to the defined standard structure of each data format and the field mapping relationship, introducing a compression method based on adaptive coding, and storing the processed data in a relational database.

[0014] Furthermore, determining a node set and a directed edge set based on the comprehensive operation and maintenance data includes using a directed acyclic graph to represent a decision tree, determining a node set based on business logic and resource management requirements in the comprehensive operation and maintenance data, wherein the node set includes an editing level, a policy parameter name, and a parameter value, and then determining a directed edge set based on an execution sequence and dependencies between nodes, and adding directed edges by calculating dependencies between nodes.

[0015] Furthermore, the adding of directed edges by calculating the dependency between nodes includes analyzing the probability of node u being executed after node v is successfully executed in historical data. To calculate the dependency, set a dependency threshold, and when the dependency is higher than the set threshold, add a directed edge from v to u in the strategy tree.

[0016] Furthermore, the strategy tree data is input into the control layer to parse the strategy tree structure and divide it into several subtrees, including inputting the constructed strategy tree data into the control layer, completely traversing the node set and the directed edge set, extracting the node features and the dependencies of the nodes in the directed edges, defining the node association and node similarity according to the node features and the dependencies, constructing the similarity matrix of the strategy tree and calculating the degree matrix and the Laplace matrix, performing eigenvalue decomposition on the Laplace matrix, taking the eigenvectors corresponding to the first k smallest eigenvalues ​​to form a matrix, clustering the matrix using the k-means algorithm, and the node set corresponding to each clustering result constitutes a subtree.

[0017] Furthermore, the node association degree is defined according to the node characteristics and dependency relationships, including determining the shortest path length between different nodes and the number of common neighbor nodes according to the result of a complete traversal of the node set and the directed edge set, and calculating the node association degree using the shortest path length and the number of common nodes. The node association degree calculation formula is:

[0018] ,

[0019] in, It is expressed as the shortest path length between node u and node v, Expressed as the number of common neighbor nodes, and are the neighbor node sets of nodes u and v respectively.

[0020] Further, the transferring of the initialization state to the subtree execution state includes setting a corresponding execution state in the control layer control loop for each subtree, initializing the control layer state set and entering the initialization state. , introduces a resource pre-allocation mechanism to pre-allocate system resources according to the complexity of the strategy tree. The internal resource allocation and parameter setting are completed according to the pre-allocated resource amount. The strategy tree complexity calculation formula is:

[0021] ,

[0022] in, , , and is the weight coefficient, and , is the average out-degree of the node, The maximum logical level.

[0023] Furthermore, the error information layer, strategy layer and mapping relationship layer are constructed, including an error information layer storing error data in task execution, using a word embedding model to convert natural language descriptions into vectors, using named entity recognition technology to extract key entity sets, and further updating feature vectors. The strategy layer stores a set of response strategies for solving each error, and the mapping relationship layer stores a many-to-many relationship between error information and response strategies. By setting adaptation weights and defining a fuzzy rule base, the similarity between the error feature vector and the applicable feature vector of the strategy is calculated, and the updated adaptation weight is obtained through a fuzzy reasoning engine and defuzzification.

[0024] Furthermore, the candidate strategies are sorted and the optimal strategy is selected to perform error recovery operations, including using cosine similarity to calculate the similarity between the error feature vector and the existing error feature vector in the fault processing database, introducing local sensitive hashing to map the error feature vector to a hash bucket, selecting error records with a similarity greater than a preset threshold according to the hash bucket, obtaining the corresponding strategy set at the mapping relationship layer, generating a strategy candidate set, calculating the priority score of the strategy candidate set, and selecting the optimal strategy according to the priority score to perform error recovery operations.

[0025] In a second aspect, a multi-cloud storage autonomous intention deployment system includes:

[0026] The data acquisition module is configured to: acquire the comprehensive operation and maintenance data of the multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format;

[0027] The decision tree module is configured to: construct a policy tree using a directed acyclic graph, including determining a set of nodes and a set of directed edges based on the comprehensive operation and maintenance data;

[0028] The conversion module is configured to: input the policy tree data into the control layer to parse the policy tree structure and divide it into several subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state;

[0029] The processing module is configured to: transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically;

[0030] The model module is configured to: construct a fault processing database model according to the error data in the node task, including constructing an error information layer, a strategy layer, and a mapping relationship layer;

[0031] The transformation module is configured to: perform feature matching based on error information, generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operation.

[0032] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, for example, in a multi-cloud storage autonomous intention deployment method.

[0033] In a fourth aspect, the present invention provides a terminal device, comprising a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being suitable for being loaded by the processor and executing the multi-cloud storage autonomous intention deployment method.

[0034] In summary, the present invention has the following beneficial technical effects:

[0035] 1. The present invention generates a hash value of a fixed length and uses multiple different hash functions for processing, and uses a Bloom filter to remove duplicate data. It can quickly and accurately identify and eliminate duplicate information in comprehensive operation and maintenance data. This not only reduces data redundancy and storage costs, but also avoids errors and waste of computing resources caused by duplicate data in subsequent processing, thereby improving overall processing efficiency.

[0036] 2. The present invention adopts a filling method based on association rule mining and weighted average to fill in missing data values. It can reasonably infer and supplement missing data according to the intrinsic correlation and statistical characteristics between the data, which ensures the integrity of the comprehensive operation and maintenance data, making subsequent analysis and decision-making based on these data more reliable and accurate.

[0037] 3. The present invention adds directed edges by calculating the dependencies between nodes, clearly defining the execution sequence and dependencies between nodes, which helps to reasonably arrange the task execution sequence when executing strategy tree node tasks, avoid errors and conflicts caused by confusion in task dependencies, and improve the reliability and efficiency of strategy execution.

[0038] 4. The present invention uses node features and dependencies to define node association and similarity. By constructing a similarity matrix, a degree matrix and a Laplace matrix, and performing eigenvalue decomposition and k-means clustering, nodes with similar features and dependencies can be divided into the same subtree. This division method conforms to the inherent logical structure of the strategy tree and improves the rationality and effectiveness of subtree division.

[0039] 5. The present invention sets a corresponding execution state in the control layer control loop for each subtree, transfers the initialization state to the subtree execution state, and completes internal resource allocation and parameter setting in the initialization state, so that the execution process of the strategy tree proceeds in an orderly manner. This state transition mechanism helps to ensure the stability and reliability of strategy execution and improves the overall performance of the system.

[0040] 6. The strategy layer of the present invention stores a set of response strategies, and the mapping relationship layer dynamically adjusts the matching relationship between the strategy and the error information by setting adaptation weights, defining a fuzzy rule base, and calculating similarities. The strategy management mechanism can continuously optimize strategy selection according to actual conditions, thereby improving the success rate and efficiency of fault handling.

[0041] 7. The present invention calculates the priority scores of the strategy candidate sets and selects the optimal strategy to perform error recovery operations based on the scores. It can select the strategy that best suits the current error from among many candidate strategies, thereby improving the success rate of error recovery, reducing the impact of errors on system operation, and ensuring the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the execution status of a multi-cloud storage autonomous intention deployment method according to an embodiment of the present invention.

[0043] Figure 2 It is a schematic diagram of the overall process of a multi-cloud storage autonomous intention deployment method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0045] Example 1

[0046] Reference Figure 1 , a multi-cloud storage autonomous intention deployment method of this embodiment includes:

[0047] Obtain comprehensive operation and maintenance data of multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format;

[0048] Constructing a strategy tree using a directed acyclic graph, including determining a set of nodes and a set of directed edges based on comprehensive operation and maintenance data;

[0049] Input the strategy tree data into the control layer to parse the strategy tree structure and divide it into several subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state;

[0050] Transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically;

[0051] Construct a fault handling database model based on the error data in the node task, including constructing the error information layer, strategy layer and mapping relationship layer;

[0052] Perform feature matching based on error information to generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operations.

[0053] Specifically, a multi-cloud storage autonomous intention deployment method includes the following steps:

[0054] like Figure 2 As shown, S1, obtaining the comprehensive operation and maintenance data of multi-cloud storage, and preprocessing the comprehensive operation and maintenance data to obtain a unified data format;

[0055] Collect multi-cloud storage comprehensive operation and maintenance data from various components and related systems of the multi-cloud platform. This data covers storage resource usage, system performance indicators, historical error logs, and user operation records. In order to obtain data more comprehensively and accurately, a data collection strategy based on adaptive sampling rate is adopted. This strategy dynamically adjusts the frequency of data collection according to the real-time operation status of the system. Specifically, when the system is in a high-load and abnormal period, the sampling rate is increased to capture more detailed data; when the system is running stably, the sampling rate is reduced to reduce the overhead of data collection. The real-time load index of the system is L through comprehensive calculations such as CPU usage and memory usage. The calculation formula of the sampling rate r is as follows:

[0056] ,

[0057] The initial sampling rate is , the maximum sampling rate is , the minimum sampling rate is , the load threshold is .

[0058] After that, duplicate data is identified and removed. An improved Bloom filter combined with a hash algorithm is used to identify duplicate data. Traditional Bloom filters may have a certain misjudgment rate when judging whether data exists. The improved Bloom filter reduces the misjudgment rate by introducing multiple hash functions and dynamically adjusting the filter size. First, a fixed-length hash value h is generated for each data record, and k different hash functions are used. Process the hash value h to get k index values , and then set the bits in the Bloom filter corresponding to these index values ​​to 1. When judging whether a new data record is duplicate data, k index values ​​are also calculated. If the bits corresponding to these index values ​​are all 1, it is considered that the data record may be duplicate data. For further confirmation, accurate hash value comparison is used to finally determine whether it is duplicate data. Assuming the size of the Bloom filter is m, the total number of data records is n, and in order to minimize the false positive rate p, the number of hash functions k is calculated as follows:

[0059] ,

[0060] Where m represents the size of the Bloom filter, n represents the total number of data records, and the calculation formula for the size of the Bloom filter m is:

[0061] ,

[0062] Where p is the misjudgment rate of the Bloom filter.

[0063] For incomplete data, a filling method based on association rule mining and weighted average is adopted. First, the association relationship between data fields is found using the association rule mining algorithm (Apriori algorithm). Suppose there are n records in the data set, and each record contains d fields. ,in, Represents each field in the record, and uses the Apriori algorithm to find frequent item sets and association rules, such as finding fields and Fields There is a strong correlation between the missing fields. According to the association rules, find the set of records S that are similar to the record in other fields, and then calculate the field in the set S. As the filling value, suppose there are m records in the set S, and the weight of each record is (The weight can be calculated based on the similarity between the record and the target record), then fill in the value The calculation formula is:

[0064] ,

[0065] in, Represents the value of the i-th field of the j-th record in the set S. If no suitable association or fill value can be found, the record is deleted directly.

[0066] The cleaned data is unified into Parquet format. In order to achieve the unification of data format, this embodiment designs a method based on data mapping and conversion rules. First, the standard structure and field mapping relationship of each data format are defined. Assume that the original data format is , the target data format is , the field mapping relationship is M, where M is a dictionary with keys The field name in , and the value is For each data record, the fields are renamed and reorganized according to the field mapping relationship M, and the data is converted to the target format. In order to improve the efficiency of data processing, a compression technology based on adaptive coding is introduced. This technology dynamically selects the appropriate encoding method for compression according to the distribution characteristics and statistical information of the data. For example, for numerical data, if the data distribution is relatively concentrated, run-length encoding is used; for text data, if there are a large number of repeated strings, Lempel-Ziv-Welch (LZW) encoding is used. Suppose the data block D contains n data elements, and the entropy of the data is calculated. To evaluate the complexity and randomness of data. The entropy calculation formula is:

[0067] ,

[0068] Where n represents the number of data elements. is the probability of data i appearing. The encoding method is selected for compression according to the entropy value to achieve the best compression effect. Finally, the processed data is stored in a relational database to provide a reliable data basis for subsequent analysis and use.

[0069] S2. constructing a strategy tree using a directed acyclic graph, including determining a node set and a directed edge set according to the comprehensive operation and maintenance data;

[0070] The strategy tree is constructed based on the collected comprehensive operation and maintenance data. The strategy tree model is represented by a directed acyclic graph G=(V,E), where V is a node set and E is a directed edge set. The node set is determined according to the business logic and resource management requirements in the data. Each node With logical hierarchy , Strategy parameter name and parameter values Attributes, set up nodes based on resource indicators: according to the capacity and performance indicators of storage resource usage, set up storage resource allocation related nodes, set the capacity indicator of storage resources as C, and the performance indicator as P (read and write speed). When C is lower than a certain threshold Or P is below the threshold When setting up storage resource expansion nodes or storage resource optimization node , storage resource expansion node The policy parameter name It can be "capacity expansion", parameter value Based on the current capacity C and expected usage It is calculated that , storage resource optimization node The policy parameter name You can select the optimization algorithm, parameter value It is obtained based on the performance index P and the effect evaluation of different optimization algorithms.

[0071] Establish nodes based on user operations: According to the data backup and access permission setting requirements in the user operation records, establish corresponding function nodes. For example, suppose the frequency of data backup operations of users within a period of time T is ,when Above a certain threshold When setting up an automatic backup strategy to adjust the node , the node's policy parameter name Can be "Backup cycle adjustment", parameter value According to the operating frequency Perform dynamic adjustment, the adjustment formula is:

[0072] ,

[0073] Where T represents the length of the time period used to count the frequency of user data backup operations. The directed edge set E is determined based on the execution sequence and dependency relationship between nodes to ensure the accuracy of the task execution logic. Expressed as the operating frequency, Dependency quantification: In order to more accurately determine the dependency between nodes, the dependency D(u,v) is introduced to measure the degree of dependency of node u on node v. Assuming that there is a certain business association between nodes u and v, the probability of node u executing after node v successfully executes in historical data is analyzed. To calculate the dependency, the formula is ,when Above a certain threshold When , we consider node u to be dependent on node v, and add a directed edge (v,u) from v to u in the strategy tree. For nodes with a sequential execution order, we determine the directed edge through time series analysis. Let the operations corresponding to nodes u and v be and ,if And meet a certain time interval threshold ,Right now , then node v is considered to be executed before node u, and a directed edge (v,u) is added from v to u.

[0074] S3, inputting the strategy tree data into the control layer to parse the strategy tree structure and divide it into several subtrees, setting the execution state for each subtree, and initializing the control layer state set to enter the initialization state;

[0075] The constructed strategy tree data is input into the control layer. After receiving the data, the control layer needs to parse the strategy tree structure in order to divide it into several subtrees. The control layer first traverses the node set V and directed edge set E in the strategy tree G=(V,E). For each node , extract its logical hierarchy ), strategy parameter name and parameter values Equal attributes, for each directed edge , records the dependency relationship between nodes u and v.

[0076] In order to improve the accuracy and flexibility of the division, the graph clustering algorithm is used to automatically divide the strategy tree into different subtrees according to the association and similarity between nodes. , through the shortest path length between them And the number of common neighbor nodes To measure, the correlation formula is as follows:

[0077] ,

[0078] in, and are the neighbor node sets of nodes u and v, and the similarity between nodes u and v Calculated according to their strategy parameter values, Expressed as the shortest path length, assuming that the strategy parameter value vectors of nodes u and v are and , then the similarity is calculated using cosine similarity:

[0079] ,

[0080] in, and They are represented as the strategy parameter value vectors of nodes u and v respectively. Then, the strategy tree is divided using the spectral clustering algorithm. First, the similarity matrix W of the strategy tree is constructed, where , Represented as a node and nodes Then, we calculate the degree matrix D, whose diagonal elements , then, calculate the Laplacian matrix , perform eigenvalue decomposition on the Laplace matrix L, take the eigenvectors corresponding to the first k smallest eigenvalues ​​to form the matrix U, regard each row of the matrix U as a k-dimensional vector, use the k-means algorithm to cluster these vectors, and obtain k clustering results. The node set corresponding to each clustering result constitutes a subtree .

[0081] like Figure 1As shown, S4, transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically;

[0082] For each subtree , set the corresponding execution state in the control loop of the control layer, that is, These execution states represent the different states of the control layer when executing the subtree tasks, and initialize the control layer state set , and enter the initialization state , the control layer is initialized from the Transfer to the subtree execution state, activate the control layer loop state machine and send an enable signal to the execution layer state machine. If no error occurs, the state transition is: ,in, For subtree If If an error occurs or an error occurs in other subtree execution states, the control layer enters the error handling state. ,Right now .

[0083] exist In the state, the control layer queries the fault processing database K according to the feedback information e of the execution layer to find the optimal response strategy .in, Represents a specific coping strategy, belonging to the set of all optional coping strategies , Represents error information passed from the execution loop, Indicates known error information Under the conditions of and fault database F, the strategy is selected The posterior probability of .

[0084] ,

[0085] in, Represented as the set of all optional response strategies, Indicates known error information Under the conditions of and fault database F, the strategy is selected The posterior probability of Represented as finding a strategy among all the optional response strategies , so that under the condition of known error information e and fault database F, the posterior probability of selecting this strategy is Reached maximum value.

[0086] Then, the state transfer is determined according to the optimal strategy. If it is necessary to return to the initial state, return If you need to recover from a breakpoint, enter the breakpoint state , where the breakpoint state is the last successfully completed state in the subtree execution loop where the error occurs. This state is the most recent stable state before the error occurs, and provides a recovery point for continuing execution from this state. The breakpoint state is defined as follows:

[0087] Assume that the state set is , the state sequence is marked in chronological order as , assuming the state If it is an error state, the breakpoint The definitions are as follows. Indicates the execution path i status, Indicates the state where an error occurred (error state), Indicates in status The probability of an error occurring when Indicates the threshold of error probability. If it is greater than the threshold, it means that the state can be considered as a condition for success. max means taking the state closest to status, Make sure the breakpoint state appears before the error state, the condition Ensure that the breakpoint state is a stable state, that is, the probability of error occurrence is low enough, then The formula is:

[0088] ,

[0089] in, represents the threshold of error probability, Indicates in status The probability of an error occurring when Indicates that the breakpoint state appears before the error state, and the execution layer responds to the enable signal sent by the control layer , the signal is defined as , where n is the number of subtrees in the strategy tree, Represents a subtree The corresponding execution layer loop, Represents a subtree The corresponding execution layer state machine, and so on,

[0090] like , then the execution layer selects the corresponding subtree State machine and enters the initialization state , subtree The loop state set is:

[0091] ,

[0092] After initialization is completed, the state machine executes the first-layer node state in sequence according to the level , the second layer node state , until the leaf node or an error occurred e , if the operation fails, the state transition is , For storing errors And provide feedback to the executive level, Used according to the control layer strategy , choose to restore from the initial state or breakpoint. If the state machine runs to If it is correct, transfer to the verification state , the verification state passes the verification result v and evaluates the execution success: ,in, It is represented as the threshold used to judge whether the execution is successful. It is expressed as a conditional probability, which means the probability of error e occurring under the condition that the verification result is v. If the verification fails, it is converted to an error record state. Otherwise, return to the initial state .

[0093] In summary, the execution layer responds to the task execution. After receiving the enable signal u, the execution layer selects the corresponding subtree state machine to enter the initialization state. After the initialization is completed, the execution layer executes the subtree node tasks in order, starting from the root node, along the direction of the directed edge, and gradually executes to the leaf node. During the execution process, the task status is monitored in real time. Suppose the currently executed node is, if an error occurs during the execution process, the execution layer will feedback the error information f to the control layer and suspend the current task execution; if there is no error, it will continue to execute the next layer of node tasks until all node tasks are completed.

[0094] In order to ensure that the initialization process of the control layer is more stable and efficient, a resource pre-allocation mechanism is introduced. In the initialization phase, a certain amount of system resources are pre-allocated according to the scale and complexity of the strategy tree. The scale of the strategy tree is measured by the number of nodes |V| and the number of edges |E|, and the complexity is measured by the average out-degree of the nodes. and maximum logical level To evaluate and define the comprehensive complexity index of the strategy tree as follows:

[0095] ,

[0096] in, , , and is the weight coefficient, and , It is expressed as the average out-degree of the node, Represented as the maximum logical level,

[0097] According to the comprehensive complexity index , pre-allocate system resources, and set the total system resources to , including CPU resources , memory resources , the amount of pre-allocated resources It can be calculated according to the following formula:

[0098] ,

[0099] in, is the resource allocation coefficient, which is adjusted according to the actual system conditions. Expressed as a comprehensive complexity index, Indicates the total system resources. In this case, the control layer completes the internal resource allocation, parameter setting and other preparatory work according to the calculated pre-allocated resource amount, prepares for the subsequent task scheduling and execution, and avoids resource competition and bottleneck problems during task execution.

[0100] S5. Construct a fault processing database model according to the error data in the node task, including constructing an error information layer, a strategy layer, and a mapping relationship layer;

[0101] The fault handling knowledge base 𝐾 is a relational and hierarchical database, and its core structure can be divided into error information layer, strategy layer and mapping relationship layer.

[0102] The error information layer stores all known error information, including the description, type, and feature representation of the error. It is formalized as: , each error message ,in, is the error code, is the natural language description of the error, Represents the error feature vector The extracted features can be log content, time, context, etc. The feature vector is generated by extracting multi-dimensional information such as log content, time, context, etc. when the error occurs. Suppose the log content when the error occurs is , timestamp is timestamp, related system configuration and operation steps are context, then the feature vector is:

[0103] ,

[0104] in, Indicates the log content when the error occurs. Represented as a timestamp, Expressed as operation steps, in order to make the feature vectors more representative and comparable, the log content is preprocessed. First, stop words (such as "de", "shi", "zai", etc.) are removed, and then stemming is performed to restore words to their stem forms. For example, "running" is restored to "run". Let the processed log content be , then the feature vector is updated to:

[0105] ,

[0106] where represents the processed log content, and the policy layer stores a set of coping strategies for solving each type of error , and each strategy has , where represents the step vector for solving the error, represents the cost of solving this error, represents the probability of the success of the strategy.

[0107] The mapping relationship between error messages and coping strategies is the following many-to-many relationship:

[0108] ,

[0109] where represents the error and the strategy 's adaptation weight, reflecting the possibility or priority of the strategy being applied to the error .

[0110] According to the characteristics of error features and strategies, fuzzy rules are defined. If the error feature vector has a high similarity with the applicable feature vector of strategy s, and the success probability of strategy is high, then the adaptation weight w(e, ) of strategy to this error is high. If the severity of the error is high, and the solution cost of strategy is low and the success probability is high, then the adaptation weight w(e, ) of strategy to this error is high. Calculate the similarity between the error feature vector and the applicable feature vector of strategy , and the cosine similarity formula can be used:

[0111] ,

[0112] where Expressed as the calculated error eigenvector, Represented as a strategy The applicable feature vector is used, and the similarity, success probability and other inputs are used as the input of the fuzzy reasoning engine (such as the Mamdani reasoning method). The fuzzy output is obtained according to the fuzzy rule base. Then, the defuzzification method (such as the centroid method) is used to convert the fuzzy output into a specific adaptation weight value and update the adaptation weight.

[0113] S6. Perform feature matching based on the error information, generate a strategy candidate set, sort the candidate strategies and select the optimal strategy to perform error recovery operations.

[0114] When the control layer receives the error information f fed back by the execution layer, it needs to perform accurate and efficient feature matching in the fault processing database. First, key information is extracted from the received error information f to construct the input error feature vector This vector contains multi-dimensional information when the error occurs, such as log content, timestamp, system configuration, etc. The cosine similarity algorithm is used to measure The error feature vector in the database The similarity is:

[0115] ,

[0116] Among them, the molecule is the dot product of two vectors, the denominator It is the product of the modulus lengths of two vectors. The closer the cosine similarity value is to 1, the more similar the two vectors are. In order to improve the efficiency of feature matching, especially when processing large-scale high-dimensional feature vectors, the locality sensitive hashing (LSH) technology is introduced to define a series of hash functions. , for each error eigenvector , calculate its hash value under these hash functions , put vectors with the same hash value combination into the same hash bucket. When performing feature matching, only the input feature vector The similar vector is searched in the hash bucket and its adjacent hash buckets without traversing the entire database. This can greatly reduce the time complexity of matching, and select the vector with a similarity greater than the preset threshold. The error records constitute a matching error record set ,Right now:

[0117] ,

[0118] in, Denoted as the input feature vector, Indicates that there is an error feature vector in the database. The steps to find the corresponding policy in the database when it occurs are as follows:

[0119] Step 1: Feature matching,

[0120] Input Error Then, from the database Find the Similar error records , the similarity is calculated as ,in, is the feature vector of the input error, is in the database The feature vector of is calculated using cosine similarity:

[0121] ,

[0122] in, is the feature vector of the input error, is in the database The feature vector of All error records ,in The similarity threshold.

[0123] Step 2: Generation of strategy candidate sets,

[0124] According to the mapping relationship layer , get each matching error Corresponding strategy set , satisfying the following formula ,in, For all the mapping relationships in the layer There are associated policies .

[0125] Step 3: Strategy sorting,

[0126] For matching strategy sets , calculate the priority score of each strategy :

[0127] ,

[0128] in, is the set of errors that match the current error e, The current error e matches the i-th error in the error set The similarity of is the weight coefficient, reflecting the i-th matching error For the jth strategy The importance of Sort the strategies in descending order and select the strategy with the highest score as the optimal strategy:

[0129] ,

[0130] in, is a matching strategy set, that is, the set of strategies mentioned above that match the current error.

[0131] Step 4: Strategy return,

[0132] The system returns the optimal strategy Specific steps , This is the step vector in S5 that represents the error resolution step.

[0133] Example 2

[0134] The difference between this embodiment and embodiment 1 is that this embodiment provides a multi-cloud storage autonomous intention deployment system, including:

[0135] The data acquisition module is configured to: acquire the comprehensive operation and maintenance data of the multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format;

[0136] The decision tree module is configured to: construct a policy tree using a directed acyclic graph, including determining a set of nodes and a set of directed edges based on the comprehensive operation and maintenance data;

[0137] The conversion module is configured to: input the policy tree data into the control layer to parse the policy tree structure and divide it into several subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state;

[0138] The processing module is configured to: transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically;

[0139] The model module is configured to: construct a fault processing database model according to the error data in the node task, including constructing an error information layer, a strategy layer, and a mapping relationship layer;

[0140] The transformation module is configured to: perform feature matching based on error information, generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operation.

[0141] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device. The multi-cloud storage autonomous intention deployment method is disclosed.

[0142] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the multi-cloud storage autonomous intention deployment method.

[0143] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-cloud storage autonomous intention deployment method, characterized in that: include: Obtain comprehensive operation and maintenance data of multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format; Constructing a strategy tree using a directed acyclic graph, including determining a set of nodes and a set of directed edges based on comprehensive operation and maintenance data; The determining of the node set and the directed edge set according to the comprehensive operation and maintenance data includes using a directed acyclic graph to represent the policy tree, determining the node set according to the business logic and resource management requirements in the comprehensive operation and maintenance data, wherein the node set includes a logical level, a policy parameter name and a parameter value, and then determining the directed edge set according to the execution sequence and dependency relationship between the nodes, and adding the directed edges by calculating the dependency between the nodes; Input the policy tree data into the control layer for parsing to obtain the policy tree structure, and divide the policy tree structure into several subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state; The strategy tree data is input into the control layer for parsing, including inputting the constructed strategy tree data into the control layer, completely traversing the node set and the directed edge set, extracting the node features and the dependency relationship of the nodes in the directed edge, defining the node association and node similarity according to the node features and the dependency relationship, constructing the similarity matrix of the strategy tree and calculating the degree matrix and the Laplace matrix, performing eigenvalue decomposition on the Laplace matrix, taking the eigenvectors corresponding to the first k smallest eigenvalues ​​to form a matrix, clustering the matrix using the k-means algorithm, and the node set corresponding to each clustering result constitutes a subtree; Transfer the initialization state to the subtree execution state, and execute the subtree node tasks according to the level, wherein the transfer of the initialization state to the subtree execution state includes setting the corresponding execution state in the control layer control loop for each subtree, initializing the control layer state set and entering the initialization state , introduces a resource pre-allocation mechanism to pre-allocate system resources according to the complexity of the strategy tree. The internal resource allocation and parameter setting are completed according to the pre-allocated resource amount. The strategy tree complexity calculation formula is: ， in, , , and is the weight coefficient, and , is the average out-degree of the node, is the maximum logical level; Construct a fault handling database model based on the error data in the node task, including constructing the error information layer, strategy layer and mapping relationship layer; The error information layer, strategy layer and mapping relationship layer are constructed, including the error information layer storing error data in task execution, using a word embedding model to convert natural language descriptions into vectors, using named entity recognition technology to extract key entity sets, and further updating feature vectors, the strategy layer storing a set of response strategies for solving each error, and the mapping relationship layer storing a many-to-many relationship between error information and response strategies, calculating the similarity between the error feature vector and the applicable feature vector of the strategy by setting an adaptation weight and defining a fuzzy rule base, and obtaining an updated adaptation weight through a fuzzy reasoning engine and defuzzification; Perform feature matching based on error information to generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operations.

2. The multi-cloud storage autonomous intention deployment method according to claim 1, characterized in that: The preprocessing of the comprehensive operation and maintenance data includes generating a hash value of a fixed length according to the acquired comprehensive operation and maintenance data, processing the hash value using a plurality of different hash functions to obtain a plurality of index values ​​for setting Bloom filter bits, removing duplicate data by comparing the hash values, and then filling the missing data values ​​using a filling method based on association rule mining and weighted average, converting the cleaned data from the original format to the target format according to the defined standard structure of each data format and the field mapping relationship, introducing a compression method based on adaptive coding, and storing the processed data in a relational database.

3. The multi-cloud storage autonomous intention deployment method according to claim 1, characterized in that: The adding of directed edges by calculating the dependency between nodes includes analyzing the probability of node u being executed after node v is successfully executed in historical data. To calculate the dependency, set a dependency threshold, and when the dependency is higher than the set threshold, add a directed edge from v to u in the strategy tree.

4. The multi-cloud storage autonomous intention deployment method according to claim 1, characterized in that: The node association degree is defined according to the node characteristics and dependency relationships, including determining the shortest path length between different nodes and the number of common neighbor nodes according to the result of the complete traversal of the node set and the directed edge set, and calculating the node association degree using the shortest path length and the number of common nodes. The node association degree calculation formula is: ， in, It is expressed as the shortest path length between node u and node v, Expressed as the number of common neighbor nodes, and are the neighbor node sets of nodes u and v respectively.

5. The multi-cloud storage autonomous intention deployment method according to claim 1, characterized in that: The candidate strategies are sorted and the optimal strategy is selected to perform an error recovery operation, including using cosine similarity to calculate the similarity between an error feature vector and an existing error feature vector in a fault processing database, introducing a local sensitive hash error feature vector to map to a hash bucket, selecting error records whose similarity is greater than a preset threshold according to the hash bucket, obtaining a corresponding strategy set at a mapping relationship layer, generating a strategy candidate set, calculating a priority score of the strategy candidate set, and selecting the optimal strategy according to the priority score to perform an error recovery operation.

6. A multi-cloud storage autonomous intention deployment system, executed in the method of claim 1, characterized in that: include: The data acquisition module is configured to: acquire the comprehensive operation and maintenance data of the multi-cloud storage, and pre-process the comprehensive operation and maintenance data to obtain a unified data format; The strategy tree module is configured to: construct a strategy tree using a directed acyclic graph, including determining a node set and a directed edge set according to the comprehensive operation and maintenance data; The conversion module is configured to: input the policy tree data into the control layer for parsing, obtain the policy tree structure, divide the policy tree structure into a number of subtrees, set the execution state for each subtree, and initialize the control layer state set to enter the initialization state; The processing module is configured to: transfer the initialization state to the subtree execution state, and execute the subtree node tasks hierarchically; The model module is configured to: construct a fault processing database model according to the error data in the node task, including constructing an error information layer, a strategy layer, and a mapping relationship layer; The transformation module is configured to: perform feature matching based on error information, generate a strategy candidate set, sort the candidate strategies and select the optimal strategy for error recovery operation.

Images