A tensor-based comprehensive optimization method for consistency of streaming computing systems

Abstract

The present invention proposes a method that takes into account availability and partition tolerance while improving the consistency of streaming computing, quantitatively describes the CAP problem, and then optimizes it from a holistic level. First, tensors are used as a mathematical tool to quantitatively express the consistency of non-fixed redundant distributed storage systems. Then, based on the streaming computing architecture and update process of the eventual consistency model, a quantitative constraint relationship between C, A, and P is derived. After determining the properties and cost constraints of different parameters in the constraint relationship model, a three-objective optimization problem is formed. Finally, the improved NSGA2 genetic algorithm is used to solve the problem, and after iteration, a frontier image depicting the Pareto optimal solution set can be obtained. This method realizes the quantitative representation of the CAP problem that can only be analyzed qualitatively in the traditional way, and uses a genetic algorithm to obtain the optimal solution set, providing support for decision makers to select a satisfactory solution as a solution to the streaming computing consistency problem.

Description

Technical Field

[0001] The present invention belongs to the field of big data streaming computing based on cloud computing, and specifically is a tensor-based streaming computing system consistency comprehensive optimization method. Background Art

[0002] In a distributed system, users and computing nodes interact with each other through a series of computational data and operations. Due to the instability of network communications, some data or operations may be missed, mis-transmitted, or duplicated. Inconsistencies arise when one party assumes the state of the other party is inconsistent with its actual state. In traditional distributed computing systems, existing technologies are already effective in resolving these inconsistencies.

[0003] The emergence of streaming computing has posed new challenges to ensuring consistency. Streaming computing not only possesses the characteristics of distributed computing but also real-time computing, requiring the system to respond to computational requests within a specified timeframe. The computational object has also evolved from batches of tasks to a continuous stream of data. The characteristics of streaming computing necessitate a certain degree of real-time and parallelism in the system. Improving consistency on this basis is more difficult than in traditional, non-streaming distributed computing systems. The famous CAP theorem demonstrates that the three characteristics of a system, consistency, availability, and partition tolerance, cannot be met simultaneously; they are mutually exclusive.

[0004] Under the constraints of the CAP theorem, most current systems prioritize ensuring two of the three properties, determining their metrics, and then attempting to maintain a limited tolerance for the third. For example, the Paxos algorithm, proposed by Leslie Lamport, prioritizes consistency for the majority of nodes in the system. A few inconsistent nodes become unavailable during data updates, reducing system availability. This approach prioritizes C and P over A. These existing approaches share a common characteristic: they prioritize two of the three CAP properties, failing to consider all three. Summary of the Invention

[0005] To address the inconsistencies that arise in streaming computations, this paper proposes a solution that considers the CAP problem as a whole and optimizes three objectives. This approach comprehensively and quantitatively considers the three aspects of CAP, achieving the goal of improving the consistency of streaming tensor computations while balancing availability and partition tolerance.

[0006] Compared with traditional consistency algorithms, this approach offers the advantage of improving consistency while also balancing availability and partition tolerance. This approach uses tensors as a carrier to integrate heterogeneous data into a unified representation model to quantify consistency. It then derives the constraint relationships between C, A, and P, which can be described using expressions containing system parameters.

[0007] By deducing the expression for the CAP constraint relationship, and after determining the parameter properties and cost constraint function, a three-objective optimization problem is formed. This three-objective optimization problem is solved using a genetic algorithm based on an improved NSGA2 algorithm. During the iterative process of the genetic algorithm, the constraints are reflected in the form of penalty functions. Through non-dominated sorting, inferior solutions are gradually eliminated, ultimately obtaining a Pareto-dominant solution set and its frontier image.

[0008] In order to achieve the above technical objectives, the present invention adopts the following specific technical solutions:

[0009] A tensor-based streaming computing system consistency comprehensive optimization method includes the following steps:

[0010] S1. Represent heterogeneous data with tensors and calculate the consistency of tensor groups in the distributed storage system.

[0011] S2. Based on the tensorflow computing architecture of the eventual consistency model, derive the constraints between consistency, availability, and partition tolerance, and express the constraints using expressions containing system parameters;

[0012] S3. Determine the parameter types and cost constraints in the CAP constraint relationship expression containing system parameters to form a CAP three-objective optimization problem;

[0013] S4. Use the improved NSGA2 algorithm to solve the CAP three-objective optimization problem and obtain the Pareto optimal solution set.

[0014] Step S1 includes the following sub-steps:

[0015] S1.1. Using expansion operators, heterogeneous data is integrated into the high-order representation model of tensors in the form of dimensions according to the principle of finest granularity, thus achieving tensorization of heterogeneous data.

[0016] S1.2. Apply the above tensor model to streaming computing through distributed tensor algorithms and the Storm platform.

[0017] S1.3. Based on the definition of dependencies, calculate the number of matched dependencies and the number of unmatched dependencies contained in the tensor group in the distributed storage with indefinite redundancy, so as to obtain the consistency of the tensor group in the distributed storage system.

[0018] Step S2 specifically includes the following sub-steps:

[0019] S2.1. Tensorflow computing system architecture based on the eventual consistency model, giving a set of representations of availability A and partition tolerance P;

[0020] S2.2. Based on the system update process, derive the expression and limit range of the update cycle and calculate the expression of availability A;

[0021] S2.3. Use the consistency value C with the minimum consistency during the update process min As a parameter to measure consistency, the consistency value C is derived min , and simplify and split the obtained expression;

[0022] S2.4. Introduce the parameters in the system, connect the actual meaning of the parameters, and express C, A, and P respectively using expressions containing system parameters so that the CAP constraint relationship can be quantitatively described.

[0023] Step S3 specifically includes the following sub-steps:

[0024] S3.1. Based on their meaning and adaptability in practical application scenarios, the parameters in the CAP constraint expression are divided into independent variables, dependent variables, and constant parameters.

[0025] S3.2. Determine the form of the cost constraint function, which is proportional to the total number of nodes, directly related to the square of the hardware performance of both streaming and batch computing, and the square of the communication overhead of streaming computing, plus the number of physical connection lines within each group multiplied by the number of groups;

[0026] S3.3. List the expressions of the CAP constraint function, cost constraint function, and objective function in the form of a multi-objective optimization problem, merge the independent variables in the parameters into a decision vector, and form a CAP three-objective optimization problem.

[0027] Step S4 specifically includes the following sub-steps:

[0028] S4.1. For complex situations that cannot be handled by the linear weighted method, the basic population initialization, individual selection, and mutation operations in the genetic algorithm are completed based on the objective function and decision vector of the CAP three-objective optimization problem.

[0029] S4.2. Implement the core operation of the evolutionary iterative process, fast non-dominated sorting, and introduce the crowding distance and elite strategy of the NSGA2 algorithm into the fast non-dominated sorting process to achieve the basic CAP three-objective optimization problem solving function;

[0030] S4.3. Instead of directly eliminating infeasible solutions, we introduce constraints and cost constraints in the form of penalty functions, and normalize the congestion distances in the three CAP directions to improve the richness and uniformity of the solution set and avoid individual aggregation.

[0031] S4.4. Display the individuals in the final population in the form of a scatter plot, and draw a surface representing the Pareto front in three-dimensional space.

[0032] The beneficial effects of the present invention are:

[0033] This paper proposes a comprehensive optimization method for consistency of streaming computing systems based on tensors, which can achieve both availability and partition fault tolerance while improving system consistency. The details are explained below.

[0034] Versatility: As a mathematical tool, tensors can be represented by multidimensional arrays during storage. Algorithms exist for tensorizing heterogeneous data and integrating them into higher-order tensor representation spaces through extension operators. Introducing tensors as data carriers in a system can achieve the goal of quantifying and comparing heterogeneous data, allowing this method to be used in systems that process widely varying data types.

[0035] Reliability: Tensors are widely used in computer science. For example, common tensor calculations such as high-order singular value decomposition (HOSVD) play an important role in signal processing and data mining. When performing calculations, existing tensor algorithms can be utilized to reduce the use of custom algorithms, thus avoiding some difficult-to-find errors.

[0036] Quantification: This paper quantifies the existence of the CAP problem. By analyzing the update process of an eventually consistent streaming computing system and the consistency expression based on multi-source dependencies, we derive expressions for the minimum system consistency and availability. Combining this with existing partition tolerance expressions establishes the connections and constraints between the three elements of C, A, and P and system resources.

[0037] Flexibility: In many application scenarios, it's not feasible to simply subjectively set a set of weights and then find a solution. Decision makers prefer to obtain a complete set of Pareto-optimal solutions and then select a satisfactory optimal solution based on the set's characteristics. The NSGA2 algorithm used in this solution can gradually obtain a series of non-dominated solutions through non-dominated sorting, eliminating the need for the weights required in linear weighting methods and improving the algorithm's flexibility.

[0038] Visualization: Because the CAP problem is a three-objective optimization problem, the solution set can be depicted using a three-dimensional view. Given that the Pareto optimal solutions obtained using the improved NSGA2 genetic algorithm are distributed on a three-dimensional surface, visualizing the individuals in the population using a scatter plot allows decision makers to see a clear and intuitive Pareto optimal frontier. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart for stream computing consistency optimization based on CAP constraint relationship;

[0040] Figure 2 This is a schematic diagram of the tensor flow computing framework based on the eventual consistency model;

[0041] Figure 3 It is a diagram illustrating the parameter types in multi-objective optimization problems;

[0042] Figure 4 It is a flowchart of using NSGA2 algorithm to solve the CAP multi-objective optimization problem;

[0043] Figure 5 This is a schematic diagram of the Pareto frontier obtained using the improved NSGA2 algorithm. Specific implementation plan

[0044] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0045] To comprehensively consider the three CAP constraints, it is necessary to quantify them and derive the constraint expressions between them. In practical applications, the data processed by streaming systems often includes a variety of heterogeneous data types, and tensors are needed as mathematical tools to integrate these heterogeneous data into a unified representation model. After obtaining the CAP constraint, the next step is to determine how to allocate system resources to achieve consistency while ensuring availability and partition tolerance. This is a typical three-objective optimization problem that can be solved using methods from multi-objective optimization.

[0046] The linear weighted sum method (LWSM) is a common approach for solving multi-objective optimization problems. It assigns weights to each objective function in the problem and then combines them cumulatively, transforming it into a single-objective optimization problem. However, the LWSM is not suitable for situations where the objective function weights cannot be known a priori. In such cases, a genetic algorithm is required. A genetic algorithm can generate a set of all Pareto-optimal solutions, allowing decision makers to select a satisfactory solution based on the optimal solution set, without having to combine the objective functions using weights. Simpler scenarios can be solved directly using the LWSM.

[0047] For a system that requires distributed computing and pursues real-time performance, this solution first uses tensors as a carrier to unify the heterogeneous data in it, making the consistency between them comparable. Then, by deriving the CAP constraint relationship, a constraint relationship between consistency and availability, and partition tolerance is established. This constraint relationship can be described by an expression containing multiple system parameters. After determining the types of the above parameters, a three-objective optimization problem is formed. Finally, the improved NSGA2 genetic algorithm is used to solve the CAP three-objective optimization problem. The flowchart of the streaming computing consistency optimization is shown below. Figure 1 shown.

[0048] 1. Tensor Architecture Design

[0049] This solution uses tensors as a carrier to compare the consistency of heterogeneous data. The requester can send data directly in tensor form or in non-tensor form, which requires tensorization. Depending on the degree of structure, non-tensor data can be integrated into a unified high-order tensor representation model using feature orders, and the dimensionality can be expanded at the finest granularity using expansion operators.

[0050] In principle, the tensor computations supported by the system can be any tensor computation suitable for streaming systems. During the computational process, distributed computing is first implemented, splitting large tensors into several equal-sized sub-tensor blocks and sending them to different nodes for computation. The inner block algorithm runs on each node, and the inter-block algorithm runs across the entire node group. Round Robin hashing is used to determine node addresses during slice allocation and result synthesis. Streaming computing is then implemented, with modifications to the inner block algorithm. This can be run through a big data computing framework like Storm that supports streaming semantics, thus achieving a streaming version of tensor computation.

[0051] The constraint relationship between system consistency, availability and partition tolerance depends on the consistency model adopted by the system. The consistency model used in the streaming tensor computing in the present invention is the eventual consistency model (Eventual Consistency). It is chosen because the allocation of computing resources in this model is highly adjustable. After the optimal system resource allocation plan is obtained through multi-objective optimization, corresponding adjustments can be flexibly made according to the plan. The eventual consistency model is a weak consistency model. It requires that data copies can eventually reach a strong consistency state after a sufficiently long time, but does not require the intermediate process to have strong consistency. Therefore, in this model, data can be temporarily calculated and stored with a weak consistency processing method to ensure real-time performance. At the same time, there must be a corresponding mechanism to ensure that inconsistent data is corrected after a period of time. In the eventual consistency model, real-time and consistency can eventually be achieved but not at the same time. The tensor streaming computing framework based on the eventual consistency model is as follows. Figure 2 shown.

[0052] On the basis of the idea of the eventual consistency model, the idea of the Lambda architecture is referred to to implement the streaming tensor decomposition algorithm based on the eventual consistency model. First, consider the partition fault tolerance P, that is, after the system is divided into several partitions by a network failure, how can we ensure that each partition can complete its original function. Since it is impossible to store a copy of all information on each single node, in order to ensure limited partition fault tolerance, a group of nodes are placed together, such as in the same computing center. Since the nodes are very close in the actual physical space, it is also easier to repair after an internal network failure. If necessary, a direct connection can be established using a dual-machine line, which can solve the problem of partitions within the same group of nodes. The computing nodes belonging to different node groups are physically far apart. When a network failure occurs in the location of one group, it is not easy to affect the other group. Therefore, whether partitions occur between node groups can be regarded as independent of each other.

[0053] 2. Methodology

[0054] The abnormal data screening of the present invention is divided into the following steps: (1) tensoring the heterogeneous data, and then calculating the consistency of the tensor group through multi-source dependencies. (2) According to the tensor flow calculation update process under the final consistency model, the CAP constraint relationship expression containing the system parameters is obtained. (3) Determine which system parameters are independent variables, which are constant parameters, and which are dependent variables, determine the form of the cost constraint expression, and form a three-objective optimization problem. (4) Use the NSGA2 genetic algorithm to solve the three-objective optimization problem, and obtain the Pareto optimal solution set by changing the independent variable parameters. The parameter types in the multi-objective optimization problem are as follows: Figure 3 shown.

[0055] 1. Establishment of the CAP Relationship Model

[0056] Before studying the CAP constraint relationship of the system, we need to give a quantitative representation method for the three elements C, A, and P. The first is consistency C. In a streaming system, a tensor group is distributed and stored by multiple nodes, and each tensor is split into several sub-tensors.

[0057] The storage of sub-tensors has certain redundancy and vacancies. In practical applications, the tensor information stored in each node consists of a triple, which is represented as (i, j, G) the jth slice of the i-th tensor. For tensor G i The jth sub-tensor of , if it is in node S k There is a copy on the node S, then the copy is called tensor G on node S j A dependency Φ on ijk The dependencies themselves are blocks of sub-tensors on a single node and need to be given an inconsistency inco(G ij ,Φ ijk ) to help determine whether the dependency matches. For a dependency Φ ijk , if and only if it is consistent with the slice G of the original tensor ij The inconsistency inco(G ij ,Φ ijk ) is less than or equal to the given maximum value μ, then the dependency is called a matched dependency, otherwise it is called an unmatched dependency. In practical applications, the value of μ can be adjusted according to specific needs.

[0058] The inconsistency between a tensor block and its corresponding sub-tensor on a node is inco(G ij ,Φ ijk ) is 1 when it is less than or equal to the given maximum value μ, otherwise it is 0. Thus, we obtain a quantitative calculation method for the consistency of tensor groups in streaming tensor computing systems, and the consistency function can be used to describe the consistency C in CAP.

[0059] After determining the quantitative calculation method for consistency, the partition tolerance and real-time requirements in the CAP constraint are described and their parameter symbolic representations are obtained. First, availability. Consider a real-time distributed system S, whose set of availability requirements is denoted as A. A consists of a series of processing time requirements and response requirements for different actions. In general system design, the system needs to respond correctly to the client's request within a given time. In this case, A can be represented by two parameters, namely the response time requirement t and the availability rate a, that is, A=(t,a). The quantitative method of partition tolerance is represented by a parameter set, in which the parameters involved are the minimum number of nodes n contained in the partition, the average duration of the partition T, and the average interval between partitions d, that is: P=(n,T,d).

[0060] The system periodically merges and updates the batch computing results with the streaming computing results. In a system with a total of N computing nodes, the average value of the availability rate a within an update cycle is:

[0061]

[0062] The moment before the system completes the batch update is the time when the consistency is the lowest. Since the harm caused by the system inconsistency is the greatest when the consistency is the lowest, the focus here is on the minimum consistency value c within an update cycle. min . Calculate c according to the tensor group consistency calculation formula based on dependency given in the previous section min .

[0063] The batch processing rate and streaming computing rate of the node are represented by E and e respectively, and the minimum consistency value is:

[0064]

[0065] 2. Use multi-objective optimization algorithms for consistency optimization

[0066] Step 1: Determine the nature of the parameters. To solve a multi-objective optimization problem, it is necessary to determine which parameters in the system are constant parameters, which are decision variables, and which are dependent variables whose values are determined based on existing decision variables and constraints. In a tensor flow computing system based on the eventual consistency model, there are 11 component parameters. Among these parameters, the system's total tensor data flow rate r and real-time requirement t are design targets that the system must meet. They cannot be arbitrarily modified in specific engineering scenarios and are therefore considered constant parameters. The system's communication error rate p, average partition interval d, and average partition duration T vary depending on the network environment and are often the responsibility of network operators. Software system designers cannot change these factors and can also be considered fixed constant parameters. After isolating the constant parameters, the next step is to determine the decision variables: the total number of system nodes N, the data flow computation rate e and batch computation rate E of a single node, and the minimum number of partitioned nodes n in the system. The decision vector used in the multi-objective optimization process is composed of these four parameters. In the CAP constraint, the availability a and minimum consistency c are also considered fixed constant parameters. min It can be determined by the decision variables and is therefore considered a dependent variable.

[0067] Step 2: Determine the cost constraints. Before solving a multi-objective optimization problem, it is necessary not only to determine the parameter type but also the cost constraints. Cost constraints are set to limit the resources consumed by the system. Otherwise, simply increasing the overall performance of the nodes and the number of nodes in each node group without limit can improve the consistency of the system. However, in actual application scenarios, resources and costs are obviously not infinite. According to current rules, the form of the cost constraint function should be:

[0068]

[0069] Among them, c1, c2, c3, and c4 are constant coefficients related to cost, which are determined by market prices and can be replaced with different values according to actual conditions. The reason for expressing it in this objective function form is that, first, when other parameters remain unchanged, the cost of the system is proportional to the total number of nodes invested. Secondly, according to the current performance-price law of hardware and rented cloud servers, when the performance of a single machine is high, the cost tends to double with each increase in the computing power of a single node. In the case of low performance, the cost of a single node has a lower threshold. The node's batch processing rate and streaming computing rate both require the node's hardware computing performance, so they are both placed in the exponential part. In addition, the node's streaming computing rate also requires network communication bandwidth. Network communication bandwidth is proportional to the cost price, so a linear term related to the streaming computing rate needs to be added. To ensure consistency of communication within the group, nodes in the same group need to establish a physical connection. A group containing n nodes has a total of 1+2+..+n= between each other. connections, and N nodes have a total of node groups, which determines the cost representation of the intra-group connections in the above expression.

[0070] Step 3: Use the NSGA2 genetic algorithm to solve the multi-objective optimization problem. The core of applying genetic algorithms to multi-objective optimization problems is the Pareto dominance relationship and the Pareto rank. If a solution is not dominated by any other solution, the solution is called a non-dominated solution. We can define the Pareto rank of all non-dominated solutions in the current set of all solutions as 1 and temporarily remove them from the solution set. Then recalculate the dominance relationship of the remaining solutions, and define the Pareto rank of the non-dominated solutions in the remaining solutions as 2 and remove them. After that, the Pareto rank of the non-dominated solutions is increased by 1 until the solution set is empty. The Pareto rank of all solutions can be obtained. Sorting the solutions in the solution set according to the Pareto rank is called non-dominated sorting. The Pareto rank can be used as a fitness evaluation function in the genetic algorithm. At each iteration, the solution set with the lowest Pareto rank is deleted, and the offspring is generated using the solutions with the higher rank.

[0071] The NSGA2 algorithm has made some improvements based on the non-dominated sort. First, it introduces crowding and crowding comparison operators to solve the problem of solution set concentration. This makes it easier for solutions that are similar to other solutions to be eliminated, thereby improving the uniformity of the final solution set distribution. At the same time, the NSGA2 algorithm also introduces an elite strategy, combining the parent population and the child population it produces, and jointly produces the next generation population through competition. This is conducive to maintaining the excellent individuals in the parent generation and ensuring that those excellent individuals are not discarded during the evolution process. The process of the NSGA2 algorithm to solve the CAP multi-objective optimization problem is as follows: Figure 4 shown.

[0072] Step 4: Improve the NSGA2 algorithm. First, a penalty function is introduced. Because the problem being solved is an optimization problem with constraints, constraints are necessary. Because infeasible solutions in a genetic algorithm may become feasible solutions in the future, infeasible solutions cannot be discarded outright. Instead, their fitness is reduced using a penalty function. The penalty function chosen here uses an attenuation function. For any portion exceeding the constraints, the excess value is first scaled. Then, an attenuation term is added based on the importance of the constraint and multiplied by a coefficient. Finally, the three objective functions C, A, and P are reduced by the inverse of the exp function of the attenuation term. The attenuation term for feasible solutions is set to zero. This ensures that solutions with low constraint violations but high fitness are retained. If these solutions later become feasible solutions during evolution, they can enrich the final solution set.

[0073] Then, we will improve the crowding distance algorithm. Due to the complexity of the constraint function, the traditional crowding distance formula of the NSGA2 algorithm is not enough to make the results have a sufficiently uniform distribution. At the same time, because the scales of the three objective functions C, A, and P are quite different, where the values of C and A are between 0 and 1, and the value of P can exceed 10, we need an improvement that can normalize the crowding distance. Otherwise, simply adding the crowding distances calculated by the three objective functions will result in the size of the crowding distance being dominated by P. Here, we improve the crowding distance formula to make it more uniform than the traditional crowding distance i d To distinguish, the improved crowding distance is recorded as i D .

[0074]

[0075] Among them, m is the total number of individuals, i d This improved crowding distance is based on the concept of variance in its design. It no longer considers only adjacent individuals but considers all individuals. This can prevent individual clustering while also avoiding the occurrence of sparse intervals to a certain extent, further improving the uniformity of the result distribution and helping to obtain a clearer boundary for the optimal solution set.

[0076] Step 5: Visualization. Use visualization software such as Matlab or the QtDataVisualization class in the C++ Qt interface library to display the individuals in the population after multiple evolutionary iterations through a scatter plot. Since these points are distributed on a three-dimensional surface, a visual image of the Pareto frontier surface can be obtained. The Pareto frontier surface image obtained by the visualization tool is as follows: Figure 5 shown.

Claims

1. A tensor-based streaming computing system consistency comprehensive optimization method, characterized in that: The following steps are involved: S1. Represent heterogeneous data with tensors and calculate the consistency of tensor groups in the distributed storage system. S2. Based on the tensorflow computing architecture of the eventual consistency model, derive the constraints between consistency, availability, and partition tolerance, and express the constraints using expressions containing system parameters; S3. Determine the parameter types and cost constraints in the CAP constraint relationship expression containing system parameters to form a CAP three-objective optimization problem; S4. Using the improved NSGA2 algorithm to solve the CAP three-objective optimization problem, obtain the Pareto optimal solution set; Step S3 specifically includes the following sub-steps: S3.

1. Based on their meaning and adaptability in practical application scenarios, the parameters in the CAP constraint expression are divided into independent variables, dependent variables, and constant parameters. S3.

2. Determine the form of the cost constraint function, which is proportional to the total number of nodes, directly related to the square of the hardware performance of both streaming and batch computing, and the square of the communication overhead of streaming computing, plus the number of physical connection lines within each group multiplied by the number of groups; S3.

3. List the expressions of the CAP constraint function, cost constraint function, and objective function in the form of a multi-objective optimization problem, merge the independent variables in the parameters into a decision vector, and form a CAP three-objective optimization problem; Step S4 specifically includes the following sub-steps: S4.

1. For complex situations that cannot be handled by the linear weighted method, the basic population initialization, individual selection, and mutation operations in the genetic algorithm are completed based on the objective function and decision vector of the CAP three-objective optimization problem. S4.

2. Implement the core operation of the evolutionary iterative process, fast non-dominated sorting, and introduce the crowding distance and elite strategy of the NSGA2 algorithm into the fast non-dominated sorting process to achieve the basic CAP three-objective optimization problem solving function; S4.

3. Instead of directly eliminating infeasible solutions, we introduce constraints and cost constraints in the form of penalty functions, and normalize the congestion distances in the three CAP directions to improve the richness and uniformity of the solution set and avoid individual aggregation. S4.

4. Display the individuals in the final population in the form of a scatter plot, and draw a surface representing the Pareto front in three-dimensional space.

2. The tensor-based streaming computing system consistency comprehensive optimization method according to claim 1 is characterized in that: Step S1 includes the following sub-steps: S1.

1. Using expansion operators, heterogeneous data is integrated into the high-order representation model of tensors in the form of dimensions according to the principle of finest granularity, thus achieving tensorization of heterogeneous data. S1.

2. Apply the above tensor model to streaming computing through distributed tensor algorithms and the Storm platform. S1.

3. Based on the definition of dependencies, calculate the number of matched dependencies and the number of unmatched dependencies contained in the tensor group in the distributed storage with indefinite redundancy, so as to obtain the consistency of the tensor group in the distributed storage system.

3. The tensor-based streaming computing system consistency comprehensive optimization method according to claim 1 is characterized in that: Step S2 specifically includes the following sub-steps: S2.

1. Tensorflow computing system architecture based on the eventual consistency model, giving a set of representations of availability A and partition tolerance P; S2.

2. Based on the system update process, derive the expression and limit range of the update cycle and calculate the expression of availability A; S2.

3. Use the consistency value C with the minimum consistency during the update process min As a parameter to measure consistency, the consistency value C is derived min , and simplify and split the obtained expression; S2.

4. Introduce the parameters in the system, connect the actual meaning of the parameters, and express C, A, and P respectively using expressions containing system parameters so that the CAP constraint relationship can be quantitatively described.

Images