Microservice component development method, medium and system based on standard data analysis

Abstract

The invention provides a micro-service component development method based on standard data analysis, a medium and a system, and belongs to the technical field of development methods.The micro-service component development method based on standard data analysis comprises the steps that a standard data analysis model is established for data hierarchical processing, a historical service request is processed by adopting a multi-dimensional Joseph ring algorithm to obtain an access calling sequence, service access parameters are calculated based on the access calling sequence, and an optimal communication flow distribution scheme is calculated by adopting a communication flow optimization equation set. The communication flow optimization equation set comprises a resource constraint equation, a flow balance equation, a time delay optimization equation and an overhead optimization equation, the resource constraint equation is used for limiting the total resource use amount, the flow balance equation is used for balancing the data flow, the time delay optimization equation is used for reducing the communication time delay, and the overhead optimization equation is used for reducing the total overhead of system operation; and constructing a service calling routing table and establishing a micro-service component registration center, and optimizing service resource allocation by adopting a multi-dimensional knapsack algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of development methods, and in particular, relates to a microservice component development method, medium and system based on standard data analysis. Background Art

[0002] With the deepening of enterprise digital transformation, microservice architecture has become the mainstream choice for building complex application systems. Compared with traditional monolithic applications, microservice architecture has the advantages of high modularity, loose coupling, and easy scalability, and can better adapt to the rapid changes in business needs. In this architecture, applications are split into many independently deployed and self-governed microservice components, which call and collaborate with each other through standardized APIs.

[0003] At present, the development of microservice components mainly includes the following methods:

[0004] 1. Monolithic architecture splitting method. This method is to gradually split the original monolithic application into microservices according to functional modules. This method is simple to implement, but lacks systematicity and is prone to problems such as uneven service granularity and unreasonable interface design.

[0005] 2. Domain-driven design. This method divides the boundaries of microservices by identifying bounded contexts in the business domain. Although it can map business requirements well, it is often difficult to accurately grasp the boundaries and granularity of services in actual implementation.

[0006] 3. Development method based on service contract. This method emphasizes defining the service interface first and then implementing it. This method is conducive to service decoupling, but it is difficult to cope with the rapid changes in business and the dynamic expansion requirements of services.

[0007] These existing methods generally have the following key problems: The basis for service splitting is unclear, and it is difficult to ensure the rationality of service granularity and interface design. Resource allocation is not optimized, and system performance is limited. Existing methods often use simple linear weighting or threshold control methods, which are difficult to cope with complex dynamic scenarios. The service call relationship is complex, and it is difficult to achieve automatic optimization of service call paths. Existing service registration and discovery mechanisms are mostly static, which is difficult to adapt to scenarios where the service scale is rapidly growing and frequently changing. The lack of global performance evaluation and continuous optimization mechanisms makes it difficult to ensure that the system runs stably and reliably in the best state. Therefore, there is an urgent need for a systematic, data-driven microservice component development method that can effectively solve the above key problems and meet the needs of enterprise microservice transformation. Summary of the invention

[0008] In view of this, the present invention provides a microservice component development method, medium and system based on standard data analysis, which solves the problem that the basis for service splitting in microservice development is unclear and it is difficult to ensure the rationality of service granularity and the rationality of interface design.

[0009] The present invention is achieved in that:

[0010] The first aspect of the present invention provides a microservice component development method, medium and system based on standard data analysis, which includes: establishing a standard data analysis model for data layering processing, using a multidimensional Joseph ring algorithm to process historical service requests to obtain an access call sequence, calculating service access parameters based on the access call sequence, using a communication traffic optimization equation group to calculate the optimal communication traffic allocation plan, the communication traffic optimization equation group includes a resource constraint equation, a traffic balance equation, a delay optimization equation, and an overhead optimization equation, the resource constraint equation is used to limit the total amount of resource usage, the traffic balance equation is used to balance data traffic, the delay optimization equation is used to reduce communication delay, the overhead optimization equation is used to reduce the total system operation overhead, build a service call routing table and establish a microservice component registration center, and use a multidimensional backpack algorithm to optimize service resource allocation.

[0011] The communication traffic optimization equation group includes a resource constraint equation, a traffic balance equation, a delay optimization equation, and a cost optimization equation;

[0012] The resource constraint equation is used to limit the total resource usage of the microservice component, the input parameters include the computing resource requirement, storage resource requirement, and network resource requirement in the service access parameter, and the output parameter is the resource constraint feasible domain, and the resource constraint feasible domain is used to determine the deployment plan of the microservice component;

[0013] The traffic balance equation is used to balance the data traffic between the microservice components, the input parameters include the service input traffic, service output traffic, and service transit traffic in the historical service request, and the output parameter is the node traffic balance value, which is used to optimize the service call path;

[0014] The delay optimization equation is used to reduce the communication delay between the microservice components, the input parameters include the communication transmission delay, service processing delay, and task queuing delay in the access call sequence, and the output parameter is the optimal communication traffic allocation scheme, which is used to construct the service call routing table;

[0015] The overhead optimization equation is used to reduce the total system operation overhead. The input parameters include the service deployment overhead, service operation and maintenance overhead, and service communication overhead in the service access parameters. The output parameter is the optimal resource configuration plan, and the optimal resource configuration plan is used to guide the resource allocation of the microservice component.

[0016] In the prior art, the development methods of microservice components mainly include the traditional monolithic architecture splitting method, domain-driven design method and service contract-based development method. The monolithic architecture splitting method is to gradually split the original monolithic application into microservices according to functional modules. This method is simple to implement but lacks systematicity, and is prone to problems such as uneven service granularity and unreasonable interface design. The domain-driven design method divides the boundaries of microservices by identifying the bounded context in the business domain. Although it can better map business requirements, it is often difficult to accurately grasp the boundaries and granularity of services in the actual implementation process. The development method based on service contracts emphasizes defining the service interface first and then implementing it. This method is conducive to service decoupling, but it is difficult to cope with the rapid changes in business and the dynamic expansion requirements of services. These existing methods generally have problems such as unclear basis for service splitting, inefficient resource allocation, and complex service call relationships, making it difficult to achieve the optimal performance of microservice systems.

[0017] In contrast, the microservice component development method based on standard data analysis proposed in the present invention effectively solves multiple key problems in the prior art by introducing innovative technologies such as red-black tree structure, multidimensional Joseph ring algorithm and communication traffic optimization equation group. First, the red-black tree structure is used to organize enterprise business data, realize the self-balancing of data access, and significantly improve the data reading efficiency. Compared with the traditional data storage method, the red-black tree structure can stabilize the data access time complexity at the O(lgn) level, and can still maintain a good balance when the data changes dynamically. Secondly, the scheduling problem of service requests is processed by the multidimensional Joseph ring algorithm, and the optimal scheduling of service requests is achieved under the consideration of multiple constraints such as processing time, resources and priority. Compared with the traditional first-come-first-served or priority scheduling algorithm, this method can better balance the system resource utilization and service response time.

[0018] In the development of microservice components, a classic problem is how to accurately evaluate and optimize the communication overhead between services. Traditional methods often use simple linear weighting or threshold control methods, which are difficult to cope with complex dynamic scenarios. The communication traffic optimization equations proposed in this invention establish a complete mathematical model by comprehensively considering four dimensions: resource constraints, traffic balance, delay optimization, and cost optimization. Among them, the resource constraint equation introduces error terms to deal with the volatility of resource usage, making resource allocation more flexible and stable. The traffic balance equation innovatively introduces time and space partial derivative terms, which can accurately describe the dynamic distribution characteristics of traffic and provide a theoretical basis for traffic scheduling. The delay optimization equation decomposes transmission delay, propagation delay, and queuing delay, and introduces the delay change rate to achieve accurate modeling and optimization of end-to-end delay. The cost optimization equation comprehensively considers multiple cost factors such as deployment, operation and maintenance, and communication, providing an economic guarantee for system optimization.

[0019] Another classic problem is the optimization of service call relationships. The existing technologies mostly use static service registration and discovery mechanisms, which are difficult to adapt to scenarios where the service scale grows rapidly and changes frequently. The present invention achieves dynamic optimization of service dependencies by constructing a service call binary heap and introducing a minimum spanning tree algorithm. At the same time, the consistent hashing algorithm is used to allocate service nodes, which greatly improves the scalability of the service registration center. This method not only reduces the complexity of service calls, but also improves the fault tolerance and maintainability of the system.

[0020] In addition, the present invention also shows good scalability and adaptability in practice. Through the cooperation of the distributed transaction manager and the performance evaluation matrix, the system can monitor the service operation status in real time and make timely adjustments when the performance indicators do not meet the standards. This closed-loop optimization mechanism ensures that the system can continue to operate in the optimal state and provide enterprises with a stable and reliable microservice support platform. Compared with the existing technology, the present invention not only solves the key technical problems in microservice development, but also provides a complete set of solutions, providing strong support for the microservice transformation of enterprises.

[0021] Among them, the step of establishing a standard data analysis model includes: collecting enterprise business data, establishing a red-black tree structure for the enterprise business data, the red-black tree structure is used for data reading balance, and constructing a data collection layer based on the red-black tree structure, a data cleaning layer of the data collection layer, a data analysis layer of the data cleaning layer, and a data visualization layer of the data analysis layer.

[0022] Furthermore, the multidimensional Joseph ring algorithm includes processing duration constraints, processing resource constraints, and processing priority constraints; the constraints of the multidimensional backpack algorithm include computing resource constraints, storage resource constraints, network resource constraints, and time resource constraints.

[0023] Furthermore, the input parameters of the resource constraint equation include the computing resource requirements, storage resource requirements, and network resource requirements in the service access parameters; the input parameters of the traffic balance equation include the service input traffic, service output traffic, and service transit traffic in the historical service requests; the input parameters of the delay optimization equation include the communication transmission delay, service processing delay, and task queuing delay in the access call sequence; the input parameters of the overhead optimization equation include the service deployment overhead, service operation and maintenance overhead, and service communication overhead in the service access parameters.

[0024] Furthermore, it also includes building a distributed transaction manager, which uses a two-phase commit protocol to determine the transaction execution order, uses a directed acyclic graph to record the transaction execution status, generates a task execution batch number and a task execution version number, and records a version log.

[0025] Furthermore, the step of constructing a service call routing table includes: using a shortest path algorithm to calculate a service call path between the microservice components, and determining a service registration requirement and a service coordination strategy based on the service call path.

[0026] Furthermore, it also includes calculating the optimized call path between the microservice components based on the service interface indicators of the microservice components, using the full-graph shortest path algorithm, and establishing a call distance matrix, which records the service call delay, service call overhead, and service call reliability.

[0027] Furthermore, it also includes deploying data analysis applications, using queuing theory models to calculate service operation indicators, building a performance evaluation matrix to monitor the operation of the microservice components, and when the service operation indicators do not reach a preset threshold, performing a system rollback based on the version log.

[0028] The equations or calculation processes involved in the present invention are described in detail as follows:

[0029] 1. In step S01, the balance factor calculation of the red-black tree structure is specifically expressed as follows:

[0030]

[0031] Where β is the balance factor; h l is the height of the left subtree; h r is the height of the right subtree.

[0032] Among them, the parameters are obtained by recursively traversing the depth of the tree. The specific steps are:

[0033] 1. Starting from the root node, recursively calculate the depth of the left and right subtrees;

[0034] 2. Take the maximum path length as the tree height.

[0035] The balance factor ranges from -1 to 1 and is used to determine whether a rotation operation is required;

[0036] The formula uses subtraction calculation to reflect the imbalance of the left and right subtrees, and the division normalization makes trees of different sizes comparable.

[0037] 2. In step S02, the multidimensional Joseph ring algorithm involves the following calculations:

[0038] The circular sequence processing model is specifically expressed as follows:

[0039] P(i,k)=(P(i-1,k)+m)%i;

[0040] Where P(i,k) is the serial number of the kth position in the i-th round; m is the step size; i is the current round number; k is the position number.

[0041] Processing time constraints:

[0042] T i ≤T max ;

[0043] Where, T i is the processing time of the i-th request; T max The maximum allowed processing time.

[0044] Dealing with resource constraints:

[0045] R i ≤R available ;

[0046] In the formula, R i is the resource requirement of the i-th request; R available The amount of available resources.

[0047] Handling precedence constraints:

[0048] Priority i ≥Priority min ;

[0049] In the formula, Priority i is the priority of the i-th request; Priority min Is the lowest allowed priority.

[0050] 3. In step S03, the communication flow optimization equation group includes the following equations:

[0051] The resource constraint equation is specifically expressed as follows:

[0052]

[0053] In the formula, C i is the computing resource requirement of the i-th component; S i N is the storage resource requirement; i is the network resource demand; R total is the total resource capacity; 1 is the error term, ranging from 0 to 0.1.

[0054] This equation takes into account additive constraints on resource usage.

[0055] The flow balance equation is specifically expressed as follows:

[0056]

[0057] In the formula, F ij is the flow from node i to node j; F jk is the flow from node j to node k; The term represents the rate of change of flow over time; The term represents the rate of change of flow with space; k 3 ,k 4 is the weight coefficient of time and space change rate, ranging from 0.1 to 1; ε 2 is the error term, ranging from -0.05 to 0.05.

[0058] This equation embodies the principle of node inflow and outflow balance, while taking into account the temporal and spatial distribution characteristics of traffic.

[0059] The delay optimization equation is specifically expressed as follows:

[0060]

[0061] Where, D total is the total delay; F ij is the flow rate; C ij is the link capacity; L ij is the physical distance; V is the propagation speed; Q ij is the queuing delay; The term represents the rate of change of the queuing delay; k 1 is the delay change rate weight coefficient, ranging from 0.1 to 1; ε 3 is the error term, ranging from 1 to 10 ms.

[0062] The equation takes into account transmission delay, propagation delay and queuing delay, and introduces differential terms to reflect the dynamic characteristics of the system.

[0063] The cost optimization equation is specifically expressed as follows:

[0064]

[0065] In the formula, Cost total is the total cost; D i M is the deployment overhead; i T is the operation and maintenance cost; i is the communication overhead; w 1 ,w 2 ,w 3 is the weight coefficient; ε 4 is the error term, ranging from 0 to 100.

[0066] This equation takes into account various types of expenses in the form of weighted sum.

[0067] 4. In step S04, the service call binary heap involves the following calculations:

[0068] The calculation of the sort key value of the heap is specifically expressed as follows:

[0069] K i =αF i +βP i +γR i ;

[0070] In the formula, K i The key value of the i-th service; F i is the functional characteristic value; P i is the performance requirement value; R i is the resource occupancy value; α, β, γ are weight coefficients.

[0071] 5. In step S07, the service reliability index calculation is specifically expressed as follows:

[0072]

[0073] In the formula, R is the system reliability; i is the failure rate of the i-th component; μ i is the repair rate; t i is the running time; The term represents the historical cumulative effect of failure rate; k 2 is the weight coefficient of historical cumulative effect, ranging from 0.01 to 0.1.

[0074] The formula is based on the reliability series-parallel theory, takes into account the two processes of failure and repair, and introduces the historical cumulative effect through the integral term.

[0075] 6. In step S08, the constraints of the multidimensional knapsack algorithm are specifically expressed as follows:

[0076]

[0077] In the formula, x i is a 0-1 decision variable; Ci ,S i ,N i ,T i are computing, storage, network, and time resource requirements respectively; C max ,S max ,N max ,T max The corresponding resource upper limit.

[0078] 7. In step S09, the service quality model is specifically expressed as follows:

[0079]

[0080] Where Q is the service quality indicator; D r is the data transmission rate; M r is the message delivery rate; T r is the transaction completion rate; w 1 ,w 2 ,w 3 is the weight coefficient; the reciprocal term uses the Sigmoid function to describe the saturation characteristics of the performance index; k 5 is the saturation effect weight coefficient, ranging from 0.1 to 1.

[0081] 8. In step S11, the distance matrix is ​​called as follows:

[0082]

[0083] Where, d ij is the calling distance from node i to node j, and the calculation formula is:

[0084] d ij =αt ij +βc ij +γ(1-γ ij );

[0085] Among them, t ij is the delay; c ij For expenses; ij is the reliability; α, β, γ are the weight coefficients.

[0086] 9. In step S12, the performance evaluation matrix is ​​specifically expressed as follows:

[0087]

[0088] Where, p ij is the value of the jth performance indicator of the ith component.

[0089] These equations and matrices form a complete microservice component development framework, which mainly considers the following aspects:

[0090] The red-black tree is used to ensure the balance of data access; the Joseph ring algorithm is used to handle the scheduling problem of service requests; multiple optimization equations are used to solve resource allocation and traffic optimization problems; and various matrices are used to record and evaluate system performance.

[0091] The following is a detailed analysis of the construction process of each equation in the communication traffic optimization equation group:

[0092] The first equation is the resource constraint equation:

[0093]

[0094] The idea of ​​constructing this equation is: First, consider the computing resources C required by a single microservice component. i , its value is obtained by monitoring the container CPU usage, the monitoring sampling period is 1 minute, and the sampling data is calculated by exponential moving average to obtain a stable value; then consider the storage resource demand S i , including runtime memory usage and persistent storage space, obtain real-time usage through system calls; then add network resource requirements N i , including bandwidth usage and number of connections, obtained through network interface statistics; these three resources are linearly superimposed to form the overall demand, and its physical meaning is to unify resources of different dimensions into standardized units; R total Indicates the total system resource capacity, determined according to the actual hardware configuration; ε 1 It is an error compensation item used to deal with fluctuations in resource usage, and its value range is determined by the standard deviation of long-term operating data.

[0095] The second equation is the flow balance equation:

[0096]

[0097] The idea of ​​constructing this equation is: first establish the node flow conservation relationship, that is, the inflow Equal to outflow Considering the time-varying characteristics of flow, the time partial derivative is introduced where k 3 is the time sensitivity coefficient, which is determined by the matching relationship between the flow rate change rate and the system response capability; since the flow in the distributed system also has spatial distribution characteristics, the spatial partial derivative is introduced where k 4 is the spatial sensitivity coefficient, which is determined by the relationship between the network topology and transmission delay; ε 2 is the flow fluctuation error term, and its value range is determined by the statistical distribution characteristics of flow fluctuation.

[0098] The third equation is the delay optimization equation:

[0099]

[0100] The idea of ​​constructing this equation is: first consider the transmission delay where F ij is the flow rate, C ij is the link capacity, and the ratio reflects the time required for data transmission; then the propagation delay is considered Where L ij is the physical link length, V is the signal propagation speed, which reflects the physical transmission delay; then add the queuing delay Q ij , calculated by the M / M / 1 queue model; considering the dynamic characteristics of the system, the queue delay change rate term is introduced where k 1 Determined by system response characteristics; ε 3 is the delay measurement error, and the value range is determined by the standard deviation of the delay jitter.

[0101] The fourth equation is the cost optimization equation:

[0102]

[0103] The idea of ​​constructing this equation is: Consider the deployment cost D i , including computing resource costs, storage resource costs, etc., calculated through the cloud service billing model; operation and maintenance expenses M i Including monitoring cost, maintenance cost, etc., estimated by labor cost and tool cost; communication cost T i Including bandwidth cost, traffic cost, etc., calculated through the network service billing model; weight coefficient w 1 ,w 2 ,w 3 Determine the relative importance of each expense through the analytic hierarchy process; 4 For cost estimation error, the value range is determined by the variance of historical cost data.

[0104] These four equations together constitute a complete description of communication traffic optimization, where the resource constraint equation ensures that the system operation does not exceed the hardware capabilities, the traffic balance equation ensures the continuity of data transmission, the delay optimization equation minimizes the end-to-end delay, and the overhead optimization equation ensures economy. The parameters in each equation have clear physical meanings and measurability. The introduction of error terms improves the robustness of the model, and the use of differential terms and partial derivatives enhances the ability to characterize the dynamic characteristics of the system. The solution results of this set of equations can directly guide the optimal deployment of microservice components, achieving efficient resource utilization while ensuring service quality.

[0105] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the above-mentioned microservice component development method based on standard data analysis.

[0106] A third aspect of the present invention provides a microservice component development system based on standard data analysis, comprising the above-mentioned computer-readable storage medium.

[0107] Compared with the prior art, the microservice component development method based on standard data analysis provided by the present invention has the following beneficial effects:

[0108] The present invention proposes a microservice component development method, medium and system based on standard data analysis, which makes full use of innovative technologies such as red-black trees, Joseph ring algorithms, communication flow optimization, minimum spanning trees, consistent hashing, multidimensional backpack algorithms, linear programming, directed acyclic graphs, etc., and effectively solves key problems in microservice development;

[0109] First, the red-black tree structure is used to organize enterprise business data, achieving efficient and self-balanced data access, laying the foundation for subsequent data analysis and microservice development. Compared with traditional data storage methods, the red-black tree can stabilize the data access time complexity at the O(logn) level and maintain good balance during dynamic changes;

[0110] Secondly, the scheduling problem of service requests is handled by the multi-dimensional Joseph ring algorithm, and the optimal scheduling of service requests is achieved under the consideration of multiple constraints such as processing time, resources and priority. Compared with the traditional first-come-first-served or priority scheduling algorithm, this method can better balance the system resource utilization and service response time;

[0111] In the development of microservice components, a classic problem is how to accurately evaluate and optimize the communication overhead between services. The communication traffic optimization equations proposed in this paper establish a complete mathematical model by comprehensively considering factors such as resource constraints, traffic balance, latency optimization, and overhead optimization. This can not only guide the optimal deployment and scheduling of microservices, but also significantly improve the resource utilization efficiency and service quality of the system;

[0112] Another key issue is the optimization of service call relationships. The existing technologies mostly use static service registration and discovery mechanisms, which are difficult to adapt to scenarios where the service scale grows rapidly and changes frequently. The present invention dynamically optimizes service dependencies and call paths by constructing a service call binary heap and a minimum spanning tree. At the same time, the consistent hashing algorithm is used to allocate service nodes, which greatly improves the scalability of the system.

[0113] In addition, the present invention also introduces a distributed transaction manager and a performance evaluation matrix to ensure the reliability and continuous optimization of the system. Through this closed-loop optimization mechanism, the service operation status can be monitored in real time, and timely adjustments can be made when performance indicators do not meet the standards, ensuring that the system can continue to operate in the optimal state;

[0114] In general, the present invention provides a systematic, data-driven microservice component development method, which not only solves the key pain points in the existing technology, but also can greatly improve the performance, reliability and scalability of the microservice system, providing strong support for the enterprise's microservice transformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0116] Figure 1 Develop a method flow chart for a microservice component based on standard data analysis;

[0117] Figure 2 A red-black tree balance factor distribution diagram according to an embodiment of the present invention;

[0118] Figure 3 A resource demand distribution diagram of microservice components in an embodiment of the present invention;

[0119] Figure 4 A service call relationship network diagram of an embodiment of the present invention;

[0120] Figure 5 The figure is a heat map of the performance evaluation of the embodiment of the present invention. DETAILED DESCRIPTION

[0121] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0122] A specific implementation method of a microservice component development method based on standard data analysis provided by the present invention is as follows: Figure 1 As shown, the following steps are included:

[0123] S01. Collect enterprise business data, establish a standard data analysis model, establish a red-black tree structure for the enterprise business data, the red-black tree structure is used for data reading balance, and construct a data collection layer, a data cleaning layer of the data collection layer, a data analysis layer of the data cleaning layer, and a data visualization layer of the data analysis layer based on the red-black tree structure;

[0124] The specific implementation method of step S01 is: first, collect various business data of the enterprise. These data may come from different systems and data sources, including sales records, financial information, customer information, etc. In order to effectively organize and manage these data, a red-black tree structure is used to establish a standard data analysis model.

[0125] The red-black tree is a self-balancing binary search tree. It has good properties and can ensure that the time complexity of data access is stable at the level of O(logn). It can maintain good balance even when the data changes dynamically. The specific implementation process is as follows:

[0126] 1. Starting from the root node, recursively calculate the depth of the left and right subtrees. Take the maximum path length as the height h of the entire tree.

[0127] 2. Calculate the balance factor β based on the difference between the left and right subtree heights:

[0128]

[0129] Among them, h l and h r are the heights of the left and right subtrees, respectively. When |β|<1, the tree is in a balanced state; when |β|≥1, a rotation operation is required to rebalance the tree structure.

[0130] In this way, an efficient and stable data access model is established, laying the foundation for subsequent data analysis and microservice development.

[0131] On this basis, four functional components were constructed to meet the needs of data analysis: data acquisition layer, data cleaning layer, data analysis layer and data visualization layer.

[0132] The data collection layer is responsible for collecting raw data from various business systems to ensure data integrity. The data cleaning layer performs cleaning operations such as format conversion and missing value processing on the collected data to ensure data consistency and accuracy. The data analysis layer conducts in-depth analysis of the cleaned data based on the red-black tree structure to mine valuable insights and patterns. Finally, the data visualization layer presents the analysis results in the form of charts, dashboards, etc., so that decision makers can quickly understand the business situation.

[0133] These four levels of components are interconnected through standardized interfaces to form a complete data analysis system. The red-black tree structure is used to ensure the efficiency and balance of data access in the system, providing a solid foundation for subsequent microservice development.

[0134] S02, obtaining historical service requests of users to access microservice components, and using a multidimensional Joseph ring algorithm to perform ring sequence processing on the historical service requests, wherein the multidimensional Joseph ring algorithm includes processing time constraints, processing resource constraints, and processing priority constraints, generating an access call sequence, and calculating the service access parameters of the microservice component based on the access call sequence;

[0135] The specific implementation method of step S02 is: first, obtain the user's past access history data to the microservice component. These historical service requests record the user's access mode, resource requirements, priority and other information. In order to effectively process these request sequences, a method called the Multi-Dimensional Josephus Ring algorithm is adopted.

[0136] The multidimensional Joseph ring algorithm is a cyclic sequence generation algorithm that can generate an optimal service request processing sequence while satisfying multiple constraints such as processing time, resources and priority.

[0137] The specific implementation process is as follows:

[0138] 1. According to the number i and step length m of the current request, calculate the position number P(i,k) of the next request:

[0139] P(i,k)=(P(i-1,k)+m)%i;

[0140] Among them, P(i,k) represents the serial number of the kth position in the i-th round. This position serial number satisfies the cyclic property.

[0141] 2. Then, filter and optimize this loop sequence to ensure that the processing time of each request is T i Does not exceed the maximum threshold T max (e.g. 2 seconds), resource requirement R i No more than available resources R available (e.g. CPU 80%, memory 90%), Priority i Not less than the minimum priority min (e.g. medium priority).

[0142] The access call sequence generated in this way not only makes full use of system resources, but also ensures service quality and response time, which lays the foundation for subsequent communication traffic optimization.

[0143] S03, calculating the expected communication flow of the microservice component based on the service access parameter, calculating the optimal communication flow allocation scheme of the microservice component using a communication flow optimization equation group, and constructing a communication flow model;

[0144] The specific implementation of step S03 is: based on the access call sequence obtained in the previous step, the expected communication flow of the microservice component is calculated. This process involves a set of mathematical models called communication flow optimization equations, which include the following four key equations:

[0145] 1. Resource constraint equation:

[0146]

[0147] Among them, C i , S i 、N i are the computing resources, storage resources and network resource requirements of the i-th component respectively; R total is the total resource capacity of the system; 1 It is a small error term, and its value range is 0 to 0.1.

[0148] This equation ensures that the total resource usage of the microservice components does not exceed the system's capacity.

[0149] 2. Flow balance equation:

[0150]

[0151] Among them, F ij is the flow from node i to node j; k 3 , k 4 is the weight coefficient of the temporal and spatial change rate, ranging from 0.1 to 1; ε 2 It is a small error term, and its value range is -0.05~0.05.

[0152] This equation ensures the continuity and balance of data flow between nodes, while also taking into account the dynamic characteristics of flow in time and space.

[0153] 3. Delay optimization equation:

[0154]

[0155] Among them, D total is the total delay; C ij is the link capacity; L ij is the physical distance; V is the propagation speed; Q ij is the queuing delay; k 1is the weight coefficient of the queuing delay change rate, and its value range is 0.1 to 1; ε 3 is a small error term, and its value range is 1 to 10 milliseconds.

[0156] This equation minimizes the end-to-end communication delay between microservice components, including three parts: transmission delay, propagation delay, and queuing delay.

[0157] 4. Overhead optimization equation:

[0158]

[0159] where Cost total is the total operating overhead; D i , M i , T i are the deployment overhead, operation and maintenance overhead, and communication overhead respectively; w 1 , w 2 , w 3 are the corresponding weight coefficients; ε 4 is a small error term, and its value range is 0 to 100.

[0160] This equation minimizes the overall operating cost of the microservice system, including factors such as deployment cost, operation and maintenance cost, and communication cost.

[0161] By solving these four optimization equations, the optimal communication traffic allocation scheme for microservice components can be obtained. This scheme not only meets the resource constraint conditions but also minimizes the system delay and overhead to the greatest extent. It provides an important basis for subsequent service call routing planning and performance evaluation.

[0162] S04. Based on the communication traffic model, establish a service call binary heap, classify and sort the microservice components, generate service function features and service performance requirements, and establish a service classification model;

[0163] The specific implementation method of step S04 is: First, based on the optimal communication traffic allocation scheme obtained in the previous step, a data structure called Service Call Binary Heap is established. The binary heap is a complete binary tree with good properties, which can efficiently perform operations such as insertion, deletion, and search.

[0164] When constructing the binary heap, calculate a comprehensive score K i for each microservice component as the sorting key value of the component in the heap:

[0165] K i = αF i + βP i + γR i ;

[0166] Among them, F i is the functional characteristic value, P i is the performance requirement value, R i is the resource occupancy value; α, β, and γ are the corresponding weight coefficients.

[0167] By adjusting these three weight coefficients, the relative importance of functions, performance, and resources in the comprehensive score can be flexibly controlled. Components at the top of the heap often have more prominent functional characteristics, higher performance requirements, and lower resource occupancy, and they are often the core services of the system. While components at the bottom of the heap are relatively ordinary and of low priority.

[0168] With this service call binary heap, the microservice components can be effectively classified and sorted. This classification information provides an important basis for subsequent service development and deployment.

[0169] S05. Develop algorithm microservice components according to the service classification model, and use the minimum spanning tree algorithm to construct the service dependency relationship of the algorithm microservice components. The algorithm microservice components include a data collection component, a data cleaning component of the data collection component, a data analysis component of the data cleaning component, and a data visualization component of the data analysis component;

[0170] The specific implementation of step S05 is: start to develop microservice components according to the service classification model obtained in the previous step. First, use the Minimum Spanning Tree algorithm to construct the dependency relationship between service components.

[0171] The minimum spanning tree is an acyclic connected graph that can connect all nodes with the minimum edge weight value. In this scenario, the nodes represent each microservice component, and the edge weight reflects the degree of dependency between components. By constructing this minimum spanning tree, the call logic between microservices can be clearly described, laying a foundation for subsequent service call routing planning.

[0172] Based on this service dependency relationship, four core microservice components are designed and developed: a data collection component, a data cleaning component, a data analysis component, and a data visualization component.

[0173] The data collection component is responsible for collecting raw data from each business system; the data cleaning component is responsible for cleaning operations such as format conversion and missing value processing on the collected data; the data analysis component conducts in-depth analysis on the cleaned data based on the red-black tree structure to discover valuable insights and patterns; finally, the data visualization component presents the analysis results in the form of charts, dashboards, etc.

[0174] These four components call each other through standardized interfaces and jointly constitute a complete microservice architecture. By adopting the minimum spanning tree algorithm, it is ensured that the dependency relationships between these components are optimal, which is conducive to the expansion and maintenance of the system.

[0175] S06. Calculate the service call paths between the microservice components using the shortest path algorithm, construct a service call routing table, and determine the service registration requirements and service coordination strategies;

[0176] The specific implementation of step S06 is as follows: To optimize the call paths between microservice components, the All-Pairs Shortest Path algorithm is adopted. This algorithm can find the shortest path between any two nodes in a given weighted graph.

[0177] Specifically, first, a call distance matrix D is constructed, where the element d ij represents the call distance from node i to node j:

[0178] d ij = αt ij + βc ij + γ(1 - r ij );

[0179] where, t ij is the delay, c ij is the overhead, r ij is the reliability; α, β, and γ are the corresponding weight coefficients.

[0180] With this call distance matrix, the All-Pairs Shortest Path algorithm can be applied to calculate the optimal call paths between any two microservice components. These paths can not only minimize the communication delay to the greatest extent but also reduce the overall operating overhead of the system and improve the reliability of the service.

[0181] Record the information of these optimized call paths in the service call routing table to provide a basis for subsequent service orchestration and scheduling. At the same time, this path information also provides important input data for performance evaluation.

[0182] Through the above steps, the service registration requirements and service coordination strategies of each microservice component are determined. The service registration requirements describe which interfaces the component needs to register on the service registry; the service coordination strategy stipulates how to select a suitable service instance for processing when a new service request arrives. These information provide a basis for subsequent service orchestration and scheduling.

[0183] S07. Establish a microservice component registration center, use the consistent hashing algorithm for service node allocation, generate service connection classes, calculate the service reliability index of the service connection classes using a probability statistical model, and generate a microservice component connection software development kit.

[0184] The specific implementation of step S07 is as follows: A microservice component registration center is established to manage and coordinate all microservice instances in the system. In this registration center, a so-called consistent hashing algorithm is used for service node allocation.

[0185] Consistent hashing is a distributed hash table algorithm that can minimize data migration as much as possible when nodes are dynamically added or removed. Specifically, first, all microservice instances are mapped to a virtual hash ring. When a new request arrives, the position on the hash ring is calculated based on the request identifier, and then the nearest service instance is found in the clockwise direction for forwarding.

[0186] This method can not only ensure the load balancing of requests but also quickly adjust the routing when service nodes change dynamically, improving the scalability of the system.

[0187] At the same time, the reliability index R is also calculated for each microservice connection class:

[0188]

[0189] where λ i is the failure rate of the i-th component, μ i is the repair rate, t i is the running time; k 2 is the weight coefficient of the historical cumulative effect, and its value range is 0.01 to 0.1.

[0190] When the reliability of a certain connection class is lower than a preset threshold (such as 95%), the operation and maintenance personnel will be warned in time for inspection and maintenance. This monitoring mechanism ensures the stability and availability of the entire microservice system.

[0191] S08. Optimize service resource allocation using the multidimensional knapsack algorithm. The constraint conditions of the multidimensional knapsack algorithm include computing resource constraints, storage resource constraints, network resource constraints, and time resource constraints. Generate service collection classes, integrate the service collection classes to generate a service collection software development kit, and calculate the data transfer rate, message passing rate, and transaction completion rate of the microservice components based on the service collection software development kit.

[0192] The specific implementation of step S08 is as follows: In terms of resource allocation, an algorithm called Multi-Dimensional Knapsack is adopted. This algorithm can find the optimal resource allocation scheme under given resource constraints.

[0193] The specific constraint conditions are as follows:

[0194]

[0195] Among them, x i is a 0-1 decision variable; C i , S i , N i , T i are the computing, storage, network, and time resource requirements respectively; C max , S max , N max , T max are the upper limits of the corresponding resources.

[0196] By solving this multi-dimensional knapsack problem, the optimal resource quota is allocated to each microservice component. This can not only maximize the resource utilization efficiency but also ensure the overall performance indicators of the system, including data transfer rate, message passing rate, and transaction completion rate, etc.

[0197] These calculation results are integrated into a service collection software development kit, providing data support for subsequent service quality evaluation.

[0198] S09. Build a service quality model based on the data transfer rate, the message passing rate, and the transaction completion rate. The service quality model adopts the linear programming method to generate a transaction management benchmark value;

[0199] The specific implementation of step S09 is as follows: Based on the service performance indicators obtained in the previous step, a mathematical model called Service Quality Model is established. This model adopts the linear programming method and comprehensively considers three key factors: data transfer rate D r , message passing rate M r , and transaction completion rate T r .

[0200] The specific expression is as follows:

[0201]

[0202] Among them, Q is the service quality index; w 1 , w 2 , w 3 are the corresponding weight coefficients; k 5is the weight coefficient of the saturation effect, and its value range is 0.1 to 1.

[0203] This model can not only comprehensively reflect the overall service quality of the system, but also uses the Sigmoid function to describe the saturation characteristics of performance indicators. By adjusting the weight coefficient, the relative importance of each indicator in the overall evaluation can be flexibly controlled according to business requirements.

[0204] By solving this service quality model, a comprehensive service quality evaluation index Q can be obtained. When the actually measured service quality is lower than the preset threshold (e.g., 90%), corresponding adjustment measures can be taken in a timely manner to ensure that the system can continuously provide stable and reliable services.

[0205] S10. Construct a distributed transaction manager. The distributed transaction manager uses the two-phase commit protocol to determine the transaction execution order. The transaction execution order is based on the transaction management benchmark value. A directed acyclic graph is used to record the transaction execution status. The nodes of the directed acyclic graph are constructed based on the service call routing table, generating a task execution batch number and a task execution version number, and recording the version log;

[0206] The specific implementation of step S10 is: constructing a Distributed Transaction Manager to coordinate and manage distributed transactions in the entire microservice system.

[0207] Distributed transactions are a complex issue because participants may be located on different service nodes, and it is necessary to ensure the ACID (atomicity, consistency, isolation, and durability) characteristics of transactions. The two-phase commit protocol is used to determine the transaction execution order.

[0208] Specifically, in the first phase, the distributed transaction manager sends a prepare request to all participants to ask if the transaction can be executed. Only when all participants respond that it can be executed will the manager enter the second phase and send a commit request for the participants to officially execute the transaction. This method can effectively avoid the atomicity and consistency problems of distributed transactions.

[0209] At the same time, a directed acyclic graph (DAG) is also used to record the transaction execution status. The nodes of the DAG correspond to each service component, and the edges describe the call relationships between them. Based on this DAG structure, the execution process of the transaction can be clearly traced, and rollback can be performed in a timely manner in case of failures.

[0210] In addition, a task execution batch number and a version number are generated to record the historical changes of transactions and provide a basis for subsequent system rollback.

[0211] Through this distributed transaction management mechanism, the correctness and consistency of complex business logics in the microservice system are ensured, and the reliability of the application is improved.

[0212] S11. Based on the service interface metrics of the microservice components, the all-pairs shortest path algorithm is used to calculate the optimized call paths between the microservice components. Based on the optimal communication traffic allocation scheme, a call distance matrix is established, and the call distance matrix records the service call latency, service call overhead, and service call reliability.

[0213] The specific implementation of step S11 is as follows: To further optimize the call paths between microservice components, the all-pairs shortest path algorithm is adopted. This algorithm can find the shortest path between any two nodes in a given weighted graph.

[0214] Specifically, a call distance matrix D is first constructed:

[0215]

[0216] where d ij is the call distance from node i to node j, and the calculation formula is:

[0217] d ij = αt ij + βc ij + γ(1 - r ij );

[0218] where t ij is the latency, c ij is the overhead, r ij is the reliability; α, β, and γ are the corresponding weight coefficients.

[0219] With this call distance matrix, the all-pairs shortest path algorithm can be applied to calculate the optimal call paths between any two microservice components. These paths can not only minimize the communication latency to the greatest extent but also reduce the overall operation overhead of the system and improve the reliability of the service.

[0220] Record the information of these optimized call paths in the service call routing table to provide a basis for subsequent service orchestration and scheduling. At the same time, this path information also provides important input data for performance evaluation.

[0221] S12. Deploy a data analysis application, calculate service operation metrics using a queuing theory model. The service operation metrics include service call latency, service call overhead, and service call reliability. Construct a performance evaluation matrix to monitor the operation of the microservice components. The performance evaluation matrix is based on the service quality model. When the service operation metrics do not reach the preset thresholds, perform system rollback based on the version log.

[0222] The specific implementation of step S12 is as follows: A data analysis application is deployed to monitor the running status of the microservice system in real time. This application first uses a queuing theory model to calculate three key performance metrics: service call latency t, service call overhead c, and service call reliability r.

[0223] The queuing theory model is a classic method for system performance analysis. It can accurately describe the queuing and processing process of service requests in the system. In this scenario, each microservice component corresponds to a service window, and requests flow between these windows, subject to resource and time constraints. By solving the queuing theory model, the values of various performance metrics in the system can be obtained.

[0224] After obtaining these performance data, a performance evaluation matrix P is constructed, where the element p ij represents the score of the I-th microservice component on the J-th performance metric. This matrix provides a global performance perspective and can intuitively reflect the bottlenecks and problems in the system.

[0225] When it is found that some performance metrics are lower than the preset thresholds (for example, latency is less than 50 ms, overhead is less than 5 yuan, and reliability is greater than 99.9%), the system will be rolled back according to the previously recorded version log. The rollback operation can restore the system to a previous stable state and avoid the impact of performance degradation on the business. At the same time, according to the performance evaluation results, resource allocation will be adjusted in a timely manner, the call path will be optimized, etc., so that the system can continue to run in the optimal state.

[0226] Through the above 12 steps, a microservice component development method based on standard data analysis is constructed. This method makes full use of a series of innovative technologies such as red-black trees, Josephus rings, communication traffic optimization, minimum spanning trees, consistent hashing, multi-dimensional knapsack algorithms, linear programming, and directed acyclic graphs, effectively solving the key problems in microservice development. It can not only improve the performance and reliability of microservices, but also greatly enhance the resource utilization efficiency, providing a stable and reliable microservice support platform for enterprises.

[0227] An embodiment of a specific application scenario of the present invention is provided below: In order to build this microservices platform, an enterprise first collected the original data of each business system, covering order information, financial statements, customer files, etc. The total amount of this data reached 200 GB and came from data sources with different formats and structures.

[0228] In the data preparation stage, a red-black tree structure was used to organize this enterprise data. Specifically, starting from the root node, the heights of the left and right subtrees were recursively calculated. According to the difference in the heights of the left and right subtrees, the balance factor β was calculated:

[0229]

[0230] where h l and h r are the heights of the left and right subtrees respectively. It was found that |β| was basically less than 0.5, indicating that the entire red-black tree structure maintained good balance. In this way, the time complexity of data access was stabilized at the O(logn) level, and high access performance could be maintained even when the data was constantly updated.

[0231] On this basis, four functional components were constructed: a data collection layer, a data cleaning layer, a data analysis layer, and a data visualization layer. The data collection layer is responsible for collecting the original data from each business system, and the data volume is about 200 GB. The data cleaning layer performs operations such as format conversion and missing value processing on this data, and the data volume after cleaning is reduced to 150 GB. The data analysis layer conducts in-depth analysis on the cleaned data based on the red-black tree structure and discovers some valuable business insights. Finally, the data visualization layer presents the analysis results in the form of charts, dashboards, etc., facilitating decision-makers to quickly understand the business situation of the enterprise.

[0232] Figure 2 Shows the balance analysis of the red-black tree data structure in the microservices platform. The horizontal axis represents different node numbers, and the vertical axis represents the balance factor β value. The red scatter points in the figure represent the balance factor values of each node, and the dashed line represents the threshold range of the balance factor (±0.5). It can be seen that the balance factors of most nodes are within the range of ±0.5, indicating that the red-black tree structure maintains good balance.

[0233] During the development of the microservices components, the access history records of users to the system in the past year were first obtained. Analysis found that there were a total of 100 components in the enterprise's microservices system, and each component was accessed 500 times per day on average. The processing duration of each access was between 0.5 seconds and 2 seconds, the resource requirements varied between 10% and 50% of the CPU and 20% and 80% of the memory, and the priority was between medium and high.

[0234] To handle these service requests, the multi-dimensional Josephus ring algorithm is adopted. Specifically, according to the number i of the current request and the step size m = 3, the position serial number P(i,k) of the next request is calculated:

[0235] P(i,k) = (P(i - 1,k) + 3) % i;

[0236] Then, this cyclic sequence is screened and optimized to ensure that the processing duration T of each request i does not exceed 2 seconds, the resource requirement R i does not exceed 50% of the CPU, 80% of the memory, and the priority Priority i is not lower than medium. After optimization, an optimal request processing sequence that meets the above constraints is obtained.

[0237] Figure 3 shows the CPU and memory resource usage of 100 microservice components. The red curve represents the CPU usage rate (range 10% - 50%), and the blue curve represents the memory usage rate (range 20% - 80%). The semi-transparent filled area below the curve intuitively shows the fluctuation range of resource usage. It can be seen from the figure the differences in resource requirements of different components, which helps with resource planning and optimization.

[0238] Next, the expected communication traffic between microservice components is calculated. This process involves a set of mathematical models called communication traffic optimization equations, which include the following four key equations:

[0239] 1. Resource constraint equation:

[0240]

[0241] where C i 、S i 、N i are respectively the computing resource (CPU), storage resource (memory), and network resource (bandwidth) requirements of the i-th component; 100 is the total system resource capacity, and 0.05 is a small error term.

[0242] 2. Traffic balance equation:

[0243]

[0244] where F ij is the data traffic from node i to node j; 0.2 and 0.3 are the weight coefficients of the time and space change rates; 0.01 is a small error term.

[0245] 3. Delay optimization equation:

[0246]

[0247] Among them, D total is the total communication delay; 10 is the link capacity; 10 / 2 is the propagation delay; 2 is the queuing delay; 0.5 is the weight coefficient of the queuing delay variation rate; 5 is a small error term.

[0248] 4. Overhead Optimization Equation:

[0249]

[0250] Among them, Cost total is the total operating overhead; 1, 2, and 3 are the deployment overhead, operation and maintenance overhead, and communication overhead respectively; 50 is a small error term.

[0251] By solving these four optimization equations, the optimal communication traffic allocation scheme for microservice components is obtained. For example, the expected traffic from component 1 to component 20 is 20 MB / s, and the expected traffic from component 20 to component 50 is 30 MB / s, and so on. This scheme not only meets the resource constraint conditions but also can minimize the system delay and overhead.

[0252] After having the communication traffic allocation scheme, start to construct the service call binary heap. Calculate a comprehensive score K i for each microservice component as its sorting key value in the heap:

[0253] K i = 0.4F i + 0.3P i + 0.3R i ;

[0254] Among them, F i is the functional feature value (range 0 - 100), P i is the performance requirement value (range 0 - 100), R i is the resource occupancy value (range 0 - 100). After calculation, the comprehensive scores of the 5 components at the top of the heap are above 90 points and are considered the core services of the system; while the scores of the 20 components at the bottom are below 70 points and are relatively ordinary functional services.

[0255] Based on this service call binary heap, the dependency relationship between microservice components is constructed using the minimum spanning tree algorithm. Specifically, 100 components are regarded as nodes in the graph, and the weights of the edges are determined according to their call relationships. Then, applying the minimum spanning tree algorithm, an acyclic connected graph is obtained, which minimizes the resource overhead and call complexity between components.

[0256] Figure 4The calling relationships between microservice components are shown in the form of a force-directed graph. Nodes represent individual service components, the size of the nodes reflects the importance of the services, and the thickness of the edges represents the calling frequency. Through this network graph, the dependency relationships between services and the location of core services can be intuitively seen.

[0257] In terms of service registration, the consistent hashing algorithm is adopted. First, all 100 microservice instances are mapped onto a virtual hash ring. When a new request arrives, the position on the hash ring is calculated based on the request identifier, and then the nearest service instance is found in the clockwise direction for forwarding. This method can not only ensure the load balancing of requests but also quickly adjust the routing when service nodes change dynamically, improving the scalability of the system.

[0258] At the same time, a reliability index R is also calculated for each microservice connection class:

[0259]

[0260] where t i is the average running time of the i-th component, approximately 1 hour. When the reliability of a certain connection class is lower than 95%, the operation and maintenance personnel will be promptly warned to conduct inspections and maintenance.

[0261] In terms of resource allocation, the multidimensional knapsack algorithm is adopted. The specific constraints are as follows:

[0262]

[0263] where x i is a 0-1 decision variable, indicating whether to allocate resources to the i-th component; 20, 50, 10, and 1 are the CPU, memory, bandwidth, and time resource requirements respectively; 2000, 5000, 1000, and 100 are the upper limits of the corresponding resources.

[0264] By solving this multidimensional knapsack problem, optimal resource quotas are allocated to 100 microservice components. For example, the core service component is allocated 40% of the CPU, 70% of the memory, 20% of the bandwidth, and 100% of the time; the ordinary service component is allocated 20% of the CPU, 40% of the memory, 10% of the bandwidth, and 50% of the time. This can not only maximize the resource utilization efficiency but also ensure the overall performance indicators of the system, including the data transfer rate (average 90MB / s), message passing rate (average 2000 messages / s), and transaction completion rate (average 99.5%).

[0265] With these performance data, a service quality evaluation matrix P is constructed, where the element p ij represents the score of the i-th microservice component on the j-th performance indicator:

[0266] Table 1 Service Quality Assessment Matrix

[0267] Components Data transfer rate Message delivery rate Transaction completion rate Core Service 1 95 98 99.8 Core Service 2 92 96 99.7 Core Service 3 90 95 99.6 General service 1 85 88 99.3 Normal service 2 83 86 99.2 … … … …

[0268] It is found that the performance indicators of most components have reached the expected goals, but the transaction completion rate of some ordinary service components is slightly lower than the threshold of 99.5%.

[0269] To further optimize the service call path, a call distance matrix D is constructed:

[0270] Table 2 Call Distance Matrix

[0271] Components Core Service 1 Core Service 2 Core Service 3 Ordinary service 1 Normal service 2 Core Service 1 0 20 30 40 45 Core Service 2 20 0 25 35 40 Core Service 3 30 25 0 30 35 General service 1 40 35 30 0 20 Normal service 2 45 40 35 20 0 … … … … … …

[0272] where d ij represents the call distance from node i to node j, taking into account three factors: latency (10 ms), overhead (2 yuan), and reliability (99%).

[0273] Applying the all-pairs shortest path algorithm, the optimal call paths between any two microservice components are calculated. For example, the optimal path from Core Service 1 to Ordinary Service 2 is: Core Service 1 > Core Service 3 > Ordinary Service 1 > Ordinary Service 2, with a total distance of 130. These optimized call path information is recorded in the service call routing table to provide a basis for subsequent service orchestration and scheduling.

[0274] Figure 5 The heatmap is used to show the scores of different microservice components on various performance indicators. The horizontal axis represents different performance indicators (data transfer rate, message passing rate, transaction completion rate, etc.), and the vertical axis represents different service components. The darker the color, the higher the performance score, with a numerical range between 80 - 100. Through this heatmap, components with better and worse performance can be quickly identified.

[0275] Meanwhile, a distributed transaction manager is also constructed to coordinate and manage distributed transactions in the entire microservice system. This manager adopts the two-phase commit protocol, sending requests to all participants in the first phase, and only entering the second phase to send requests when all participants have feedback that they can execute. A directed acyclic graph is also used to record the execution status of transactions, and task execution batch numbers and version numbers are generated to track transaction changes.

[0276] Finally, a data analysis application was deployed to monitor the running status of the microservice system in real time. This application first used queuing theory models to calculate three key performance indicators: service call latency (average 50 ms), service call overhead (average 3 yuan), and service call reliability (99.5%). Then, a performance evaluation matrix P was constructed, and the performance indicators of most components reached the expected goals, only the transaction completion rate of individual ordinary service components was slightly lower.

[0277] When it is found that some performance indicators are lower than the preset thresholds, the system will be rolled back according to the previously recorded version logs. At the same time, according to the performance evaluation results, the resource allocation will be adjusted in a timely manner, the call path will be optimized, etc., so that the system can continue to run in the optimal state.

[0278] Through the above steps, a microservice support platform based on standard data analysis was successfully constructed. This platform uses innovative technologies to solve key problems in microservice development, and can not only meet the business needs of enterprises, but also has significant improvements in terms of performance, reliability, and scalability.

[0279] Table 3 Variables of the present invention and their explanations

[0280]

[0281] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention.

[0282] Specifically, the principle of the present invention is:

[0283] The core technical principle of the present invention is: based on enterprise business data, through a standardized data analysis model, the optimal development plan for microservice components is derived. This data analysis-driven method can better fit business needs and uses a series of innovative algorithms to solve key problems in microservice development;

[0284] First, the enterprise business data is organized using a red-black tree structure. A red-black tree is a self-balancing binary search tree with good properties that can stabilize the data access time complexity at the O(logn) level. This is very important for large-scale dynamic data and can greatly improve the system's response speed. In addition, the red-black tree can automatically adjust when the data changes to ensure that the tree structure always remains balanced, avoiding a sharp decline in search performance. This lays a foundation for subsequent data analysis and microservice development;

[0285] Secondly, the multi-dimensional Josephus ring algorithm is adopted to handle the scheduling problem of service requests. This algorithm can comprehensively consider the processing duration, resource requirements, and priorities of requests, and generate an optimal processing sequence that meets these constraints. Compared with traditional first-come-first-served or priority scheduling, it can better balance system resource utilization and service quality. This is because the multi-dimensional Josephus ring algorithm is essentially a cyclic scheduling mechanism that can make full use of system resources and ensure that critical requests can be processed preferentially;

[0286] In terms of the communication optimization of microservice components, the present invention proposes a complete set of communication traffic optimization equations. This set of equations is modeled from four dimensions: resource constraints, traffic balance, delay optimization, and cost optimization, which can not only ensure that the system operates within the resource carrying capacity but also minimize the end-to-end communication delay and the overall operating cost. Among them, the resource constraint equation introduces an error term to cope with the volatility of actual resource usage; the traffic balance equation considers the dynamic characteristics of traffic in space and time; the delay optimization equation combines three parts: transmission delay, propagation delay, and queuing delay, and introduces a differential term to describe the dynamic characteristics of the system; the cost optimization equation comprehensively considers deployment costs, operation and maintenance costs, and communication costs. By solving this set of equations, the optimal deployment and scheduling scheme of microservices can be obtained, greatly improving the performance and economy of the system;

[0287] Another key issue is the optimization of service call relationships. The present invention adopts two data structures and algorithms: service call binary heap and minimum spanning tree. The binary heap can efficiently classify and sort microservices to ensure that core services are preferentially deployed and called. The minimum spanning tree can construct the optimal dependency relationship between service components with the least resource overhead. This can not only simplify the call logic of the system but also facilitate dynamic expansion and rapid response to business changes. At the same time, the consistent hashing algorithm is used for service node allocation, greatly improving the scalability of the system and enabling it to cope with the rapid growth of service scale;

[0288] Finally, the present invention designs a distributed transaction manager and a performance evaluation matrix to ensure the reliability and continuous optimization of the system. The distributed transaction manager adopts a two-phase commit protocol, which can effectively coordinate complex distributed transactions and avoid atomicity and consistency problems. The performance evaluation matrix provides a basis for the real-time monitoring and dynamic adjustment of the system operation status, ensuring the continuous satisfaction of service quality.

Claims

1. A microservice component development method based on standard data analysis, characterized in that: include: A standard data analysis model is established for data stratification processing, and a multidimensional Joseph ring algorithm is used to process historical service requests to obtain an access call sequence. Service access parameters are calculated based on the access call sequence, and a communication traffic optimization equation group is used to calculate the optimal communication traffic allocation plan. The communication traffic optimization equation group includes a resource constraint equation, a traffic balance equation, a delay optimization equation, and a cost optimization equation. The resource constraint equation is used to limit the total amount of resource usage, the traffic balance equation is used to balance data traffic, the delay optimization equation is used to reduce communication delay, and the cost optimization equation is used to reduce the total system operation cost. A service call routing table is constructed and a microservice component registration center is established. A multidimensional backpack algorithm is used to optimize service resource allocation.

2. According to a method for developing microservice components based on standard data analysis according to claim 1, it is characterized in that: The step of establishing a standard data analysis model includes: collecting enterprise business data, establishing a red-black tree structure for the enterprise business data, the red-black tree structure is used for data reading balance, and constructing a data collection layer, a data cleaning layer of the data collection layer, a data analysis layer of the data cleaning layer, and a data visualization layer of the data analysis layer based on the red-black tree structure.

3. According to a method for developing microservice components based on standard data analysis according to claim 2, it is characterized in that: The multidimensional Joseph ring algorithm includes processing time constraints, processing resource constraints, and processing priority constraints; the constraints of the multidimensional backpack algorithm include computing resource constraints, storage resource constraints, network resource constraints, and time resource constraints.

4. According to a method for developing microservice components based on standard data analysis according to claim 3, it is characterized in that: The input parameters of the resource constraint equation include the computing resource requirements, storage resource requirements, and network resource requirements in the service access parameters; the input parameters of the traffic balance equation include the service input traffic, service output traffic, and service transit traffic in the historical service requests; the input parameters of the delay optimization equation include the communication transmission delay, service processing delay, and task queuing delay in the access call sequence; The input parameters of the cost optimization equation include the service deployment cost, service operation and maintenance cost, and service communication cost in the service access parameters.

5. According to a method for developing microservice components based on standard data analysis according to claim 4, it is characterized in that: It also includes building a distributed transaction manager, which uses a two-phase commit protocol to determine the transaction execution order, uses a directed acyclic graph to record the transaction execution status, generates a task execution batch number and a task execution version number, and records a version log.

6. According to a method for developing microservice components based on standard data analysis according to claim 5, it is characterized in that: The step of constructing a service call routing table includes: using a shortest path algorithm to calculate a service call path between the microservice components, and determining a service registration requirement and a service coordination strategy based on the service call path.

7. A method for developing microservice components based on standard data analysis according to claim 6, characterized in that: It also includes calculating the optimized call path between the microservice components based on the service interface indicators of the microservice components using the full-graph shortest path algorithm, and establishing a call distance matrix, which records the service call delay, service call overhead, and service call reliability.

8. A method for developing microservice components based on standard data analysis according to claim 7, characterized in that: It also includes deploying a data analysis application, using a queuing theory model to calculate service operation indicators, building a performance evaluation matrix to monitor the operation of the microservice components, and when the service operation indicators do not reach a preset threshold, performing a system rollback based on the version log.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed in a computer, they are used to execute the microservice component development method based on standard data analysis according to any one of claims 1 to 8.

10. A microservice component development system based on standard data analysis, comprising the computer-readable storage medium of claim 9.

Images