Data analysis system and method based on distributed cache technology

Abstract

The invention discloses a data analysis system and method based on a distributed cache technology, and relates to the technical field of data analysis. Service nodes are positioned based on target service data, and service node state data are obtained; analyzing a business node data request demand degree in real time based on the state data of each business node, and judging a database access state; according to a database access state judgment result, constructing a data cache node for each service node, and analyzing cache memory allocation based on data request demand data of the corresponding service node; analyzing the memory utilization degree of the real-time cache node according to the data reading state of each service node cycle to the cache node, and performing cycle memory dynamic regulation and control on the cache node; according to the dynamic regulation and control data of the periodic memory of the cache node, generating a regulation and control instruction and feeding back to the management port; according to the invention, flexible regulation, control and analysis of cache demands under different demand scenes in a service node data calling process are realized.

Description

Technical Field

[0001] The present invention relates to the field of data analysis, and specifically to a data analysis system and method based on distributed cache technology. Background Art

[0002] Distributed cache technology is a technical solution for coping with high concurrency and big data scenarios. By dispersing data storage across multiple server nodes, it can improve system throughput and significantly reduce data read / write latency and high concurrency requests. However, the current distribution and regulation of cache nodes are relatively simple. It cannot adaptively regulate cache nodes for a dynamic node data request environment, nor can it perform its own capacity regulation and node status regulation for complex data change scenarios. This results in a situation where most current cache nodes have unreasonable allocation of capacity resources, causing waste of memory resources. Summary of the Invention

[0003] The purpose of the present invention is to provide a data analysis system and method based on distributed cache technology to solve the problems raised in the prior art.

[0004] To achieve the above purpose, the present invention provides the following technical solutions: A data analysis method based on distributed cache technology, the method includes the following steps: Determine the target business data, locate the business nodes based on the target business data, and obtain the business node status data; Based on the status data of each business node, analyze the degree of data request requirements of the business nodes in real time, and judge the database access status according to the analysis of the data request requirements of the business nodes; according to the judgment result of the database access status, construct data cache nodes for each business node, and allocate cache memory based on the analysis of the data request requirements of the corresponding business node; According to the data reading status of the cache nodes in each business node cycle, analyze the real-time memory utilization degree of the cache nodes, and perform periodic memory dynamic regulation on the cache nodes according to the real-time memory utilization degree of the cache nodes; generate a regulation instruction based on the periodic memory dynamic regulation data of the cache nodes and feedback it to the management port.

[0005] Further, determine the target business data by retrieving the business data through the management port; the business data includes business name data and business node data; the business node data is the business node number; Locate the business nodes through the target business node data, and obtain the status data of the corresponding business nodes by obtaining the corresponding business node request data; the status data of the business nodes includes the number of request data packets and the capacity of the request data packets of the business nodes.

[0006] Further, based on the status data of each business node of the target business, construct the corresponding business node status set; analyze the real-time data request requirements of the target business according to the business node status set, and obtain the real-time data request data capacity of the target business; its calculation formula is ; P(t) is the data request demand capacity of the target business at the current time t; p is the single data packet capacity; Mj is the number of data packet requests of the business node with the corresponding number j at time t; n is the number of business nodes of the target business; based on the analysis of the actual target business data request demand data, judge and analyze the real-time access status of the database; calculate the maximum response request data packet capacity of the database at a single moment by obtaining the maximum number of request data packets responded by the database at a single moment; determine the access status of the database at the current moment by comparing the real-time data request demand analysis data of the target business at a single moment with the maximum response request data packet capacity of the database at a single moment; the access status includes busy and idle; the specific comparison steps are as follows: determine the early warning coefficient k for the maximum response request data packet capacity of the database at a single moment, and obtain the early warning capacity P of the response request data packet of the database at a single moment e,s , and its calculation is P e,s = k * P max,s ; where P max,s is the maximum response request data packet capacity of the database at a single moment; compare with the current time target business data request demand capacity P(t). If P(t) ≥ P e,s , then judge that the current database access status is busy; otherwise, judge that the current database access status is busy and idle; When the access status of the database is busy, construct cache nodes for each business node of the target business; based on the real-time business data request demand data of each business node, conduct capacity prediction analysis on each cache node, and obtain the real-time capacity data of the cache nodes corresponding to each business node; its calculation is ; Among them, C(j,t) is the predicted value of the capacity of the cache node corresponding to the service node numbered j at time t; r is the pre-storage coefficient of the single-data-packet cache capacity; the pre-storage coefficient of the single-data-packet cache capacity refers to the additional storage space required for the storage space when caching and storing a single data packet to prevent data storage loss; by dividing the observation period T1, the capacity prediction analysis of each cache node within the observation period is coordinated, and the maximum value is taken as the memory capacity value of the cache node of the corresponding service node within the period T1; according to the request demand data of the corresponding service node within the period T1, type division and coordination are carried out, the number of request data of each type within the period T1 is determined respectively, and the proportion of the request data volume of each type of request data within the period T1 is analyzed; according to the analysis results, a proportion threshold is introduced, and the proportion of the request data volume of each type of request data is compared with the proportion threshold to determine high-request-type data and low-request-type data; among them, the request data type with the proportion of the request data volume of the corresponding type of request data greater than or equal to the proportion threshold is judged as high-request-type data, and vice versa is judged as low-request data type; according to the comparison analysis data of each type of request data in the corresponding service node, the cache space of the cache node is divided into a primary cache space and a secondary cache space; among them, the primary cache space is used to store high-request-type data; the secondary cache space is used to store low-request-type data.

[0007] Further, based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each service node of the target service, a cache node update period T2 is set, and the call situation of each service node for the data stored in the cache node within the period is analyzed to analyze the utilization degree of the primary cache space and the secondary cache space in real time; it analyzes the call hit rates of the data stored in the primary cache space and the secondary cache space respectively; its calculation formula is ; Among them, hit1 and hit2 respectively correspond to the call hit rates of the data stored in the primary storage space and the secondary storage space of the cache node of each service node within the period T2; m(1,T2) and m(1,T2) respectively correspond to the call numbers of the data stored in the primary storage space and the secondary storage space of the cache node of each service node within the period T2; m(T2) is the call number of the data stored in the cache node of each corresponding service node within the period T2; Based on the analysis data of the call situation of each service node for the data stored in the cache node within the cache node update period T2, dynamic regulation and control analysis of the cache node storage memory of each service node is carried out; it analyzes the memory occupancy of the primary storage space and the secondary storage space of the corresponding cache node within the period T2 respectively based on the call hit rates of the data in the primary storage space and the secondary storage space of the corresponding cache node; its calculation formula is ; Among them, C(1,j,T2) and C(2,j,T2) are respectively the memory occupation amounts within a period in the primary storage space and the secondary storage space in the cache nodes corresponding to the service node with number j within period T2; C(1,j) and C(2,j) are respectively the memory capacities of the primary storage space and the secondary storage space in the cache nodes corresponding to the service node with number j; S is the cache fragmentation rate parameter of the cache node; by analyzing the memory update capacities Cg1 of the primary storage space and Cg2 of the secondary storage space in the cache nodes corresponding to each service node, and setting the cache prompt capacity Cv, the memory occupation amounts of the primary storage space and the secondary storage space in each cache node at consecutive time points within a period are updated and judged, and a memory capacity regulation strategy for the corresponding cache node is generated according to the judgment result; among them, when at all consecutive time points within a certain period, the memory occupation amount of the primary storage space in the cache node is greater than the cache prompt capacity and less than or equal to the memory update capacity of the cache node, which is correspondingly Cv < C(1,j,T2) ≤ Cg1, the memory capacity of the corresponding primary storage space is regulated to the memory update capacity Cg of the cache node; if C(1,j,T2) ≤ Cv, it indicates that the current primary storage space is in an idle state; according to the above judgment method, the occupation amount of the secondary storage space is judged and the memory capacity is regulated; among them, if both the primary storage space and the secondary storage space of a certain cache node are in an idle state, a cache node sleep instruction is generated, and the service node requests data to directly call data from the database; and when the primary storage space or the secondary storage space is greater than the cache node update capacity, no memory regulation is performed; the calculation formulas for the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space corresponding to the cache node are 。

[0008] Further, the service node status data of the target service and the status data of the corresponding cache nodes are fed back through a visualization window; the cache node status data includes the capacity data of the cache node and the data access records of the responding service nodes; The generated memory capacity regulation strategies corresponding to each cache node are output to the management port, and the memory capacity regulation strategies corresponding to each cache node are executed.

[0009] A data analysis system based on a distributed cache technology, the system includes a node positioning module, a node analysis module, a dynamic regulation module and a data feedback module; The node location module determines target service data, locates service nodes based on the target service data, and obtains service node status data; the node analysis module analyzes the degree of data request requirements of service nodes in real time based on the status data of each service node, and judges the database access status according to the analysis of the data request requirements of service nodes; according to the judgment result of the database access status, data cache nodes are constructed for each service node, and cache memory allocation is analyzed based on the data request requirement data of the corresponding service node; the dynamic regulation module analyzes the memory utilization degree of the cache nodes in real time according to the data reading status of the cache nodes in each service node cycle, and performs periodic memory dynamic regulation on the cache nodes according to the real-time memory utilization degree of the cache nodes; according to the periodic memory dynamic regulation data of the cache nodes, a regulation instruction is generated and fed back to the management port; the data feedback module outputs the service node status data and the status data of the corresponding cache nodes through a visualization window and executes the regulation instruction.

[0010] Further, the node location module includes a target service determination unit and a service node location unit; The target service determination unit determines target service data based on the management port by retrieving service data; the service data includes service name data and service node data; the service node data is the service node number. The service node location unit locates service nodes through the target service node data, and obtains the status data of the corresponding service nodes by obtaining the corresponding service node request data; the status data of the service nodes includes the number of request data packets and the capacity of request data packets of the service nodes.

[0011] Further, the node analysis module includes a service node requirement analysis unit and a cache node construction unit; The service node requirement analysis unit constructs a corresponding service node status set based on the status data of each service node of the target service; analyzes the real-time data request requirements of the target service according to the service node status set, and obtains the real-time data request data capacity of the target service; based on the analysis data of the actual target service data request requirements, judges and analyzes the real-time access status of the database; calculates the capacity of the maximum response request data packet of the database at a single moment by obtaining the number of request data packets with the maximum response at a single moment of the database; determines the access status of the database at the current moment by comparing the analysis data of the target service data request requirements at a single moment with the capacity of the maximum response request data packet of the database at a single moment; the access status includes busy and idle. When the access status of the database is busy, the cache node construction unit constructs cache nodes for each service node of the target service; based on the real-time service data request demand data of each service node, it performs capacity prediction analysis on each cache node to obtain the real-time capacity data of the cache nodes corresponding to each service node; by dividing the observation period T1, it coordinates the capacity prediction analysis of each cache node within the observation period, and takes the maximum value as the memory capacity value of the cache node of the corresponding service node within the period T1; according to the request demand data of the corresponding service node within the period T1, it conducts type division and coordination, respectively determines the quantity of each type of request data within the period T1, and analyzes the proportion of the request data volume of each type of request data within the period T1; according to the analysis results, it introduces a proportion threshold, compares the proportion of the request data volume of each type of request data with the proportion threshold to determine high-request type data and low-request type data; according to the comparative analysis data of each type of request data in the corresponding service node, it divides the cache space of the cache node into a primary cache space and a secondary cache space; where the primary cache space is used to store high-request type data; the secondary cache space is used to store low-request type data.

[0012] Furthermore, the dynamic regulation module includes a cache node memory analysis unit and a cache node memory regulation unit; Based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each service node of the target service, the cache node memory analysis unit sets the cache node update period T2, analyzes the call situation of each service node to the data stored in the cache node within the period, and real-time analyzes the utilization degree of the primary cache space and the secondary cache space; it analyzes the data call hit rate of the data stored in the primary cache space and the secondary cache space respectively; Based on the analysis data of the call situation of each service node to the data stored in the cache node within the cache node update period T2, the cache node memory regulation unit conducts dynamic regulation analysis on the memory of the cache node stored by each service node; it analyzes the memory occupancy of the primary storage space and the secondary storage space of the corresponding cache node within the period T2 respectively based on the data call hit rate of the primary storage space and the secondary storage space in the corresponding cache node; by analyzing the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space in the cache nodes of each service node, and setting the cache prompt capacity Cv, it updates and judges the memory occupancy of the primary storage space and the secondary storage space in each cache node at consecutive time points within the period, and generates a memory capacity regulation strategy for the corresponding cache node according to the judgment results.

[0013] Furthermore, the data feedback module includes a visualization unit and a policy execution unit; The visualization unit feeds back the status data of the service nodes of the target service and the status data of the corresponding cache nodes through a visualization window; the cache node status data includes the capacity data of the cache node and the data access records of the responding service nodes. The policy execution unit outputs the memory capacity regulation policies generated for each corresponding cache node to the management port and executes the memory capacity regulation policies for each corresponding cache node.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention determines nodes for the target service, and through a series of analyses of node request volumes and database access statuses, combined with cache node construction and cache node memory prediction and regulation analysis, realizes flexible regulation and analysis of cache requirements in different demand scenarios during the data access process of service nodes; this application can realize adaptive cache node regulation for a dynamic node data request environment, respond to complex data change scenarios for cache capacity regulation and status regulation, and improve the situation where most current cache nodes have unreasonable allocation of capacity resources, resulting in waste of memory resources. Description of the Drawings

[0015] Figure 1 It is a schematic structural diagram of a data analysis system based on distributed cache technology of the present invention; Figure 2 It is a schematic flowchart of a data analysis method based on distributed cache technology of the present invention. Detailed Embodiments

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] Embodiment: As Figure 1 shown, the present invention provides a technical solution: A data analysis system based on distributed cache technology, where the system includes a node positioning module, a node analysis module, a dynamic regulation module, and a data feedback module; Among them, the node positioning module determines the target service data, locates the service nodes based on the target service data, and obtains the service node status data; the node analysis module analyzes the degree of data request requirements of the service nodes in real time based on the status data of each service node, and judges the database access status according to the data request requirements analysis data of the service nodes; according to the database access status judgment result, data cache nodes are constructed for each service node, and cache memory allocation is analyzed based on the data request requirements analysis data of the corresponding service nodes; the dynamic regulation module analyzes the memory utilization degree of the cache nodes in real time according to the data reading status of the cache nodes in each service node cycle, and performs dynamic cycle memory regulation on the cache nodes according to the real-time memory utilization degree of the cache nodes; a regulation instruction is generated according to the dynamic cycle memory regulation data of the cache nodes and fed back to the management port; the data feedback module outputs the service node status data and the status data of the corresponding cache nodes through a visualization window and executes the regulation instruction.

[0018] Further, the node positioning module includes a target service determination unit and a service node positioning unit; The target service determination unit determines the target service data based on the management port by retrieving the service data; the service data includes service name data and service node data; the service node data is the service node number. The service node positioning unit locates the service nodes through the target service node data, and obtains the status data of the corresponding service nodes by obtaining the corresponding service node request data; the status data of the service nodes includes the number of request data packets and the capacity of the request data packets of the service nodes.

[0019] Further, the node analysis module includes a service node requirement analysis unit and a cache node construction unit; The service node requirement analysis unit constructs a corresponding service node status set based on the status data of each service node of the target service; analyzes the real-time data request requirements of the target service according to the service node status set, and obtains the real-time data request data capacity of the target service; based on the actual target service data request requirements analysis data, judges and analyzes the real-time access status of the database; calculates the maximum response request data packet capacity of the database at a single moment by obtaining the number of request data packets with the maximum response at a single moment of the database; determines the access status of the database at the current moment by comparing the single-moment target service data request requirements analysis data with the maximum response request data packet capacity of the database at a single moment; the access status includes busy and idle; When the access status of the database is busy, the cache node construction unit constructs cache nodes for each service node of the target service; based on the real-time service data request demand data of each service node, it performs capacity prediction analysis on each cache node to obtain the real-time capacity data of the cache nodes corresponding to each service node; by dividing the observation period T1, it coordinates the capacity prediction analysis of each cache node within the observation period, and takes the maximum value as the memory capacity value of the cache node of the corresponding service node within the period T1; according to the request demand data of the corresponding service node within the period T1, it conducts type division and coordination, respectively determines the quantity of each type of request data within the period T1, and analyzes the proportion of the request data volume of each type of request data within the period T1; according to the analysis results, it introduces a proportion threshold, compares the proportion of the request data volume of each type of request data with the proportion threshold to determine high-request type data and low-request type data; according to the comparison analysis data of each type of request data in the corresponding service node, it divides the cache space of the cache node into a primary cache space and a secondary cache space; where the primary cache space is used to store high-request type data; the secondary cache space is used to store low-request type data.

[0020] Furthermore, the dynamic regulation module includes a cache node memory analysis unit and a cache node memory regulation unit; Based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each service node of the target service, the cache node memory analysis unit sets the cache node update period T2, analyzes the invocation situation of the data stored in the cache nodes by each service node within the period, and real-time analyzes the utilization degree of the primary cache space and the secondary cache space; it analyzes the hit rate of the data invocation in the primary cache space and the secondary cache space respectively; Based on the analysis data of the invocation situation of the data stored in the cache nodes by each service node within the cache node update period T2, the cache node memory regulation unit conducts dynamic regulation analysis on the memory of the cache nodes stored by each service node; it analyzes the memory occupancy of the primary storage space and the secondary storage space of the corresponding cache node within the period T2 respectively based on the hit rate of the data invocation in the primary storage space and the secondary storage space of the corresponding cache node; by analyzing the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space of the cache nodes of each service node, and setting the cache prompt capacity Cv, it updates and judges the memory occupancy of the primary storage space and the secondary storage space of each cache node at consecutive time points within the period, and generates a memory capacity regulation strategy for the corresponding cache node according to the judgment results.

[0021] Furthermore, the data feedback module includes a visualization unit and a policy execution unit; The visualization unit feeds back the status data of the service nodes of the target service and the status data of the corresponding cache nodes through the visualization window; the cache node status data includes the capacity data of the cache node and the data retrieval records of the response service nodes. The policy execution unit outputs the memory capacity regulation policies generated for each cache node to the management port and executes the memory capacity regulation policies for each corresponding cache node. As Figure 2 shown, the present invention provides another technical solution: A data analysis method based on distributed cache technology, the method comprising the following steps: Determine the target service data, locate the service nodes based on the target service data, and obtain the status data of the service nodes. Analyze the degree of data request requirements of the service nodes in real time based on the status data of each service node, and judge the database access status according to the analysis of the data request requirements of the service nodes; according to the judgment result of the database access status, construct data cache nodes for each service node, and analyze the cache memory allocation based on the data request requirements analysis of the corresponding service nodes. Analyze the memory utilization degree of the real-time cache nodes according to the data reading status of the cache nodes in each service node cycle, and perform periodic memory dynamic regulation on the cache nodes according to the memory utilization degree of the real-time cache nodes; generate a regulation instruction according to the periodic memory dynamic regulation data of the cache nodes and feedback it to the management port.

[0022] Further, determine the target service data by retrieving the service data through the management port; the service data includes service name data and service node data; the service node data is the service node number. Locate the service nodes through the target service node data, and obtain the status data of the corresponding service nodes by obtaining the request data of the corresponding service nodes; the status data of the service nodes includes the number of request data packets and the capacity of the request data packets of the service nodes.

[0023] Further, based on the status data of each service node of the target service, construct a corresponding service node status set; analyze the real-time data request requirements of the target service according to the service node status set, and obtain the real-time data request data capacity of the target service; its calculation formula is ; P(t) is the data request demand capacity of the target service at the current time t; p is the single data packet capacity; Mj is the number of data packet requests of the service node with the corresponding number j at time t; n is the number of service nodes of the target service; based on the analysis of the actual target service data request demand, the real-time access status of the database is judged and analyzed; by obtaining the maximum number of request data packets that the database can respond to at a single moment, the maximum response request data packet capacity of the database at a single moment is calculated; by comparing the data of the target service data request demand analysis at a single moment with the maximum response request data packet capacity of the database at a single moment, the access status of the database at the current moment is determined; the access status includes busy and idle; the specific comparison steps are as follows. By determining the warning coefficient k for the maximum response request data packet capacity of the database at a single moment, the warning capacity P of the response request data packet of the database at a single moment is obtained. e,s It is calculated as P e,s = k * P max,s ; where P max,s is the maximum response request data packet capacity of the database at a single moment; compare with the data request demand capacity P(t) of the target service at the current moment. If P(t) ≥ P e,s , it is determined that the current database access status is busy; otherwise, it is determined that the current database access status is idle. When the access status of the database is busy, cache nodes are constructed for each service node of the target service; based on the real-time service data request demand data of each service node, capacity prediction and analysis are performed on each cache node to obtain the real-time capacity data of the cache nodes corresponding to each service node. The calculation is as follows ; Among them, C(j,t) is the predicted capacity value of the cache node corresponding to the service node numbered j at time t; r is the pre-storage coefficient of the single data packet cache capacity; the pre-storage coefficient of the single data packet cache capacity refers to the additional storage space required for the storage space when caching and storing a single data packet to prevent data storage loss; by dividing the observation period T1, the capacity prediction analysis of each cache node within the observation period is coordinated, and the maximum value is taken as the memory capacity value of the cache node of the corresponding service node within the period T1; according to the type division and coordination of the request demand data within the period T1 of the corresponding service node, the number of request data of each type within the period T1 is determined respectively, and the proportion of the request data volume of each type of request data within the period T1 is analyzed; according to the analysis results, a proportion threshold is introduced, and the proportion of the request data volume of each type of request data is compared with the proportion threshold to determine the high-request type data and the low-request type data; among them, the request data type with the proportion of the request data volume of the corresponding type of request data greater than or equal to the proportion threshold is judged as the high-request type data, and vice versa is judged as the low-request data type; according to the comparative analysis data of each type of request data in the corresponding service node, the cache space of the cache node is divided into a first-level cache space and a second-level cache space; among them, the first-level cache space is used to store high-request type data; the second-level cache space is used to store low-request type data.

[0024] Furthermore, based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each service node of the target service, a cache node update period T2 is set, and the call situation of each service node for the data stored in the cache node within the period is analyzed to analyze the utilization degree of the first-level cache space and the second-level cache space in real time; it analyzes the call hit rates of the data stored in the first-level cache space and the second-level cache space respectively; its calculation formula is ; Among them, hit1 and hit2 respectively correspond to the call hit rates of the data stored in the first-level storage space and the second-level storage space in the cache node of each service node within the period T2; m(1,T2) and m(1,T2) respectively correspond to the call quantities of the data stored in the first-level storage space and the second-level storage space in the cache node of each service node within the period T2; m(T2) is the call quantity of the data stored in the cache node of each corresponding service node within the period T2; Based on the analysis data of the call situation of each service node for the data stored in the cache node within the cache node update period T2, the dynamic regulation and control analysis of the storage memory of the cache node by each service node is carried out; it respectively analyzes the memory occupancy of the first-level storage space and the second-level storage space of the corresponding cache node within the period T2 based on the call hit rates of the data in the first-level storage space and the second-level storage space of the corresponding cache node; its calculation formula is ; Among them, C(1, j, T2) and C(2, j, T2) are respectively the memory occupation amounts within a period in the primary storage space and the secondary storage space in the cache nodes corresponding to the service node with number j within the period T2; C(1, j) and C(2, j) are respectively the memory capacities of the primary storage space and the secondary storage space in the cache nodes corresponding to the service node with number j; S is the cache fragmentation rate parameter of the cache node; by analyzing the memory update capacities Cg1 of the primary storage space and Cg2 of the secondary storage space in the cache nodes of each service node and setting the cache prompt capacity Cv, the memory occupation amounts of the primary storage space and the secondary storage space in each cache node at consecutive time points within the period are updated and judged, and a memory capacity regulation strategy for the corresponding cache node is generated according to the judgment result; among them, when at all consecutive time points within a certain period, the memory occupation amount of the primary storage space in the cache node is greater than the cache prompt capacity and less than or equal to the memory update capacity of the cache node, which corresponds to Cv < C(1, j, T2) ≤ Cg1, the memory capacity of the corresponding primary storage space is regulated to the memory update capacity Cg of the cache node; if C(1, j, T2) ≤ Cv, it indicates that the current primary storage space is in an idle state; according to the above judgment method, the occupation amount of the secondary storage space is judged and the memory capacity is regulated; among them, if both the primary storage space and the secondary storage space of a certain cache node are in an idle state, a cache node sleep instruction is generated, and the service node requests data to directly call data from the database; and when the primary storage space or the secondary storage space is greater than the cache node update capacity, no memory regulation is performed; the calculation formulas for the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space corresponding to the cache node are 。

[0025] Further, the service node status data of the target service and the status data of the corresponding cache node are fed back through a visualization window; the cache node status data includes the capacity data of the cache node and the data access record for responding to the service node Output the generated memory capacity regulation strategies of the corresponding cache nodes to the management port and execute the memory capacity regulation strategies of the corresponding cache nodes

[0026] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights

Claims

1. A data analysis method based on distributed caching technology, characterized in that: The method comprises the following steps: Determine target business data, locate business nodes based on the target business data, and obtain business node status data; Analyze the data request demand of business nodes in real time based on the status data of each business node, and judge the database access status based on the business node data request demand analysis data; build a data cache node for each business node based on the database access status judgment result, and analyze the cache memory allocation based on the corresponding business node data request demand data; According to the data reading status of the cache node in each business node cycle, the real-time cache node memory utilization is analyzed, and the cache node periodic memory dynamic regulation is performed according to the real-time cache node memory utilization; the regulation instructions are generated based on the cache node periodic memory dynamic regulation data and fed back to the management port.

2. The data analysis method based on distributed caching technology according to claim 1, characterized in that: Determine target business data by searching business data based on the management port; the business data includes business name data and business node data; the business node data is the business node number; The service node is located by the target service node data, and the status data of the corresponding service node is obtained by obtaining the corresponding service node request data; the status data of the service node includes the number of request data packets and the capacity of the request data packets of the service node.

3. The data analysis method based on distributed caching technology according to claim 2, characterized in that: Based on the status data of each business node of the target business, a corresponding business node status set is constructed; Analyze the real-time data request requirements of the target business based on the business node status set, and obtain the real-time data request data capacity of the target business; Based on the actual target business data request demand analysis data, the real-time access status of the database is judged and analyzed; by obtaining the maximum number of request packets that the database responds to at a single moment, the maximum request packet capacity of the database response at a single moment is calculated; Determine the access status of the database at the current moment by comparing the single-moment target business data request demand analysis data with the database's single-moment maximum response request data packet capacity; the access status includes busy and idle; When the access status of the database is busy, cache nodes are constructed for each business node of the target service; based on the real-time business data request demand data of each business node, capacity prediction analysis is performed on each cache node to obtain the real-time capacity data of the cache nodes corresponding to each business node; by dividing the observation period T1, the capacity prediction analysis of each cache node within the observation period is coordinated, and the maximum value is taken as the memory capacity value of the cache node within the period T1 corresponding to the business node; according to the request demand data within the period T1 corresponding to the business node, type division and coordination are performed to respectively determine the quantity of each type of request data within the period T1, and the proportion of the request data volume of each type of request data within the period T1 is analyzed; according to the analysis results, a proportion threshold is introduced, and the proportion of the request data volume of each type of request data is compared with the proportion threshold to determine high-request type data and low-request type data; according to the comparison analysis data of each type of request data in the corresponding business node, the cache space of the cache node is divided into a primary cache space and a secondary cache space; wherein the primary cache space is used to store high-request type data; the secondary cache space is used to store low-request type data.

4. The data analysis method based on the distributed cache technology according to claim 3, wherein: Based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each business node of the target service, a cache node update period T2 is set, and the call situation of each business node for the data stored in the cache node within the period is analyzed to analyze the utilization degree of the primary cache space and the secondary cache space in real time; It analyzes the data call hit rates of the data stored in the primary cache space and the secondary cache space respectively; Based on the analysis data of the call situation of each business node for the data stored in the cache node within the cache node update period T2, dynamic regulation analysis is performed on the memory of the cache node stored by each business node; it respectively analyzes the memory occupation of the primary storage space and the secondary storage space of the corresponding cache node within the period T2 based on the data call hit rates of the primary storage space and the secondary storage space in the corresponding cache node; by analyzing the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space in the cache node of each business node, and setting a cache prompt capacity Cv, the memory occupation of the primary storage space and the secondary storage space in each cache node at consecutive time points within the period is updated and judged, and a memory capacity regulation strategy for the corresponding cache node is generated according to the judgment results.

5. The data analysis method based on the distributed cache technology according to claim 4, wherein: The status data of the business nodes of the target service and the status data of the corresponding cache nodes are fed back through a visualization window; the status data of the cache node includes the capacity data of the cache node and the data retrieval records of the responding business nodes; The generated memory capacity regulation strategies of the corresponding cache nodes are output to the management port to execute the memory capacity regulation strategies of the corresponding cache nodes.

6. A data analysis system based on distributed caching technology, characterized by: The system includes a node positioning module, a node analysis module, a dynamic regulation module, and a data feedback module; The node positioning module determines the target service data, locates the service nodes based on the target service data, and obtains the service node status data; the node analysis module analyzes the degree of data request requirements of the service nodes in real time based on the status data of each service node, and judges the database access status according to the analysis of the data request requirements of the service nodes; according to the judgment result of the database access status, data cache nodes are constructed for each service node, and the cache memory allocation is analyzed based on the data request requirement data of the corresponding service node; the dynamic regulation module analyzes the memory utilization degree of the real-time cache nodes according to the data reading status of the cache nodes in each service node cycle, and performs periodic memory dynamic regulation on the cache nodes according to the memory utilization degree of the real-time cache nodes; a regulation instruction is generated according to the periodic memory dynamic regulation data of the cache nodes and fed back to the management port; The data feedback module uses a visualization window to output the service node status data and the status data of the corresponding cache nodes and executes the regulation instruction.

7. A data analysis system based on distributed cache technology according to claim 6, characterized in that: The node positioning module includes a target service determination unit and a service node positioning unit; The target service determination unit determines the target service data by retrieving the service data through the management port; the service data includes service name data and service node data; the service node data is the service node number; The service node positioning unit locates the service nodes through the target service node data, and obtains the status data of the corresponding service nodes by obtaining the corresponding service node request data; the status data of the service nodes includes the number of request data packets and the capacity of the request data packets of the service nodes.

8. The data analysis system based on distributed cache technology according to claim 7, characterized in that: The node analysis module includes a service node requirement analysis unit and a cache node construction unit; The service node requirement analysis unit constructs a corresponding service node status set based on the status data of each service node of the target service; According to the service node status set, the real-time data request requirements of the target service are analyzed to obtain the real-time data request data capacity of the target service; Based on the analysis data of the actual target service data request requirements, the real-time access status of the database is judged and analyzed; by obtaining the number of request data packets with the maximum response at a single moment of the database, the capacity of the request data packets with the maximum response at a single moment of the database is calculated; By comparing the analysis data of the target service data request requirements at a single moment with the capacity of the request data packets with the maximum response at a single moment of the database, the access status of the database at the current moment is determined; the access status includes busy and idle; When the access status of the database is busy, the cache node construction unit constructs cache nodes for each service node of the target service; based on the real-time service data request demand data of each service node, it performs capacity prediction analysis on each cache node to obtain the real-time capacity data of the cache node corresponding to each service node; by dividing the observation period T1, it coordinates the capacity prediction analysis of each cache node within the observation period, and takes the maximum value as the memory capacity value of the cache node of the corresponding service node within the period T1; according to the request demand data of the corresponding service node within the period T1, it conducts type division and coordination, respectively determines the quantity of each type of request data within the period T1, and analyzes the proportion of the request data volume of each type of request data within the period T1; according to the analysis results, it introduces a proportion threshold, compares the proportion of the request data volume of each type of request data with the proportion threshold to determine high-request type data and low-request type data; according to the comparison analysis data of each type of request data in the corresponding service node, it divides the cache space of the cache node into a primary cache space and a secondary cache space; where the primary cache space is used to store high-request type data; the secondary cache space is used to store low-request type data.

9. The data analysis system based on distributed cache technology according to claim 8, characterized in that: The dynamic regulation module includes a cache node memory analysis unit and a cache node memory regulation unit; Based on the capacity prediction analysis and data storage division results of the cache nodes corresponding to each service node of the target service, the cache node memory analysis unit sets the cache node update period T2, analyzes the call situation of each service node to the data stored in the cache node within the period, and real-time analyzes the utilization degree of the primary cache space and the secondary cache space; It analyzes the data call hit rates of the data stored in the primary cache space and the secondary cache space respectively; Based on the analysis data of the call situation of each service node to the data stored in the cache node within the cache node update period T2, the cache node memory regulation unit conducts dynamic regulation analysis on the memory of the cache node stored by each service node; it respectively analyzes the memory occupancy of the primary storage space and the secondary storage space of the corresponding cache node within the period T2 based on the data call hit rates of the primary storage space and the secondary storage space in the corresponding cache node; by analyzing the memory update capacity Cg1 of the primary storage space and the memory update capacity Cg2 of the secondary storage space of the corresponding cache node of each service node, and setting the cache prompt capacity Cv, it updates and judges the memory occupancy of the primary storage space and the secondary storage space of each cache node at consecutive time points within the period, and generates a memory capacity regulation strategy for the corresponding cache node according to the judgment results.

10. A data analysis system based on distributed cache technology according to claim 9, characterized in that: The data feedback module includes a visualization unit and a policy execution unit; The visualization unit feeds back the service node status data of the target service and the status data of the corresponding cache node through a visualization window; the status data of the cache node includes the capacity data of the cache node and the data retrieval records of the responding service nodes; The policy execution unit outputs the memory capacity regulation strategies generated for each corresponding cache node to the management port and executes the memory capacity regulation strategies of each corresponding cache node.

Images