A distributed cache method based on hash algorithm

Abstract

The present invention relates to the field of distributed cache technology, and in particular to a distributed cache method based on a hash algorithm. The method comprises the following steps: obtaining data to be cached; performing consistent hash calculation on the data to be cached to obtain keyword hash value data; performing hash ring node allocation on the keyword hash value data to generate corresponding storage node data; performing high-voltage simulation on the corresponding storage node data based on preset time period data to generate node failure rate data; performing box plotting on the node failure rate data to generate a node failure box plot; performing abnormal analysis on the node failure box plot to generate isolated failure node abnormal data; comprehensively using consistent hash allocation, node failure rate prediction, node abnormality detection and processing, node optimization and reconstruction and other technologies to form a complete set of node management and optimization solutions, thereby improving the stability, availability and performance of the distributed storage system.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed cache, and in particular to a distributed cache method based on a hash algorithm. Background Art

[0002] The earliest distributed cache method uses consistent hashing algorithm to achieve data distribution and load balancing. This method maps cache nodes and data to a hash ring, calculates the hash value of the data through a hash function, and maps it to a specific node on the ring. This method is simple and effective, but there is a problem of data migration when nodes are dynamically added and removed, which may lead to a large amount of cache failure and data migration overhead. In order to solve the data migration problem when nodes are dynamically expanded and reduced, scientists introduced the concept of virtual nodes, mapping a physical node to multiple virtual nodes, thereby reducing the scope and impact of data migration. In order to solve the data migration problem of consistent hashing algorithm when nodes change dynamically, hybrid consistent hashing algorithm was introduced. This algorithm distributes data to multiple consistent hashing rings, each ring corresponds to the state of a cache node. When the node changes, only part of the data needs to be migrated, thereby reducing the overall data migration overhead. Although consistent hashing algorithm is widely used in distributed cache, it still has some limitations. For example, the problem of unbalanced load of nodes. When the processing capacity of nodes is inconsistent, some nodes will be overloaded. In addition, in the case of node failure or network partition, data loss or inconsistency may occur. In existing methods, the analysis of node failures is usually limited to simple fault diagnosis, lacking a comprehensive analysis of the causes and impacts of failures, and unable to identify potential node failures in advance, resulting in the system being unable to respond in a timely manner when a failure occurs. Summary of the invention

[0003] Based on this, it is necessary to provide a distributed caching method based on a hash algorithm to solve at least one of the above technical problems.

[0004] To achieve the above object, a distributed caching method based on a hash algorithm is provided, the method comprising the following steps:

[0005] Step S1: Obtain data to be cached; perform consistent hash calculation on the data to be cached to obtain keyword hash value data; perform hash ring node allocation on the keyword hash value data to generate corresponding storage node data;

[0006] Step S2: Based on the preset time period data, high voltage simulation is performed on the corresponding storage node data to generate node failure rate data; a box plot is drawn on the node failure rate data to generate a node failure box plot;

[0007] Step S3: Perform abnormal analysis on the node fault box line diagram to generate isolated fault node abnormal data; when the isolated fault node abnormal data is dangerous abnormal node data, perform abnormal node isolation on the dangerous abnormal node data to obtain updated storage node data; when the isolated fault node abnormal data is ordinary abnormal node data, perform fault prevention on the ordinary abnormal node data to obtain preventive storage node data;

[0008] Step S4: remove identical nodes from update storage node data and preventive storage node data to generate differential node data; replace corresponding storage node data based on differential node data to generate optimized storage node data; perform dependency reconstruction on optimized storage node data to generate system storage node data.

[0009] The present invention can evenly distribute the data to be cached to different storage nodes through consistent hash calculation and node allocation, so as to realize load balancing and distributed storage. By performing high-voltage simulation on the storage node data, the failure of the node in a preset time period can be simulated, and the node failure rate data can be obtained, which is helpful to evaluate the reliability and stability of the system. By drawing a box-line diagram of the node failure rate data, the distribution and abnormality of the node failure can be intuitively understood. By performing abnormal analysis on the node failure box-line diagram, the abnormal data of the isolated fault node can be identified, which is helpful for further processing and optimization. According to the type of the abnormal data of the isolated fault node (dangerous abnormal node or ordinary abnormal node), different measures can be taken for processing. The dangerous abnormal node data can be isolated to prevent it from causing serious impact on the system. The ordinary abnormal node data can be prevented from fault prevention, and corresponding measures can be taken to avoid the occurrence of potential faults. By performing node replacement on the difference node data, the storage node data can be updated and the performance and reliability of the system can be optimized. At the same time, the dependency reconstruction can ensure the consistency and integrity of the system storage node data. By analyzing and processing the abnormal data, the configuration and performance of the storage node are optimized, and the reliability and stability of the system are improved. It helps to identify and handle isolated faulty nodes and take appropriate measures for fault isolation and prevention, thereby improving the overall efficiency and availability of the system.

[0010] The invention is beneficial in that the data to be cached is mapped to keyword hash value data through consistent hash calculation. The characteristics of consistent hashing ensure relatively uniform data distribution, avoid the problem of uneven distribution of hot data on storage nodes, and help improve the load balancing of the system. Mapping the keyword hash value data to the nodes on the hash ring realizes the allocation of storage nodes, so that adjacent nodes are close to each other on the hash ring, which helps to achieve data locality, improve cache hit rate, and reduce network delay during data access. According to the allocation of hash ring nodes, corresponding storage node data is generated, which helps to establish a mapping relationship between data and nodes, and provides basic data for subsequent fault simulation, analysis and processing. High-voltage simulation can simulate the working state of nodes under different load conditions, thereby generating node failure rate data, so that the system can predict possible failures of nodes, and can take corresponding preventive measures before the failure occurs. Through abnormal analysis, the system can perform more in-depth diagnosis and analysis of node failures. Performing abnormal analysis on isolated fault nodes helps to distinguish dangerous abnormal nodes from ordinary abnormal nodes, so as to adopt different processing strategies. Abnormal node isolation, fault prevention, and node replacement operations are intelligent decisions made based on the system's comprehensive analysis and prediction of node failures, which can reduce the manual intervention of system maintenance personnel and improve the system's automation and operating efficiency. Node replacement and reconstruction, the system can replace faulty nodes with optimized nodes and rebuild the system storage node data, thereby improving the stability and performance of the system. The system can detect, diagnose, and handle node failures in a timely manner, thereby improving the system's reliability and fault tolerance and reducing the impact of system failures on users and businesses. The system uses a comprehensive set of technologies such as consistent hashing allocation, node failure rate prediction, node anomaly detection and processing, node optimization and reconstruction, and forms a complete set of node management and optimization solutions to improve the stability, availability, and performance of distributed storage systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of the steps of a distributed caching method based on a hash algorithm;

[0012] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0013] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0014] Figure 4 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0015] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0016] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0017] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0018] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0019] To achieve this, please refer to Figures 1 to 4 , a distributed caching method based on a hash algorithm, comprising the following steps:

[0020] Step S1: Obtain data to be cached; perform consistent hash calculation on the data to be cached to obtain keyword hash value data; perform hash ring node allocation on the keyword hash value data to generate corresponding storage node data;

[0021] Step S2: Based on the preset time period data, high voltage simulation is performed on the corresponding storage node data to generate node failure rate data; a box plot is drawn on the node failure rate data to generate a node failure box plot;

[0022] Step S3: Perform abnormal analysis on the node fault box line diagram to generate isolated fault node abnormal data; when the isolated fault node abnormal data is dangerous abnormal node data, perform abnormal node isolation on the dangerous abnormal node data to obtain updated storage node data; when the isolated fault node abnormal data is ordinary abnormal node data, perform fault prevention on the ordinary abnormal node data to obtain preventive storage node data;

[0023] Step S4: remove identical nodes from update storage node data and preventive storage node data to generate differential node data; replace corresponding storage node data based on differential node data to generate optimized storage node data; perform dependency reconstruction on optimized storage node data to generate system storage node data.

[0024] The invention is beneficial in that the data to be cached is mapped to keyword hash value data through consistent hash calculation. The characteristics of consistent hashing ensure relatively uniform data distribution, avoid the problem of uneven distribution of hot data on storage nodes, and help improve the load balancing of the system. Mapping the keyword hash value data to the nodes on the hash ring realizes the allocation of storage nodes, so that adjacent nodes are close to each other on the hash ring, which helps to achieve data locality, improve cache hit rate, and reduce network delay during data access. According to the allocation of hash ring nodes, corresponding storage node data is generated, which helps to establish a mapping relationship between data and nodes, and provides basic data for subsequent fault simulation, analysis and processing. High-voltage simulation can simulate the working state of nodes under different load conditions, thereby generating node failure rate data, so that the system can predict possible failures of nodes, and can take corresponding preventive measures before the failure occurs. Through abnormal analysis, the system can perform more in-depth diagnosis and analysis of node failures. Performing abnormal analysis on isolated fault nodes helps to distinguish dangerous abnormal nodes from ordinary abnormal nodes, so as to adopt different processing strategies. Abnormal node isolation, fault prevention, and node replacement operations are intelligent decisions made based on the system's comprehensive analysis and prediction of node failures, which can reduce the manual intervention of system maintenance personnel and improve the system's automation and operating efficiency. Node replacement and reconstruction, the system can replace faulty nodes with optimized nodes and rebuild the system storage node data, thereby improving the stability and performance of the system. The system can detect, diagnose, and handle node failures in a timely manner, thereby improving the system's reliability and fault tolerance and reducing the impact of system failures on users and businesses. The system uses a comprehensive set of technologies such as consistent hashing allocation, node failure rate prediction, node anomaly detection and processing, node optimization and reconstruction, and forms a complete set of node management and optimization solutions to improve the stability, availability, and performance of distributed storage systems.

[0025] In the embodiment of the present invention, reference Figure 1The above is a schematic flow chart of the steps of a distributed caching method based on a hash algorithm of the present invention. In this example, the distributed caching method based on a hash algorithm includes the following steps:

[0026] Step S1: Obtain data to be cached; perform consistent hash calculation on the data to be cached to obtain keyword hash value data; perform hash ring node allocation on the keyword hash value data to generate corresponding storage node data;

[0027] In an embodiment of the present invention, data to be cached is obtained, including any content that needs to be cached, such as web page content, database query results, and files. For each data to be cached, the system will perform a consistent hash calculation, and the calculation process includes: selecting a suitable hash function, such as MD5, SHA-1, etc., to convert the data to be cached into a hash value, and mapping the calculated hash value to a fixed range, such as the range of 0 to 2^32-1. Each hash value on the hash ring is associated with a storage node to determine which node each data should be cached on, and the name or IP address of the node is hashed by a hash function to obtain the identifier of the storage node. The identifier of the storage node is mapped to the hash ring and compared with the hash value of the data to be cached to determine which node the data should be stored on. According to the result of the hash ring node allocation, the data to be cached is stored on the corresponding node.

[0028] Step S2: Based on the preset time period data, high voltage simulation is performed on the corresponding storage node data to generate node failure rate data; a box plot is drawn on the node failure rate data to generate a node failure box plot;

[0029] In the embodiment of the present invention, a time period is defined according to actual needs, covering a sufficiently long period of system operation. During the preset time period, high-voltage simulation is performed on the corresponding storage node data, for example, simulating a large number of concurrent accesses, sudden load increases, and network anomalies. During the high-voltage simulation, the number or time of failures of each storage node is recorded, and the node failure rate can be calculated by the following formula: Distribution by time period to get a more detailed understanding of the failure rate of the system at different time points. Using the collected node failure rate data, draw a box plot to show the distribution and statistical information of the node failure rate. Box plots usually include the following elements: median, which represents the middle value of the failure rate, upper and lower quartiles, which show the distribution of the failure rate, and outliers, which represent data points that are out of a certain range, indicating that the system has an abnormality during a specific period of time. By using different colors or marks to distinguish each node, draw the box plot according to each storage node to compare the failure rate distribution of different nodes.

[0030] Step S3: Perform abnormal analysis on the node fault box line diagram to generate isolated fault node abnormal data; when the isolated fault node abnormal data is dangerous abnormal node data, perform abnormal node isolation on the dangerous abnormal node data to obtain updated storage node data; when the isolated fault node abnormal data is ordinary abnormal node data, perform fault prevention on the ordinary abnormal node data to obtain preventive storage node data;

[0031] In an embodiment of the present invention, an abnormal analysis is performed on a node failure box plot, including identifying abnormal nodes in the box plot, that is, nodes whose failure rate obviously deviates from the normal range. Abnormal nodes can be divided into two types: dangerous abnormal nodes and ordinary abnormal nodes. For the identified abnormal nodes, the fault data and the time period when the fault occurs are recorded. For the identified abnormal nodes, further judgment is performed to determine whether they belong to dangerous abnormal nodes or ordinary abnormal nodes. Dangerous abnormal nodes include nodes with abnormally high and abnormally prominent failure rates, which have a serious impact on system stability and reliability. Ordinary abnormal nodes are relatively minor, but measures are still taken to prevent them. If the abnormal node is determined to be a dangerous abnormal node, that is, its failure rate poses a major threat to system security, it is immediately isolated, and the node is temporarily removed from the system or other measures are taken to reduce its impact on the overall system. After isolating the abnormal node, the data of the storage node needs to be updated to ensure that the system can continue to operate normally, including reallocating data or adjusting system configuration operations; if the abnormal node is determined to be an ordinary abnormal node, preventive measures are taken to reduce the possibility of its failure, including increasing redundancy, optimizing node configuration, and improving monitoring and early warning mechanisms.

[0032] Step S4: remove identical nodes from update storage node data and preventive storage node data to generate differential node data; replace corresponding storage node data based on differential node data to generate optimized storage node data; perform dependency reconstruction on optimized storage node data to generate system storage node data.

[0033] In an embodiment of the present invention, the updated storage node data and the preventive storage node data are compared to find the same nodes, including the processed abnormal nodes, which need to be eliminated to avoid repeated processing. After eliminating the same nodes, the difference node data of the updated storage node data and the preventive storage node data is obtained, including the newly added nodes in the updated storage node data and the nodes removed in the preventive storage node data. Based on the difference node data, the corresponding storage node data is replaced with nodes, including adding the newly added nodes in the updated storage node data to the system and removing the nodes removed in the preventive storage node data. After completing the node replacement, the data obtained is the optimized storage node data, which reflects the latest node configuration in the system, including the newly added nodes and the deleted nodes. For the optimized storage node data, a dependency reconstruction operation is performed, including reconfiguring the dependency relationship between nodes, redistributing data, and updating the network topology. After completing the dependency reconstruction, the data obtained by the final configuration after node replacement and dependency reconstruction is the system storage node data.

[0034] Preferably, step S1 comprises the following steps:

[0035] Step S11: Obtain data to be cached;

[0036] Step S12: Perform keyword mining on the cached data to obtain cached item keyword data;

[0037] Step S13: performing consistent hash calculation on the cache item keyword data to obtain keyword hash value data;

[0038] Step S14: Perform hash ring node allocation on the keyword hash value data to generate corresponding storage node data.

[0039] The present invention ensures the timely acquisition of data to be cached, and extracts the keyword data of cache items through keyword mining, which helps the system to better understand and organize the data to be cached, and lays a foundation for subsequent processing and storage. Consistent hash calculation can effectively map keywords to hash value space, ensure the uniform distribution of cache items, reduce the occurrence of hot data, and improve the system's processing ability for load balancing. According to the keyword hash value data, the cache items are allocated to the hash ring nodes, and the distributed storage and management of data are realized, which has certain fault tolerance. When a node fails or a new node is added, the data balance can be achieved by simple redistribution. The hash ring node allocation method can make the system easier to expand. When it is necessary to increase storage nodes, it is only necessary to adjust the distribution of the hash ring without large-scale migration or modification of the entire system, thereby reducing the maintenance cost and complexity of the system. The cache system can be made more intelligent and efficient. Keyword mining and consistent hash calculation can better utilize the characteristics and distribution of data, so that the cache items can be more accurately located and hit, thereby improving the hit rate of the cache system and accelerating the data access speed. It optimizes the data processing flow of the cache system, improves the performance, scalability and intelligence of the system, and thus better meets the needs for efficient data management and access in actual application scenarios.

[0040] In an embodiment of the present invention, the data to be cached is obtained through a suitable data source (such as a database, a network request, a file system), and the cached data is subjected to text processing, word segmentation or other forms of data preprocessing to extract keywords. Use a suitable keyword extraction algorithm (such as TF-IDF, word frequency statistics, part-of-speech tagging) to extract keywords from the data. Use a consistent hashing algorithm (such as MurmurHash, SHA-1) to perform hash calculations on each keyword, and map the value obtained by the hash calculation to a fixed hash value space to ensure that the distribution of the hash value is uniform and sufficiently random, and to ensure that the same keyword is always mapped to the same hash value to ensure data consistency. Use a circular array or other data structure to build a hash ring structure. Map the hash value to the node position on the hash ring, determine the storage node for each keyword, ensure that the hash value range between adjacent nodes is as uniform as possible, so as to improve the distribution and load balancing of the data, and allocate the cache item to the corresponding storage node according to the hash value of the node, and generate the corresponding storage node data structure.

[0041] Preferably, step S2 comprises the following steps:

[0042] Step S21: recording the data throughput of the corresponding storage node data based on the preset time period data to obtain the access frequency data of each node;

[0043] Step S22: performing high voltage simulation on the corresponding storage node data through the access frequency data of each node to generate node failure rate data;

[0044] Step S23: performing five-number summarization on the node failure rate data to obtain node failure rate summary data;

[0045] Step S24: Draw a box plot for the node failure rate summary data to generate a node failure box plot.

[0046] By recording the throughput of storage node data, the present invention can understand the access frequency of each node, which is helpful to evaluate the usage and load of the node and provide a basis for subsequent high-voltage simulation. By performing high-voltage simulation on the storage node data, the failure of the node within a preset time period can be simulated, and the node failure rate data can be obtained, which is helpful to evaluate the reliability and stability of the node. By summarizing the node failure rate data in five numbers (such as minimum value, first quartile, median, third quartile, maximum value), the summary statistics of the node failure rate can be obtained. The distribution and abnormal conditions of the node failure rate can be more comprehensively understood. By drawing a box-and-whisker plot of the node failure rate summary data, the distribution of the node failure rate can be intuitively displayed. The box-and-whisker plot can display statistical indicators such as outliers, medians, upper and lower quartiles, etc., which is helpful to discover the abnormal conditions and trends of node failures. By recording the data throughput of the storage node, simulating the node failure condition, and summarizing and visually analyzing the node failure rate, the usage, reliability and stability of the node can be more comprehensively understood, which helps the system administrator or operation and maintenance team to promptly discover and solve the node failure problem and improve the availability and performance of the system.

[0047] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0048] Step S21: recording the data throughput of the corresponding storage node data based on the preset time period data to obtain the access frequency data of each node;

[0049] In an embodiment of the present invention, a time period for recording data throughput is determined, such as one day, one week, or one month, and a range of storage nodes to be recorded is determined. Appropriate tools or software are used to record the data throughput of storage nodes. Monitoring tools are configured to ensure that they can capture the required data, including setting the monitoring frequency, selecting the storage nodes to be monitored, and defining indicators of data throughput. Data recording is started within a preset time period, ensuring that the monitoring tool operates normally and records data within the time period. After the preset time period ends, the recorded data is collected, including exporting the data to a file, storing it in a database, or obtaining the data through an API. The collected data is analyzed using a data analysis tool or script to obtain access frequency data of each node, including calculating data throughput, generating statistical reports, or visualizing data.

[0050] Step S22: performing high voltage simulation on the corresponding storage node data through the access frequency data of each node to generate node failure rate data;

[0051] In an embodiment of the present invention, the access frequency data of each node is used as the basis for simulation, an algorithm or model for simulating node failure rate is determined, and the selected simulation algorithm is used to simulate and generate node failure rate data based on the access frequency data of each node, and this is achieved by simulating the frequency and probability of node failure events. Configure the parameters of the simulation environment and the simulation algorithm to ensure that the simulation results can accurately reflect the actual situation. Run the simulation process of the node failure rate. During the simulation process, monitor the simulation environment and record the simulation results. Collect the generated node failure rate data, including recording the time, type, and duration information of the node failure. Use data analysis tools or scripts to analyze the simulation results, including evaluating the distribution of node failure rates and identifying potential failure modes.

[0052] Step S23: performing five-number summarization on the node failure rate data to obtain node failure rate summary data;

[0053] In the embodiment of the present invention, the node failure rate data are arranged in ascending order, the minimum value in the data set is calculated, that is, the minimum failure rate value in the data set, the maximum value in the data set is calculated, that is, the maximum failure rate value in the data set, the median is calculated as the middle value of the data set, that is, the value located in the middle position after the data set is arranged by size, the first quartile (Q1) is calculated as the value at the 25% position in the data set, that is, the maximum value located in the first part after the data set is divided into four equal parts, and the third quartile (Q3) is calculated as the value at the 75% position in the data set, that is, the maximum value located in the third part after the data set is divided into four equal parts. The calculated minimum value, Q1, median, Q3 and maximum value are sorted into a data set, which is the five-number summary data of the node failure rate. The five-number summary data is analyzed to understand the distribution and central trend of the failure rate data.

[0054] Step S24: Draw a box plot for the node failure rate summary data to generate a node failure box plot.

[0055] In an embodiment of the present invention, summary data of node failure rate is collected, including minimum value, first quartile (Q1), median, third quartile (Q3) and maximum value. Use drawing tools (such as Matplotlib in Python, ggplot2 in R language) or spreadsheet software (such as Excel) to draw box plots, and create an appropriate drawing coordinate system to accommodate the display of node failure rate data. Draw a box in the coordinate system, whose upper and lower boundaries correspond to the upper quartile (Q3) and lower quartile (Q1) of the data set, respectively, and the line segment in the middle of the box represents the median. Draw two "tentacles" at the upper and lower boundaries of the box, extending to the minimum and maximum values ​​of the data set respectively. Display outliers outside the box in the form of dots or crosses. Add titles, coordinate axis labels, and legends, analyze the generated node failure box plots, and understand the distribution of failure rate data and outliers.

[0056] Preferably, step S22 includes the following steps:

[0057] Step S221: Calculate the frequency fluctuation of the access frequency data of each node to obtain access frequency fluctuation data;

[0058] Step S222: performing low-fluctuation filtering on the access frequency data of each node based on the access frequency fluctuation data to generate access frequency data with large fluctuations;

[0059] Step S223: using the access frequency data with large fluctuations to perform fatigue access simulation on the corresponding storage node data to obtain node access status data, wherein the node access status data includes node fault data and node normal data;

[0060] Step S224: Calculate the failure rate of the node failure data based on the node normal data to generate node failure rate data.

[0061] The present invention can identify nodes with large access frequency fluctuations by calculating the frequency fluctuations of the access frequency data of each node, which is helpful to find nodes with abnormal access frequencies. Low fluctuation filtering is performed based on the access frequency fluctuation data to generate access frequency data with large fluctuations, which can reduce noise and interference in the access frequency data and improve data quality. Fatigue access simulation is performed on the corresponding storage node data using the access frequency data with large fluctuations to obtain node access situation data, including node failure data and node normal data. It is helpful to simulate the access situation in a real environment and find the potential failure situation of the storage node. The failure rate calculation of the node failure data based on the node normal data can quantitatively evaluate the failure risk of the node in the storage system, which is helpful to take preventive or repair measures in time and improve the stability and reliability of the system. Preventive maintenance of the storage system can be achieved, potential problems can be discovered in time and corresponding measures can be taken to reduce the risk of system failures and improve the availability and stability of the system. It is helpful to improve the management and maintenance efficiency of the storage system, reduce the impact of system failures on the business, and thus improve the reliability and performance of the overall system.

[0062] In the embodiment of the present invention, the access frequency data of each node is collected, and the access frequency data is fluctuated using a suitable algorithm or method (such as standard deviation, variance) to measure the change of the access frequency. The calculated access frequency fluctuation data is stored or recorded. The node data with large access frequency fluctuation is screened out. The node data with small access frequency fluctuation is filtered out, and only the access frequency data with large fluctuation is retained. The filtered access frequency data with large fluctuation is stored or recorded for use in subsequent steps. The access behavior of the storage node is simulated using the access frequency data with large fluctuation. During the simulation process, the load of the node, the change of access frequency, and the durability factor of the storage medium are simulated. According to the simulation results, the access situation data of the node is recorded, including the normal access situation and the possible failure situation. According to the node access situation data, the normal node data and the node failure data are distinguished, the normal node data is statistically analyzed, the node failure rate is calculated, and the calculated node failure rate data is stored or recorded for use in system management and maintenance.

[0063] Preferably, step S3 comprises the following steps:

[0064] Step S31: isolating outliers on the node fault box plot to obtain isolated fault node data;

[0065] Step S32: performing abnormal analysis on the isolated fault node data to generate isolated fault node abnormal data;

[0066] Step S33: when the isolated fault node abnormal data is dangerous abnormal node data, the dangerous abnormal node data is isolated from the abnormal nodes to obtain updated storage node data;

[0067] Step S34: When the isolated fault node abnormal data is common abnormal node data, fault prevention is performed on the common abnormal node data to obtain preventive storage node data.

[0068] The present invention uses a box plot to isolate outliers of node failures, which helps to identify abnormal nodes in the storage system. By identifying abnormal nodes, timely and targeted measures can be taken to prevent the failure from further spreading or affecting the normal operation of other nodes. The isolated fault node data is analyzed for abnormality, and abnormal data of the isolated fault node is generated. It helps to gain an in-depth understanding of the specific causes and characteristics of the failure, and provides data support and basis for subsequent processing. The abnormal nodes are isolated for the identified dangerous abnormal node data to avoid a greater impact on the entire system. Isolating dangerous nodes and updating storage node data helps to ensure the stability and reliability of the system. Fault prevention for ordinary abnormal node data means that the system can take preventive measures to reduce or eliminate potential failure risks, thereby improving the overall performance and availability of the storage system. It helps to timely identify, process and prevent node failures in the storage system, improve the stability, reliability and availability of the system, and ensure the security and integrity of data.

[0069] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0070] Step S31: isolating outliers on the node fault box plot to obtain isolated fault node data;

[0071] In an embodiment of the present invention, a box plot is analyzed to identify outliers in the box plot. An outlier refers to a node that is significantly different from other node data, specifically an abnormal situation caused by a fault, abnormal operation or other reasons. Based on the identification result of the outlier, the node data identified as an outlier is extracted. These data are isolated fault node data, representing nodes with potential faults in the storage system. The isolated fault node data is recorded and a corresponding report is generated. The report should include detailed information of the identified outlier nodes, such as node identification, description of the abnormal situation, and possible causes.

[0072] Step S32: performing abnormal analysis on the isolated fault node data to generate isolated fault node abnormal data;

[0073] In an embodiment of the present invention, the data of isolated fault nodes is ensured to be complete and accurate, and feature analysis is performed on the data of isolated fault nodes, including but not limited to performance indicators, log files, and error codes. The changing trend of the isolated fault node data over time is analyzed, and any unusual patterns or mutations are identified by drawing a time series graph and calculating a moving average. The relationship between the isolated fault node data and other nodes or system events is studied. Through correlation analysis, the potential factors or triggering conditions behind the fault can be identified, and statistical models or machine learning algorithms can be applied to identify and confirm anomalies. Combined with the analysis results, the potential root causes of the isolated fault nodes are explored in depth.

[0074] Step S33: when the isolated fault node abnormal data is dangerous abnormal node data, the dangerous abnormal node data is isolated from the abnormal nodes to obtain updated storage node data;

[0075] In an embodiment of the present invention, when the abnormal data of the isolated fault node is the data of the dangerous abnormal node, an isolation strategy for the dangerous abnormal node is formulated to ensure that other parts of the system are not affected by it. The isolation strategy includes temporary disabling, restricting its access rights, and reducing its priority. According to the formulated isolation strategy, the dangerous abnormal node is isolated, including configuration management system, network isolation, and traffic redirection technical means. After the isolation operation is performed, the operating status and performance of the system are closely monitored to ensure that the isolation effect meets expectations. When the dangerous abnormal node is successfully isolated, the storage node data needs to be updated to reflect the results of the isolation operation. Including updating system configuration files, database records, and network routing tables. The isolated system is verified, and functional testing, performance testing, and security audit methods are used to ensure that the dangerous abnormal node has been effectively isolated and no longer poses a threat to the system.

[0076] Step S34: When the isolated fault node abnormal data is common abnormal node data, fault prevention is performed on the common abnormal node data to obtain preventive storage node data.

[0077] In an embodiment of the present invention, when the abnormal data of the isolated fault node is ordinary abnormal node data, the abnormal node data is analyzed and diagnosed in detail to determine the cause and characteristics of the abnormality. Based on the analysis results of the abnormal node data, targeted preventive measures are formulated, including repairing software defects, optimizing system configuration, increasing redundant backup, and improving network security. According to the determined preventive measures, the system is modified and adjusted accordingly, including updating software patches, adjusting network settings, and strengthening access control operations. After implementing the preventive measures, the system is tested and verified to ensure the effectiveness and stability of the preventive measures, including using functional testing, stress testing, and security vulnerability scanning methods to verify the overall performance and security of the system. Continuously monitor the system operation status to promptly discover and handle new abnormal situations.

[0078] Preferably, step S32 includes the following steps:

[0079] Step S321: highlighting the characteristics of the isolated fault node data to obtain the characteristic data of the isolated fault node;

[0080] Step S322: capturing abnormal patterns of characteristic data of isolated fault nodes to generate abnormal pattern data of fault nodes;

[0081] Step S323: Analyze the abnormal causes of the abnormal pattern data of the faulty node and generate abnormal data of the isolated faulty node.

[0082] The present invention can highlight the key features or abnormal signals in the data by highlighting the features of the isolated fault node data. It helps to quickly locate the anomaly and provide useful clues and basis for subsequent processing. Based on the feature data, the abnormal pattern is captured and extracted, which helps to form an analyzable pattern for the abnormal situation of the isolated fault node. It makes the abnormal situation easier to understand and handle, and lays the foundation for further analysis. The abnormal pattern data is deeply analyzed to identify the root cause of the anomaly. It helps to discover potential problems or defects in the system and take corresponding measures to repair or prevent them to ensure the stability and reliability of the system. A set of systematic methods is provided to obtain useful information from the isolated fault node data, providing actionable clues and directions for solving system anomalies. Abnormal situations in the system can be identified, located and solved more efficiently, thereby improving the performance and reliability of the system.

[0083] In an embodiment of the present invention, data containing isolated fault nodes are collected, and feature points are extracted from the data, including mutations, abnormal peaks, and periodic changes in the data. The extracted feature points are highlighted or marked for subsequent analysis and processing. Feature data of isolated fault nodes for abnormal pattern capture are prepared, and an abnormal pattern recognition algorithm suitable for the problem is selected, such as based on statistical methods, machine learning algorithms, and time series analysis. The selected algorithm is used to capture abnormal patterns on the feature data to identify abnormal patterns or abnormal behaviors. The captured abnormal patterns are organized into abnormal pattern data for subsequent abnormal cause analysis. An abnormal cause analysis method suitable for the problem is selected, such as root cause analysis, statistical analysis, and expert system. An in-depth analysis of the abnormal pattern data is performed to explore the root cause of the abnormality. The results of the abnormal cause analysis are organized into isolated fault node abnormal data.

[0084] Preferably, step S33 includes the following steps:

[0085] Step S331: when the isolated fault node abnormal data is dangerous abnormal node data, the abnormal point of the corresponding storage node data is isolated based on the dangerous abnormal node data to generate abnormal isolation storage node data;

[0086] Step S332: backing up the abnormal isolation storage node data to obtain abnormal isolation storage node backup data;

[0087] Step S333: Perform node construction on the backup data of the abnormal isolated storage node to generate replacement storage node data;

[0088] Step S334: Replace the abnormal isolated storage node data by replacing the storage node data to generate updated storage node data.

[0089] When the present invention finds that the abnormal data of the isolated fault node is a dangerous abnormality, the abnormal point isolation of the corresponding storage node data is immediately performed, which can prevent the abnormal data from further affecting the normal operation of the system, thereby improving the fault tolerance of the system. By backing up the abnormal isolation storage node data and constructing the replacement node data, the system can quickly recover to a normal state after an abnormal situation occurs, reducing the loss caused by system shutdown or data loss due to failure. Automatic execution of the abnormal handling process can reduce the need for manual intervention, reduce the cost of system maintenance and management, and can respond to and handle abnormal situations more quickly. By backing up the abnormal isolation storage node data and constructing the replacement storage node data, the system can ensure the integrity and security of the data, and can restore the data in time even if an abnormal situation occurs. Handling the abnormal situation in time and restoring to the normal state can reduce the performance degradation or service interruption of the system caused by the abnormal situation, thereby improving the overall performance and user experience of the system. It helps to ensure that the system can quickly and effectively recover to a normal operating state when facing an abnormal situation, thereby ensuring the stability, reliability and security of the system.

[0090] In an embodiment of the present invention, when the abnormal data of the isolated fault node is dangerous abnormal node data, the corresponding storage node is determined based on the detected dangerous abnormal node data. The dangerous abnormal node data is isolated from the corresponding storage node to ensure that it no longer has a negative impact on the system. The isolated abnormal node data is saved in a new storage area to form abnormal isolation storage node data. A system backup tool or mechanism is used to perform a complete backup of the generated abnormal isolation storage node data. Using the backup data, a new node is established, which contains the backup data of the abnormal isolation storage node. Replacement storage node data is generated in the newly created node, and the data is consistent with the original storage node data. The newly generated replacement storage node data replaces the original storage node data to ensure that the replacement process is seamless. Update system records to ensure that the system has completed the replacement of the storage node and that the new storage node data has taken effect.

[0091] Preferably, step S34 includes the following steps:

[0092] Step S341: when the abnormal data of the isolated fault node is common abnormal node data, backing up the common abnormal node data to obtain common abnormal node backup data;

[0093] Step S342: performing hash sharding on the common abnormal node backup data based on the corresponding storage node data to obtain sharded common abnormal node backup data;

[0094] Step S343: dynamically and evenly distribute the backup data of the sharded common abnormal nodes to generate updated storage node data.

[0095] The present invention backs up the data of common abnormal nodes to ensure the security and integrity of the data. Even if an abnormal situation occurs, the system can restore the data in time to avoid data loss or damage. Hash sharding can divide the backup data into multiple fragments, making the storage and management of data more efficient. Through hash sharding, the data can be evenly distributed to different storage nodes, improving the parallel processing capability and load balancing of the system. Dynamic balanced allocation can dynamically adjust the distribution of data according to the real-time load situation of the system and the performance characteristics of the storage nodes, making the resource utilization of the system more balanced and avoiding single point failure or performance bottleneck. Common abnormal node data can be quickly and effectively identified and processed, thereby reducing the impact of failures on the system. Through measures such as backup, sharding and dynamic balanced allocation, the system can respond to abnormal situations more flexibly and improve the availability and stability of the system. It can be adjusted and optimized according to the needs and actual conditions of the system. At the same time, through sharding and dynamic balanced allocation, the system can better adapt to the growth of data volume and changes in nodes, and maintain the efficient operation of the system. A processing strategy that comprehensively considers data backup, sharding and dynamic balanced allocation is provided, which can effectively respond to the challenges brought by common abnormal node data and improve the stability, reliability and performance of the system.

[0096] In an embodiment of the present invention, when the abnormal data of the isolated fault node is ordinary abnormal node data, a backup process is started, including transferring the abnormal data from the fault node to the backup storage node and creating a copy on the backup storage node to ensure the reliability and redundant backup of the data. An appropriate hash function (MD5) is selected to shard the backup data. Using the selected hash function, the backed-up ordinary abnormal node data is sharded and the data is dispersed to different storage nodes. The load of each storage node is monitored in real time, including storage capacity utilization and network traffic. The movement strategy of the sharded ordinary abnormal node backup data is calculated based on the monitoring data to achieve dynamic balanced distribution, including transferring some sharded data from nodes with higher loads to nodes with lower loads to balance the load of the entire system. The sharded backup data is dynamically moved between storage nodes according to the calculated strategy to optimize the performance and resource utilization of the system.

[0097] Preferably, step S4 comprises the following steps:

[0098] Step S41: performing node full attribute comparison on the update storage node data and the prevention storage node data to obtain a node comparison data set;

[0099] Step S42: removing identical nodes from the update storage node data and the prevention storage node data through the node comparison data set to generate difference node data;

[0100] Step S43: performing node replacement on corresponding storage node data based on the difference node data to generate optimized storage node data;

[0101] Step S44: Dependency reconstruction is performed on the optimized storage node data to generate system storage node data.

[0102] The present invention ensures the consistency of updated data with original data by performing full attribute comparison on storage node data. It helps to prevent data loss, damage or inconsistency, and improves the integrity of system data. The updated data is subjected to node comparison data set to eliminate identical nodes, thereby generating difference node data. The cost of storing and transmitting difference data can be reduced, and storage space and network bandwidth can be saved. The corresponding storage node data is replaced according to the difference node data to generate optimized storage node data. It helps to optimize the data storage structure and improve the access efficiency and performance of data. The optimized storage node data is rebuilt based on dependencies to generate system storage node data. It helps to ensure the consistency and integrity of system data, and further improves the performance and stability of the system. It helps to ensure the consistency, integrity and performance of system data, and can effectively optimize the data storage and transmission process, reduce costs, and improve the overall efficiency of the system.

[0103] As an example of the present invention, refer to Figure 4As shown, in this example, step S4 includes:

[0104] Step S41: performing node full attribute comparison on the update storage node data and the prevention storage node data to obtain a node comparison data set;

[0105] In an embodiment of the present invention, update storage node data and prevention storage node data are collected, and all attributes are extracted from the data of each node. Including text, numbers, and date data. Node-level comparison is performed on the update storage node data and the prevention storage node data. All attributes of each node are compared one by one to determine whether they are the same or different. Based on the comparison results, a node comparison data set is generated. The data set records the comparison results of each node, indicating which nodes are the same and which nodes are different. The node comparison data set is output for use in subsequent steps, including writing the data to a database, file, or memory for access by other components or modules.

[0106] Step S42: removing identical nodes from the update storage node data and the prevention storage node data through the node comparison data set to generate difference node data;

[0107] In an embodiment of the present invention, a comparison result is obtained from a node comparison data set, which records the same or different relationship at the node level between the update storage node data and the prevention storage node data. The node comparison data set is traversed to check the comparison result of each node. For nodes marked as the same in the comparison data set, they are removed from the update storage node data and the prevention storage node data, which is achieved by marking the same nodes in the data set and excluding these nodes in subsequent processing. After removing the same nodes, the remaining nodes are difference nodes. The generated difference node data is output for use in subsequent steps, including writing the data to a database, a file or a memory for access by other components or modules.

[0108] Step S43: performing node replacement on corresponding storage node data based on the difference node data to generate optimized storage node data;

[0109] In an embodiment of the present invention, the node information that needs to be replaced is obtained from the difference node data. The corresponding storage node data initially generated is obtained from the system, that is, the data before the update or the original data of the backup. For each difference node, the corresponding node is found from the original storage node data, and replaced with the information in the difference node data. This includes updating the attribute value of the node and adding / deleting node operations, and the specific operation depends on the nature of the node difference. After the node replacement is completed, the optimized storage node data is generated, which includes the changes for replacing the difference node. The generated optimized storage node data is output for use in subsequent steps, including writing the data to a database, file or memory for access by other components or modules.

[0110] Step S44: Dependency reconstruction is performed on the optimized storage node data to generate system storage node data.

[0111] In an embodiment of the present invention, optimized storage node data is obtained, which is an updated version that includes replacement changes of difference node data. The dependencies between various data in the system are analyzed, including checking the relationship between data models and database tables, or analyzing the references and usage of data in the code. Based on the analysis of optimized storage node data and data dependencies, the dependencies of data in the system are rebuilt. Including updating other data objects that reference storage node data, or adjusting the data structure to reflect the new data state. After rebuilding the dependencies, data consistency is verified. This is achieved by performing data integrity checks, system function tests or other verification methods to ensure that the updated system data is consistent with expectations. After completing dependency reconstruction and data consistency verification, system storage node data is generated. After updating and optimization, it has the correct dependencies. The generated system storage node data is output for use by other components or modules of the system, and the data is written to a database, file or memory.

[0112] Preferably, step S44 includes the following steps:

[0113] Step S441: performing data flow analysis on corresponding storage node data to obtain node data flow data;

[0114] Step S442: performing dependency analysis on the node data flow data to generate node dependency relationship data;

[0115] Step S443: Based on the node dependency data, dependency reconstruction is performed on the optimized storage node data to generate system storage node data.

[0116] The present invention can understand the flow mode of data in the system by performing data flow analysis on storage node data. It is helpful to determine the generation, use and transmission path of data. By obtaining node data flow data, the system can more clearly understand the changes of data at various stages, so as to better support subsequent dependency analysis and dependency reconstruction. Dependency analysis on node data flow data helps to identify the relationship between storage node data. Including understanding which nodes depend on other nodes and the dependency relationship of data in the system. It helps to establish node dependency data, which provides a basis for subsequent dependency reconstruction. Dependency reconstruction is performed on optimized storage node data based on node dependency data, ensuring the consistency of the system after updating node data. By considering the dependency relationship, the system can update the associated data in the correct order to avoid destroying the dependency relationship, thereby ensuring the correctness and stability of the system. A systematic method is provided to handle data updates and ensure that after updating the storage node data, the system can adapt to these changes while maintaining the integrity and consistency of the data. Through data flow analysis, dependency analysis and dependency reconstruction, the system can more intelligently understand and manage data changes, thereby improving the maintainability, scalability and stability of the system.

[0117] In an embodiment of the present invention, determining the target of data flow analysis includes understanding the source, destination and change process of data in the system. Obtaining corresponding storage node data, including updated data or original data, depending on system requirements. Based on the storage node data, constructing a data flow graph to represent the flow path of data in the system. Identifying nodes where data is generated, transmitted and consumed. For each node, recording the changes in its attribute values. Determining the main flow path of data in the system, including identifying the main data change points and data transfer processes. Converting the results of data flow analysis into node data flow data, including node ID, data attributes and change information. Based on the node data flow data, constructing a dependency graph to represent the dependency relationship between storage nodes. Determining which nodes depend on other nodes. Identifying key dependencies, including determining the nodes where data is generated, which nodes are used, and which nodes have direct or indirect dependencies. Recording the dependencies of each storage node, including the dependent node ID, dependency type and other information. Converting the results of dependency analysis into node dependency data, including node ID and its dependency information. Obtaining dependency information from the acquired node dependency data. Traverse the node dependency data and update the storage node data in the correct dependency order. Make sure to update those nodes that have no dependencies or whose dependencies have been updated first. During the update process, ensure that the update of each node does not destroy the dependencies or introduce inconsistencies. Perform necessary data verification, including data integrity check and system function test. After completing the dependency reconstruction, generate the latest storage node data of the system to ensure that it reflects all relevant data updates and changes in the system. Output the generated system storage node data for use by other components or modules of the system.

[0118] The invention is beneficial in that the data to be cached is mapped to keyword hash value data through consistent hash calculation. The characteristics of consistent hashing ensure relatively uniform data distribution, avoid the problem of uneven distribution of hot data on storage nodes, and help improve the load balancing of the system. Mapping the keyword hash value data to the nodes on the hash ring realizes the allocation of storage nodes, so that adjacent nodes are close to each other on the hash ring, which helps to achieve data locality, improve cache hit rate, and reduce network delay during data access. According to the allocation of hash ring nodes, corresponding storage node data is generated, which helps to establish a mapping relationship between data and nodes, and provides basic data for subsequent fault simulation, analysis and processing. High-voltage simulation can simulate the working state of nodes under different load conditions, thereby generating node failure rate data, so that the system can predict possible failures of nodes, and can take corresponding preventive measures before the failure occurs. Through abnormal analysis, the system can perform more in-depth diagnosis and analysis of node failures. Performing abnormal analysis on isolated fault nodes helps to distinguish dangerous abnormal nodes from ordinary abnormal nodes, so as to adopt different processing strategies. Abnormal node isolation, fault prevention, and node replacement operations are intelligent decisions made based on the system's comprehensive analysis and prediction of node failures, which can reduce the manual intervention of system maintenance personnel and improve the system's automation and operating efficiency. Node replacement and reconstruction, the system can replace faulty nodes with optimized nodes and rebuild the system storage node data, thereby improving the stability and performance of the system. The system can detect, diagnose, and handle node failures in a timely manner, thereby improving the system's reliability and fault tolerance and reducing the impact of system failures on users and businesses. The system uses a comprehensive set of technologies such as consistent hashing allocation, node failure rate prediction, node anomaly detection and processing, node optimization and reconstruction, and forms a complete set of node management and optimization solutions to improve the stability, availability, and performance of distributed storage systems.

[0119] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0120] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A distributed caching method based on a hash algorithm, characterized in that: The following steps are involved: Step S1: Obtain data to be cached; Perform consistent hash calculation on cached data to obtain keyword hash value data; Perform hash ring node allocation on keyword hash value data to generate corresponding storage node data; Step S2: performing high voltage simulation on corresponding storage node data based on preset time period data to generate node failure rate data; Draw a box plot of the node failure rate data to generate a node failure box plot; Step S3: Perform abnormal analysis on the node fault box line diagram to generate isolated fault node abnormal data; when the isolated fault node abnormal data is dangerous abnormal node data, perform abnormal node isolation on the dangerous abnormal node data to obtain updated storage node data; when the isolated fault node abnormal data is ordinary abnormal node data, perform fault prevention on the ordinary abnormal node data to obtain preventive storage node data; Step S4: remove identical nodes from the update storage node data and the prevention storage node data to generate difference node data; Perform node replacement on corresponding storage node data based on differential node data to generate optimized storage node data; Rebuild the dependency of optimized storage node data to generate system storage node data; Step S1 includes the following steps: Step S11: Obtain data to be cached; Step S12: Perform keyword mining on the cached data to obtain cached item keyword data; Step S13: performing consistent hash calculation on the cache item keyword data to obtain keyword hash value data; Step S14: performing hash ring node allocation on the keyword hash value data to generate corresponding storage node data; Step S2 includes the following steps: Step S21: recording the data throughput of the corresponding storage node data based on the preset time period data to obtain the access frequency data of each node; Step S22: performing high voltage simulation on the corresponding storage node data through the access frequency data of each node to generate node failure rate data; Step S23: performing five-number summarization on the node failure rate data to obtain node failure rate summary data; Step S24: draw a box plot of the node failure rate summary data to generate a node failure box plot; Step S22 includes the following steps: Step S221: Calculate the frequency fluctuation of the access frequency data of each node to obtain the access frequency fluctuation data; Step S222: Filter the access frequency data of each node based on the access frequency fluctuation data to generate the access frequency data with large fluctuation; Step S223: Use the access frequency data with large fluctuation to perform fatigue access simulation on the corresponding storage node data to obtain the node access situation data, wherein the node access situation data includes node fault data and node normal data; Step S224: Calculate the failure rate of the node fault data based on the node normal data to generate the node failure rate data; Step S3 includes the following steps: Step S31: Isolate outliers on the node fault box plot to obtain isolated fault node data; Step S32: Perform abnormal analysis on the isolated fault node data to generate isolated fault node abnormal data; Step S33: When the isolated fault node abnormal data is dangerous abnormal node data, perform abnormal node isolation on the dangerous abnormal node data to obtain updated storage node data; Step S34: When the isolated fault node abnormal data is ordinary abnormal node data, perform fault prevention on the ordinary abnormal node data to obtain preventive storage node data; Step S32 includes the following steps: Step S321: feature lighting the isolated fault node data to obtain the isolated fault node feature data; Step S322: abnormal pattern capture of the isolated fault node feature data to generate fault node abnormal pattern data; Step S323: abnormal cause analysis of the fault node abnormal pattern data to generate isolated fault node abnormal data; Step S33 includes the following steps: Step S331: when the abnormal data of the isolated fault node is dangerous abnormal node data, the abnormal point of the corresponding storage node data is isolated based on the dangerous abnormal node data to generate abnormal isolation storage node data; Step S332: data backup of the abnormal isolation storage node data to obtain abnormal isolation storage node backup data; Step S333: node construction of the abnormal isolation storage node backup data to generate replacement storage node data; Step S334: node replacement of the abnormal isolation storage node data by replacing the storage node data to generate updated storage node data; Step S34 includes the following steps: Step S341: when the abnormal data of the isolated fault node is common abnormal node data, backing up the common abnormal node data to obtain common abnormal node backup data; Step S342: performing hash sharding on the common abnormal node backup data based on the corresponding storage node data to obtain sharded common abnormal node backup data; Step S343: dynamically balancing and distributing the sharded common abnormal node backup data to obtain preventive storage node data; Step S4 includes the following steps: Step S41: performing node full attribute comparison on the update storage node data and the prevention storage node data to obtain a node comparison data set; Step S42: removing identical nodes from the update storage node data and the prevention storage node data through the node comparison data set to generate difference node data; Step S43: performing node replacement on corresponding storage node data based on the difference node data to generate optimized storage node data; Step S44: Dependency reconstruction is performed on the optimized storage node data to generate system storage node data; Step S44 includes the following steps: Step S441: perform data flow analysis on the corresponding storage node data to obtain node data flow data; Step S442: perform dependency analysis on the node data flow data to generate node dependency relationship data; Step S443: perform dependency reconstruction on the optimized storage node data based on the node dependency relationship data to generate system storage node data.

Images