Effective weighted density peak clustering analysis method based on rough evidence theory

Abstract

The invention discloses an effective weighted density peak clustering analysis method based on a rough evidence theory, and relates to the technical field of electric digital data processing. The method comprises the following steps: analyzing a data preprocessing real-time deviation index according to preprocessing deviation data, and judging whether to carry out preprocessing adjustment or not; analyzing a data storage real-time deviation index according to the storage deviation data, and judging whether to carry out storage adjustment or not; and analyzing a data access real-time deviation index according to the access deviation data, and judging whether to perform access adjustment or not. According to the method, whether preprocessing adjustment is performed or not is analyzed through preprocessing deviation data, whether storage adjustment is performed or not is analyzed according to storage deviation data, and whether access adjustment is performed or not is analyzed according to access deviation data, so that the effect of improving the real-time performance of network personalized recommendation by applying an effective weighted density peak clustering analysis method is achieved; the problem that in the prior art, real-time performance is poor when an effective weighted density peak clustering analysis method is applied to network personalized recommendation is solved.

Description

Technical Field

[0001] The invention relates to the technical field of electronic digital data processing, and in particular to an effective weighted density peak clustering analysis method based on rough evidence theory. Background Art

[0002] In social network analysis, the relationships between users are complex and changeable, with a large number of overlapping communities and fuzzy attribution. Traditional user classification methods are difficult to accurately classify users, so an effective weighted density peak clustering method based on rough evidence theory was invented. This method can handle this uncertainty and ambiguity, accurately identify the community structure in social networks, and provide strong support for social network recommendations, user behavior analysis, and social network mining. For example, this method can be used to analyze users' social behaviors, discover user groups with similar interests and behaviors, and provide a basis for precision marketing and personalized recommendations.

[0003] The existing user classification system analyzes the user's behavior data and classifies users with high similarity in behavior characteristics into one category, thereby realizing the user classification function.

[0004] For example, the real-time personalized recommendation system disclosed in the patent application with publication number: CN118568335A includes a data collection module: responsible for collecting users' real-time behavior data; a real-time data processing module: processing the collected real-time data to extract key features, identify user interests and behavior patterns; a user modeling module: establishing a user model based on the user's historical behavior and real-time data to understand the user's preferences and interests; a recommendation algorithm module: implementing a personalized recommendation algorithm based on the user model and product / service information to provide users with customized product and service recommendations; a feedback processing module: processing user feedback information, including click feedback, purchase feedback, evaluation, etc., for adjusting and optimizing the recommendation algorithm; a recommendation result display module: displaying personalized recommendation results on the front-end interface in a user-friendly manner; a real-time update module: ensuring that the recommendation system can respond to users' real-time behaviors and changes in a timely manner, and update user models and recommendation results.

[0005] For example, a patent application with publication number: CN116226520A discloses a real-time recall method for a personalized recommendation system, including: in this recall process, first use the 1-fold variance vector to update the 2-fold variance vector; then use the t-fold variance vector to update the t+1-fold variance vector in sequence until the highest multiple, i.e., the z-fold variance vector, is obtained; then the highest multiple, i.e., the z-fold variance vector is scaled to a 1-fold variance vector, and the 1-fold variance vector is updated and stored to complete this recall; in the next recall process, the updated 1-fold variance vector is used for the next recall.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems: In the prior art, when obtaining relevant data for network personalized recommendation (such as user behavior data, user attribute data, product feature data and transaction data, etc.), processing and analysis process delays may occur, and there is a problem of poor real-time performance when applying effective weighted density peak clustering analysis method for network personalized recommendation. Summary of the invention

[0007] The embodiment of the present application solves the problem of poor real-time performance of applying the effective weighted density peak clustering analysis method for network personalized recommendation in the prior art by providing an effective weighted density peak clustering analysis method based on rough evidence theory, and improves the timeliness of applying the effective weighted density peak clustering analysis method for network personalized recommendation.

[0008] The embodiment of the present application provides an effective weighted density peak clustering analysis method based on rough evidence theory, comprising the following steps: obtaining preprocessing deviation data corresponding to the data to be clustered and analyzed, analyzing a data preprocessing real-time deviation index based on the preprocessing deviation data and judging whether to perform preprocessing adjustment; storing the data to be clustered and analyzed after the preprocessing adjustment is qualified, obtaining corresponding storage deviation data, analyzing a data storage real-time deviation index based on the storage deviation data and judging whether to perform storage adjustment; monitoring the access situation corresponding to the data to be clustered and analyzed after the storage adjustment is qualified, obtaining corresponding access deviation data, analyzing a data access real-time deviation index based on the access deviation data and judging whether to perform access adjustment.

[0009] Furthermore, the preprocessing deviation data includes: data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time and data partitioning time; the data preprocessing real-time deviation index is obtained by processing the preprocessing deviation data and the reference preprocessing deviation data; the data preprocessing real-time deviation index is used to reflect the degree of deviation in preprocessing of the data to be clustered analysis; the reference preprocessing deviation data includes: reference data acquisition time, reference data preprocessing delay, CPU usage during reference preprocessing, reference data simplification time and reference data partitioning time.

[0010] Furthermore, the specific method for obtaining the data preprocessing real-time deviation index is: processing the data acquisition duration with the reference data acquisition duration to obtain a duration score; processing the data preprocessing delay with the reference data preprocessing delay to obtain a preprocessing delay score; processing the CPU usage during the preprocessing process with the reference preprocessing process to obtain a CPU usage score; processing the data simplification duration with the reference data simplification duration to obtain a simplification duration score; processing the data partition duration with the reference data partition duration to obtain a partition duration score; obtaining the data preprocessing real-time deviation index based on the duration score, preprocessing delay score, CPU usage score, simplification duration score and partition duration score.

[0011] Furthermore, the specific limiting expression of the data preprocessing real-time deviation index is: ; In the formula, It is represented as the real-time deviation index of data preprocessing for the data to be clustered. Expressed as The duration score of a preset time period, , Indicates the number of the preset time period. Expressed as the total number of preset time periods, Expressed as The fraction of preprocessing delays for a preset time period, Expressed as The CPU usage score for a preset time period. Expressed as The simplified duration score of a preset time period, Expressed as The duration score of each preset time period.

[0012] Furthermore, the specific process of determining whether to perform preprocessing adjustment is as follows: comparing the data preprocessing real-time deviation index with the preprocessing deviation threshold preset in the database; when the data preprocessing real-time deviation index is less than the preprocessing deviation threshold preset in the database, no preprocessing adjustment is performed; when the data preprocessing real-time deviation index is not less than the preprocessing deviation threshold preset in the database, preprocessing adjustment is performed, and the preprocessing adjustment includes data cleaning deviation adjustment and rough set theory application deviation adjustment, and when the data preprocessing real-time deviation index monitored after the preprocessing adjustment is less than the preprocessing deviation threshold preset in the database, the preprocessing adjustment is stopped.

[0013] Furthermore, the storage deviation data includes: real-time deviation index of data preprocessing after preprocessing adjustment, data storage duration, data storage speed, data time complexity and data space complexity; the data storage real-time deviation index is obtained by processing the storage deviation data and the reference storage deviation data; the data storage real-time deviation index is used to reflect the degree of deviation of the data storage to be clustered analysis; the reference storage deviation data includes: reference data storage duration, reference data storage speed, reference data time complexity and reference data space complexity.

[0014] Furthermore, the specific method for obtaining the data storage real-time deviation index is: obtaining a storage duration score according to the relative relationship between the data storage duration and the reference data storage duration; obtaining a storage speed score according to the relative relationship between the data storage speed and the reference data storage speed; obtaining a time complexity score according to the relative relationship between the data time complexity and the reference data time complexity; obtaining a space complexity score according to the relative relationship between the data space complexity and the reference data space complexity; obtaining a data storage real-time deviation index according to the preprocessing real-time deviation index of the data after preprocessing adjustment, the storage duration score, the storage speed score, the time complexity score and the space complexity score.

[0015] Furthermore, the specific process of determining whether to perform storage adjustment is as follows: comparing the data storage real-time deviation index with the storage deviation threshold preset in the database to determine whether the data storage real-time deviation index is less than the storage deviation threshold preset in the database; when the data storage real-time deviation index is less than the storage deviation threshold preset in the database, no storage adjustment is performed; when the data storage real-time deviation index is not less than the storage deviation threshold preset in the database, storage adjustment is performed, and when the data storage real-time deviation index monitored after the storage adjustment is less than the storage deviation threshold preset in the database, the storage adjustment is stopped.

[0016] Furthermore, the access deviation data includes: a real-time deviation index of data preprocessing after qualified preprocessing adjustment, a real-time deviation index of data storage after qualified storage adjustment, data query response time, data access frequency, data access time and number of data access interruptions; the data access real-time deviation index is obtained by processing the access deviation data and the reference access deviation data; the data access real-time deviation index is used to reflect the degree of access deviation of the data to be clustered and analyzed; the reference access deviation data includes: reference data query response time, reference data access frequency, reference data access time and number of reference data access interruptions.

[0017] Furthermore, the specific method for obtaining the data access real-time deviation index is: obtaining a query response time score by processing the relative relationship between data query response time and reference data query response time; obtaining an access frequency score by processing the relative relationship between data access frequency and reference data access frequency; obtaining an access time score by processing the relative relationship between data access time and reference data access time; obtaining an access interruption number score by processing the relative relationship between data access interruption times and reference data access interruption times; obtaining a data access real-time deviation index based on the data preprocessing real-time deviation index after preprocessing adjustment, the data storage real-time deviation index after storage adjustment, and the query response time score, access frequency score, access time score and access interruption number score obtained by processing access deviation data and reference access deviation data.

[0018] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Analyze the data preprocessing real-time deviation index through preprocessing deviation data and judge whether to make preprocessing adjustment. After the preprocessing adjustment is qualified, analyze the data storage real-time deviation index according to the storage deviation data and judge whether to make storage adjustment. After the storage adjustment is qualified, analyze the data access real-time deviation index according to the access deviation data and judge whether to make access adjustment. Then, the real-time performance of network personalized recommendation is improved by applying the effective weighted density peak clustering analysis method, which effectively solves the problem of poor real-time performance of network personalized recommendation by applying the effective weighted density peak clustering analysis method in the prior art. 2. The duration score, preprocessing delay score, CPU usage score, simplified duration score and partition duration score are obtained by preprocessing deviation data and reference preprocessing deviation data, and the data preprocessing real-time deviation index is analyzed to determine whether to make preprocessing adjustments based on the data preprocessing real-time deviation index, thereby achieving more efficient and accurate preprocessing adjustments; 3. By processing the storage deviation data and the reference storage deviation data, the storage duration score, storage speed score, time complexity score and space complexity score are obtained. The data storage real-time deviation index is obtained by combining the preprocessing real-time deviation index of the qualified data after preprocessing adjustment. Therefore, it is judged whether to make storage adjustment according to the data storage real-time deviation index, thereby achieving more efficient and accurate storage adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of an effective weighted density peak clustering analysis method based on rough evidence theory provided in an embodiment of the present application; Figure 2 A graph showing the real-time deviation index of data preprocessing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiment of the present application solves the problem of poor real-time performance of network personalized recommendation using the effective weighted density peak clustering analysis method in the prior art by providing an effective weighted density peak clustering analysis method based on rough evidence theory. The real-time deviation index of data preprocessing is analyzed by preprocessing deviation data to determine whether to perform preprocessing adjustment. Then, after the preprocessing adjustment is qualified, the real-time deviation index of data storage is analyzed according to the storage deviation data to determine whether to perform storage adjustment. Then, after the storage adjustment is qualified, the real-time deviation index of data access is analyzed according to the access deviation data to determine whether to perform access adjustment, thereby achieving the improvement of the real-time performance of network personalized recommendation using the effective weighted density peak clustering analysis method.

[0021] The technical solution in the embodiment of the present application is to solve the problem of poor real-time performance of the above-mentioned effective weighted density peak clustering analysis method for network personalized recommendation. The overall idea is as follows: By obtaining the preprocessing deviation data corresponding to the data to be clustered and analyzed, the data preprocessing real-time deviation index is analyzed according to the preprocessing deviation data and it is determined whether to perform preprocessing adjustment. The data to be clustered and analyzed after the preprocessing adjustment is qualified is stored and the corresponding storage deviation data is obtained. Then, the data storage real-time deviation index is analyzed according to the storage deviation data and it is determined whether to perform storage adjustment. Then, the access situation corresponding to the data to be clustered and analyzed after the storage adjustment is qualified is monitored and the corresponding access deviation data is obtained. Finally, the data access real-time deviation index is analyzed according to the access deviation data and it is determined whether access adjustment is performed, thereby achieving the effect of improving the real-time performance of network personalized recommendation using the effective weighted density peak clustering analysis method.

[0022] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0023] like Figure 1 As shown, it is a flowchart of an effective weighted density peak clustering analysis method based on rough evidence theory provided by an embodiment of the present application, and the method includes the following steps: obtaining preprocessing deviation data corresponding to the data to be clustered and analyzed, analyzing the data preprocessing real-time deviation index according to the preprocessing deviation data and judging whether to perform preprocessing adjustment; storing the data to be clustered and analyzed after the preprocessing adjustment is qualified, obtaining the corresponding storage deviation data, analyzing the data storage real-time deviation index according to the storage deviation data and judging whether to perform storage adjustment; monitoring the access situation corresponding to the data to be clustered and analyzed after the storage adjustment is qualified, obtaining the corresponding access deviation data, analyzing the data access real-time deviation index according to the access deviation data and judging whether to perform access adjustment.

[0024] It should be supplemented that the preprocessing deviation data include: data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time and data partitioning time; the data preprocessing real-time deviation index is obtained by processing the preprocessing deviation data and the reference preprocessing deviation data; the data preprocessing real-time deviation index is used to reflect the degree of deviation in the preprocessing of the data to be clustered analysis; the reference preprocessing deviation data include: reference data acquisition time, reference data preprocessing delay, CPU usage during reference preprocessing, reference data simplification time and reference data partitioning time.

[0025] In this embodiment, the data to be clustered and analyzed include user behavior data, user attribute data, product feature data and transaction data, among which user behavior data include user browsing history, click history, purchase history, collection history, etc., which can be obtained through the background system, log system, etc. of the website or application; user attribute data include user's age, gender, occupation, interests and hobbies, region, etc., which can be obtained through user registration information, questionnaires, third-party data services, etc.; product feature data include product name, price, brand, category, description, picture, etc., which can be obtained through product management system, product database, etc.; transaction data include transaction time, transaction amount, transaction quantity, etc., which can be obtained through transaction system, payment system, etc.

[0026] The data acquisition duration indicates the duration used to acquire the data to be clustered in the preset time period, the data preprocessing delay indicates the delay duration of preprocessing the data to be clustered in the preset time period, the CPU usage rate during preprocessing refers to the system CPU usage rate when preprocessing the data to be clustered in the preset time period, the data simplification duration indicates the duration of simplification of the data to be clustered in the preset time period, and the data partitioning duration indicates the duration of partitioning the data to be clustered in the preset time period. The reference data acquisition duration indicates the duration consumed to acquire the data to be clustered when the preprocessing is qualified in the preset time period, the reference data preprocessing delay indicates the delay duration of preprocessing the data to be clustered when the preprocessing is qualified in the preset time period, the reference preprocessing CPU usage rate indicates the system CPU usage rate when preprocessing the data to be clustered when the preprocessing is qualified in the preset time period, the reference data simplification duration indicates the duration of simplification of the data to be clustered when the preprocessing is qualified in the preset time period, and the reference data partitioning duration indicates the duration of partitioning the data to be clustered when the preprocessing is qualified in the preset time period.

[0027] The specific process of the above simplification is as follows. Assume that there is a network personalized recommendation system whose data set contains multiple attributes of users, such as age, gender, occupation, browsing history, purchase history, etc. The goal is to recommend suitable goods or services to users based on their attributes. The attribute simplification method of rough set theory is used to remove redundant attributes that have little impact on classification (i.e., recommendation). For example, by calculating the dependency or importance between attributes, it is found that the "occupation" attribute contributes little to the classification ability of the recommendation system, so it can be simplified. The simplified data set only contains attributes that have an important impact on classification, such as age, gender, browsing history, and purchase history. Result analysis: The simplified data set is more concise, reducing the amount of calculation and storage requirements. At the same time, due to the removal of redundant attributes, the classification ability of the recommendation system (i.e., recommendation accuracy) may be improved.

[0028] The specific process of the above division is as follows. The concepts of lower approximation and upper approximation of rough sets are used to divide the simplified data set. The lower approximation represents the part where the user clearly belongs to a certain category, while the upper approximation represents the part where the user's purchased items may belong to a certain category. For example, the items purchased by the user can be divided into the lower approximation set of "expensive items" (higher than 200 yuan) and the upper approximation set of "low-priced items" (less than 200 yuan).

[0029] Use the system's built-in logging function to record the start and end time of data acquisition, calculate the difference between the two time points, and get the data acquisition duration. Use specialized data preprocessing tools or frameworks, such as Apache Flink, Apache Spark, etc., to measure the delay of data preprocessing and get the preprocessing delay. Use the operating system's built-in monitoring tools (such as Windows' Task Manager, Linux's top or htop command) to monitor CPU usage and get the CPU usage during preprocessing. Add timing logic to the key parts of the data reduction and partitioning code, record the start and end time of the operation, calculate the difference to get the duration, and get the data reduction duration and data partitioning time.

[0030] The data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time and data partitioning time of the same data volume as the data to be clustered analysis within a historical preset time period are obtained from the database when the data preprocessing is qualified, and the reference data acquisition time, reference data preprocessing delay, reference CPU usage during preprocessing, reference data simplification time and reference data partitioning time are obtained by mean processing.

[0031] The data preprocessing real-time deviation index is obtained by comprehensively considering the data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time and data partitioning time. This makes the consideration of the data preprocessing real-time deviation index more comprehensive, which is conducive to more accurately analyzing the deviation degree of data preprocessing to be clustered analysis based on the data preprocessing real-time deviation index, so as to more accurately adjust the preprocessing and improve the real-time of preprocessing, which is conducive to improving the real-time of network personalized recommendation using the effective weighted density peak clustering analysis method.

[0032] Furthermore, the specific method for obtaining the real-time deviation index of data preprocessing is: the data acquisition time and the reference data acquisition time are processed to obtain the time score (i.e. ); Process the data preprocessing delay and the reference data preprocessing delay to obtain the preprocessing delay score (i.e. ); Process the CPU usage rate in the preprocessing process with the CPU usage rate in the reference preprocessing process to obtain the CPU usage rate score (i.e. ); Process the data simplification duration and the reference data simplification duration to obtain the simplification duration score (i.e. ); Process the data partition duration and the reference data partition duration to obtain the partition duration score (i.e. ); The data preprocessing real-time deviation index is obtained according to the duration score, preprocessing delay score, CPU usage score, simplification duration score and partitioning duration score.

[0033] Specifically, the time periods for collecting cluster analysis data are numbered. , Indicates the number of the preset time period. Expressed as the total number of preset time periods, the specific limiting expression of the data preprocessing real-time deviation index is: ; ; ; ; ; ; In the formula, It is represented as the real-time deviation index of data preprocessing for the data to be clustered. Expressed as The duration of data acquisition for a preset time period, Expressed as Data preprocessing delay for a preset time period, Expressed as CPU usage during preprocessing for a preset time period, Expressed as The data reduction duration for a preset time period, Expressed as The duration of the data division for each preset time period, Indicates the reference data acquisition time. W represents the reference data preprocessing delay, It is expressed as the CPU usage during the reference preprocessing process. It is represented as the reference data simplification duration, Indicates the duration of reference data division. Expressed as The duration score of a preset time period, Expressed as The fraction of preprocessing delays for a preset time period, Expressed as The CPU usage score for a preset time period. Expressed as The simplified duration score of a preset time period, Expressed as The duration score of each preset time period.

[0034] In this embodiment, a long data acquisition time may increase the time that subsequent preprocessing tasks wait for data, thereby increasing the preprocessing delay. If the data acquisition time is too long and the amount of data is large, then after the data arrives, the preprocessing task may consume a large amount of CPU resources in a short period of time, resulting in a surge in CPU usage. A long data acquisition time may cause the subsequent data reduction and data partitioning task processing time to be relatively extended because the amount of data to be processed may be larger.

[0035] Long preprocessing delays may mean that the preprocessing task encountered a bottleneck during execution, such as high algorithm complexity, insufficient resources, etc., which may cause CPU usage to fluctuate or continue to run at a high level. Long preprocessing delays may cause subsequent data reduction and data partitioning tasks to fail to start in time, thereby extending the time of the entire data preprocessing process.

[0036] Excessive CPU usage may cause data reduction and data partitioning tasks to be resource-constrained during execution, thereby prolonging processing time. High CPU usage may cause the overall system performance to degrade, affecting the execution of other concurrent tasks, which may indirectly affect the efficiency of data acquisition and data preprocessing.

[0037] A shorter data reduction time usually means that the data is compressed and simplified more effectively, which helps reduce the processing time of subsequent data partitioning tasks.

[0038] In summary, data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time, and data partitioning time are interrelated and influence each other in the data preprocessing process, and together constitute a comprehensive evaluation of the real-time performance of data preprocessing, thereby achieving accurate preprocessing adjustments.

[0039] Therefore, when the data acquisition duration, data preprocessing delay, CPU usage during preprocessing, data simplification duration and data partitioning duration are respectively greater than and deviate from the reference data acquisition duration, reference data preprocessing delay, reference preprocessing CPU usage, reference data simplification duration and reference data partitioning duration, the larger the duration score, preprocessing delay score, CPU usage score, simplification duration score and partitioning duration score, the larger the data preprocessing real-time deviation index, the greater the degree of deviation of data preprocessing, and the more preprocessing adjustment is needed; and vice versa.

[0040] like Figure 2 As shown, it is a data preprocessing real-time deviation index curve graph provided by an embodiment of the present application, with the preprocessing delay score as the independent variable and the data preprocessing real-time deviation index as the dependent variable. For example, the preset time period number is 1, the reference data acquisition duration, the reference data preprocessing delay, the CPU usage rate during the reference preprocessing process, the reference data simplification duration, and the reference data division duration are 20 milliseconds, 15 milliseconds, 50%, 10 milliseconds, and 10 milliseconds, respectively. Through the increase and decrease analysis of curve one, curve two, and curve three, it can be seen that when the preprocessing delay score is larger, the data preprocessing real-time deviation index is larger; when the preprocessing delay score is smaller, the data preprocessing real-time deviation index is smaller. The statistical table of data preprocessing real-time deviation index related data is shown in Table 1: Table 1 Statistics of data preprocessing real-time deviation indicators:

[0041] When the parameters of curve 1, curve 2 and curve 3 are as shown in the above table, curve 1 and curve 2 only have inconsistent division time scores and other parameters are consistent. The smaller the division time score, the smaller the corresponding data preprocessing real-time deviation index. When the simplified time scores of curve 2 and curve 3 are inconsistent and other data are consistent, the smaller the simplified time score, the smaller the corresponding data preprocessing real-time deviation index. The smaller the data preprocessing real-time deviation index, the less preprocessing adjustment is required. The larger the data preprocessing real-time deviation index, the more preprocessing adjustment is required, thereby realizing the application of the effective weighted density peak clustering analysis method to improve the real-time performance of network personalized recommendation.

[0042] Furthermore, the specific process of determining whether to perform preprocessing adjustment is as follows: comparing the data preprocessing real-time deviation index with the preprocessing deviation threshold preset in the database; when the data preprocessing real-time deviation index is less than the preprocessing deviation threshold preset in the database, no preprocessing adjustment is performed; when the data preprocessing real-time deviation index is not less than the preprocessing deviation threshold preset in the database, preprocessing adjustment is performed, and the preprocessing adjustment includes data cleaning deviation adjustment and rough set theory application deviation adjustment, and when the data preprocessing real-time deviation index monitored after the preprocessing adjustment is less than the preprocessing deviation threshold preset in the database, the preprocessing adjustment is stopped. The preprocessing deviation threshold preset in the database can be represented by the average value of the preprocessing real-time deviation index corresponding to the historical data.

[0043] In this embodiment, data cleaning refers to cleaning the collected data to remove duplicate, invalid or erroneous data. Data cleaning deviation adjustment includes: checking and reprocessing duplicate data: using tools such as unique identification methods or hash functions to recheck the data to ensure that duplicate data is accurately identified and removed; verifying and filling missing values: rechecking the identification process of missing values ​​to ensure that all missing values ​​are accurately marked, and using methods such as mean, median, mode, regression prediction, etc. to fill missing values ​​according to the distribution of the data, and verifying whether the filled data is reasonable. For example, suppose there is a data set containing user purchase records for training a personalized recommendation system. The data set contains the following features: user ID, product ID, purchase quantity, purchase time, product category, product price, etc. First, it is necessary to identify the missing values ​​in the data set. Using Python's Pandas library, the output results are: user ID 0, product ID 5, purchase quantity 10, purchase time 2, product category 3, product price 8. This indicates that there are missing values ​​in the data set, because when the purchase quantity is 10, the corresponding product price should be 10, but it is 8 at this time. It needs to be processed. Different methods are used to fill the missing values ​​according to the distribution of the data. Mean filling: For numerical data (such as purchase quantity, product price), if the data distribution is relatively uniform, the mean can be used to fill the missing values. Median filling: If the data distribution is skewed or has extreme values, the median can be used to fill the missing values. Here, it is assumed that the purchase quantity has a skewed distribution, so the median is used for filling. Mode filling: For categorical data (such as product category), the mode can be used to fill the missing values. Regression prediction filling: For missing values ​​that have a linear relationship with other features (such as product price may be related to product category), regression prediction can be used to fill. But in this example, it is assumed that the product price has been filled by the mean, so regression prediction filling is no longer performed. Verify the rationality of the filled data and use statistical methods (such as t-test) to compare the difference between the filled data and the original data (non-missing part). But in this example, since the mean or median has been used to fill the missing values ​​and the data distribution is relatively uniform, it can be considered that the filled data is statistically consistent with the original data. Correct erroneous data: Use automated tools or algorithms to identify and correct erroneous data, and manually review and correct key data (data that has a significant impact on business decisions and analysis. For example, customer name, order amount, purchase date, etc. are all key data in the e-commerce field. Or obviously erroneous data (referring to errors in the data that are illogical or obviously inconsistent with the actual situation. For example, a negative age value is obviously erroneous data) (assuming there is a sales data set that contains fields such as customer name, order amount, purchase date, etc. Automated tools identify that a customer's order amount is "-1,000 yuan", which is obviously erroneous data.After manual review, it was found that the negative amount was caused by a system error, so it was corrected to the correct amount of "1,000 yuan"); handle outliers: use binning methods, statistical methods or combine business conditions to identify outliers (data points in the data set that are significantly different from most data points. For example, the prices of all goods are below 200, and only one exceeds 50%, then this price is an outlier). Depending on the impact of the outliers on the overall analysis, choose to delete, replace or deeply analyze the outliers and decide how to deal with them; improve the accuracy and efficiency of data cleaning tools: use more advanced data cleaning tools and technologies, such as automated cleaning tools (Python, SQL, etc.) or professional data cleaning software (such as Trifacta, OpenRefine, etc.).

[0044] The application of rough set theory refers to using rough set theory to simplify data, remove redundant attributes, and retain attributes that have an important impact on classification. Through the concepts of lower approximation and upper approximation of rough sets, data is divided to form different knowledge particles. The deviation adjustment of rough set theory application includes: re-evaluation of attribute simplification: using the attribute importance evaluation method in rough set theory (such as information gain, Gini index, etc.) to re-evaluate attributes to ensure that only redundant attributes are deleted; adjust the division of lower approximation and upper approximation: re-examine the distribution of data and classification requirements, adjust the division criteria of lower approximation and upper approximation, and use more sophisticated division methods (such as clustering-based division methods) to improve the accuracy of division; verify and optimize knowledge particles: verify the rationality of knowledge particles to ensure that they can accurately reflect the classification characteristics of data, and optimize the size and shape of knowledge particles according to classification requirements and data distribution; introduce other machine learning algorithms for assistance: introduce other machine learning algorithms (such as decision trees, support vector machines, etc.) to assist classification and improve the accuracy of classification results.

[0045] Furthermore, the storage deviation data includes: the real-time deviation index of data preprocessing after qualified preprocessing adjustment, data storage duration, data storage speed, data time complexity and data space complexity; the data storage real-time deviation index is obtained by processing the storage deviation data and the reference storage deviation data; the data storage real-time deviation index is used to reflect the degree of deviation of the data storage to be clustered analysis; the reference storage deviation data includes: reference data storage duration, reference data storage speed, reference data time complexity and reference data space complexity.

[0046] In this embodiment, data storage duration refers to the time consumed to complete storage of each type of data in the data to be clustered and analyzed, data storage speed refers to the speed at which each type of data in the data to be clustered and analyzed is stored, data time complexity refers to the time complexity of each type of data in the data to be clustered and analyzed, and time complexity describes the relationship between the time consumed in the data storage process and the input data scale, data space complexity refers to the space complexity of each type of data in the data to be clustered and analyzed, and space complexity describes the relationship between the size of the storage space temporarily occupied by the data in the storage process and the input data scale. Reference data storage duration refers to the time consumed to complete storage of each type of data in the data to be clustered and analyzed when data storage is qualified, reference data storage speed refers to the speed at which each type of data in the data to be clustered and analyzed is stored when data storage is qualified, reference data time complexity refers to the time complexity of each type of data in the data to be clustered and analyzed when data storage is qualified, and reference data space complexity refers to the space complexity of each type of data in the data to be clustered and analyzed when data storage is qualified.

[0047] During the data storage process, the timestamps of the start and end of storage are recorded through the log system, and then the time difference is calculated to obtain the data storage duration. The storage speed is evaluated by writing a large amount of data to the storage device and measuring the amount of data written per unit time to obtain the data storage speed. The algorithm used for data storage is analyzed through software tools for algorithm analysis and performance evaluation, and its time complexity is evaluated to obtain the data time complexity. For example, the algorithm used is the bubble sort algorithm. The time complexity analysis of the bubble sort algorithm: the bubble sort contains two layers of nested loops, the outer loop is executed n times, and the inner loop is executed ni-1 times (i is the counter of the outer loop). Therefore, the total time complexity is O(n^2) (using the big O notation to simplify the time complexity function, only retaining the highest order terms, and ignoring coefficients and low-order terms). The algorithm for storing data is analyzed through software tools for storage demand evaluation and space complexity analysis to obtain the data space complexity. For example, the storage algorithm is the bubble sort algorithm. The space complexity analysis of the bubble sort algorithm: the bubble sort mainly uses an array to store the data to be sorted, as well as a small number of local variables. Therefore, its space complexity is O(n) (n is the length of the array) (using big O notation to simplify the space complexity function).

[0048] Obtain data from the database for the same amount of data in the historical time period as the data to be clustered for analysis, and store the data in a qualified manner (the specific acquisition method is the same as above), and perform mean processing to obtain the reference data storage duration, reference data storage speed, reference data time complexity, and reference data space complexity.

[0049] By combining the data preprocessing real-time deviation index, data storage duration, data storage speed, data time complexity and data space complexity, reference data storage duration, reference data storage speed, reference data time complexity and reference data space complexity to analyze the data storage real-time deviation index, it is beneficial to more accurately analyze the degree of data storage deviation based on the data storage real-time deviation index, and to more accurately make storage adjustments.

[0050] Furthermore, the specific method for obtaining the data storage real-time deviation index is: according to the relative relationship between the data storage duration and the reference data storage duration, the storage duration score (i.e. ); According to the relative relationship between the data storage speed and the reference data storage speed, the storage speed score is obtained (i.e. ); According to the relative relationship between the data time complexity and the reference data time complexity, the time complexity score is obtained (i.e. ); The space complexity score is obtained according to the relative relationship between the data space complexity and the reference data space complexity (i.e. ); The data storage real-time deviation index is obtained according to the preprocessing real-time deviation index, storage duration score, storage speed score, time complexity score and space complexity score of the qualified data after preprocessing adjustment.

[0051] Specifically, obtain each type of data in the data to be clustered and number the data types. , A number representing the data type, Expressed as the total number of data types, the specific limiting expression of the data storage real-time deviation index is: ; ; ; ; ; In the formula, It is represented as the data storage real-time deviation index corresponding to the data to be clustered. It represents the real-time deviation index of data preprocessing after preprocessing adjustment. Expressed as The data storage duration corresponding to the data to be clustered, Expressed as The data storage speed corresponding to the data to be clustered, Expressed as The time complexity of the data corresponding to the data to be clustered, Expressed as The complexity of the data space corresponding to the data to be clustered, Expressed as Storage duration of reference data corresponding to the data to be clustered, Expressed as The reference data storage speed corresponding to the data to be clustered, Expressed as The time complexity of the reference data corresponding to the data to be clustered, Expressed as The complexity of the reference data space corresponding to the data to be clustered. Expressed as The storage duration fraction corresponding to the data to be clustered. Expressed as The storage speed score corresponding to the data to be clustered. Expressed as The time complexity score corresponding to the data to be clustered. Expressed as The space complexity score corresponding to the data to be clustered.

[0052] In this embodiment, the data storage time is negatively correlated with the data storage speed. When the data storage speed is faster, the data storage time required for each type of data in the data to be clustered and analyzed will be relatively shorter; conversely, when the data storage speed is slower, the data storage time required for each type of data in the data to be clustered and analyzed will be relatively longer. Longer data storage time may cause delays in data processing, affecting the real-time performance and response speed of the system. Shorter data storage time helps improve the efficiency and performance of the system.

[0053] Data storage speed is affected by data time complexity. Data time complexity describes the trend of algorithm execution time as the amount of input data increases. Higher time complexity means longer algorithm execution time, which may affect data storage speed. Higher data time complexity may lead to lower data storage speed because the algorithm needs more time to process data. Lower data time complexity helps to increase data storage speed.

[0054] Although there is no direct proportional relationship between data storage speed and data space complexity, there is a certain mutual influence between them. For example, some algorithms may require more storage space to optimize their execution speed, thereby increasing the data space complexity. Higher data space complexity may mean that more storage resources are required to support data storage, which may have a certain negative impact on data storage speed. However, in some cases, by optimizing the storage structure and algorithm, the data storage speed can be improved without increasing the data space complexity.

[0055] There is usually a certain trade-off between data time complexity and data space complexity. In some cases, in order to reduce time complexity, it may be necessary to increase space complexity; and vice versa. This trade-off has an important impact on the degree of deviation in data storage. If you pursue low time complexity too much and sacrifice space complexity, it may lead to waste of storage resources and reduced efficiency; conversely, if you pursue low space complexity too much and sacrifice time complexity, it may lead to a decrease in data processing speed and loss of real-time performance.

[0056] The smaller the data preprocessing real-time deviation index of each type of data in the data to be clustered after the preprocessing adjustment is qualified, the smaller the deviation of data preprocessing, the better the real-time performance of data preprocessing, and the smaller the impact on data storage. When the data storage speed is slow, the data storage time is long, the data time complexity is high, or the data space complexity is large, the degree of deviation of data storage may increase. Therefore, when the data preprocessing real-time deviation index is larger, the data storage time, data time complexity, and data space complexity are respectively greater than the reference data storage time, reference data time complexity, and reference data space complexity, and the reference data storage speed is less than the storage speed score, the larger the storage time score, storage speed score, time complexity score, and space complexity score, the larger the data storage real-time deviation index, the greater the degree of deviation of data storage of each type of data in the data to be clustered, and the more storage adjustment is needed.

[0057] The data storage real-time deviation index is obtained by combining the preprocessing real-time deviation index, storage duration score, storage speed score, time complexity score and space complexity score of the data after preprocessing adjustment, so that the data storage real-time deviation index is more accurate, which is conducive to more accurate storage adjustment.

[0058] Furthermore, the specific process of determining whether to perform storage adjustment is as follows: comparing the data storage real-time deviation index with the storage deviation threshold preset in the database, determining whether the data storage real-time deviation index is less than the storage deviation threshold preset in the database; when the data storage real-time deviation index is less than the storage deviation threshold preset in the database, no storage adjustment is performed; when the data storage real-time deviation index is not less than the storage deviation threshold preset in the database, storage adjustment is performed, and when the data storage real-time deviation index monitored after the storage adjustment is less than the storage deviation threshold preset in the database, storage adjustment is stopped. The storage deviation threshold preset in the database can be represented by the average value of the data storage real-time deviation index corresponding to the historical data.

[0059] In this embodiment, storage adjustment specifically includes: optimizing data storage speed: upgrading storage devices: using higher performance storage devices, such as SSD (Solid State Drive) to replace HDD (Hard Disk Drive) to improve data read and write speed; optimizing storage strategies: reasonably setting storage strategies, such as using cache, distributed storage and other technologies to reduce delays and bottlenecks in the data storage process; optimizing data structures: selecting suitable data structures, such as hash tables, linked lists, etc., to improve data access and storage efficiency. Adjusting data storage duration: setting a reasonable storage cycle: setting a reasonable storage cycle according to data type and storage requirements to avoid deviations caused by too long or too short data storage time; monitoring storage duration: establishing a storage duration monitoring system to track and record data storage duration in real time, and promptly discover and correct deviations; optimizing storage processes: shortening data storage duration by optimizing storage processes, such as reducing unnecessary storage steps and improving storage efficiency. Reduce the time complexity of data: Optimize algorithms: Select or develop algorithms with lower time complexity, such as binary search, hash table, and quick sort, to improve the efficiency of data processing and storage; Reduce the amount of data processing: Reduce the amount of data processing through data preprocessing, data compression and other technologies, thereby reducing time complexity; Parallel processing: Use multi-core processors or distributed computing and other technologies to achieve parallel processing of data and improve processing speed. Optimize data space complexity: Data compression: Use data compression technology, such as gzip (GNUzip, GNU file compression program), LZMA (Lempel-Ziv-Markovchain-Algorithm, Lempel-Ziv-Markov chain algorithm), etc., to reduce data storage space, thereby reducing space complexity; Data type optimization: Select the appropriate storage format and size according to the data type, such as using smaller data types to store fields that do not require a lot of space; Data archiving and clearing: Archive data that is no longer needed on low-cost storage devices to free up space on high-cost storage devices and reduce storage costs.

[0060] Furthermore, the access deviation data include: the real-time deviation index of data preprocessing after qualified preprocessing adjustment, the real-time deviation index of data storage after qualified storage adjustment, the data query response time, the data access frequency, the data access time and the number of data access interruptions; the data access real-time deviation index is obtained by processing the access deviation data and the reference access deviation data; the data access real-time deviation index is used to reflect the degree of access deviation of the data to be clustered and analyzed; the reference access deviation data include: the reference data query response time, the reference data access frequency, the reference data access time and the number of reference data access interruptions.

[0061] In this embodiment, the data query response time refers to the query response time when accessing the data type to be clustered, the data access frequency refers to the access frequency of accessing the data type to be clustered, the data access time refers to the time to complete the access to the data type to be clustered, and the number of data access interruptions refers to the number of interruptions in accessing the data type to be clustered. The reference data query response time refers to the query response time when accessing the data type to be clustered when the data access is qualified, the reference data access frequency refers to the access frequency of accessing the data type to be clustered when the data access is qualified, the reference data access time refers to the time to complete the access to the data type to be clustered when the data access is qualified, and the reference data access interruption number refers to the number of interruptions in accessing the data type to be clustered when the data access is qualified.

[0062] The built-in tools and functions provided by database management systems such as MySQL, such as SHOW PROFILES (before MySQL 8.0), EXPLAIN to analyze query execution plans, and the slow query log function, are used to measure and record query execution time and obtain data query response time. By analyzing database access logs, the frequency of access to the data type to be clustered and analyzed within a specific time period can be counted to obtain the data access frequency. User behavior analysis: Through web statistics tools (such as Google Analytics, Baidu Statistics, etc.) or server logs, the time from the user initiating a data access request to completing the access can be obtained to obtain the data access time. Network monitoring tools: Use network monitoring tools (such as Pingdom, Zabbix, Nagios, etc.) to monitor the stability of network connections and data transmission, so as to detect interruptions in the data access process and obtain the number of data access interruptions.

[0063] The data with the same amount of data as the data to be clustered for analysis within the historical time period is obtained from the database for data storage. After the data storage is qualified, the data query response time, data access frequency, data access time and number of data access interruptions in the preset time period are calculated and averaged to obtain the reference data query response time, reference data access frequency, reference data access time and number of reference data access interruptions.

[0064] The data access real-time deviation index is obtained by comprehensively analyzing the data preprocessing real-time deviation index, data storage real-time deviation index, data query response time, data access frequency, data access time and data access interruption times. This makes the data access real-time deviation index more specific, helps to more accurately analyze the degree of data access deviation, and facilitates accurate access adjustment.

[0065] Furthermore, the specific method for obtaining the data access real-time deviation index is: the query response time score (i.e. ); The access frequency score is obtained by processing the relative relationship between the data access frequency and the reference data access frequency (i.e. ); The access time score is obtained by processing the relative relationship between the data access time and the reference data access time (i.e. ); The access interruption number score is obtained by processing the relative relationship between the number of data access interruptions and the number of reference data access interruptions (i.e. ); obtain the data access real-time deviation index according to the preprocessing real-time deviation index of the data after preprocessing adjustment, the data storage real-time deviation index after storage adjustment, and the query response time score, access frequency score, access time score and access interruption number score obtained by processing the access deviation data and referring to the access deviation data; the specific process of judging whether to make access adjustment is: compare the data access real-time deviation index with the access real-time deviation threshold preset in the database; when the data access real-time deviation index is less than the access real-time deviation threshold preset in the database, no access adjustment is made; when the data access real-time deviation index is not less than the access real-time deviation threshold preset in the database, access adjustment is made; when the data access real-time deviation index monitored after access adjustment is less than the access real-time deviation threshold preset in the database, the access adjustment is stopped. The access real-time deviation threshold preset in the database is represented by the average value of the data access real-time deviation index corresponding to the historical data.

[0066] Specifically, the specific restriction expression of the data access real-time deviation indicator is: ; ; ; ; ; In the formula, It is represented as the data access real-time deviation index of the data to be clustered. It is the real-time deviation index of data storage after storage adjustment. It represents the real-time deviation index of data preprocessing after preprocessing adjustment. Represents the first Data query response time for class data, Represents the first Data access frequency of class data, Represents the first Data access duration of class data, Represents the first Number of data access interruptions for class data, Represents the first The reference data query response time of class data, Represents the first The reference data access frequency of the class data, Represents the first The reference data access duration of the class data, Represents the first The number of interruptions in accessing reference data of class data, Represents the first The query response time score of the class data, Represents the first The access frequency score of the class data, Represents the first The access duration score of the class data, Represents the first The number of interruptions in accessing class data, Expressed as a natural constant.

[0067] In this embodiment, a short data query response time means that the database can quickly respond to the user's request, which may increase the user's data access frequency. Conversely, if the query response time is long, the user may reduce the data access frequency due to the long waiting time.

[0068] Data query response time is part of data access time. A shorter query response time helps shorten the overall data access time. If the query response time is too long, it may cause the user-perceived data access time to become longer.

[0069] High data access frequency may mean that users need to access the database frequently, which may increase the overall data access time. However, if each access can be completed quickly (that is, the query response time is short), the overall data access time may not increase significantly.

[0070] The number of data access interruptions will affect the user's data access experience and may cause users to reduce the frequency of data access. Too many interruptions may also cause each data access to take longer, because users need to wait for the interruption to be resolved before they can continue accessing.

[0071] The instability of data query response time may lead to deviations in user experience, such as sometimes the response is fast and sometimes it is slow. There may be significant differences in the frequency of data access for different users or in different time periods, which reflects the diversity and variability of user needs. Due to the influence of multiple factors such as query response time, data volume, and user behavior, there may be large deviations in data access time. The number and frequency of data access interruptions reflect the stability and reliability of the database system and the stability of the network environment. In summary, there is a close correlation and influence between data query response time, data access frequency, data access time, and data access interruptions. The obtained data access real-time deviation index reflects the degree of deviation of data access, which is conducive to more accurate access adjustment according to the degree of deviation of data access.

[0072] Therefore, the larger the data preprocessing real-time deviation index and the data storage real-time deviation index are, the greater the impact on data access. When the data query response time, data access frequency, data access time, and data access interruption times are respectively greater than the reference data query response time, reference data access frequency, reference data access time, and reference data access interruption times, the query response time score, access frequency score, access time score, and access interruption times score are greater, which in turn leads to a larger data access real-time deviation index, a larger data access deviation, and a greater need for access adjustment, and vice versa.

[0073] Access adjustments include: Regarding data query response time: optimize database query statements to improve query efficiency; increase server resources, such as CPU, memory, etc., to cope with high load conditions; optimize network configuration to reduce network latency. Regarding data access frequency: monitor and identify malicious attacks or crawler behaviors, and take corresponding security measures; analyze user behavior patterns, optimize system functions and user experience, and improve user activity. Regarding data access duration: monitor and analyze long-term data access to understand users' specific needs and behavior patterns; optimize system architecture and data processing procedures to improve data access speed and processing efficiency. Regarding the number of data access interruptions: strengthen system stability monitoring, promptly detect and handle hardware / software failures; introduce redundancy mechanisms and backup strategies to improve the system's fault tolerance and recovery speed.

[0074] The technical solution in the above-mentioned embodiment of the present application analyzes the data preprocessing real-time deviation index through preprocessing deviation data and determines whether to perform preprocessing adjustment. After the preprocessing adjustment is qualified, the data storage real-time deviation index is analyzed according to the storage deviation data and determines whether to perform storage adjustment. After the storage adjustment is qualified, the data access real-time deviation index is analyzed according to the access deviation data and determines whether to perform access adjustment, thereby realizing the improvement of the real-time performance of network personalized recommendation using the effective weighted density peak clustering analysis method, and effectively solving the problem of poor real-time performance of network personalized recommendation using the effective weighted density peak clustering analysis method in the prior art.

[0075] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An effective weighted density peak clustering analysis method based on rough evidence theory, characterized in that: The following steps are involved: Obtain preprocessing deviation data corresponding to the data to be clustered, analyze the data preprocessing real-time deviation index based on the preprocessing deviation data, and determine whether to perform preprocessing adjustment; The data to be clustered after preprocessing and adjustment is stored, the corresponding storage deviation data is obtained, and the data storage real-time deviation index is analyzed according to the storage deviation data to determine whether to perform storage adjustment; Monitor the access situation corresponding to the data to be clustered and analyzed after the storage adjustment is qualified, obtain the corresponding access deviation data, analyze the data access real-time deviation index based on the access deviation data, and determine whether to adjust the access.

2. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 1, characterized in that: The preprocessing deviation data includes: data acquisition time, data preprocessing delay, CPU usage during preprocessing, data simplification time and data division time; The data preprocessing real-time deviation index is obtained by processing the preprocessing deviation data and the reference preprocessing deviation data; The data preprocessing real-time deviation index is used to reflect the deviation degree of preprocessing of the data to be clustered; The reference preprocessing deviation data includes: reference data acquisition duration, reference data preprocessing delay, CPU usage during reference preprocessing, reference data simplification duration and reference data division duration.

3. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 2, characterized in that: The specific method for obtaining the data preprocessing real-time deviation index is as follows: The data acquisition time and the reference data acquisition time are processed to obtain a time score; Processing the data preprocessing delay and the reference data preprocessing delay to obtain a preprocessing delay score; The CPU usage rate in the preprocessing process is processed with the CPU usage rate in the reference preprocessing process to obtain a CPU usage rate score; Process the data simplification duration and the reference data simplification duration to obtain a simplification duration score; Process the data partition duration and the reference data partition duration to obtain a partition duration score; The data preprocessing real-time deviation index is obtained based on the duration score, preprocessing delay score, CPU usage score, simplification duration score and partitioning duration score.

4. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 3, characterized in that: The specific limiting expression of the data preprocessing real-time deviation index is: ； In the formula, It is represented as the real-time deviation index of data preprocessing for the data to be clustered. Expressed as The duration score of a preset time period, , Indicates the number of the preset time period. Expressed as the total number of preset time periods, Expressed as The fraction of preprocessing delays for a preset time period, Expressed as The CPU usage score for a preset time period. Expressed as The simplified duration score of a preset time period, Expressed as The duration score of each preset time period.

5. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 1, characterized in that: The specific process of determining whether to perform preprocessing adjustment is as follows: Compare the data preprocessing real-time deviation index with the preprocessing deviation threshold preset in the database; When the data preprocessing real-time deviation index is less than the preprocessing deviation threshold preset in the database, no preprocessing adjustment is performed; When the data preprocessing real-time deviation index is not less than the preprocessing deviation threshold preset in the database, preprocessing adjustment is performed, and the preprocessing adjustment includes data cleaning deviation adjustment and rough set theory application deviation adjustment. When the data preprocessing real-time deviation index monitored after the preprocessing adjustment is less than the preprocessing deviation threshold preset in the database, the preprocessing adjustment is stopped.

6. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 1, characterized in that: The storage deviation data includes: preprocessing real-time deviation index of qualified data after preprocessing adjustment, data storage duration, data storage speed, data time complexity and data space complexity; The data storage real-time deviation index is obtained by processing the storage deviation data and the reference storage deviation data; The data storage real-time deviation index is used to reflect the degree of deviation of the data storage to be clustered and analyzed; The reference storage deviation data includes: reference data storage duration, reference data storage speed, reference data time complexity and reference data space complexity.

7. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 6, characterized in that: The specific method for obtaining the data storage real-time deviation indicator is: A storage duration score is obtained according to the relative relationship between the data storage duration and the reference data storage duration; A storage speed score is obtained according to the relative relationship between the data storage speed and the reference data storage speed; The time complexity score is obtained according to the relative relationship between the data time complexity and the reference data time complexity; The spatial complexity score is obtained according to the relative relationship between the data spatial complexity and the reference data spatial complexity; The data storage real-time deviation index is obtained based on the preprocessing real-time deviation index, storage duration score, storage speed score, time complexity score and space complexity score of the data after preprocessing adjustment.

8. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 1, characterized in that: The specific process of determining whether to perform storage adjustment is as follows: Compare the data storage real-time deviation index with the storage deviation threshold preset in the database to determine whether the data storage real-time deviation index is less than the storage deviation threshold preset in the database; When the data storage real-time deviation index is less than the storage deviation threshold preset in the database, no storage adjustment is performed; When the data storage real-time deviation index is not less than the storage deviation threshold preset in the database, storage adjustment is performed; when the data storage real-time deviation index monitored after storage adjustment is less than the storage deviation threshold preset in the database, storage adjustment is stopped.

9. The effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 1, characterized in that: The access deviation data includes: the preprocessing real-time deviation index of data after preprocessing adjustment, the data storage real-time deviation index after storage adjustment, the data query response time, the data access frequency, the data access time and the number of data access interruptions; The data access real-time deviation index is obtained by processing access deviation data and reference access deviation data; The data access real-time deviation index is used to reflect the degree of access deviation of the data to be clustered; The reference access deviation data includes: reference data query response time, reference data access frequency, reference data access time and reference data access interruption times.

10. An effective weighted density peak clustering analysis method based on rough evidence theory as claimed in claim 9, characterized in that: The specific method for obtaining the data access real-time deviation indicator is: The query response time score is obtained by processing the relative relationship between the data query response time and the reference data query response time; The access frequency score is obtained by processing the relative relationship between the data access frequency and the reference data access frequency; The access time score is obtained by processing the relative relationship between the data access time and the reference data access time; The access interruption number score is obtained by processing the relative relationship between the data access interruption number and the reference data access interruption number; The data access real-time deviation index is obtained based on the data preprocessing real-time deviation index after preprocessing adjustment, the data storage real-time deviation index after storage adjustment, and the query response time score, access frequency score, access time score and access interruption number score obtained by processing the access deviation data and referring to the access deviation data.

Images