Prediction method and system for service data analysis platform

Abstract

The invention belongs to the technical field of data processing and analysis, and provides a prediction method and system for a business data analysis platform, and the method comprises the steps: collecting log data from logs of different businesses; performing feature extraction on each piece of business data by using a first model; performing association analysis on the first business feature set by using a second model; updating the service data and the first service feature set in real time, and when the fluctuation of a first feature of a first service in the first service feature set exceeds a preset value, updating the weight of the first feature in the first model and the second model; and updating the first service feature set by using the first model after the weight is updated, performing association analysis on the updated first service feature set by using the second model after the weight is updated, and displaying a result of the association analysis, the result of the association analysis comprising the first feature.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing and analysis, and specifically relates to a prediction method and system for a business data analysis platform. Background Art

[0002] With the booming development of cloud computing technology, cloud service centers have become the core platform for enterprises and organizations to conduct business operations and data processing. Cloud service centers integrate multiple businesses such as computing, storage, networking, and security to provide customers with flexible and efficient resource allocation and service support. Due to the diversity of cloud services and the complexity of customer needs, cloud service centers need to deal with a large number of different types of businesses, including but not limited to: virtual machine deployment, container management, database services, content distribution network (CDN), artificial intelligence services, etc.

[0003] In the current market environment, customers' business needs and usage patterns are constantly changing. For example, some customers may increase their demand for computing resources in a specific period of time, or some new business functions are introduced into the cloud service center due to market trends. These changes cause the characteristics of business data to be highly dynamic and uncertain. Traditional static data analysis and prediction methods cannot effectively adapt to this rapidly changing business environment.

[0004] Cloud service centers need to dynamically monitor business data to timely understand the operation status, resource usage and potential risks of each business. This is crucial to ensure service quality, optimize resource allocation and improve customer satisfaction. However, existing technical methods have the following shortcomings in processing dynamic monitoring of multiple business data in cloud service centers:

[0005] Traditional data analysis models are usually based on pre-defined feature sets and cannot automatically identify and incorporate emerging business features. When customers introduce new services or the characteristics of existing services change, the model's prediction accuracy will drop significantly.

[0006] The cloud service center includes multiple businesses. Traditional methods often analyze a single business and lack the ability to comprehensively process multi-business data, making it impossible to fully grasp the overall business dynamics.

[0007] Due to the fixed nature of model structure and parameters, traditional methods have a slow update speed when faced with changes in data distribution or the introduction of new features, and are unable to respond to business changes in a timely manner, resulting in delayed prediction results.

[0008] Fixed feature extraction and model training methods limit the mining of potential connections and patterns between data, fail to fully utilize the deep information contained in business data, and affect the identification of business trends and anomalies.

[0009] When business data features fluctuate or new features emerge, the model cannot automatically adjust the weights of the features, and key features may be ignored, affecting the accuracy and reliability of the prediction.

[0010] In order to meet the above challenges and satisfy the cloud service center's dynamic monitoring needs for business data, a new prediction method is urgently needed to deeply explore the potential rules and patterns among data to support the early identification of business changes and anomalies. Summary of the invention

[0011] In order to solve the problems in the prior art, the present invention provides a prediction method for a business data analysis platform, the method comprising the following steps:

[0012] Collect log data from logs of different businesses, perform data cleaning, preprocessing and integration on the log data, and obtain business data corresponding to each business;

[0013] Using the first model to extract features from each of the business data, to obtain a first business feature set, wherein each business in the first business feature set corresponds to a plurality of features;

[0014] Performing association analysis on the first service feature set using a second model, and displaying results of the association analysis, wherein the results of the association analysis include associated features and corresponding services;

[0015] updating the service data and the first service feature set in real time, and when a first feature of a first service in the first service feature set fluctuates beyond a preset value, updating a weight of the first feature in the first model and the second model;

[0016] The first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, wherein the result of the association analysis includes the first feature.

[0017] Furthermore, the first model is a deep autoencoder.

[0018] Furthermore, the second model is an association rule mining model.

[0019] Further, updating the weight of the first feature in the first model includes:

[0020] In the input layer of the neural network, for the first feature Multiply by a weight factor :

[0021]

[0022] in, Represents the feature after multiplying by the weight coefficient;

[0023] when When the fluctuation exceeds the threshold, increase The value of

[0024]

[0025] in, is the updated weight, is the old weight, Determined by the degree of fluctuation.

[0026] Further, updating the weight of the first feature in the second model includes:

[0027] In the process of association rule mining, each feature is given a weight and its support and confidence calculations are adjusted;

[0028] For the itemset containing the first feature, calculate the weighted support:

[0029]

[0030] Increase the weight of the first feature , increasing the importance of association rules containing this feature.

[0031] Another aspect of the present invention further provides a prediction system for a business data analysis platform, the system comprising the following modules:

[0032] The acquisition module is used to collect log data from logs of different businesses, clean, pre-process and integrate the log data, and obtain business data corresponding to each business;

[0033] A feature extraction module, configured to extract features from each of the service data using a first model to obtain a first service feature set, wherein each service in the first service feature set corresponds to a plurality of features;

[0034] an association analysis module, configured to perform an association analysis on the first service feature set using a second model, and display a result of the association analysis, wherein the result of the association analysis includes associated features and corresponding services;

[0035] An updating module is used to update the business data and the first business feature set in real time. When the fluctuation of the first feature of the first business in the first business feature set exceeds a preset value, the weight of the first feature in the first model and the second model is updated; the first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, and the result of the association analysis includes the first feature.

[0036] Furthermore, the first model is a deep autoencoder.

[0037] Furthermore, the second model is an association rule mining model.

[0038] Further, updating the weight of the first feature in the first model includes:

[0039] In the input layer of the neural network, for the first feature Multiply by a weight factor :

[0040]

[0041] in, Represents the feature after multiplying by the weight coefficient;

[0042] when When the fluctuation exceeds the threshold, increase The value of

[0043]

[0044] in, is the updated weight, is the old weight, Determined by the degree of fluctuation.

[0045] Further, updating the weight of the first feature in the second model includes:

[0046] In the process of association rule mining, each feature is given a weight and its support and confidence calculations are adjusted;

[0047] For the itemset containing the first feature, calculate the weighted support:

[0048]

[0049] Increase the weight of the first feature , increasing the importance of association rules containing this feature.

[0050] The prediction method and system of the business data analysis platform proposed in the present invention have the following beneficial effects:

[0051] It can extract and update features from log data of different businesses in real time to adapt to the dynamic changes of business data features. It can timely identify newly added features and dynamically adjust the feature list and weights to ensure that the model always focuses on the most important features.

[0052] When key features fluctuate significantly, their weights in the model are adjusted in real time to enhance the model's sensitivity to data changes. By dynamically adjusting model parameters and structure, the model can more effectively utilize the information of important features and improve the accuracy and stability of predictions.

[0053] By using the important feature changes identified in the fast-reacting business, we can adjust the feature weights in other slow-reacting businesses to achieve cross-business knowledge transfer. We can also identify potential changes and risks in the slow-reacting business in advance to improve the overall prediction performance.

[0054] Through iterative training and association analysis, we can capture potential associations and patterns among data and reveal hidden business rules. We can provide data support for business strategy formulation and optimization and promote scientific decision-making.

[0055] By identifying and resolving potential problems in advance, we can improve service quality and enhance user satisfaction and loyalty. We can also warn and identify business changes in advance to help companies optimize resource allocation and avoid unnecessary waste. We can improve the overall efficiency of business operations through real-time adjustment and optimization of models.

[0056] The model has the ability to continuously learn new data and new features, and maintain adaptability to changes in the business environment. The model has shown good prediction performance in different business scenarios and has strong generalization capabilities.

[0057] Deeply explore the deep-level value in massive business log data and improve the benefits of data assets. Discover new business opportunities: Through association analysis and pattern recognition, explore new business opportunities and market demands.

[0058] By monitoring changes in key features, potential business risks and abnormal situations can be warned in a timely manner. Monitoring of high-risk associations can be strengthened to ensure business and data security.

[0059] The present invention significantly improves the prediction accuracy and responsiveness of the model through dynamic feature management, real-time model adjustment and cross-business collaborative optimization. It can not only deeply mine the potential laws and patterns in the data, support business decision-making and risk management, but also improve customer satisfaction and operational efficiency, and maximize the value of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0062] Below, the invention is preferably described in conjunction with the accompanying drawings and specific implementation methods.

[0063] This embodiment solves the above problem through the following steps:

[0064] In one embodiment, reference Figure 1 The present invention provides a prediction method for a business data analysis platform, aiming to solve the data analysis challenges brought about by the coexistence of multiple businesses and the continuous changes in customer businesses in a cloud service center.

[0065] First, log data is collected from logs of different businesses, and the log data is cleaned, preprocessed, and integrated to obtain business data corresponding to each business, including:

[0066] Collect log data from different business logs:

[0067] Identify the different business systems that need to be analyzed, such as e-commerce platforms, customer relationship management (CRM) systems, financial management systems, etc.

[0068] For each business, determine the type of logs it generates, such as access logs, transaction logs, error logs, etc.

[0069] Use data collection tools (such as Fluentd, Logstash) to collect logs in real time to capture the latest business dynamics.

[0070] For historical data or non-real-time data, log files are exported from the business system regularly for processing.

[0071] Ensure the security of data transmission and use encryption protocols and authentication mechanisms to prevent data leakage or loss.

[0072] For example, Fluentd is used to collect user access logs and transaction logs in real time on the server of an online shopping platform and transmit them to centralized storage through a secure channel.

[0073] Detect and process missing values ​​in log data, and fill or delete incomplete data records.

[0074] Identify and address outliers in your data, such as unreasonable transaction amounts or incorrect timestamps.

[0075] Remove duplicate log entries caused by system errors or duplicate records to ensure data uniqueness.

[0076] Convert log data from different business systems into a unified format (such as JSON, CSV) to facilitate subsequent processing.

[0077] Extract important fields from the logs and rename the fields to unify the naming convention.

[0078] Convert timestamps from different systems to a unified time zone and format to ensure consistent order of events.

[0079] Convert the timestamp field in the logs of the cloud storage service to UTC time and format it in the format of YYYY-MM-DD HH:MM.

[0080] Convert text categorical variables to numerical encoding (such as One-Hot encoding) for easier model processing.

[0081] Normalize or standardize numerical data to eliminate the impact of dimension.

[0082] For example, converting product categories of an e-commerce platform from text (such as "electronics", "clothing") into numerical codes.

[0083] Merge log data from different businesses according to predefined keys (such as user ID, session ID) to form a complete user behavior record.

[0084] Handle data inconsistencies between different data sources to ensure accurate matching of data.

[0085] Through user ID, the browsing history of online shopping platforms and the payment records of mobile payment services are merged to form complete shopping and payment behavior data.

[0086] The cleaned and preprocessed data are grouped according to the business system to obtain the business data set for each business.

[0087] The integrated data is divided into three data sets according to the business source fields: "online shopping business data", "mobile payment business data" and "cloud storage business data".

[0088] Store the data of each business in the corresponding database table or subject domain of the data warehouse to facilitate subsequent query and analysis.

[0089] Perform version management on business data, record the time and method of data processing, and ensure data traceability.

[0090] Verify the integrity of each business data set to ensure that key fields are not missing.

[0091] Check whether the data complies with business rules, such as transaction amount cannot be negative and order status must be within a predefined range.

[0092] Use the first model to extract features from each of the business data to obtain a first business feature set, wherein each business in the first business feature set corresponds to multiple features.

[0093] After cleaning, preprocessing and integrating the log data of different businesses, it is necessary to extract features from the data of each business in order to capture the key information and potential patterns in the business data. To this end, the first model is used to extract features from each business data to obtain the first business feature set. This feature set contains multiple features corresponding to each business, which can effectively represent the characteristics of the business data and provide a basis for subsequent association analysis and prediction.

[0094] The first model uses a deep autoencoder, which is an unsupervised deep learning model that is often used for feature extraction and dimensionality reduction of high-dimensional data. It consists of two parts, an encoder and a decoder, and can learn a low-dimensional representation of the data while retaining the main information of the data.

[0095] Feature extraction process:

[0096] Convert the preprocessed log data into numerical form. For example, convert text fields into word vectors and perform one-hot encoding on categorical data.

[0097] For example, for the "action type" field, possible actions (such as login, order, payment) are one-hot encoded.

[0098] In the model, the encoder consists of a multi-layer neural network that compresses high-dimensional input data into a low-dimensional latent space. The decoder reconstructs the output of the encoder and tries to restore the original input data. The bottleneck layer is the minimum dimensional layer between the encoder and the decoder, that is, the extracted feature vector.

[0099] During the model training process, reconstruction error (such as mean square error) is used as the loss function to minimize the difference between the input data and the reconstructed data. Unsupervised training is performed using business data so that the model can learn the main features of the data.

[0100] After the model training is completed, the business data is input, and the output of the bottleneck layer is extracted as the feature vector through the encoder part. The feature vectors of all business data are aggregated to form the first business feature set.

[0101] Assume that the following features are extracted for the business data of "online shopping platform": Table 1 Business data table of "Online shopping platform"

[0102] Record ID User ID Timestamp Operation Type Product ID category price Payment Methods 1 1001 2023-11-17 10:20:45 Place an order 5001 Electronics 999.99 credit card 2 1002 2023-11-17 11:15:30 View Products 6001 clothing 199.99 (null) 3 1003 2023-11-17 12:05:15 add to the cart 7001 Home Furnishings 49.99 (null)

[0103] Feature vector extracted by the encoder (bottleneck layer output): Table 2 Feature vector table extracted by encoder

[0104] Record ID Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 1 0.82 0.15 0.05 0.70 0.60 2 0.30 0.65 0.05 0.20 0.25 3 0.50 0.40 0.10 0.35 0.30

[0105] Feature meaning:

[0106] Feature 1 (Purchase Intention): The higher the value, the stronger the user's purchase intention.

[0107] Feature 2 (Browsing Activity): The higher the value, the more frequently users browse products.

[0108] Feature 3 (High Price Sensitivity): The higher the value, the more users pay attention to high-priced products.

[0109] Feature 4 (Category Preference): The degree of association with a specific product category.

[0110] Feature 5 (payment method preference): The higher the value, the more likely the user is to use a certain payment method.

[0111] Use the second model to perform association analysis on the first business feature set, and display the results of the association analysis, which include associated features and corresponding businesses.

[0112] After extracting the features of each business data and obtaining the first business feature set, it is necessary to further explore the relationship between these features and their association with the corresponding business. In this step, the second model is used to perform association analysis on the first business feature set. This process aims to discover the potential association rules and patterns between features, reveal which features often appear together, which features have strong associations, and how these associations affect different businesses. By displaying the results of the association analysis, it is possible to understand the associated features and their corresponding businesses, providing strong support for business optimization and decision-making.

[0113] Select the association rule mining model as the second model, specifically the Apriori algorithm or the FP-Growth algorithm. Association rule mining is an unsupervised machine learning method that aims to discover frequent item sets and association rules from large data sets. It is widely used in market basket analysis, recommendation systems and other fields.

[0114] Since the association rule mining model usually processes discrete categorical data, if there are continuous numerical features in the first business feature set, they need to be discretized. For example, they can be divided into three categories of "high", "medium" and "low" according to the distribution of feature values.

[0115] Each data record is considered as a transaction, and the items in the transaction consist of features and their values.

[0116] Assume that part of the data in the first service feature set is as follows: Table 3 Partial data table of the first service feature set

[0117] Record ID Purchase intention Browsing activity High price sensitivity Category Preference Payment method preference 1 0.82 0.15 0.05 Electronics credit card 2 0.30 0.65 0.05 clothing Mobile Payment 3 0.50 0.40 0.10 Home Furnishings unknown

[0118] Discretization processing:

[0119] Purchase intention:

[0120] High (≥0.7)

[0121] Medium (0.4–0.7)

[0122] Low (<0.4)

[0123] Browsing activity:

[0124] High (≥0.6)

[0125] Medium (0.3–0.6)

[0126] Low (<0.3)

[0127] High price sensitivity:

[0128] Sensitive (≥0.7)

[0129] Average (0.4–0.7)

[0130] Insensitive (<0.4)

[0131] The converted transaction data: Table 4 Transaction data table

[0132] Record ID Transaction Item 1 {Purchase intention = high, Browsing activity = low, High price sensitivity = insensitive, Category preference = electronic products, Payment method preference = credit card} 2 {Purchase intention = low, Browsing activity = high, High price sensitivity = insensitive, Category preference = clothing, Payment method preference = mobile payment} 3 {Purchase intention = Medium, Browsing activity = Medium, High price sensitivity = Insensitive, Category preference = Household goods, Payment method preference = Unknown}

[0133] The specific process includes:

[0134] Generate frequent itemsets:

[0135] Calculate the support of an item: The support indicates the proportion of a certain item set contained in the transaction set.

[0136] Filter frequent item sets: According to the set minimum support threshold, retain the item sets whose support is not less than the threshold.

[0137] Generate association rules:

[0138] Calculate the confidence of the rule: The confidence indicates the proportion of transactions that contain the rule's consequent among transactions that contain the rule's antecedent.

[0139] Filter strong association rules: According to the set minimum confidence threshold, retain the rules whose confidence is not less than the threshold.

[0140] Evaluation rule quality:

[0141] Lift: Measures the independence of the antecedent and consequent of a rule. The greater the lift, the stronger the correlation.

[0142] Leverage: measures the difference between the probability that the antecedent and consequent of a rule occur simultaneously and the probability that they occur independently.

[0143] Parameter settings:

[0144] Minimum support threshold: For example, set it to 0.3, which means that the item set appears in at least 30% of the transactions.

[0145] Minimum confidence threshold: For example, set it to 0.6, indicating that the confidence of the rule is at least 60%.

[0146] The example belongs to, and the associated result can be similar to the following:

[0147] If {high price sensitivity = insensitive, purchase intention = high}, then {category preference = electronic products}

[0148] If {browsing activity = high}, then {purchase intention = low}

[0149] After determining the association relationship, a visual display can be performed:

[0150] Optionally, the association relationship between features is displayed, with nodes representing features, lines representing association rules, and line thickness indicating association strength.

[0151] Optionally, the degree of association between features is displayed, with the color depth indicating the support or confidence level.

[0152] The service data and the first service feature set are updated in real time, and when a first feature of a first service in the first service feature set fluctuates beyond a preset value, a weight of the first feature in the first model and the second model is updated.

[0153] In the prediction method of the present invention, in order to ensure that the model can reflect the changes in business data in a timely manner, it is necessary to update the business data and the first business feature set in real time. When it is detected that the fluctuation of a key feature (referred to as the first feature) of a business (referred to as the first business) in the first business feature set exceeds a preset threshold, it is necessary to update the weight of the feature in the first model and the second model. This process is intended to increase the sensitivity of the model to the change of the feature, thereby improving the accuracy of the prediction.

[0154] First, it is necessary to continuously monitor the value change of the first feature to determine whether it exceeds a preset threshold.

[0155] Fluctuation detection conditions:

[0156] if , it is considered that the first characteristic has fluctuated significantly.

[0157] in represents the current value of the first feature, The historical average value of the first feature, T is the preset value, , is the historical standard deviation of the first characteristic, is a constant, which can be 2 or 3.

[0158] The first model is used to extract features from business data, such as neural network models such as deep autoencoders. When the first feature fluctuates significantly, the weight of the feature in the model needs to be adjusted to enhance the model's attention to the feature.

[0159] In the input layer of the neural network, for the first feature Multiply by a weight factor :

[0160]

[0161] in, Represents the feature after multiplying by the weight coefficient;

[0162] when When the fluctuation exceeds the threshold, increase The value of

[0163]

[0164] in, is the updated weight, is the old weight, Determined by the degree of fluctuation.

[0165] The second model is used to perform association analysis on the feature set, such as an association rule mining model. In this model, the weight of the first feature in the association analysis needs to be adjusted to highlight its importance in the association rule.

[0166] In the process of association rule mining, a weight is assigned to each feature to adjust its support and confidence calculations.

[0167] For the itemset containing the first feature, calculate the weighted support:

[0168]

[0169] Increase the weight of the first feature , increasing the importance of association rules containing this feature.

[0170] The increase in weighted support indicates that the importance of the association rule has increased. In the subsequent association analysis, the rule is more likely to be selected as a strong association rule.

[0171] By monitoring the fluctuation of the first feature in real time, when it exceeds the preset threshold, the weight of the feature in the model is adjusted in time. In the first model, by increasing the weight of the input layer, the impact of the feature on the model output is amplified to ensure that the model can more keenly capture the changes in the feature. In the second model, through weighted association rule mining, the importance of the association rules containing the feature is enhanced to better reflect the potential rules in the business data.

[0172] By adjusting the weights, the model can more effectively utilize the information of important features and improve the predictive performance of the feature in other businesses. This advantage comes from the fact that different businesses may react differently to market changes and feature fluctuations:

[0173] Some businesses respond faster to market changes, user behavior or other influencing factors, and their characteristic fluctuations are more obvious; while other businesses may respond slower and their characteristic changes lag behind.

[0174] In fast-response businesses, when a significant fluctuation in a feature is detected and its weight is adjusted, this adjustment can help the model more sensitively capture the changing pattern of the feature.

[0175] By paying more attention to this feature in the model, the model can apply the change trends learned from fast-response businesses to slow-response businesses and identify potential changes in advance.

[0176] This cross-business weight adjustment strategy enables the model to more comprehensively understand the impact of features and improve prediction accuracy and robustness in different business scenarios.

[0177] Assume that among multiple services in a cloud service center, service A (for example, real-time communication service) is very sensitive to changes in network latency, and fluctuations in latency will immediately affect user experience. Service B (for example, file storage service) is less sensitive to network latency, and changes in user experience may lag behind changes in latency.

[0178] In business A, a significant fluctuation in network latency (the first feature) was detected. The model adjusted the weights to increase attention to network latency and optimize prediction and response in a timely manner.

[0179] Apply adjustments to business B: Because the model has strengthened its focus on network latency, even if the user experience of business B has not been significantly affected, the model can predict potential problems in advance and remind operations personnel to take preventive measures.

[0180] The first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, wherein the result of the association analysis includes the first feature.

[0181] After adjusting the weight of the first feature in the model, the first business feature set needs to be updated using the first model with the updated weight. Next, the updated first business feature set is subjected to association analysis using the second model with the updated weight, and the results of the association analysis are displayed. This result will include the first feature of interest, reflecting the impact of the feature weight adjustment on the model output and association relationship.

[0182] Update the first service feature set using the first model after updating the weights:

[0183] In the previous step, the weight of the first feature in the model has been adjusted according to its fluctuation. The latest business data is input into the updated first model. The model extracts the updated feature vector through forward propagation to form a new first business feature set. As the weight of the first feature increases, the extracted feature vector will emphasize the information of the first feature more. The updated first business feature set contains new feature vectors, reflecting the impact of the latest business data and weight adjustment.

[0184] Use the second model with updated weights to perform association analysis on the updated first service feature set:

[0185] According to the weight adjustment of the first feature, the corresponding parameters in the model are updated to increase the importance of the first feature in the association analysis. The new feature set is used as the input of the association analysis. The weighted association rule mining algorithm is adopted to consider the feature weight. Since the weight of the first feature is increased, the result of the association analysis includes the first feature.

[0186] For example, a cloud service center provides a variety of business services, including:

[0187] Business A: Live video streaming service

[0188] Users can watch high-definition real-time video online, which is very sensitive to network bandwidth and latency.

[0189] User viewing time and number of freezes are key indicators.

[0190] Respond quickly to changes in network conditions; network fluctuations immediately affect user experience.

[0191] Business B: Cloud storage backup service

[0192] Users automatically back up data in the background and are insensitive to short-term fluctuations in network bandwidth and latency.

[0193] Data backup success rate and speed are the main concerns.

[0194] It reacts slowly to changes in network conditions. Short-term network fluctuations have little impact on backup, but long-term fluctuations may cause problems.

[0195] Scenario description:

[0196] Recently, the cloud service center has detected significant fluctuations in network bandwidth, resulting in a decline in overall network performance.

[0197] Significant fluctuations in key characteristics were detected in Business A:

[0198] The first characteristic: network bandwidth

[0199] Observed changes:

[0200] The network bandwidth dropped from an average of 100 Mbps to 60 Mbps, exceeding the preset fluctuation threshold.

[0201] Users of service A began to frequently encounter problems such as video freezes and image quality degradation, seriously impairing the user experience.

[0202] Adjust the weight of the model in business A:

[0203] Increase the weight of the network bandwidth feature:

[0204] In the first model (for feature extraction) and the second model (for association analysis), the weight of the network bandwidth feature is increased.

[0205] Model adjustment effect:

[0206] The model can more sensitively capture changes in network bandwidth and more accurately predict user viewing time and the number of freezes.

[0207] Apply the adjustments to Business B:

[0208] Impact of network bandwidth on service B:

[0209] Cloud storage backup services are not sensitive to short-term network bandwidth fluctuations, but long-term bandwidth drops may cause backup delays or failures.

[0210] Adjust the model weight of business B:

[0211] Similarly, the weight of the network bandwidth feature in business B is increased so that the model can pay attention to the potential impact of bandwidth fluctuations on backup services in advance.

[0212] The model can predict problems such as slow backups and increased failure rates before they become apparent.

[0213] The results for business A are:

[0214] The model accurately predicted a decline in user experience, and the operations team took immediate action:

[0215] Optimize video encoding compression rate and reduce bandwidth requirements.

[0216] Negotiate with network service providers to adjust network resource allocation.

[0217] The results for business B are:

[0218] The model predicts possible future backup delays and failures, allowing the operations team to take proactive action:

[0219] Notify users of possible backup delays and adjust backup plans to network idle periods.

[0220] Add data compression and incremental backup strategies to reduce bandwidth usage.

[0221] The model adjustment of service A makes the prediction of user experience more timely and accurate. The weight adjustment of network bandwidth characteristics is applied to service B, so that its model can perceive potential problems in advance. Service B takes preventive measures before the problem occurs, avoiding backup failures and user complaints. The operation team can reasonably arrange network resources and optimize strategies to improve the overall service quality.

[0222] On the other hand, the present invention also provides a prediction system for a business data analysis platform, comprising:

[0223] The acquisition module is used to collect log data from logs of different businesses, clean, pre-process and integrate the log data, and obtain business data corresponding to each business;

[0224] A feature extraction module, configured to extract features from each of the service data using a first model to obtain a first service feature set, wherein each service in the first service feature set corresponds to a plurality of features;

[0225] an association analysis module, configured to perform an association analysis on the first service feature set using a second model, and display a result of the association analysis, wherein the result of the association analysis includes associated features and corresponding services;

[0226] An updating module is used to update the business data and the first business feature set in real time. When the fluctuation of the first feature of the first business in the first business feature set exceeds a preset value, the weight of the first feature in the first model and the second model is updated; the first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, and the result of the association analysis includes the first feature.

[0227] The prior art mentioned in the above background technology section and specific embodiment section of the present invention can be used as part of the present invention to understand the meaning of some technical features or parameters.

Claims

1. A prediction method for a business data analysis platform, characterized in that: The method comprises the following steps: Collect log data from logs of different businesses, perform data cleaning, preprocessing and integration on the log data, and obtain business data corresponding to each business; Using the first model to extract features from each of the business data, to obtain a first business feature set, wherein each business in the first business feature set corresponds to a plurality of features; Performing association analysis on the first service feature set using a second model, and displaying results of the association analysis, wherein the results of the association analysis include associated features and corresponding services; updating the service data and the first service feature set in real time, and when a first feature of a first service in the first service feature set fluctuates beyond a preset value, updating a weight of the first feature in the first model and the second model; The first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, wherein the result of the association analysis includes the first feature.

2. The prediction method of the business data analysis platform according to claim 1, characterized in that: The first model is a deep autoencoder.

3. The prediction method of the business data analysis platform according to claim 1, characterized in that: The second model is an association rule mining model.

4. The prediction method of the business data analysis platform according to claim 1, characterized in that: Updating the weight of the first feature in the first model includes: In the input layer of the neural network, for the first feature Multiply by a weight factor : ； in, Represents the feature after multiplying by the weight coefficient; when When the fluctuation exceeds the threshold, increase The value of ； in, is the updated weight, is the old weight, Determined by the degree of fluctuation.

5. The prediction method of the business data analysis platform according to claim 1, characterized in that: Updating the weight of the first feature in the second model includes: In the process of association rule mining, each feature is given a weight and its support and confidence calculations are adjusted; For the itemset containing the first feature, calculate the weighted support: ； Increase the weight of the first feature .

6. A prediction system for a business data analysis platform, characterized in that: The system includes the following modules: The acquisition module is used to collect log data from logs of different businesses, clean, pre-process and integrate the log data, and obtain business data corresponding to each business; A feature extraction module, configured to extract features from each of the service data using a first model to obtain a first service feature set, wherein each service in the first service feature set corresponds to a plurality of features; an association analysis module, configured to perform an association analysis on the first service feature set using a second model, and display a result of the association analysis, wherein the result of the association analysis includes associated features and corresponding services; An updating module, configured to update the service data and the first service feature set in real time, and when a fluctuation of a first feature of a first service in the first service feature set exceeds a preset value, update a weight of the first feature in the first model and the second model; The first business feature set is updated using the first model after the weight is updated, and the updated first business feature set is subjected to association analysis using the second model after the weight is updated, and the result of the association analysis is displayed, wherein the result of the association analysis includes the first feature.

7. The prediction system of the business data analysis platform according to claim 6, characterized in that: The first model is a deep autoencoder.

8. The prediction system of the business data analysis platform according to claim 6, characterized in that: The second model is an association rule mining model.

9. The prediction system of the business data analysis platform according to claim 6, characterized in that: Updating the weight of the first feature in the first model includes: In the input layer of the neural network, for the first feature Multiply by a weight factor : ； in, Represents the feature after multiplying by the weight coefficient; when When the fluctuation exceeds the threshold, increase The value of ； in, is the updated weight, is the old weight, Determined by the degree of fluctuation.

10. The prediction system of the business data analysis platform according to claim 6, characterized in that: Updating the weight of the first feature in the second model includes: In the process of association rule mining, each feature is given a weight and its support and confidence calculations are adjusted; For the itemset containing the first feature, calculate the weighted support: ； Increase the weight of the first feature .

Images