Cloud computing analysis method, device and equipment of SaaS (Software as Service) system and storage medium
Through distributed log collection and multi-dimensional real-time analysis, dynamic resource scheduling and the construction of a multi-dimensional indicator system, the problems of high log data processing complexity and untimely resource scheduling response in the cloud computing environment are solved, and the system performance is improved and efficiency, systematic index evaluation is achieved.
Patent Information
- Application Number
- CN202411318293.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-09-20
AI Technical Summary
The existing SaaS system faces the problems of high complexity in log data processing, untimely resource scheduling response, and lack of a systematic indicator evaluation system in the cloud computing environment, resulting in low log data utilization, difficult to detect abnormal behaviors in a timely manner, and low resource scheduling efficiency. .
Through distributed log collection and preprocessing, a centralized log aggregate index is established, a multi-dimensional real-time analysis model is input to generate abnormal behavior reports, determine user operation mode, perform dynamic resource scheduling, build a multi-cloud collaborative computing framework, and monitor fault detection in real time, establish a Multi-dimensional KPI index system.
It improves the system's log data utilization and analysis accuracy, improves resource scheduling efficiency, realizes accurate evaluation and optimization of system performance, and enhances the timely discovery and processing capabilities of abnormal behaviors.
Smart Images

Figure CN119201620A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud computing analysis, and in particular to a cloud computing analysis method, device, equipment and storage medium for a SaaS system. Background Art
[0002] With the widespread application of cloud computing technology, more and more companies choose to use Software as a Service (SaaS) systems to manage and operate their businesses. The flexibility and scalability of SaaS systems give them significant advantages in meeting diverse business needs. However, with the continuous growth of user scale and data volume, the management and performance optimization of SaaS systems face many challenges. In particular, in a distributed cloud computing environment, how to effectively collect and process a large amount of system log data, and how to perform anomaly detection, resource scheduling, user behavior analysis, and system performance evaluation based on this data have become important issues in the operation of cloud computing platforms.
[0003] Traditional SaaS system log analysis methods usually rely on a single data source, or simply perform simple log storage and retrieval. This method often has difficulty coping with the needs of high concurrency and large-scale data processing in the face of complex distributed cloud computing environments, resulting in low log data utilization, difficulty in timely detection of system abnormal behavior, and inefficient resource scheduling. In addition, traditional methods usually lack in-depth analysis of user operation patterns and the construction of behavioral portraits, making it difficult to provide accurate user behavior predictions and system performance optimization recommendations.
[0004] In order to solve these problems, the industry has begun to explore how to build an efficient log collection, data processing and analysis framework in a cloud computing environment. By introducing distributed log collection, centralized log aggregation, multi-dimensional analysis models and dynamic resource scheduling strategies, the system's log data utilization and analysis accuracy can be greatly improved. However, these technologies still face problems such as high data processing complexity, untimely resource scheduling response and lack of a systematic indicator evaluation system in actual applications. Summary of the invention
[0005] The main purpose of the present invention is to solve the technical problems of the existing SaaS system, such as high complexity of cloud computing analysis data processing, untimely resource scheduling response, and lack of a systematic indicator evaluation system; A first aspect of the present invention provides a cloud computing analysis method for a SaaS system, the cloud computing analysis method for a SaaS system comprising: Perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; According to the full system log, a preset multi-dimensional real-time analysis model is input to obtain an abnormal behavior report, and a hierarchical storage strategy is planned for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; Determine a data lineage relationship diagram of the standardized log data based on the hierarchical query optimization structure, and determine a user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship diagram to obtain a user portrait; According to the abnormal behavior report and user portrait, dynamically allocate computing resources to obtain a resource scheduling strategy, and build a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster; The elastic computing cluster is monitored and fault detected in real time to obtain a cluster health status report, and a multi-dimensional KPI indicator system is constructed based on the full system log, abnormal behavior report, user portrait and cluster health status report, and the SaaS system is analyzed and evaluated through the multi-dimensional KPI indicator system.
[0006] Optionally, in a first implementation of the first aspect of the present invention, the distributed log collection and data preprocessing of the cloud computing nodes of the SaaS system to obtain standardized log data, and establishing a centralized log aggregation index based on the standardized log data to obtain a queryable full system log includes: Deploy lightweight log collection agents on the cloud computing nodes of the SaaS system to collect multi-source heterogeneous log data and obtain the original log stream; Performing timestamp normalization and field extraction on the original log stream to obtain structured log data, and storing the structured log data in a preset distributed log storage cluster; Index construction and optimization are performed on the structured log data in the distributed log storage cluster to obtain a full-text search index, and a distributed query engine is implemented based on the full-text search index to obtain a full amount of system logs that can be queried.
[0007] Optionally, in a second implementation of the first aspect of the present invention, the multi-dimensional real-time analysis model preset according to the full system log input is used to obtain an abnormal behavior report, and a hierarchical storage strategy is planned for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure, including: Perform time window segmentation and multi-dimensional feature extraction on the full system log to obtain a real-time feature vector stream, and apply a pre-trained multi-dimensional real-time analysis model to the real-time feature vector stream to perform event clustering and correlation analysis to obtain an abnormal event map; According to the abnormal event map, priority sorting and severity assessment are performed in combination with predefined business rules to obtain an abnormal behavior report; Performing data lifecycle analysis on the full system logs and abnormal behavior reports to obtain an access frequency attenuation curve, and performing cold and hot stratification and storage medium matching on the full system logs and abnormal behavior reports according to the access frequency attenuation curve to obtain a multi-level storage solution; According to the multi-level storage solution, the full system logs and abnormal behavior reports are compressed, encoded and stored in blocks to obtain a data storage structure, and based on the data storage structure, a distributed index and cache strategy is constructed to obtain a hierarchical query optimization structure.
[0008] Optionally, in a third implementation of the first aspect of the present invention, the determining of the data lineage relationship graph of the standardized log data based on the hierarchical query optimization structure, and determining the user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship graph, to obtain the user portrait includes: Applying a graph traversal algorithm to the hierarchical query optimization structure, tracing the data flow path, obtaining an initial data dependency relationship, and performing topological sorting and cycle detection on the initial data dependency relationship to obtain a data lineage relationship diagram of the standardized log data; Performing session identification and behavior serialization processing on the full system log to obtain user operation time series data, and applying a sequence pattern mining algorithm based on the user operation time series data and abnormal behavior reports to obtain an initial user operation pattern; Performing association analysis on the initial user operation mode and the data lineage relationship diagram, identifying key data nodes and operation paths, obtaining enhanced user operation modes, constructing user behavior feature vectors based on the enhanced user operation modes, and applying unsupervised clustering algorithms to obtain user grouping results; Perform multi-dimensional feature extraction and weight calculation on the user grouping results to obtain a user tag set, and apply a collaborative filtering algorithm based on the user tag set and historical interaction data to obtain an initial user portrait; A time series analysis method is applied to the initial user portrait to identify user behavior trends and periodic patterns to obtain a user portrait.
[0009] Optionally, in a fourth implementation of the first aspect of the present invention, dynamically allocating computing resources according to the abnormal behavior report and the user portrait to obtain a resource scheduling strategy, and constructing a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster includes: Based on the abnormal behavior report and user portrait, a multi-dimensional prediction of the future load of the SaaS system is performed to obtain a resource demand prediction result; Performing performance evaluation and utilization analysis on the existing computing resources of the SaaS system to obtain a resource status matrix, and applying a multi-objective optimization algorithm based on the resource status matrix and resource demand prediction results to perform resource allocation calculations to obtain a resource scheduling strategy; According to the resource scheduling strategy, a distributed consistency protocol is applied to the SaaS system to build a cross-cloud resource collaboration mechanism to obtain a multi-cloud collaborative computing framework; According to the multi-cloud collaborative computing framework, computing tasks are intelligently scheduled and load balanced to obtain an initial elastic computing cluster, and a fault tolerance mechanism and an automatic expansion and contraction strategy are applied to the initial elastic computing cluster to perform real-time resource adjustment to obtain an elastic computing cluster.
[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the performance evaluation and utilization analysis of the existing computing resources are performed to obtain a resource status matrix, and a multi-objective optimization algorithm is applied according to the resource status matrix and the resource demand prediction result to perform resource allocation calculation to obtain a resource scheduling strategy, including: Perform multi-dimensional testing on existing computing resources to obtain a resource performance indicator set, and construct a dynamic resource utilization curve based on the resource performance indicator set and historical load data; Applying a time series decomposition algorithm to the dynamic resource utilization curve, extracting trends, cycles and random components in the dynamic resource utilization curve to obtain a resource utilization pattern, and constructing a multidimensional resource state vector based on the resource utilization pattern and a resource performance indicator set to obtain a resource state matrix; Performing difference analysis on the resource status matrix and the resource demand forecast results, identifying resource surplus and shortage areas, obtaining a resource balance demand graph, and constructing a multi-objective optimization problem model based on the resource balance demand graph to obtain an optimization problem expression; An improved genetic algorithm is applied to the optimization problem expression to perform multiple generations of iterative optimization calculations to obtain a Pareto optimal solution set, and a resource scheduling strategy is generated based on the Pareto optimal solution set in combination with preset decision rules.
[0011] Optionally, in a sixth implementation of the first aspect of the present invention, real-time monitoring and fault detection are performed on the elastic computing cluster to obtain a cluster health status report, and a multi-dimensional KPI indicator system is constructed based on the full system log, abnormal behavior report, user portrait and cluster health status report. The analysis and evaluation of the SaaS system through the multi-dimensional KPI indicator system includes: Deploy distributed probes on the elastic computing cluster to collect real-time performance indicators and resource utilization data to obtain original monitoring data streams, and apply anomaly detection algorithms and fault diagnosis rules based on the original monitoring data streams to perform real-time fault identification and location to obtain a cluster health status report; Perform data fusion and feature extraction on the full system log, abnormal behavior report, user portrait and cluster health status report to obtain a multi-source heterogeneous data feature set, and apply the analytic hierarchy process and Delphi method based on the multi-source heterogeneous data feature set to build a multi-level KPI indicator system; Applying the Monte Carlo simulation method to the multi-level KPI indicator system, conducting large-scale data sampling and indicator sensitivity analysis, obtaining the KPI fluctuation range and threshold, and constructing an adaptive KPI evaluation benchmark based on the KPI fluctuation range and threshold and combining historical data; The adaptive KPI evaluation benchmark is applied to the real-time operation data of the SaaS system to conduct multi-dimensional performance evaluation and health analysis to obtain a comprehensive system score.
[0012] A second aspect of the present invention provides a cloud computing analysis device for a SaaS system, the cloud computing analysis device for the SaaS system comprising: The log processing module is used to perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and to establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; A query structure building module, which is used to obtain an abnormal behavior report according to a preset multi-dimensional real-time analysis model inputted from the full system log, and to plan a hierarchical storage strategy for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; A graph structure construction module, for determining a data lineage relationship graph of the standardized log data based on the hierarchical query optimization structure, and determining a user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship graph to obtain a user portrait; A computing cluster construction module is used to dynamically allocate computing resources according to the abnormal behavior report and user profile, obtain a resource scheduling strategy, and build a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster; The analysis and evaluation module is used to perform real-time monitoring and fault detection on the elastic computing cluster, obtain a cluster health status report, and build a multi-dimensional KPI indicator system based on the full system log, abnormal behavior report, user portrait and cluster health status report, and analyze and evaluate the SaaS system through the multi-dimensional KPI indicator system.
[0013] A third aspect of the present invention provides a cloud computing analysis device for a SaaS system, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the cloud computing analysis device of the SaaS system executes the steps of the above-mentioned cloud computing analysis method for the SaaS system.
[0014] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the cloud computing analysis method of the SaaS system described above.
[0015] The cloud computing analysis method, device, equipment and storage medium of the above-mentioned SaaS system generate standardized log data by performing distributed log collection and preprocessing on cloud computing nodes, and establish a centralized log aggregation index to obtain the full system log. The full system log is input into the real-time analysis model to generate an abnormal behavior report, and a hierarchical storage strategy is planned for the log and report to form a query structure. Based on the structure, a data lineage relationship diagram is determined to generate a user portrait. According to the abnormal behavior report and user portrait, the allocation of computing resources is planned, and a multi-cloud collaborative elastic computing cluster is constructed, which is monitored and fault detected in real time to generate a cluster health status report. The system is analyzed and evaluated through a multi-dimensional indicator system. The method improves the performance and efficiency of the system through distributed log collection, multi-dimensional real-time analysis, dynamic resource scheduling and the construction of a multi-dimensional indicator system.
[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a schematic diagram of a first embodiment of a cloud computing analysis method for a SaaS system according to an embodiment of the present invention; Figure 2 A schematic diagram of an embodiment of a cloud computing analysis device of a SaaS system in an embodiment of the present invention; Figure 3 Schematic diagram of an embodiment of a cloud computing analysis device of a SaaS system in an embodiment of the present invention. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0020] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.
[0021] To facilitate understanding of this embodiment, a cloud computing analysis method for a SaaS system disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; In one embodiment of the present invention, the distributed log collection and data preprocessing of the cloud computing nodes of the SaaS system to obtain standardized log data, and the centralized log aggregation index is established based on the standardized log data to obtain the queryable full system log, including: deploying a lightweight log collection agent on the cloud computing nodes of the SaaS system to collect multi-source heterogeneous log data to obtain an original log stream; performing timestamp normalization and field extraction on the original log stream to obtain structured log data, and storing the structured log data in a preset distributed log storage cluster; indexing and optimizing the structured log data in the distributed log storage cluster to obtain a full-text retrieval index, and implementing a distributed query engine based on the full-text retrieval index to obtain a queryable full system log.
[0022] Specifically, first deploy lightweight log collection agents on each cloud computing node of the SaaS system. These agents are usually customized based on open source tools such as Filebeat or Fluentd to adapt to different types of log sources. The agent is configured to listen to specific log files or data streams and can recognize multiple log formats, including application server logs, database logs, network device logs, etc. During the collection process, the agent reads the newly generated log entries in real time and packages them into small data packets to reduce the burden of network transmission. After obtaining the original log stream, the system will perform preliminary preprocessing on it. The first is the normalization of timestamps, which involves converting timestamps of different formats into a standard UTC time format to ensure that all log entries are comparable and sorted in time. Then, field extraction is performed, using predefined regular expressions or parsing rules to extract key information from unstructured log text, such as user ID, operation type, IP address, etc. This step converts the original log into a structured data format, usually JSON or a similar key-value pair structure. The structured log data is then transmitted to a pre-set distributed log storage cluster. This cluster is usually built on Elasticsearch or a similar distributed storage system with high availability and scalability. Data is automatically sharded and replicated during the writing process to ensure data integrity and fast access. The storage cluster also implements data lifecycle management strategies, such as regularly archiving cold data to low-cost storage to balance performance and cost. The structured log data stored in the cluster then enters the index building and optimization phase. The system creates an inverted index for each field to support full-text search and field-level queries. Various optimization techniques are applied during the indexing process, such as index compression, text analyzer selection, stop word filtering, etc., to improve retrieval efficiency. In addition, a multi-level cache mechanism is established, including node-level cache and cluster-level cache, to accelerate access to hot data. Based on the optimized full-text search index, the system further implements a distributed query engine. The engine can parse complex query statements and convert them into distributed execution plans. The query process uses the characteristics of the index to quickly locate relevant data and execute them in parallel on multiple nodes. The query engine also implements an intelligent result aggregation function, which can perform preliminary aggregation on data nodes to reduce the amount of data transmitted over the network. At the same time, the query engine integrates a cache mechanism and a result reuse strategy to further improve query performance. In order to support large-scale concurrent queries, the query engine uses load balancing and request routing mechanisms. Incoming query requests are dynamically distributed to the nodes with lighter loads in the cluster for processing. In addition, the query engine also implements a query optimizer that can automatically select the optimal query plan based on data distribution and query mode. For complex analytical queries, the system will adopt a phased execution strategy, first quickly returning some results, and then continuing to process in the background to gradually improve the query results.
[0023] 102. According to the full system log input, a preset multi-dimensional real-time analysis model is used to obtain an abnormal behavior report, and a hierarchical storage strategy is planned for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; In one embodiment of the present invention, the method of obtaining an abnormal behavior report by inputting a preset multi-dimensional real-time analysis model according to the full system log, and planning a hierarchical storage strategy for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure includes: performing time window segmentation and multi-dimensional feature extraction on the full system log to obtain a real-time feature vector stream, and applying a pre-trained multi-dimensional real-time analysis model to the real-time feature vector stream to perform event clustering and association analysis to obtain an abnormal event map; according to the abnormal event map, priority sorting and severity assessment are performed in combination with predefined business rules to obtain an abnormal behavior report; data life cycle analysis is performed on the full system log and the abnormal behavior report to obtain an access frequency attenuation curve, and according to the access frequency attenuation curve, the full system log and the abnormal behavior report are subjected to cold and hot stratification and storage medium matching to obtain a multi-level storage solution; according to the multi-level storage solution, the full system log and the abnormal behavior report are compressed and encoded and stored in blocks to obtain a data storage structure, and according to the data storage structure, a distributed index and cache strategy are constructed to obtain a hierarchical query optimization structure.
[0024] Specifically, the full system log is first segmented into time windows and multi-dimensional features are extracted. The system first segments the continuous log stream according to predefined time intervals (such as 5 minutes, 1 hour, etc.) to form discrete time windows. For the log data in each time window, the system applies a series of feature extraction algorithms, including but not limited to statistical features (such as log quantity, error rate), text features (such as keyword frequency, semantic vectors), performance indicators (such as response time, resource utilization), etc. These features are combined into high-dimensional vectors to form a real-time feature vector stream. Subsequently, the system inputs these real-time feature vector streams into a pre-trained multi-dimensional real-time analysis model. The model is usually based on a deep learning architecture, such as a long short-term memory network (LSTM) or a variational autoencoder (VAE), which can capture complex patterns in time series data. When processing feature vectors, the model considers both the data in the current time window and historical data to identify abnormal patterns. The anomaly score output by the model triggers an event clustering algorithm, such as DBSCAN or hierarchical clustering, to aggregate similar abnormal events together. Next, the system applies association analysis algorithms, such as frequent pattern mining or causal reasoning, to explore potential associations between different abnormal events. This process ultimately generates an abnormal event graph, where nodes represent abnormal events and edges represent associations between events. After generating the abnormal event graph, the system evaluates the events in the graph based on predefined business rules. These rules are usually developed by domain experts and include definitions of severity for different types of anomalies, business impact assessment criteria, etc. The system uses these rules to calculate the priority score and severity level for each abnormal event. The evaluation process also considers factors such as the duration of the anomaly, the scope of impact, and the degree of connectivity in the graph. Finally, the system generates a structured abnormal behavior report that contains detailed information, priority, severity, and recommended treatment measures for each abnormal event. Next, the system performs data lifecycle analysis on the full system logs and abnormal behavior reports. This step involves analyzing the access patterns of historical data and calculating the access frequency of data in different time periods. Through statistical analysis of a large amount of historical data, the system draws an access frequency decay curve that describes the changing trend of the probability of data being accessed over time. Based on this curve, the system formulates a hot and cold tiered storage strategy. Typically, recent and frequently accessed data (hot data) is stored on high-performance storage devices, such as solid-state drives or memory; older and less frequently accessed data (warm data) may be stored on ordinary hard drives; and rarely accessed historical data (cold data) may be archived to low-cost object storage or tape libraries. The system compresses and encodes the full system logs and abnormal behavior reports and stores them in blocks based on a multi-level storage solution. For hot data, the system can use lightweight compression algorithms, such as LZ4 or Snappy, to save storage space while maintaining fast decompression speeds.For cold data, algorithms with higher compression ratios, such as GZIP or LZMA, can be used. The data is divided into fixed-size blocks, and metadata information is attached to each block, including the time range, the type of events contained, etc. This block partitioning strategy facilitates parallel processing and quick location of data for a specific time period. Finally, the system builds a distributed index and cache strategy based on this data storage structure. The index structure adopts a multi-level design, including block-level index, time index, and event type index. These indexes are distributed on different nodes of the cluster, and load balancing is ensured through algorithms such as consistent hashing. The system also implements an intelligent caching strategy to cache frequently accessed data blocks and index information in memory. The cache strategy is dynamically adjusted to optimize the cache content based on real-time access patterns and system load conditions.
[0025] 103. Determine the data lineage relationship diagram of the standardized log data based on the hierarchical query optimization structure, and determine the user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship diagram to obtain the user portrait; In one embodiment of the present invention, the data lineage relationship graph of the standardized log data is determined based on the hierarchical query optimization structure, and the user operation mode is determined based on the queryable full system log, abnormal behavior report and data lineage relationship graph to obtain the user portrait, which includes: applying a graph traversal algorithm to the hierarchical query optimization structure to track the data flow path to obtain the initial data dependency relationship, and performing topological sorting and cycle detection on the initial data dependency relationship to obtain the data lineage relationship graph of the standardized log data; performing session identification and behavior serialization processing on the full system log to obtain user operation time series data, and performing Based on the report, a sequence pattern mining algorithm is applied to obtain the initial user operation pattern; the initial user operation pattern and the data lineage relationship diagram are associated with analysis to identify key data nodes and operation paths to obtain enhanced user operation patterns, and a user behavior feature vector is constructed based on the enhanced user operation pattern, and an unsupervised clustering algorithm is applied to obtain user grouping results; multi-dimensional feature extraction and weight calculation are performed on the user grouping results to obtain a user tag set, and a collaborative filtering algorithm is applied based on the user tag set and historical interaction data to obtain an initial user portrait; a time series analysis method is applied to the initial user portrait to identify user behavior trends and periodic patterns to obtain a user portrait.
[0026] Specifically, first, the data lineage graph of the standardized log data is determined based on the hierarchical query optimization structure. The system applies graph traversal algorithms, such as depth-first search (DFS) or breadth-first search (BFS), to the hierarchical query optimization structure to track the flow path of data in the system. This process involves analyzing the source, transformation, and storage location of the log data to build initial data dependencies. Subsequently, the system topologically sorts these initial data dependencies to ensure the order of data processing, and performs loop detection to identify potential data loop dependencies. Through these steps, the system finally generates a complete data lineage graph that clearly shows the dependencies and flow relationships between data. Next, the system determines the user operation mode based on the queryable full system logs, abnormal behavior reports, and the data lineage graph just generated. First, the system performs session identification on the full system logs and divides the user's operation sessions by setting time thresholds or using specific session identifiers. In each session, the system arranges the user's operations in chronological order to form a behavior sequence, thereby obtaining the time series data of the user's operations. The system then combines these time series data with abnormal behavior reports and applies sequential pattern mining algorithms, such as PrefixSpan or SPADE, to discover frequently occurring operation patterns and abnormal operation sequences, thereby obtaining the initial user operation pattern. The system further performs association analysis on the initial user operation pattern and the data lineage graph. This step involves mapping the user's operation sequence to the data lineage graph, identifying the key data nodes involved in the user operation and the common operation paths. In this way, the system can understand the impact of user operations on the data flow and the correlation between different operations, thereby obtaining an enhanced user operation pattern. Based on this enhanced user operation pattern, the system constructs a user behavior feature vector, which contains information such as the user's operation frequency, operation sequence, and accessed data nodes. The system then applies unsupervised clustering algorithms, such as K-means or DBSCAN, to perform cluster analysis on these feature vectors to obtain user grouping results. For the obtained user grouping results, the system performs multi-dimensional feature extraction and weight calculation. This process involves analyzing the unique features of each user group, such as common functions, operation habits, data access patterns, etc., and assigning weights to these features. In this way, the system obtains a set of descriptive user tags. The system then combines these user tag sets with historical interaction data and applies collaborative filtering algorithms, such as user-based collaborative filtering or item-based collaborative filtering, to generate an initial user profile. This initial user profile contains the user's preferences, behavior patterns, and possible points of interest. Finally, the system applies time series analysis methods, such as autoregressive integrated moving average model (ARIMA) or Prophet, to the initial user profile to identify long-term trends and periodic patterns in user behavior.This step can capture changes in user behavior over time, including seasonal patterns, differences between weekdays and weekends, and long-term behavioral evolution. By integrating information from these time dimensions, the system ultimately obtains a comprehensive and dynamic user profile.
[0027] 104. According to the abnormal behavior report and user portrait, dynamically allocate computing resources to obtain resource scheduling strategies, and build a multi-cloud collaborative computing framework based on the resource scheduling strategies to obtain elastic computing clusters; In one embodiment of the present invention, the method of dynamically allocating computing resources according to the abnormal behavior report and user portrait to obtain a resource scheduling strategy, and constructing a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster includes: performing multi-dimensional prediction of the future load of the SaaS system based on the abnormal behavior report and user portrait to obtain a resource demand prediction result; performing performance evaluation and utilization analysis on the existing computing resources of the SaaS system to obtain a resource status matrix, and applying a multi-objective optimization algorithm to perform resource allocation calculation based on the resource status matrix and the resource demand prediction result to obtain a resource scheduling strategy; applying a distributed consistency protocol to the SaaS system based on the resource scheduling strategy to construct a cross-cloud resource collaboration mechanism to obtain a multi-cloud collaborative computing framework; intelligently scheduling and load balancing computing tasks based on the multi-cloud collaborative computing framework to obtain an initial elastic computing cluster, and applying a fault-tolerant mechanism and an automatic expansion and contraction strategy to the initial elastic computing cluster to perform real-time resource adjustment to obtain an elastic computing cluster.
[0028] Specifically, firstly, an in-depth analysis of abnormal behavior reports and user portraits is performed to predict the future load of the SaaS system. The system first performs a time series analysis on abnormal behavior reports to identify the occurrence pattern and periodicity of abnormal events, and at the same time combines the behavioral trend information in the user portrait to build a multi-dimensional prediction model. This model can use machine learning algorithms, such as LSTM (Long Short-Term Memory Network) or ARIMA (Autoregressive Integrated Moving Average Model), to consider historical load data, user activity, business peak period and other factors to generate resource demand forecast results for a period of time in the future (such as the next 24 hours or a week). The forecast results include multiple dimensions such as CPU usage, memory usage, and network bandwidth requirements. Subsequently, the system conducts a comprehensive performance evaluation and utilization analysis of the existing computing resources of the SaaS platform to generate a resource status matrix. This matrix contains information such as the current performance indicators, historical utilization data, and available capacity of each computing node. The system combines the resource status matrix with the previously obtained resource demand forecast results, and applies a multi-objective optimization algorithm (such as genetic algorithm or particle swarm optimization) to perform resource allocation calculations. The optimization objectives usually include maximizing resource utilization, minimizing energy consumption, and balancing load distribution. The output of this process is a detailed resource scheduling strategy that specifies how to dynamically allocate and adjust computing resources in future time periods. Based on the formulated resource scheduling strategy, the system begins to build a multi-cloud collaborative computing framework. This process first involves the unified abstraction and encapsulation of resources from different cloud platforms (such as public clouds, private clouds, and hybrid cloud environments) to create a standardized resource pool. The system then applies a distributed consistency protocol, such as Raft or Paxos, to ensure the consistency of resource management and task scheduling in a multi-cloud environment. This protocol ensures that the entire system can maintain consistent resource allocation and task execution status even in the event of network partitions or partial node failures. In this way, the system establishes a cross-cloud resource coordination mechanism that can uniformly manage and schedule computing resources distributed on different cloud platforms. Based on the multi-cloud collaborative computing framework, the system implements intelligent task scheduling and load balancing strategies. The scheduling algorithm considers multiple factors, including task priority, resource requirements, data location, network latency, etc., to allocate computing tasks to the most appropriate resource nodes. The load balancing mechanism monitors the load of each node in real time and dynamically adjusts task allocation to avoid local overload or resource idleness. This process forms the initial elastic computing cluster, which can automatically adjust the allocation of computing resources according to changes in workload. In order to further enhance the reliability and adaptability of the system, fault tolerance mechanisms and automatic expansion and contraction strategies are applied to the initial elastic computing cluster. The fault tolerance mechanism includes functions such as real-time monitoring, fault detection, and automatic recovery, which can quickly reallocate tasks when nodes fail or the network is interrupted to ensure service continuity. The automatic expansion and contraction strategy dynamically increases or decreases computing nodes based on real-time load conditions and predefined rules.For example, when the system detects that the load continues to increase, it will automatically start a new computing instance; when the load decreases, it will shut down the redundant nodes to save costs. This process is continuous and dynamic. The system constantly evaluates the current resource utilization and makes real-time adjustments based on the preset performance indicators and cost targets.
[0029] Furthermore, the performance evaluation and utilization analysis of existing computing resources are performed to obtain a resource status matrix, and a multi-objective optimization algorithm is applied based on the resource status matrix and resource demand prediction results to perform resource allocation calculations to obtain a resource scheduling strategy, including: performing multi-dimensional testing on existing computing resources to obtain a resource performance indicator set, and constructing a dynamic resource utilization curve based on the resource performance indicator set and historical load data; applying a time series decomposition algorithm to the dynamic resource utilization curve to extract trends, cycles and random components in the dynamic resource utilization curve to obtain a resource utilization pattern, and constructing a multi-dimensional resource status vector based on the resource utilization pattern and the resource performance indicator set to obtain a resource status matrix; performing a difference analysis on the resource status matrix and the resource demand prediction results to identify areas of excess and insufficient resources to obtain a resource balance demand diagram, and constructing a multi-objective optimization problem model based on the resource balance demand diagram to obtain an optimization problem expression; applying an improved genetic algorithm to the optimization problem expression to perform multi-generation iterative optimization calculations to obtain a Pareto optimal solution set, and generating a resource scheduling strategy based on the Pareto optimal solution set in combination with preset decision rules.
[0030] Specifically, the existing computing resources are first tested in a comprehensive multi-dimensional manner. This testing process involves evaluating key indicators such as CPU performance, memory read and write speed, storage I / O capacity, and network throughput. The testing method includes running standardized benchmark programs such as LINPACK for CPU testing, STREAM for memory testing, Iozone for storage testing, and Netperf for network performance testing. These test results are summarized to form a comprehensive set of resource performance indicators. Next, the system combines this set of resource performance indicators with historical load data to construct a dynamic resource utilization curve. Historical load data usually comes from system monitoring tools and contains the usage of various resources over a period of time (such as several months or a year). The system uses time series analysis techniques, such as moving average or exponential smoothing, to associate historical data with current performance indicators and generate a curve that reflects the dynamic changes in resource usage. For the resulting dynamic resource utilization curve, the system applies time series decomposition algorithms, such as STL (Seasonal and Trend decomposition using Loess) or X-12-ARIMA. These algorithms can decompose time series data into trend, cycle, and random components. The trend component reflects the direction of long-term changes in resource usage, the periodic component reveals the periodic pattern of resource usage (such as daily, weekly or monthly regularity), and the random component represents unpredictable fluctuations. By analyzing these components, the system obtains detailed resource utilization patterns. The system then combines the resource utilization pattern with the previously obtained set of resource performance indicators to construct a multi-dimensional resource state vector. This vector not only contains the current resource performance data, but also incorporates the historical pattern and predicted trend of resource usage. By applying this process to all computing resources, the system finally generates a comprehensive resource state matrix that provides a real-time snapshot of the entire computing environment and future trend predictions. Next, the system performs a difference analysis on the resource state matrix with the previously obtained resource demand forecast results. This process involves comparing the difference between the predicted resource demand and the currently available resources to identify areas of excess and insufficient resources. The analysis results are visualized as a resource balance demand graph, which clearly shows which resource types need to be increased or decreased when and where. Based on the resource balance demand graph, the system constructs a multi-objective optimization problem model. This model considers multiple objectives, such as maximizing resource utilization, minimizing energy consumption, and ensuring service quality. Each objective is expressed as a mathematical function and assigned a corresponding weight. At the same time, the model also contains various constraints, such as total resource capacity, service level agreement (SLA) requirements, etc. These goals and constraints together form a complex optimization problem expression. In order to solve this multi-objective optimization problem, the system applies an improved genetic algorithm. This algorithm simulates the biological evolution process and searches for the optimal solution through multiple generations of iterations.In each generation, the algorithm generates multiple candidate solutions (i.e., different resource allocation schemes), evaluates their fitness (i.e., the degree to which each goal is met), and then generates the next generation of solutions through selection, crossover, and mutation operations. This process is repeated until the preset number of iterations is reached or the termination condition is met. After multiple generations of iterative optimization calculations, the algorithm finally generates a set of Pareto optimal solutions. A Pareto optimal solution is a solution that inevitably leads to a decrease in the performance of at least one other goal when the performance of any one goal is improved. This set of solutions provides a series of optional solutions that weigh different goals. Finally, the system selects the final solution from the Pareto optimal solution set according to preset decision rules. These decision rules may consider factors such as current business priorities, cost constraints, and performance requirements. For example, during periods of resource shortage, the system may tend to choose a solution that focuses more on resource utilization; while during performance-critical periods, it may choose a solution that focuses more on service quality. By applying these rules, the system finally generates a specific resource scheduling strategy.
[0031] 105. Perform real-time monitoring and fault detection on the elastic computing cluster to obtain the cluster health status report, and build a multi-dimensional KPI indicator system based on the full system log, abnormal behavior report, user portrait and cluster health status report, and analyze and evaluate the SaaS system through the multi-dimensional KPI indicator system.
[0032] In one embodiment of the present invention, the elastic computing cluster is monitored in real time and fault detection is performed to obtain a cluster health status report, and a multi-dimensional KPI indicator system is constructed based on the full system log, abnormal behavior report, user portrait and cluster health status report. The analysis and evaluation of the SaaS system through the multi-dimensional KPI indicator system includes: deploying distributed probes on the elastic computing cluster, collecting real-time performance indicators and resource utilization data, obtaining original monitoring data streams, and applying anomaly detection algorithms and fault diagnosis rules according to the original monitoring data streams to perform real-time fault identification and location to obtain a cluster health status report; the full system log, Data fusion and feature extraction are performed on abnormal behavior reports, user portraits, and cluster health status reports to obtain a multi-source heterogeneous data feature set, and based on the multi-source heterogeneous data feature set, the analytic hierarchy process and the Delphi method are applied to construct a multi-level KPI indicator system; the Monte Carlo simulation method is applied to the multi-level KPI indicator system to perform large-scale data sampling and indicator sensitivity analysis to obtain the KPI fluctuation range and threshold, and based on the KPI fluctuation range and threshold, combined with historical data, an adaptive KPI evaluation benchmark is constructed; the adaptive KPI evaluation benchmark is applied to the real-time operation data of the SaaS system to perform multi-dimensional performance evaluation and health analysis to obtain a comprehensive system score.
[0033] Specifically, first, a comprehensive distributed probe deployment is performed on the elastic computing cluster. These probes are lightweight monitoring agents that are installed on each node of the cluster and are responsible for continuously collecting various real-time performance indicators and resource utilization data. The collected data includes key indicators such as CPU usage, memory usage, disk I / O, network throughput, and process status. These probes are configured to collect data at a high frequency (such as every second or every minute) and transmit the data to the central monitoring system in real time to form a raw monitoring data stream. Subsequently, the system applies a series of anomaly detection algorithms and predefined fault diagnosis rules to these raw monitoring data streams. Anomaly detection algorithms can include statistical methods (such as Z-score or InterquartileRange), machine learning methods (such as Isolation Forest or One-Class SVM), etc. These algorithms can identify data points that deviate from normal patterns. At the same time, the system also applies a set of complex fault diagnosis rules, which are based on the knowledge of domain experts and historical fault data, and can associate detected anomalies with specific types of system faults. Through this process, the system can identify potential faults in real time, accurately locate the location and type of faults, and finally generate a detailed cluster health status report. Next, the system performs in-depth data fusion and feature extraction on the full system log, abnormal behavior report, user profile, and the newly generated cluster health status report. This process involves unified processing and correlation analysis of data from different sources and formats. For example, error information in the system log may be associated with certain abnormal indicators in the cluster health status report, while usage patterns in the user profile may explain certain abnormal behaviors. Through complex data processing algorithms, such as feature engineering technology and automatic feature extraction of deep learning, the system extracts a comprehensive set of features from these heterogeneous data, which covers all aspects of system operation. Based on this multi-source heterogeneous data feature set, the system applies the analytic hierarchy process (AHP) and the Delphi method to construct a multi-level KPI indicator system. The analytic hierarchy process is used to decompose complex evaluation problems into hierarchical structures and determine the relative importance of different indicators. The Delphi method continuously optimizes and adjusts the indicator system through multiple rounds of expert opinion collection and feedback. This process combines quantitative analysis and qualitative judgment to ensure that the KPI indicator system is both scientific and reflects business needs. The final KPI indicator system includes multiple levels, such as system level (such as service availability, response time), business level (such as transaction success rate, user activity) and strategic level (such as customer satisfaction, market share). In order to determine the reasonable thresholds and evaluation criteria of these KPI indicators, the system uses the Monte Carlo simulation method to conduct large-scale data sampling and sensitivity analysis. This process involves generating a large amount of simulated data, which is based on the statistical characteristics of historical data but contains various possible changes.By analyzing these simulated data, the system can understand the range and sensitivity of each KPI indicator under different conditions. These analysis results are used to determine the normal fluctuation range and warning threshold of each KPI. Combining these KPI fluctuation ranges and thresholds obtained through simulation, as well as the actual historical operation data of the system, the system constructs an adaptive KPI evaluation benchmark. This evaluation benchmark is not static, but can be dynamically adjusted according to the actual situation of system operation. For example, during business peak periods, the acceptable range of certain performance indicators may be relaxed accordingly; while during sensitive periods, security-related indicators may be set with stricter standards. Finally, the system applies this adaptive KPI evaluation benchmark to the real-time operation data of the SaaS system to perform multi-dimensional performance evaluation and health analysis. This process involves calculating the current value of each KPI indicator in real time, comparing it with the evaluation benchmark, and comprehensively considering the importance and interrelationship of each indicator. The system not only evaluates the performance of a single indicator, but also analyzes the correlation and combined impact between indicators. For example, a slight decrease in certain performance indicators may be a normal result of an increase in the number of users, and does not necessarily indicate a deterioration in system health.
[0034] In this embodiment, by performing distributed log collection and preprocessing on cloud computing nodes, standardized log data is generated, and a centralized log aggregation index is established to obtain the full system log. The full system log is input into the real-time analysis model, an abnormal behavior report is generated, and a hierarchical storage strategy is planned for the log and report to form a query structure. Based on the structure, a data lineage relationship diagram is determined to generate a user portrait. According to the abnormal behavior report and user portrait, the allocation of computing resources is planned, a multi-cloud collaborative elastic computing cluster is built, and real-time monitoring and fault detection are performed on it to generate a cluster health status report. The system is analyzed and evaluated through a multi-dimensional indicator system. This method improves the performance and efficiency of the system through distributed log collection, multi-dimensional real-time analysis, dynamic resource scheduling, and the construction of a multi-dimensional indicator system.
[0035] The above describes the cloud computing analysis method of the SaaS system in the embodiment of the present invention. The following describes the cloud computing analysis device of the SaaS system in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a cloud computing analysis device for a SaaS system includes: The log processing module 201 is used to perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and to establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; A query structure building module 202 is used to obtain an abnormal behavior report according to the full system log input preset multi-dimensional real-time analysis model, and to plan a hierarchical storage strategy for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; A graph structure construction module 203 is used to determine a data lineage relationship graph of the standardized log data based on the hierarchical query optimization structure, and determine a user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship graph to obtain a user portrait; A computing cluster construction module 204 is used to dynamically allocate computing resources according to the abnormal behavior report and user profile, obtain a resource scheduling strategy, and build a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster; The analysis and evaluation module 205 is used to perform real-time monitoring and fault detection on the elastic computing cluster, obtain a cluster health status report, and build a multi-dimensional KPI indicator system based on the full system log, abnormal behavior report, user portrait and cluster health status report, and analyze and evaluate the SaaS system through the multi-dimensional KPI indicator system.
[0036] In an embodiment of the present invention, the cloud computing analysis device of the SaaS system runs the cloud computing analysis method of the SaaS system. The cloud computing analysis device of the SaaS system generates standardized log data by performing distributed log collection and preprocessing on cloud computing nodes, and establishes a centralized log aggregation index to obtain the full system log. The full system log is input into the real-time analysis model, an abnormal behavior report is generated, and a hierarchical storage strategy is planned for the log and report to form a query structure. Based on the structure, a data lineage relationship diagram is determined to generate a user portrait. According to the abnormal behavior report and the user portrait, the allocation planning of computing resources is carried out, a multi-cloud collaborative elastic computing cluster is constructed, and real-time monitoring and fault detection are performed on it to generate a cluster health status report. The system is analyzed and evaluated through a multi-dimensional indicator system. The method improves the performance and efficiency of the system through distributed log collection, multi-dimensional real-time analysis, dynamic resource scheduling, and the construction of a multi-dimensional indicator system.
[0037] above Figure 2 The cloud computing analysis device of the SaaS system in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The cloud computing analysis device of the SaaS system in the embodiment of the present invention is described in detail from the perspective of hardware processing.
[0038] Figure 31 is a schematic diagram of the structure of a cloud computing analysis device of a SaaS system provided by an embodiment of the present invention. The cloud computing analysis device 300 of the SaaS system may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the cloud computing analysis device 300 of the SaaS system. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the cloud computing analysis device 300 of the SaaS system to implement the steps of the cloud computing analysis method of the SaaS system described above.
[0039] The cloud computing analysis device 300 of the SaaS system may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 3 The cloud computing analysis device structure of the SaaS system shown does not constitute a limitation on the cloud computing analysis device of the SaaS system provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.
[0040] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the cloud computing analysis method of the SaaS system.
[0041] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.
[0042] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0043] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A cloud computing analysis method for a SaaS system, characterized in that: The cloud computing analysis method of the SaaS system includes: Perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; According to the full system log, a preset multi-dimensional real-time analysis model is input to obtain an abnormal behavior report, and a hierarchical storage strategy is planned for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; Determine a data lineage relationship diagram of the standardized log data based on the hierarchical query optimization structure, and determine a user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship diagram to obtain a user portrait; According to the abnormal behavior report and user portrait, dynamically allocate computing resources to obtain a resource scheduling strategy, and build a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster; The elastic computing cluster is monitored and fault detected in real time to obtain a cluster health status report, and a multi-dimensional KPI indicator system is constructed based on the full system log, abnormal behavior report, user portrait and cluster health status report, and the SaaS system is analyzed and evaluated through the multi-dimensional KPI indicator system.
2. The cloud computing analysis method of the SaaS system according to claim 1, characterized in that: The distributed log collection and data preprocessing of the cloud computing nodes of the SaaS system are performed to obtain standardized log data, and a centralized log aggregation index is established based on the standardized log data to obtain the queryable full system logs including: Deploy lightweight log collection agents on the cloud computing nodes of the SaaS system to collect multi-source heterogeneous log data and obtain the original log stream; Performing timestamp normalization and field extraction on the original log stream to obtain structured log data, and storing the structured log data in a preset distributed log storage cluster; Index construction and optimization are performed on the structured log data in the distributed log storage cluster to obtain a full-text search index, and a distributed query engine is implemented based on the full-text search index to obtain a full amount of system logs that can be queried.
3. The cloud computing analysis method of the SaaS system according to claim 1, characterized in that: The step of inputting a preset multi-dimensional real-time analysis model according to the full system log to obtain an abnormal behavior report, and performing hierarchical storage strategy planning for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure includes: Perform time window segmentation and multi-dimensional feature extraction on the full system log to obtain a real-time feature vector stream, and apply a pre-trained multi-dimensional real-time analysis model to the real-time feature vector stream to perform event clustering and correlation analysis to obtain an abnormal event map; According to the abnormal event map, priority sorting and severity assessment are performed in combination with predefined business rules to obtain an abnormal behavior report; Performing data lifecycle analysis on the full system logs and abnormal behavior reports to obtain an access frequency attenuation curve, and performing cold and hot stratification and storage medium matching on the full system logs and abnormal behavior reports according to the access frequency attenuation curve to obtain a multi-level storage solution; According to the multi-level storage solution, the full system logs and abnormal behavior reports are compressed, encoded and stored in blocks to obtain a data storage structure, and based on the data storage structure, a distributed index and cache strategy is constructed to obtain a hierarchical query optimization structure.
4. The cloud computing analysis method of the SaaS system according to claim 1, characterized in that: The step of determining the data lineage relationship diagram of the standardized log data based on the hierarchical query optimization structure, and determining the user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship diagram, to obtain the user portrait includes: Applying a graph traversal algorithm to the hierarchical query optimization structure, tracing the data flow path, obtaining an initial data dependency relationship, and performing topological sorting and cycle detection on the initial data dependency relationship to obtain a data lineage relationship diagram of the standardized log data; Performing session identification and behavior serialization processing on the full system log to obtain user operation time series data, and applying a sequence pattern mining algorithm based on the user operation time series data and abnormal behavior reports to obtain an initial user operation pattern; Performing association analysis on the initial user operation mode and the data lineage relationship diagram, identifying key data nodes and operation paths, obtaining enhanced user operation modes, constructing user behavior feature vectors based on the enhanced user operation modes, and applying unsupervised clustering algorithms to obtain user grouping results; Perform multi-dimensional feature extraction and weight calculation on the user grouping results to obtain a user tag set, and apply a collaborative filtering algorithm based on the user tag set and historical interaction data to obtain an initial user portrait; A time series analysis method is applied to the initial user portrait to identify user behavior trends and periodic patterns to obtain a user portrait.
5. The cloud computing analysis method of the SaaS system according to claim 1, characterized in that: According to the abnormal behavior report and user portrait, the computing resources are dynamically allocated and planned to obtain a resource scheduling strategy, and a multi-cloud collaborative computing framework is constructed based on the resource scheduling strategy to obtain an elastic computing cluster including: Based on the abnormal behavior report and user portrait, a multi-dimensional prediction of the future load of the SaaS system is performed to obtain a resource demand prediction result; Performing performance evaluation and utilization analysis on the existing computing resources of the SaaS system to obtain a resource status matrix, and applying a multi-objective optimization algorithm based on the resource status matrix and resource demand prediction results to perform resource allocation calculations to obtain a resource scheduling strategy; According to the resource scheduling strategy, a distributed consistency protocol is applied to the SaaS system to build a cross-cloud resource collaboration mechanism to obtain a multi-cloud collaborative computing framework; According to the multi-cloud collaborative computing framework, computing tasks are intelligently scheduled and load balanced to obtain an initial elastic computing cluster, and a fault tolerance mechanism and an automatic expansion and contraction strategy are applied to the initial elastic computing cluster to perform real-time resource adjustment to obtain an elastic computing cluster.
6. The cloud computing analysis method of the SaaS system according to claim 5, characterized in that: The performance evaluation and utilization analysis of the existing computing resources are performed to obtain a resource status matrix, and a multi-objective optimization algorithm is applied according to the resource status matrix and the resource demand prediction result to perform resource allocation calculation to obtain a resource scheduling strategy including: Perform multi-dimensional testing on existing computing resources to obtain a resource performance indicator set, and construct a dynamic resource utilization curve based on the resource performance indicator set and historical load data; Applying a time series decomposition algorithm to the dynamic resource utilization curve, extracting trends, cycles and random components in the dynamic resource utilization curve to obtain a resource utilization pattern, and constructing a multidimensional resource state vector based on the resource utilization pattern and a resource performance indicator set to obtain a resource state matrix; Performing difference analysis on the resource status matrix and the resource demand forecast results, identifying resource surplus and shortage areas, obtaining a resource balance demand graph, and constructing a multi-objective optimization problem model based on the resource balance demand graph to obtain an optimization problem expression; An improved genetic algorithm is applied to the optimization problem expression to perform multiple generations of iterative optimization calculations to obtain a Pareto optimal solution set, and a resource scheduling strategy is generated based on the Pareto optimal solution set in combination with preset decision rules.
7. The cloud computing analysis method of the SaaS system according to claim 1, characterized in that: The real-time monitoring and fault detection of the elastic computing cluster is performed to obtain a cluster health status report, and a multi-dimensional KPI indicator system is constructed based on the full system log, abnormal behavior report, user portrait and cluster health status report. The analysis and evaluation of the SaaS system through the multi-dimensional KPI indicator system includes: Deploy distributed probes on the elastic computing cluster to collect real-time performance indicators and resource utilization data to obtain original monitoring data streams, and apply anomaly detection algorithms and fault diagnosis rules based on the original monitoring data streams to perform real-time fault identification and location to obtain a cluster health status report; Perform data fusion and feature extraction on the full system log, abnormal behavior report, user portrait and cluster health status report to obtain a multi-source heterogeneous data feature set, and apply the analytic hierarchy process and Delphi method based on the multi-source heterogeneous data feature set to build a multi-level KPI indicator system; Applying the Monte Carlo simulation method to the multi-level KPI indicator system, conducting large-scale data sampling and indicator sensitivity analysis, obtaining the KPI fluctuation range and threshold, and constructing an adaptive KPI evaluation benchmark based on the KPI fluctuation range and threshold and combining historical data; The adaptive KPI evaluation benchmark is applied to the real-time operation data of the SaaS system to conduct multi-dimensional performance evaluation and health analysis to obtain a comprehensive system score.
8. A cloud computing analysis device for a SaaS system, characterized in that: The cloud computing analysis device of the SaaS system includes: The log processing module is used to perform distributed log collection and data preprocessing on the cloud computing nodes of the SaaS system to obtain standardized log data, and to establish a centralized log aggregation index based on the standardized log data to obtain a queryable full system log; A query structure building module, which is used to obtain an abnormal behavior report according to a preset multi-dimensional real-time analysis model inputted from the full system log, and to plan a hierarchical storage strategy for the full system log and the abnormal behavior report to obtain a hierarchical query optimization structure; A graph structure construction module, for determining a data lineage relationship graph of the standardized log data based on the hierarchical query optimization structure, and determining a user operation mode based on the queryable full system log, abnormal behavior report and data lineage relationship graph to obtain a user portrait; A computing cluster construction module is used to dynamically allocate computing resources according to the abnormal behavior report and user profile, obtain a resource scheduling strategy, and build a multi-cloud collaborative computing framework based on the resource scheduling strategy to obtain an elastic computing cluster; The analysis and evaluation module is used to perform real-time monitoring and fault detection on the elastic computing cluster, obtain a cluster health status report, and build a multi-dimensional KPI indicator system based on the full system log, abnormal behavior report, user portrait and cluster health status report, and analyze and evaluate the SaaS system through the multi-dimensional KPI indicator system.
9. A cloud computing analysis device for a SaaS system, characterized in that: The cloud computing analysis device of the SaaS system includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the cloud computing analysis device of the SaaS system executes the steps of the cloud computing analysis method of the SaaS system as described in any one of claims 1 to 7.
10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the cloud computing analysis method of the SaaS system as described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Privileged account management-based platform and method for protecting private data in power data and mining data
CN114372098A
Nuclear power plant PaaS platform resource quota monitoring method and system based on cloud computing
CN115048260A
Container cloud resource intelligent scheduling system based on delay perception and working method
CN118331738A
Elastic management method and system for improving resource efficiency
CN118642852A
GPU resource usage method, GPU virtualization method, and job scheduling apparatus and cluster
WO2024032587A1
Cited By
Low-altitude airspace digital base rapid generation method and device based on high-precision dense point cloud and storage medium
CN119399408A
A SaaS platform front-end intelligent reconstruction method and system
CN119759413A
Network data processing method
CN119835304A
Network anomaly detection and automatic processing method and system, storage medium and equipment
CN119892476A
Network anomaly detection and automatic handling methods, systems, storage media and devices
CN119892476B
