Architecture fault prediction and application effect evaluation system and method based on deep learning

Abstract

The invention provides a deep learning-based architecture fault prediction and application effect evaluation method and system. The method comprises the following steps of: acquiring multi-source heterogeneous operation data of heterogeneous components in a digital architecture and preprocessing the multi-source heterogeneous operation data; the fault types are input to a pre-trained deep learning model, the probability of occurrence of each fault type is obtained, and fault early warning information is generated based on comparison between the probability and a preset threshold value; constructing a multi-dimensional evaluation index system, and obtaining real-time data of each index; determining the weight of each evaluation index based on an analytic hierarchy process, and mapping the actual value of each index into a comprehensive evaluation result by adopting a fuzzy comprehensive evaluation method; and displaying the fault early warning information and the comprehensive evaluation result in a visual chart form, and generating a targeted architecture optimization suggestion according to the comprehensive evaluation result. According to the method, the accuracy of fault prediction is improved, the framework application effect evaluation is comprehensive and accurate, and the method has very high real-time performance and dynamic adaptability and good expandability and reusability.

Description

Technical Field

[0001] This invention relates to the field of information technology, and more specifically, to a system and method for predicting architectural failures and evaluating application effectiveness based on deep learning. Background Technology

[0002] In today's digital age, various digital architectures are widely used in enterprises, data centers, and other scenarios. As system complexity continues to increase, architectural failures can lead to serious business interruptions, data loss, and other problems, causing significant economic losses and reputational risks for enterprises. Traditional failure prediction methods are mostly based on preset rules and simple statistical models (such as moving averages and exponential smoothing). These methods can only capture the linear patterns on the surface of data and are ill-suited to the complex patterns of multi-source heterogeneous data in digital architectures (such as nonlinear fluctuations and sudden anomalies) and dynamic operating environments (such as dynamic changes in business load and iterative updates of components).

[0003] Meanwhile, existing solutions for evaluating the effectiveness of architecture applications mostly rely on single-dimensional indicators (such as focusing only on response time or resource utilization) or subjective human scoring, lacking a multi-dimensional and quantifiable evaluation system that covers "performance-reliability-resources-user experience", and cannot reflect the system's comprehensive performance in business support, resource allocation, user service and other aspects in real time.

[0004] Therefore, there is an urgent need for a system that can efficiently predict failures in digital architectures and accurately assess their application effectiveness in order to meet the needs of lean management in the context of digital transformation. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based system and method for predicting architectural failures and evaluating application effectiveness. This system and method improves the accuracy of failure prediction, provides comprehensive and accurate evaluation of architectural application effectiveness, and has strong real-time performance, dynamic adaptability, scalability, and reusability.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for predicting architectural failures and evaluating application effectiveness based on deep learning, the method comprising: Collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture, and perform cleaning, normalization, and feature engineering to generate a high-quality dataset suitable for model computation. The processed real-time running data is input into a pre-trained deep learning model to obtain the probability of each fault type. Based on the comparison between the probability and a preset threshold, fault warning information is generated. The deep learning model is an LSTM-CNN hybrid model trained with historical fault data and normal operation data, which is used to capture long-term dependencies and local abnormal features in time series data. A multi-dimensional evaluation index system was constructed, and real-time data of each index was obtained; the weight of each evaluation index was determined based on the analytic hierarchy process, and the actual value of each index was mapped to the comprehensive evaluation result using the fuzzy comprehensive evaluation method. The fault warning information and the comprehensive evaluation results are displayed in the form of visual charts, and targeted architecture optimization suggestions are generated based on the comprehensive evaluation results.

[0007] Preferably, the data cleaning includes: The density-based spatial clustering algorithm DBSCAN is used to identify and remove outliers in the data; and multiple imputation is used to impute missing values ​​in the data.

[0008] Preferably, the data normalization includes: According to formula (1), Z-Score normalization is used to normalize the numerical data to a standard normal distribution with a mean of 0 and a standard deviation of 1. (1) Where x is the original data value, μ is the dataset mean, and σ is the dataset standard deviation; One-hot encoding is used to convert categorical data into binary vectors suitable for model processing; The frequency domain features of time series data are extracted using Fourier Transform (FFT), and statistical features are calculated using a sliding window. It also uses word embedding to extract semantic features from log text data and Named Entity Recognition (NER) to extract key entities.

[0009] Preferably, the training process of the LSTM-CNN hybrid model includes: The dataset containing historical fault labels is divided into training, validation, and test sets proportionally. Based on formulas (2) to (4), the adaptive moment estimation algorithm is used as the optimizer, with the goal of minimizing the mean squared error loss function, to iteratively update the model parameters. (2) (3) (4) in, The gradient at the current time. and The exponential decay rate is estimated by moments. For learning rate, To prevent small constants in the division-by-zero operation, For the current first-order moment estimate, For second-order moment estimation, and These are the first and second moment estimates after bias correction, respectively. , .

[0010] Preferably, the evaluation of the effectiveness of the architecture application includes: The evaluation index system is constructed as follows: performance dimension includes average response time, transaction throughput and concurrent users; reliability dimension includes failure rate, mean time between failures (MTBF) and mean time to recovery (MTTR); resource utilization dimension includes average CPU utilization, average memory utilization and storage resource utilization; and user experience dimension includes user satisfaction score, operation success rate and user retention rate. Based on the analytic hierarchy process, a judgment matrix is ​​constructed through expert scoring, the consistency ratio is calculated and a consistency test is performed, and the weight vector of each indicator is obtained by solving the problem. Establish a fuzzy evaluation level, determine the membership degree of each indicator to the fuzzy evaluation level based on the actual value of each indicator, and construct a fuzzy relation matrix; The weight vector and the fuzzy relation matrix are used to obtain the membership vector that represents the comprehensive evaluation result, and the final evaluation level and percentage score are determined.

[0011] Preferably, generating targeted architecture optimization suggestions includes: Based on the comprehensive evaluation results, the decision tree algorithm is used to analyze the indicators that led to the low scores and generate corresponding optimization suggestions. The optimization suggestions include optimizing application code, expanding server resources, optimizing interface interaction design, or improving emergency response plans.

[0012] On the other hand, the present invention provides a deep learning-based system for predicting architectural failures and evaluating application effectiveness, for implementing the above method. This system includes: The data acquisition layer is used to collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture through multiple interfaces. A data preprocessing layer, connected to the data acquisition layer, is used to clean, normalize, and perform feature engineering on the acquired data. The data storage layer, connected to the data preprocessing layer, adopts a collaborative storage mode of graph database and time-series database, used to store architectural asset relationship data and runtime sequence indicator data, respectively. The core functional layer, connected to the data storage layer, includes: The fault prediction module has a built-in deep learning model based on the fusion of long short-term memory network and convolutional neural network, which is used to predict the fault probability and generate early warning based on the processed real-time data. The architecture application effectiveness evaluation module is used to quantitatively evaluate the effectiveness of architecture applications based on the constructed multi-dimensional evaluation index system, using the analytic hierarchy process and fuzzy comprehensive evaluation method, and to generate optimization suggestions. The application presentation layer, connected to the core functional layer, is used to visually display fault warnings, architecture topology, evaluation results, and optimization suggestions.

[0013] Preferably, the data acquisition layer is configured as follows: Server performance metrics were collected by deploying Prometheus and Exporter components. Collect network device status data via the SNMP protocol; Business performance metrics are collected by embedding APM probes in the application. Collect system logs, application logs, and security logs using log collection tools.

[0014] Import unstructured data through the manual data entry module and automatically generate a data index after import.

[0015] Preferably, the deep learning model in the fault prediction module is an LSTM-CNN hybrid model, wherein: The LSTM network part consists of multiple LSTM units connected in series, and is used to capture long-term dependencies in time series data; The CNN network consists of at least one convolutional layer and a pooling layer, used to extract local anomaly patterns from log text data and feature data.

[0016] Preferably, the architecture application effectiveness evaluation module is configured as follows: The weights of each indicator in the evaluation index system are determined using the analytic hierarchy process (AHP). The fuzzy comprehensive evaluation method is used to map the actual values ​​of each indicator to a preset fuzzy evaluation level, and the comprehensive evaluation result is obtained by combining the weights. Based on the comprehensive evaluation results, a decision tree algorithm is used to generate targeted architecture optimization suggestions.

[0017] Based on the above technical solution, deep learning technology is used to deeply mine and analyze the multi-source heterogeneous operating data of the digital architecture, breaking through the "linear dependence" and "experience limitations" of traditional methods in fault prediction, and achieving accurate fault prediction. At the same time, a scientific multi-dimensional evaluation system is established to comprehensively measure the application effectiveness of the architecture in terms of business adaptation, resource utilization, and user services, ultimately improving the stability, operating efficiency and value creation capability of the digital system.

[0018] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the architecture fault prediction and application effectiveness evaluation system based on deep learning provided by the present invention. Figure 2 This is a schematic diagram of a fault prediction model with an LSTM-CNN fusion architecture as its core, constructed in the deep learning-based architecture fault prediction and application effectiveness evaluation method provided by the present invention. Figure 3 This is a flowchart of the fault prediction method in the deep learning-based architecture fault prediction and application effectiveness evaluation method provided by the present invention. Detailed Implementation

[0020] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0021] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0022] This invention provides a deep learning-based method for predicting architectural failures and evaluating application effectiveness, the method comprising: Collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture, and perform cleaning, normalization, and feature engineering to generate a high-quality dataset suitable for model computation. The processed real-time running data is input into a pre-trained deep learning model to obtain the probability of each fault type. Based on the comparison between the probability and a preset threshold, fault warning information is generated. The deep learning model is an LSTM-CNN hybrid model trained with historical fault data and normal operation data, which is used to capture long-term dependencies and local abnormal features in time series data. A multi-dimensional evaluation index system was constructed, and real-time data of each index was obtained; the weight of each evaluation index was determined based on the analytic hierarchy process, and the actual value of each index was mapped to the comprehensive evaluation result using the fuzzy comprehensive evaluation method. The fault warning information and the comprehensive evaluation results are displayed in the form of visual charts, and targeted architecture optimization suggestions are generated based on the comprehensive evaluation results.

[0023] In this implementation, during data cleaning, a density-based spatial clustering algorithm (DBSCAN) is used to identify and filter out outliers in the data. This algorithm identifies data points in low-density areas as noise points and removes them based on the density distribution of the data points. For missing values, multiple imputation (MICE) is used. The principle is to construct multiple imputation datasets and comprehensively consider the relationships between variables to improve the accuracy of missing value imputation. Simultaneously, text processing techniques such as regular expression matching are used to correct formatting errors and garbled characters in the log text data.

[0024] Outlier removal specifically employs the DBSCAN algorithm (parameter settings: ε=0.3, minimum sample size MinPts=5) to perform cluster analysis on continuous indicators such as server CPU utilization and network packet loss rate. Data points in low-density areas are identified as outliers (e.g., CPU utilization suddenly jumps from 50% to 100% without supporting business load) and removed. For missing value imputation, for numerical data (e.g., memory utilization), the Multiple Imputation Method (MICE) is used to construct five imputation datasets. Missing values ​​are predicted using a linear regression model, and the average is taken as the final imputation result. For categorical data (e.g., port status), the .mode() function is used to select the most frequently occurring value for imputation. During log repair, regular expressions are used (e.g., matching "ERROR[(\d{4}-\d{2}-\d{2}\d{2}:\d{2}:\d{2})]" to extract error timestamps) to repair log format errors. UTF-8 encoding conversion is used to resolve garbled characters, ensuring the readability and structure of the log text.

[0025] In this embodiment, when performing data normalization, based on the numerical data, the Z-Score normalization method is used to normalize the data to a standard normal distribution with a mean of 0 and a standard deviation of 1. The formula is as follows: , where x is the original data value, μ is the dataset mean, and σ is the dataset standard deviation. For categorical data, one-hot encoding is used to convert it into a binary vector form suitable for model processing. For example, the categorical variable "operating system type" is encoded as [0,1,0], [1,0,0], etc., to represent different operating system categories.

[0026] In this implementation, during feature engineering, Fourier Transform (FFT) is used to extract frequency domain features of the time-series data to analyze its periodic variation. A sliding window algorithm is used to calculate statistical characteristics within the window, such as mean, median, variance, maximum, and minimum values, capturing local trends. For log text data, word embedding techniques from Natural Language Processing (NLP) (such as Word2Vec and GloVe) are used to map words in the text into low-dimensional dense vectors, thereby extracting semantic features. Named Entity Recognition (NER) technology is used to identify key entities in the logs, such as IP addresses, usernames, and error codes, providing support for subsequent analysis.

[0027] like Figure 2 As shown, this invention constructs a fault prediction model with an LSTM-CNN fusion architecture as its core, wherein: In the LSTM part, the LSTM network units effectively capture long-term dependencies in time-series data through the coordinated operation of forget gates, input gates, output gates, and memory units. The forget gate ft determines the dependencies from past memory units. What information is retained, and what is the calculation formula? Where σ is the sigmoid activation function, W f It is the forget gate weight matrix. h t 1 represents the output of the hidden layer at the previous moment. x t This is the input at the current moment. b f It is the forget gate bias term; input gate i t The formula for determining which new information to add to the memory unit is: Candidate memory units Used to generate potentially updated memory content. ; memory unit C t The update is , where ⊙ represents element-wise multiplication; output gate o t The information output from the control memory unit to the hidden layer Hidden layer output Multiple LSTM units are connected sequentially in time series to form an LSTM network layer, which deeply mines the data dependencies in the time dimension.

[0028] In the CNN part, the convolutional layers of a CNN extract local features from the data through a sliding convolution operation where the convolutional kernel moves across the data. Taking one-dimensional convolution as an example, let the input data be X, the convolutional kernel be K, and the formula for calculating the output feature map Y is: ,in The coordinates of the output feature map are given, (m,n) represents the kernel coordinates, M and N are the kernel sizes, and b is the bias term. In this system, a combination of multiple convolutional and pooling layers is designed for both preprocessed log text data and time-series data after feature engineering. The convolutional layers use kernels of different sizes, such as 3 and 5, to capture local features at different scales. The pooling layers employ max pooling to reduce data dimensionality while retaining key feature information. For example, the max pooling window size is set to 2, and feature downsampling is achieved by selecting the maximum value within the window.

[0029] For model training, a massive amount of historical fault data and corresponding normal operation data were used to train the LSTM-CNN hybrid model. The dataset was divided into training, validation, and test sets at a ratio of 70%, 15%, and 15%, respectively. During training, the Adaptive Moment Estimation (Adam) algorithm was used to update the model parameters. This algorithm integrates first-order and second-order moment estimation of the gradient, and can dynamically adjust the learning rate during training to accelerate model convergence. Its core update formula is: First-order moment estimation:

[0030] Second-order moment estimation:

[0031] Parameter update: (4) in, The gradient at the current time. and The exponential decay rate estimated by moments (usually) =0.9, =0.999). For learning rate, To prevent small constants in the division-by-zero operation, For the current first-order moment estimate, For second-order moment estimation, and These are the first and second moment estimates after bias correction, respectively. , The training process uses mean squared error (MSE) as the loss function. By minimizing the loss function, the deviation between the model's predicted values ​​and the true values ​​is continuously reduced, thereby improving the accuracy of the model's predictions.

[0032] To achieve a combination of "visualized architecture topology" and "intuitive fault warning," helping operations and maintenance personnel quickly locate fault nodes and their impact range, and to embody the core function of "a single dynamic architecture diagram"—the overlay display of static topology and real-time fault data—this approach clarifies the visual interaction design, provides prototype references for application presentation layer development, and ensures user-friendly operation. Figure 3 As shown, the real-time running data processed by the data acquisition and preprocessing module is input into the fully trained LSTM-CNN model. The model outputs the probability value of each fault type, and fault prediction is achieved by comparing it with a pre-set threshold. For example, fault types are divided into multiple categories such as hardware faults, software faults, and network faults. The model outputs a probability vector of the form [0.1, 0.8, 0.1], corresponding to the probability of occurrence of the above fault categories. If the pre-set warning threshold is 0.7, when the probability of a software fault reaches 0.8, the system determines that a software fault may occur and generates detailed fault warning information, including the fault type, possible fault location, and a preliminary assessment of the fault's impact range.

[0033] In this implementation, a comprehensive and detailed evaluation index system is first constructed from multiple dimensions. The performance dimension covers metrics such as response time, throughput, and concurrent users. Response time refers to the time interval from a user's request to the system's response, precisely measured using APM tools. Throughput is the number of transactions processed by the system per unit time, obtained through application log statistical analysis. Concurrent users represent the number of user requests the system can handle simultaneously, obtained using load testing tools. The reliability dimension includes metrics such as failure rate, mean time between failures (MTBF), and mean time to recovery (MTTR). The failure rate is calculated as the ratio of the number of failures to the total runtime within a specific time period. MTBF is estimated based on historical failure data using reliability statistics methods. MTTR is obtained by recording the time from each failure to normal operation and averaging the results. The resource utilization dimension involves metrics such as CPU utilization, memory utilization, and disk utilization, calculated by collecting relevant data using the operating system's built-in performance monitoring tools or server management software. User experience metrics include user satisfaction survey scores, operation success rate, and user retention rate. User satisfaction surveys collect user feedback through online questionnaires and user feedback platforms and quantify it into scores. Operation success rate is the percentage of users who successfully perform key business operations. User retention rate is measured by analyzing the percentage of users who reuse the system within a certain period of time.

[0034] Thus, as shown in Table 1, the constructed evaluation system of "4 dimensions and 12 indicators" covers the entire lifecycle performance of the digital architecture:

[0035] Secondly, the weights of each evaluation indicator are determined using the Analytic Hierarchy Process (AHP). First, a judgment matrix is ​​constructed. For the criterion layer and the indicator layer, the relative importance of each indicator is determined by expert scoring or pairwise comparison, forming a judgment matrix where represents the importance of each indicator relative to the other, satisfying aii=1. Then, the maximum eigenvalue λmax of the judgment matrix and its corresponding eigenvector W are calculated using mathematical methods such as eigenvalue decomposition. The eigenvectors are normalized to obtain the weight vectors of each indicator, ensuring that the sum of the weights is 1. Combining the fuzzy comprehensive evaluation method, the actual values ​​of each indicator are mapped to corresponding fuzzy evaluation levels, such as "excellent," "good," "medium," "poor," and "very poor," according to pre-set fuzzy evaluation standards, constructing a fuzzy relation matrix R. Finally, the matrix operation B=W is used to determine the weights. R represents the comprehensive evaluation result of the architecture application effectiveness. The elements in vector B are the membership degrees corresponding to the evaluation levels. By comparing the membership degrees, the final evaluation level of the architecture application effectiveness is determined. Specifically: In AHP weight calculation: 1. Construct a judgment matrix: Invite multiple digital architecture experts to use the 1-9 scale (1 = equally important, 3 = slightly important, 5 = significantly important, 7 = strongly important, 9 = extremely important) to score the relative importance of each dimension and indicator, forming a judgment matrix (such as the criterion-level judgment matrix: performance dimension B1, reliability dimension B2, resource utilization dimension B3, user experience dimension B4). 2. Consistency Test: Calculate the maximum eigenvalue λmax of the judgment matrix and pass the consistency index test. (where n is the matrix order) and the consistency ratio is calculated using the random consistency index RI (RI = 0.90 when n = 4). When CR < 0.1, the decision matrix satisfies the consistency requirement; 3. Weight vector calculation: The eigenvector is calculated using the summation method. After normalizing each column of the judgment matrix, the summation is performed row by row, and then normalized again to obtain the weight vector (e.g., criterion layer weights: B1=0.4, B2=0.3, B3=0.2, B4=0.1). In fuzzy comprehensive evaluation, 1. Establish evaluation levels: Divide the effectiveness levels into 5 levels: "Excellent (90-100 points)", "Good (80-89 points)", "Average (70-79 points)", "Poor (60-69 points)" and "Poor (<60 points)"; 2. Construct a fuzzy relation matrix R: Based on the matching degree between the actual value of the indicator and the evaluation standard, determine the membership degree of each indicator to the evaluation level (e.g., response time = 500ms, corresponding to a membership degree of 0.8 for "good" and 0.2 for "excellent"), forming an R matrix (12 rows and 5 columns). 3. Comprehensive evaluation calculation: The comprehensive membership vector B is obtained by matrix multiplication B=W*R ​​(W is the indicator weight vector, 1 row and 12 columns). The level with the highest membership degree is taken as the final evaluation level. At the same time, the membership vector is converted into a percentage score (e.g., B=[0.1,0.7,0.2,0,0], corresponding score=90×0.1+85×0.7+75×0.2=84 points, judged as "good").

[0036] Finally, real-time evaluation and feedback are conducted. The system collects data for each evaluation indicator at set time intervals, such as every 5 minutes. Based on the constructed evaluation model, the effectiveness of the digital architecture application is dynamically evaluated. The evaluation results are presented in an intuitive visualization format, such as using visualization libraries like Echarts and D3.js to generate dashboards, bar charts, line charts, radar charts, etc., displaying real-time values, trend changes, and comprehensive evaluation scores for each indicator. Simultaneously, based on the evaluation results, the system uses intelligent analysis technologies such as decision tree algorithms to automatically generate targeted optimization suggestions. For example, when CPU utilization remains consistently high, it suggests optimizing application algorithms, increasing server resources, or adjusting business load balancing strategies, and provides feedback to system administrators to help them efficiently adjust and optimize the digital architecture. 1. Real-time data acquisition: The system collects data on each evaluation indicator every 5 minutes and obtains real-time values ​​from data sources such as APM tools, Prometheus, and user feedback platforms through API interfaces; 2. Dynamic evaluation calculation: Based on the preset evaluation model, it automatically completes the matching of indicator weights, calculation of membership degree and generation of comprehensive score, and updates the comprehensive evaluation results every 30 minutes; 3. Visual presentation: The performance evaluation report uses a radar chart to show the score distribution of each dimension indicator, a line chart to show the historical trend of the overall score, and a heat map to show the real-time distribution of resource utilization. 4. Intelligent Optimization Suggestions: Based on the evaluation results, optimization suggestions are generated using the C4.5 decision tree algorithm. If the average CPU utilization is >80%, it is recommended to "optimize the application code (such as reducing cyclic redundancy) or expand the server CPU resources". If the user satisfaction score is less than 3.5, it is recommended to "optimize the interface interaction design (such as simplifying the operation process) or add a function tutorial"; If MTTR > 4 hours, it is recommended to "improve the emergency response plan (such as adding automatic recovery scripts) or strengthen the training of maintenance personnel"; Feedback loop: Optimization suggestions are pushed to the administrator via system messages. After the administrator processes them, he enters the results into the system. The system tracks the changes in indicators in subsequent evaluation cycles, forming a closed loop of "evaluation-suggestion-action-verification".

[0037] On the other hand, the present invention also provides a deep learning-based system for predicting architectural failures and evaluating application effectiveness, for implementing the above method. This system includes: The data acquisition layer is used to collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture through multiple interfaces. A data preprocessing layer, connected to the data acquisition layer, is used to clean, normalize, and perform feature engineering on the acquired data. The data storage layer, connected to the data preprocessing layer, adopts a collaborative storage mode of graph database and time-series database, used to store architectural asset relationship data and runtime sequence indicator data, respectively. The core functional layer, connected to the data storage layer, includes: The fault prediction module has a built-in deep learning model based on the fusion of long short-term memory network and convolutional neural network, which is used to predict the fault probability and generate early warning based on the processed real-time data. The architecture application effectiveness evaluation module is used to quantitatively evaluate the effectiveness of architecture applications based on the constructed multi-dimensional evaluation index system, using the analytic hierarchy process and fuzzy comprehensive evaluation method, and to generate optimization suggestions. The application presentation layer, connected to the core functional layer, is used to visually display fault warnings, architecture topology, evaluation results, and optimization suggestions.

[0038] like Figure 1 As shown, in this system, the data acquisition layer comprehensively collects multi-source heterogeneous architecture asset data; the data preprocessing layer transforms raw data into high-quality data suitable for model calculation through operations such as cleaning, standardization, and feature engineering; the data storage layer adopts a collaborative storage mode of "graph database + time series database" to respectively carry architecture asset relationship data and real-time operation indicator time series data; the core function layer relies on the LSTM-CNN hybrid model to realize fault prediction and the AHP-fuzzy comprehensive evaluation model to realize application effectiveness evaluation; the application display layer presents fault warning information, effectiveness evaluation results, and thematic maps to users through visualization tools, forming a closed loop of "data acquisition-processing-storage-computation-display" to provide accurate decision support for digital architecture management.

[0039] Specifically, in this implementation, the architecture assets utilize diverse interfaces, such as RESTful APIs and Syslog protocol interfaces, to conduct comprehensive data collection on heterogeneous components within the digital architecture. For the server side, open-source monitoring tools like Prometheus, along with their compatible Exporter components, are used to collect real-time data on key performance indicators such as CPU, memory, disk I / O, and network ports. Regarding network devices, the SNMP protocol is used to connect to routers, switches, and other devices to collect data on network traffic, port status, and packet loss rate. For applications, APM probes, such as those from NewRelic and Datadog, are embedded in the code to obtain key data such as application response time, transaction success rate, and database query count. Simultaneously, log collection tools, such as Filebeat and Logstash, are used to collect text data, including operation records and error stack traces, from various log sources, including system logs, application logs, and security logs.

[0040] For heterogeneous components (servers, network devices, applications, databases, etc.) in a digital architecture, multi-dimensional data collection is achieved using diverse interfaces and tools. 1. Server data collection: Deploy the Prometheus monitoring tool, along with components such as NodeExporter and ProcessExporter, to collect performance metrics such as CPU utilization, memory usage, disk I / O throughput, and network port traffic at a frequency of 15 seconds / time. The data is output in real time through the PromQL query interface. 2. Network device data collection: Based on the SNMPv3 protocol (supporting encrypted transmission and authentication), connect to devices such as routers and switches to collect data such as port status (Up / Down), network packet loss rate, latency, and bandwidth utilization. The collection cycle is set to 30 seconds / time. 3. Application Data Collection: Embed APM probes (such as NewRelicAPM and DatadogAPM) in the application code to collect business metrics such as application response time (RT), transaction success rate, database query time, and API call volume. At the same time, capture cross-service call chain data through the SkyWalking distributed tracing tool. 4. Log data collection: Filebeat is deployed on each node to collect system logs (syslog and messages in the / var / log directory), application logs (such as log4j logs for Java applications and logging logs for Python applications) and security logs (such as SSH login logs and firewall alarm logs) in real time. The log data is filtered by Logstash (such as extracting key fields and filtering redundant information) before being transmitted to subsequent modules. 5. Unstructured Data Acquisition: Import unstructured data such as architecture design documents (e.g., SG-EA architecture blueprint, system summary design specification) and standard specifications (e.g. "Digital Architecture Design Standard") through the manual data entry module. Supports formats such as PDF, Word, and Excel. Data index is automatically generated after import.

[0041] In summary, this invention employs a hybrid deep learning model of LSTM-CNN. The CNN sub-network can accurately capture local anomalies in log text and performance metrics (such as the keyword "OutOfMemoryError" and sudden spikes in CPU usage), while the LSTM sub-network can effectively mine long-term dependencies in time-series data (such as the slow growth trend of memory utilization). Compared with traditional rule-based methods (such as threshold judgment), the fault prediction accuracy is improved by 20%-30%, the false negative rate is reduced to below 5%, and potential faults can be warned 1-2 hours in advance, allowing sufficient time for operation and maintenance. The accuracy of fault prediction is greatly improved.

[0042] Secondly, this invention constructs an evaluation system of 12 indicators across 4 dimensions: performance, reliability, resources, and user experience, covering the business support capabilities, resource allocation efficiency, and user service quality of the digital architecture. It combines the AHP algorithm to achieve scientific quantification of indicator weights (avoiding subjective human bias) and uses the fuzzy comprehensive evaluation method to transform qualitative indicators (such as user satisfaction) into quantitative scores. The evaluation result error rate is controlled within 3%, which can provide accurate data support for architecture optimization and make the evaluation of architecture application effectiveness comprehensive and accurate.

[0043] Meanwhile, the data acquisition cycle of this invention is as short as 15 seconds / time, and the update frequency of fault prediction and effectiveness evaluation results is 1 minute / time and 30 minutes / time, respectively, which can reflect the changes in the architecture's operating status in real time. The model training supports incremental learning. When new components are added to the architecture (such as adding cloud servers) or the business model is adjusted, only new data needs to be added to update the model parameters without reconstructing the system. This adapts to the dynamic iteration requirements of digital architecture and greatly enhances real-time performance and dynamic adaptability.

[0044] Finally, this invention has good scalability and reusability. Each module adopts the design of "standardized interface + decoupled function". The data acquisition layer can be adapted to different types of architecture components (such as IoT devices and edge computing nodes) by adding Exporters and probes. The model algorithms of the core functional layer support plug-in replacement (such as replacing LSTM with Transformer). The evaluation index system can add indicators (such as "energy efficiency") through the configuration interface without modifying the underlying code. It can be quickly reused in digital architecture management scenarios of different industries such as finance, manufacturing, and energy.

[0045] The following provides a specific embodiment to illustrate the present invention: The digital architecture of an energy company includes 20 physical servers, 5 network switches, and 8 core application systems.

[0046] During the data acquisition and preprocessing implementation phase: Data collection and deployment: Deploy PrometheusNodeExporter on all servers to collect CPU, memory, and disk I / O data; enable SNMPv3 protocol on network switches and monitor network traffic and packet loss rate through Zabbix; embed NewRelicAPM probes in application systems such as ERP and SCADA to collect response time and transaction success rate; collect system logs and application logs of all nodes through Filebeat, and transmit log data to Elasticsearch storage after being filtered by Logstash. Data preprocessing results: Preprocessing tasks are performed daily at 2:00 AM. The DBSCAN algorithm is used to remove approximately 0.8% of outliers (such as a momentary 100% CPU utilization caused by a server restart), and the MICE method is used to fill in approximately 1.5% of missing values ​​(such as 5 minutes of missing data caused by a network interruption). Numerical indicators such as CPU utilization and response time are Z-score normalized, and categorical indicators such as operating system type and fault level are encoded. Finally, a training data format with 128 time steps and 64 features is generated.

[0047] During the fault prediction implementation phase: Model training process: Operational data from the company in 2022-2023 was collected, including 120 failure events (35 hardware failures, 55 software failures, and 30 network failures). This data was divided into a training set (84 events), a validation set (18 events), and a test set (18 events) in a 7:1.5:1.5 ratio. The Adam optimization algorithm was used with a learning rate of 0.001. After 80 epochs of training, the validation set loss stabilized at around 0.04, and the test set accuracy reached 92%. Fault warning example: At 10:00 on March 15, 2024, the system collected memory utilization data of the ERP server (increasing from 60% to 75% for 5 consecutive minutes). After inputting it into the model, the output "software failure - memory leak" probability was 0.82 (threshold 0.7), triggering a high-risk warning. The warning message showed "Memory overflow is expected to occur within 1 hour, affecting the order generation function of the marketing business". The maintenance personnel restarted the ERP application and cleared the memory cache as suggested. At 10:30, the memory utilization dropped to 40%, successfully avoiding the failure.

[0048] During the implementation phase of architecture application effectiveness evaluation: Weighting determination: Five digital experts in the energy industry were invited to score using the AHP method, and the weights of the criteria layer were finally determined as follows: performance dimension 0.4, reliability dimension 0.3, resource utilization dimension 0.2, and user experience dimension 0.1; the weights of the indicator layer include "average response time 0.15, transaction throughput 0.12, and failure rate 0.1", etc. Evaluation Results and Optimizations: The overall evaluation score in April 2024 was 78 points (above average), with "user satisfaction score of 3.2 points (low)" and "average CPU utilization of 75% (reasonable)". The system suggested "optimizing the order query interface of the ERP system and adding operation guidance pop-ups". After the company implemented the optimization, the user satisfaction score in May increased to 4.1 points, and the overall evaluation score rose to 83 points (good).

[0049] In practical applications, the fault prediction module of this invention can use a Transformer model instead of the original LSTM-CNN hybrid model. The Transformer's self-attention mechanism captures long-distance dependencies in time-series data without relying on the gating structure of LSTM. The input layer supplements the temporal information through positional encoding, the encoder layer uses a 6-layer multi-head attention and feedforward neural network (FFN), and the output layer outputs the fault probability through Softmax. The advantage of this approach is its stronger ability to capture dependencies in long-term data (such as performance indicators of more than one week).

[0050] Alternatively, an XGBoost + time series decomposition model can be used, employing a combination of "time series decomposition + ensemble learning". First, the time series data is decomposed into trend, seasonal, and residual terms using the STL (Season on the Land of the Trend, Decomposition using Loess) algorithm. The trend and seasonal terms are predicted using an ARIMA model, while the residual terms are captured using an XGBoost model (based on decision tree ensemble) to capture nonlinear features. The prediction results of the three are then fused together, and a classifier is trained using fault labels to output the fault probability. The advantage of this approach is its strong model interpretability (it can output feature importance), making it suitable for scenarios with high requirements for model interpretability (such as the financial industry).

[0051] In addition, the original storage solution of "Graph Database (NebulaGraph) + Time Series Database (InfluxDB)" can be replaced by a time series database (InfluxDB) + relational database (MySQL). InfluxDB is still used to store real-time runtime metrics time series data; architectural asset relationship data (such as "business-application-data" relationships) is stored in MySQL, and the integrity of the relationship is maintained through foreign key constraints (such as the "business ID" of the application table being associated with the primary key of the business table). The advantage of this solution is that MySQL has low deployment cost and low operation and maintenance difficulty, and is suitable for scenarios with relatively simple architectural asset relationships (such as no complex many-to-many relationships).

[0052] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0057] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0058] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0059] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0060] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for predicting architectural failures and evaluating application effectiveness based on deep learning, characterized in that, The method includes: Collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture, and perform cleaning, normalization, and feature engineering to generate a high-quality dataset suitable for model computation. The processed real-time running data is input into a pre-trained deep learning model to obtain the probability of each fault type. Based on the comparison between the probability and a preset threshold, fault warning information is generated. The deep learning model is an LSTM-CNN hybrid model trained with historical fault data and normal operation data, which is used to capture long-term dependencies and local abnormal features in time series data. A multi-dimensional evaluation index system was constructed, and real-time data of each index was obtained; the weight of each evaluation index was determined based on the analytic hierarchy process, and the actual value of each index was mapped to the comprehensive evaluation result using the fuzzy comprehensive evaluation method. The fault warning information and the comprehensive evaluation results are displayed in the form of visual charts, and targeted architecture optimization suggestions are generated based on the comprehensive evaluation results.

2. The method for predicting architectural failures and evaluating application effectiveness based on deep learning according to claim 1, characterized in that, The data cleaning includes: The density-based spatial clustering algorithm DBSCAN is used to identify and remove outliers in the data; and multiple imputation is used to impute missing values ​​in the data.

3. The method for predicting architectural failures and evaluating application effectiveness based on deep learning according to claim 1, characterized in that, The data normalization includes: According to formula (1), Z-Score normalization is used to normalize the numerical data to a standard normal distribution with a mean of 0 and a standard deviation of 1. ，（1） Where x is the original data value, μ is the dataset mean, and σ is the dataset standard deviation; One-hot encoding is used to convert categorical data into binary vectors suitable for model processing; The frequency domain features of time series data are extracted using Fourier Transform (FFT), and statistical features are calculated using a sliding window. It also uses word embedding to extract semantic features from log text data and Named Entity Recognition (NER) to extract key entities.

4. The method for predicting architectural failures and evaluating application effectiveness based on deep learning according to claim 1, characterized in that, The training process of the LSTM-CNN hybrid model includes: The dataset containing historical fault labels is divided into training, validation, and test sets proportionally. Based on formulas (2) to (4), the adaptive moment estimation algorithm is used as the optimizer, with the goal of minimizing the mean squared error loss function, to iteratively update the model parameters. ，（2） ，（3） ，（4） in, The gradient at the current time. and The exponential decay rate is estimated by moments. For learning rate, To prevent small constants in the division-by-zero operation, For the current first-order moment estimate, For second-order moment estimation, and These are the first and second moment estimates after bias correction, respectively. , .

5. The method for predicting architectural failures and evaluating application effectiveness based on deep learning according to claim 1, characterized in that, The effectiveness evaluation of architecture application includes: The evaluation index system is constructed as follows: performance dimension includes average response time, transaction throughput and concurrent users; reliability dimension includes failure rate, mean time between failures (MTBF) and mean time to recovery (MTTR); resource utilization dimension includes average CPU utilization, average memory utilization and storage resource utilization; and user experience dimension includes user satisfaction score, operation success rate and user retention rate. Based on the analytic hierarchy process, a judgment matrix is ​​constructed through expert scoring, the consistency ratio is calculated and a consistency test is performed, and the weight vector of each indicator is obtained by solving the problem. Establish a fuzzy evaluation level, determine the membership degree of each indicator to the fuzzy evaluation level based on the actual value of each indicator, and construct a fuzzy relation matrix; The weight vector and the fuzzy relation matrix are used to obtain the membership vector that represents the comprehensive evaluation result, and the final evaluation level and percentage score are determined.

6. The method for predicting architectural failures and evaluating application effectiveness based on deep learning according to claim 1, characterized in that, Generate targeted architecture optimization recommendations, including: Based on the comprehensive evaluation results, the decision tree algorithm is used to analyze the indicators that lead to low scores and generate corresponding optimization suggestions. The optimization suggestions include optimizing application code, expanding server resources, optimizing interface interaction design, or improving emergency response plans.

7. A deep learning-based system for predicting architectural failures and evaluating application effectiveness, characterized in that, The system for implementing the method as described in any one of claims 1-6 comprises: The data acquisition layer is used to collect multi-source heterogeneous operational data from heterogeneous components in the digital architecture through multiple interfaces. A data preprocessing layer, connected to the data acquisition layer, is used to clean, normalize, and perform feature engineering on the acquired data. The data storage layer, connected to the data preprocessing layer, adopts a collaborative storage mode of graph database and time-series database, used to store architectural asset relationship data and runtime sequence indicator data, respectively. The core functional layer, connected to the data storage layer, includes: The fault prediction module has a built-in deep learning model based on the fusion of long short-term memory network and convolutional neural network, which is used to predict the fault probability and generate early warning based on the processed real-time data. The architecture application effectiveness evaluation module is used to quantitatively evaluate the effectiveness of architecture applications based on the constructed multi-dimensional evaluation index system, using the analytic hierarchy process and fuzzy comprehensive evaluation method, and to generate optimization suggestions. The application presentation layer, connected to the core functional layer, is used to visually display fault warnings, architecture topology, evaluation results, and optimization suggestions.

8. The deep learning-based architecture fault prediction and application effectiveness evaluation system according to claim 7, characterized in that, The data acquisition layer is configured as follows: Server performance metrics were collected by deploying Prometheus and Exporter components. Collect network device status data via the SNMP protocol; Business performance metrics are collected by embedding APM probes in the application. Collect system logs, application logs, and security logs using log collection tools. Import unstructured data through the manual data entry module and automatically generate a data index after import.

9. The deep learning-based architecture fault prediction and application effectiveness evaluation system according to claim 7, characterized in that, The deep learning model in the fault prediction module is an LSTM-CNN hybrid model, wherein: The LSTM network part consists of multiple LSTM units connected in series, and is used to capture long-term dependencies in time series data; The CNN network consists of at least one convolutional layer and a pooling layer, used to extract local anomaly patterns from log text data and feature data.

10. The deep learning-based architecture fault prediction and application effectiveness evaluation system according to claim 7, characterized in that, The architecture application effectiveness evaluation module is configured as follows: The weights of each indicator in the evaluation index system are determined using the analytic hierarchy process (AHP). The fuzzy comprehensive evaluation method is used to map the actual values ​​of each indicator to a preset fuzzy evaluation level, and the comprehensive evaluation result is obtained by combining the weights. Based on the comprehensive evaluation results, a decision tree algorithm is used to generate targeted architecture optimization suggestions.

Images