Intranet service quality optimization method and system based on deep reinforcement learning

Abstract

The invention provides an intranet service quality optimization method and system based on deep reinforcement learning, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting multi-dimensional performance index data, carrying out the preprocessing, constructing an analytic hierarchy model, outputting the performance score of each layer, integrating the performance score of each layer, and outputting an intranet service quality score. Constructing a simulation environment, adjusting hyper-parameters of the adaptive deep reinforcement learning model, continuously learning in an actual intranet environment, determining multi-dimensional rewards, performing multi-target optimization, and generating a multi-target optimization result; and setting a hierarchical performance index threshold, if a hierarchical optimization strategy is triggered, adjusting a network topology structure, adjusting resource allocation and service configuration, adjusting weights of different indexes in the service quality evaluation model, updating a reward function, solving an optimal service decision, generating an optimal service decision sequence, and executing the optimal service decision sequence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for optimizing intranet service quality based on deep reinforcement learning. Background Art

[0002] With the continuous deepening of enterprise informatization, the intranet has become a key infrastructure to support enterprise business operations and data transmission. The quality of intranet services directly affects the operational efficiency and user experience of enterprises. However, the complexity and variability of the intranet environment, the rapid evolution of business needs, and the dynamic interaction of massive devices and applications have brought huge challenges to the optimization of intranet service quality.

[0003] Traditional intranet optimization methods mainly rely on manual configuration and rule setting, which are difficult to adapt to the dynamic changes and complexity of the intranet environment. Network administrators need to manually collect and analyze massive amounts of performance data and formulate optimization strategies based on experience. However, it is often difficult to discover and locate performance bottlenecks in a timely manner, limiting the optimization effect.

[0004] Therefore, a method is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for optimizing the quality of intranet service based on deep reinforcement learning, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for optimizing intranet service quality based on deep reinforcement learning, comprising:

[0007] Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to integrate the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model;

[0008] Acquire historical intranet performance data from a database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in an actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results;

[0009] Based on the multi-objective optimization result, hierarchical performance indicator thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, and the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and fault points are identified and the resource allocation and service configuration are adjusted in combination with the preset prediction model. User feedback data is periodically collected, and the weights of different indicators in the preset service quality evaluation model are adjusted according to the user feedback data. The reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

[0010] In an optional embodiment,

[0011] Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model, including:

[0012] Deploy a hierarchical distributed sensor network, which includes bottom-level sensing nodes, middle-level aggregation nodes, and top-level aggregation nodes. A time-based polling mechanism and a threshold-based active reporting mechanism are combined to collect multi-dimensional performance indicator data in the intranet in real time through the hierarchical distributed sensor network. The multi-dimensional performance indicator data is adaptively optimized for collection frequency, the collection frequency is dynamically adjusted according to the indicator change speed, and the collected multi-dimensional performance indicator data is compressed in combination with differential coding technology.

[0013] Using an improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data, using an adaptive weighted moving average method to smooth the detected multi-dimensional performance indicator data, and performing trend analysis on the smoothed multi-dimensional performance indicator data based on a nonlinear trend term estimation method and an adaptive seasonal cycle detection mechanism;

[0014] Constructing a hierarchical analysis model, using the hierarchical analysis method to stratify and assign weights to the processed multi-dimensional performance indicator data, dividing the multi-dimensional performance indicator data into a network layer, a service layer, and a resource layer, using the partial least squares regression method to establish a relationship model between the network layer, the service layer, and the resource layer, using an integrated learning method to construct a performance scoring model for the network layer, the integrated learning method combines random forests and gradient boosting trees, using a support vector regression model based on multi-kernel learning technology to construct a performance scoring model for the service layer, and using a long short-term memory network model based on an attention mechanism to construct a performance scoring model for the resource layer;

[0015] Construct a fuzzy set based on triangular fuzzy numbers, and use an adaptive fuzzy set generation algorithm to dynamically adjust fuzzy set parameters, design a fuzzy rule base, use a data-driven approach combined with expert knowledge and machine learning technology to automatically generate and optimize fuzzy rules, and use an improved fuzzy inference system to comprehensively evaluate the performance scores of the network layer, the service layer, and the resource layer to obtain an intranet service quality score;

[0016] A multi-scale state space is designed, and the principal component analysis method is used to reduce the dimension of the state space. A continuous action space including network parameter adjustment, service configuration optimization and resource allocation strategy is defined, a multi-objective reward function is constructed, and a reward allocation mechanism based on Pareto optimization is introduced to construct the adaptive deep reinforcement learning model.

[0017] In an optional embodiment,

[0018] Using the improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data includes:

[0019]

[0020] in, Represents the i-th data point P i The local anomaly factor is used to measure the data point P i The degree of abnormality, Represents data point P i With k i The distance neighborhood, that is, in the selected dynamic neighborhood size k i Next, the set of neighbor data points included, lrd(Pj ) represents the data point P j The local reachability density, lrd(P i ) represents the data point P i The local reachable density of Represents neighborhood The number of data points in , that is, data point P i The neighborhood size.

[0021] In an optional embodiment,

[0022] The historical intranet performance data is obtained from the database to build a simulation environment, and the adaptive deep reinforcement learning model is pre-trained offline. The hyperparameters of the adaptive deep reinforcement learning model are adjusted by a learning algorithm based on priority experience replay. The adaptive deep reinforcement learning model is optimized again by combining the adversarial network to generate multiple network abnormality scenarios. The optimized adaptive deep reinforcement learning model is used to continuously learn in the actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to the pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results including:

[0023] Extracting historical intranet performance data from a database, the historical intranet performance data including network throughput, delay, packet loss rate, CPU utilization and memory usage, cleaning, standardizing and feature engineering the historical intranet performance data, using principal component analysis to perform feature dimension reduction, and using a Monte Carlo method combined with variational inference to construct a statistical model for the historical intranet performance data after feature dimension reduction, simulating dynamic changes in the intranet environment, and obtaining a simulation environment;

[0024] For the pre-built adaptive deep reinforcement learning model, the simulation environment is used for offline pre-training. During the offline pre-training process, the behavior cloning technology is used to imitate the optimal decision in the historical data, a hierarchical experience replay pool is constructed, and the experience is classified and stored according to importance. A priority sampling strategy based on temporal difference error is used, and a Bayesian optimization method is used to search for the optimal configuration in the hyperparameter space. A Gaussian process regression model is used to predict the hyperparameter performance. The next set of hyperparameters to be evaluated is selected through the expected improvement criterion, and the hyperparameters with the best performance are selected as the hyperparameters of the adaptive deep reinforcement learning model;

[0025] Construct a generative adversarial network architecture, including a generator and a discriminator, use conditional generative adversarial network technology to control the properties of the generated scenario, generate a variety of network anomaly scenarios, input the generated network anomaly scenarios into the adaptive deep reinforcement learning model, use the policy gradient method to update the model parameters, optimize the adaptive deep reinforcement learning model again, and deploy the optimized adaptive deep reinforcement learning model to the actual intranet environment for continuous learning;

[0026] Deploy a distributed sensor network, use edge computing technology to perform preliminary data processing, collect and receive multi-level network status information in real time, use the ε-greedy strategy to select optimization actions based on the multi-level network status information, introduce a soft update mechanism to execute the optimization actions, update the network status and calculate multi-dimensional rewards after executing the optimization actions, use sliding window technology to comprehensively evaluate short-term and long-term optimization effects, use a multi-objective Q learning algorithm to simultaneously optimize multiple performance indicators, introduce a solution set update strategy based on dominance relations to maintain non-dominated solution sets, use a weighted Chebyshev divergence method to select the optimal solution in each iteration, generate a Pareto optimal solution set based on a non-dominated sorting genetic algorithm, and combine clustering technology for grouping to obtain the multi-objective optimization results.

[0027] In an optional embodiment,

[0028] The statistical model is constructed using the Monte Carlo method combined with variational inference as shown in the following formula:

[0029]

[0030] Among them, ELOB(q) represents the variational lower bound, K represents the number of Monte Carlo sampling, and p(x, y, z k ) represents the latent variable z obtained by the kth sampling k The corresponding joint probability distribution, z k represents the hidden variable obtained by the kth sampling, q(z k |x, y) represents the latent variable z obtained by the k-th sampling of the variational distribution q(z|x, y) k The approximate posterior probability density of .

[0031] In an optional embodiment,

[0032] Based on the multi-objective optimization results, hierarchical performance index thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance index threshold, a hierarchical optimization strategy is triggered, and the network topology is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and failure points are identified in combination with the pre-set prediction model, and resource allocation and service configuration are adjusted, including:

[0033] Perform cluster analysis on the multi-objective optimization results, use a clustering algorithm to divide the multi-objective optimization results into three subsets: network layer, service layer, and resource layer, apply principal component analysis to the three subsets of the network layer, service layer, and resource layer, respectively, extract the main feature vectors as the key performance indicators corresponding to each layer, use extreme value statistical theory, and establish a performance indicator probability distribution model for the network layer, service layer, and resource layer based on generalized extreme value distribution, and use a quantile regression method, combined with historical data and expert experience, to set adaptive performance indicator thresholds for the network layer, service layer, and resource layer;

[0034] Deploy a distributed monitoring agent to collect the performance parameters of the network layer, service layer and resource layer in real time, calculate the exponential moving average and standard deviation of the performance parameters of the network layer, service layer and resource layer through an anomaly detection algorithm based on a sliding window, design a multi-level trigger mechanism, and classify alarms into mild alarms, moderate alarms and severe alarms according to the relationship between the performance parameters, the exponential moving average, the standard deviation and the performance indicator threshold. Combined with a decision-making system based on a rule engine, trigger corresponding hierarchical optimization strategies according to the mild alarm, the moderate alarm and the severe alarm;

[0035] Design a network topology representation learning module based on graph neural network, encode the network topology structure into a low-dimensional vector representation, build a dynamic graph convolutional network, capture the temporal change characteristics of the network topology, fuse the output of the dynamic graph convolutional network with the traditional network state characteristics as the input of the adaptive deep reinforcement learning model, design a policy gradient algorithm based on the actor-critic architecture, in which the actor network outputs continuous network adjustment actions, the critic network evaluates the value of the action, introduces an entropy regularization term, adjusts the exploration degree of the network adjustment action, and combines the proximal policy optimization algorithm to limit the policy update step size by clipping the objective function;

[0036] Construct a multilayer perceptron, input the performance indicators of each layer, output the probability scores of potential bottlenecks and failure points, introduce the attention mechanism into the multilayer perceptron, integrate the decision tree model with the multilayer perceptron, use the Shapley additive explanatory value method to explain the output results of the integrated learning framework, use the genetic algorithm to design the resource allocation optimizer, use chromosome encoding to represent the allocation scheme of different resources, define the multi-objective fitness function, and use the multi-objective optimization process of non-dominated sorting to generate the resource allocation scheme.

[0037] In an optional embodiment,

[0038] Periodically collect user feedback data, adjust the weights of different indicators in the preset service quality evaluation model according to the user feedback data, update the reward function of the adaptive deep reinforcement learning model according to the adjusted service quality evaluation model and solve the optimal service decision based on the updated reward function, generate the optimal service decision sequence and execute it, including:

[0039] The Bayesian optimization method is used to optimize the service configuration, including defining the configuration parameter space, building the proxy model using Gaussian process regression, selecting the configuration parameters by maximizing the expected improvement criterion, exploring the joint configuration strategy, collecting user feedback through questionnaire surveys, user behavior log analysis and social media sentiment analysis, and using natural language processing to extract keywords and sentiment tendencies, using collaborative filtering algorithms to cluster users, and using time series analysis to process user satisfaction data, using the hierarchical analysis method to organize service quality indicators into a multi-level structure, and using the fuzzy comprehensive evaluation method to calculate the relative importance of each indicator;

[0040] The particle swarm optimization algorithm is used to adjust the indicator weights, and the learning step size is adjusted according to the effect of historical weight adjustment. The Durst-Schafer evidence theory is used to integrate expert opinions, historical data and user feedback. The reward function of the adaptive deep reinforcement learning model is updated according to the service quality evaluation model. The Thompson sampling strategy is used for exploration, and the recursive least squares temporal difference learning algorithm is used to solve the optimal service decision based on the updated reward function and the adaptive deep reinforcement learning model. A recurrent neural network is used to generate a decision sequence, and a beam search algorithm is used to generate multiple candidate decisions. The decision sequence is processed, and the implementation process of the decision sequence is controlled in combination with a finite state machine.

[0041] A second aspect of an embodiment of the present invention provides an intranet service quality optimization system based on deep reinforcement learning, including:

[0042] The first unit is used to deploy a distributed sensor network, collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into a network layer, a service layer and a resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model;

[0043] The second unit is used to obtain historical intranet performance data from a database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in an actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate a multi-objective optimization result;

[0044] The third unit is used to set hierarchical performance indicator thresholds for the network layer, service layer and resource layer respectively based on the multi-objective optimization result. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model, performance bottlenecks and fault points are identified in combination with a preset prediction model and resource allocation and service configuration are adjusted, user feedback data is periodically collected, the weights of different indicators in a preset service quality evaluation model are adjusted according to the user feedback data, the reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

[0045] According to a third aspect of the embodiments of the present invention,

[0046] An electronic device is provided, comprising:

[0047] processor;

[0048] a memory for storing processor-executable instructions;

[0049] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0050] According to a fourth aspect of the embodiments of the present invention,

[0051] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0052] In the present invention, by deploying a distributed sensor network, multi-dimensional performance indicator data is collected in real time, and time series analysis technology is used to perform outlier detection, data smoothing and trend analysis, thereby realizing real-time monitoring of intranet performance and early warning of potential anomalies, independently evaluating and comprehensively optimizing the performance of each layer, providing more comprehensive and fine-grained performance insights, and using priority experience replay and adversarial networks to generate abnormal scenarios, thereby improving the robustness and generalization ability of the model, and realizing multi-objective balance and hierarchical control of intranet performance by dynamically adjusting network topology, identifying bottlenecks and fault points, and optimizing resource allocation and service configuration. In summary, the present invention enhances the intelligent management capability of the intranet, improves network service quality and resource utilization efficiency, and lays a good foundation for subsequent applications and expansions. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method for optimizing intranet service quality based on deep reinforcement learning according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of an intranet service quality optimization system based on deep reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0057] Figure 1 FIG. 1 is a flow chart of a method for optimizing intranet service quality based on deep reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0058] S1. Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to integrate the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model;

[0059] The distributed sensor network is a network composed of multiple sensor nodes. The multi-dimensional performance indicator data refers to performance data collected from multiple aspects or dimensions. These data can reflect the performance of the system or equipment under different conditions, which is convenient for comprehensive evaluation and decision-making. The time series analysis technology is a method for analyzing time series data. By identifying the trend, seasonality and cyclical changes of the data, it helps to predict future values ​​or understand the dynamic characteristics of the data. The hierarchical analysis model is a decision analysis method that evaluates and compares different factors or options by building a hierarchical structure, and usually uses pairwise comparison to determine the importance of each factor. The fuzzy comprehensive evaluation model is a comprehensive evaluation method that uses fuzzy mathematics principles to combine qualitative and quantitative information, handle uncertainty and ambiguity, and give a comprehensive evaluation result.

[0060] In an optional embodiment,

[0061] Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model, including:

[0062] Deploy a hierarchical distributed sensor network, which includes bottom-level sensing nodes, middle-level aggregation nodes, and top-level aggregation nodes. A time-based polling mechanism and a threshold-based active reporting mechanism are combined to collect multi-dimensional performance indicator data in the intranet in real time through the hierarchical distributed sensor network. The multi-dimensional performance indicator data is adaptively optimized for collection frequency, the collection frequency is dynamically adjusted according to the indicator change speed, and the collected multi-dimensional performance indicator data is compressed in combination with differential coding technology.

[0063] Using an improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data, using an adaptive weighted moving average method to smooth the detected multi-dimensional performance indicator data, and performing trend analysis on the smoothed multi-dimensional performance indicator data based on a nonlinear trend term estimation method and an adaptive seasonal cycle detection mechanism;

[0064] Constructing a hierarchical analysis model, using the hierarchical analysis method to stratify and assign weights to the processed multi-dimensional performance indicator data, dividing the multi-dimensional performance indicator data into a network layer, a service layer, and a resource layer, using the partial least squares regression method to establish a relationship model between the network layer, the service layer, and the resource layer, using an integrated learning method to construct a performance scoring model for the network layer, the integrated learning method combines random forests and gradient boosting trees, using a support vector regression model based on multi-kernel learning technology to construct a performance scoring model for the service layer, and using a long short-term memory network model based on an attention mechanism to construct a performance scoring model for the resource layer;

[0065] Construct a fuzzy set based on triangular fuzzy numbers, and use an adaptive fuzzy set generation algorithm to dynamically adjust fuzzy set parameters, design a fuzzy rule base, use a data-driven approach combined with expert knowledge and machine learning technology to automatically generate and optimize fuzzy rules, and use an improved fuzzy inference system to comprehensively evaluate the performance scores of the network layer, the service layer, and the resource layer to obtain an intranet service quality score;

[0066] A multi-scale state space is designed, and the principal component analysis method is used to reduce the dimension of the state space. A continuous action space including network parameter adjustment, service configuration optimization and resource allocation strategy is defined, a multi-objective reward function is constructed, and a reward allocation mechanism based on Pareto optimization is introduced to construct the adaptive deep reinforcement learning model.

[0067] The time-based polling mechanism refers to a method of visiting and collecting data from each sensor or node in turn within a fixed time interval, which is suitable for real-time monitoring. The threshold-based active reporting is a data transmission strategy. When the sensor data exceeds the preset threshold, it actively reports to the central system to reduce unnecessary data transmission. The differential coding technology is a data compression method that reduces the cost of storage and transmission by storing the differences between data instead of absolute values. The improved local anomaly factor algorithm is an algorithm for detecting outliers. Considering the density information of local data, it can effectively identify points in the data set that are significantly different from the surrounding data points. The adaptive weighted moving average method is a moving average algorithm that dynamically adjusts weights, which can respond to data changes more flexibly and provide more accurate trend estimates. The nonlinear trend term estimation method is used to capture nonlinear trends in time series data and improve the description of data changes by fitting nonlinear functions. The hierarchical analysis method is a multi-criteria The decision analysis method evaluates and selects the best solution through hierarchical structure and pairwise comparison, and is often used in complex decision problems. The partial least squares regression method is a regression analysis technology that can handle multicollinearity problems and is suitable for modeling small samples and high-dimensional data. The support vector regression model based on multi-kernel learning technology uses the advantages of multi-kernel learning to improve the performance of support vector regression, especially when dealing with complex features. The fuzzy set based on triangular fuzzy numbers is a form of representing fuzzy numbers in a triangular shape, which is suitable for expressing uncertainty and ambiguity. The fuzzy rule base is a set of fuzzy rules used for decision-making or control based on fuzzy reasoning, and is often used in fuzzy logic systems. The data-driven method is a strategy that relies on actual data for model building and decision-making, emphasizing the extraction of useful information from data. The principal component analysis method is a dimensionality reduction technology that converts raw data into a set of new unrelated variables (principal components) through linear transformation, reducing the complexity of the data while retaining the main information.

[0068] Construct a hierarchical distributed sensor network including bottom-level sensing nodes, middle-level aggregation nodes, and top-level aggregation nodes. Use a combination of time-based polling mechanism and threshold-based active reporting to collect multi-dimensional performance indicator data in the intranet in real time through the hierarchical distributed sensor network. Adaptively optimize the collection frequency of multi-dimensional performance indicator data, dynamically adjust the collection frequency according to the indicator change speed, and use differential coding technology to compress the collected multi-dimensional performance indicator data.

[0069] The improved local anomaly factor algorithm is used to detect outliers on the compressed multi-dimensional performance indicator data, and the adaptive weighted moving average method is used to smooth the detected multi-dimensional performance indicator data. The trend analysis of the smoothed multi-dimensional performance indicator data is performed based on the nonlinear trend term estimation method and the adaptive seasonal cycle detection mechanism. A hierarchical analysis model is constructed, and the hierarchical analysis method is used to stratify and weight the processed multi-dimensional performance indicator data. The multi-dimensional performance indicator data is divided into the network layer, service layer and resource layer. The partial least squares regression method is used to establish the relationship model between the network layer, service layer and resource layer. The performance scoring model is constructed for the network layer by an ensemble learning method. The ensemble learning method is combined with random forest and gradient boosting tree. The support vector regression model based on multi-kernel learning technology is used to construct a performance scoring model for the service layer. The long short-term memory network model based on the attention mechanism is used to construct a performance scoring model for the resource layer.

[0070] A fuzzy set based on triangular fuzzy numbers is constructed, and an adaptive fuzzy set generation algorithm is used to dynamically adjust the fuzzy set parameters. A fuzzy rule base is designed, and a data-driven method is used to combine expert knowledge and machine learning technology to automatically generate and optimize fuzzy rules. An improved fuzzy inference system is used to comprehensively evaluate the performance scores of the network layer, service layer, and resource layer to obtain the intranet service quality score. A multi-scale state space is designed, and the principal component analysis method is used to reduce the dimension of the state space. A continuous action space including network parameter adjustment, service configuration optimization, and resource allocation strategy is defined. A multi-objective reward function is constructed, and a reward allocation mechanism based on Pareto optimization is introduced to construct an adaptive deep reinforcement learning model.

[0071] For example, suppose that a company's intranet includes 100 bottom-level perception nodes, 10 middle-level aggregation nodes, and 1 top-level aggregation node. Each perception node collects 10 performance indicators such as CPU utilization, memory usage, and network throughput every 5 seconds, and sends the data to the corresponding aggregation node. The aggregation node compresses the received data every 30 seconds, and sends it to the aggregation node after outlier detection and smoothing. The aggregation node performs trend analysis on the data and passes the results to the hierarchical analysis model. The hierarchical analysis model divides the performance indicators into the network layer (such as network throughput, latency), service layer (such as response time, error rate), and resource layer (such as CPU utilization, memory usage), and uses partial least squares regression to establish the relationship between them. The network layer uses random forests and gradient boosting trees to build a performance scoring model, the service layer uses multi-core support vector regression to build a performance scoring model, and the resource layer uses attention long short-term memory network to build a performance scoring model;

[0072] Construct a fuzzy set containing 27 triangular fuzzy numbers (e.g., low, medium, and high), and design a rule base containing 125 fuzzy rules (e.g., "if the network layer score is low and the service layer score is low and the resource layer score is high, then the intranet service quality score is low"), and use a fuzzy inference system to comprehensively evaluate the performance scores of the three levels to obtain the intranet service quality score;

[0073] A state space containing 100 state variables (such as the current value and historical average of each performance indicator) is designed, and its dimension is reduced to 20 dimensions using principal component analysis. An action space containing 20 continuous variables (such as network parameters, service parameters, resource allocation ratio, etc.) is defined, and a reward function containing 5 sub-goals (such as maximizing throughput, minimizing latency, minimizing error rate, maximizing resource utilization, and minimizing energy consumption) is constructed. Pareto optimization is used to weigh multiple sub-goals to obtain the final adaptive deep reinforcement learning model, which can automatically adjust network parameters, optimize service configuration, and allocate resources according to the current intranet status, thereby achieving dynamic optimization of intranet service quality.

[0074] In this embodiment, by deploying a hierarchical distributed sensor network and adopting a combination of polling and active reporting, multi-dimensional performance indicator data at all levels in the intranet can be collected in real time, and data compression is achieved through differential coding to improve data transmission efficiency, ensure the real-time and comprehensiveness of data collection, and provide a high-quality data basis for subsequent analysis. By using an improved local anomaly factor algorithm to detect outliers, adaptive weighted moving average to perform smoothing, nonlinear trend term estimation and adaptive seasonal cycle detection to perform trend analysis, it is possible to effectively identify and remove noise and anomalies in the data, improve data quality and reliability, and provide more accurate input for subsequent modeling. By constructing a hierarchical analysis model, the multi-dimensional performance indicator data is divided into a network layer, a service layer and a resource layer using a hierarchical analysis method, and a relationship model is established between them, the influencing factors of the intranet service quality can be understood from a global perspective, and targeted insights can be obtained at different levels, providing a multi-angle basis for optimizing decisions. The fuzzy rules have good interpretability and are easy for human experts to understand and adjust. In summary, this embodiment forms a complete set of intranet service quality assurance methods, which can comprehensively improve the intelligent level of intranet service quality management, provide strong support for the rapid development of enterprise business, and have significant technical advantages and application value.

[0075] In an optional embodiment,

[0076] Using the improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data includes:

[0077]

[0078] in, Represents the i-th data point P i The local anomaly factor is used to measure the data point P i The degree of abnormality, Represents data point P i With k i The distance neighborhood, that is, in the selected dynamic neighborhood size k i Next, the set of neighbor data points included, lrd(P j ) represents the data point P j The local reachability density, lrd(P i )

[0079] Represents data point P i The local reachable density of Represents neighborhood The number of data points in , that is, data point P i The neighborhood size.

[0080] In this embodiment, by calculating the density of each data point relative to its neighborhood, local outliers can be effectively identified, that is, data points whose density in the local neighborhood is significantly lower than that of their neighbors. This is particularly important for detecting local anomalies in multi-dimensional performance indicator data. The adaptive neighborhood selection method can adjust the neighborhood size according to the local density characteristics of the data point, select a smaller neighborhood for a higher density area, and select a larger neighborhood for a lower density area, which helps to improve the accuracy and robustness of anomaly detection. The comparison method based on relative density can adapt to the density differences in different areas and better reflect the degree of anomaly of the data point in the local neighborhood. In summary, this embodiment can effectively identify local anomalies, adaptively select neighborhoods, improve detection accuracy and robustness, and has good interpretability, providing a high-quality data foundation for subsequent data analysis and optimization decisions, and helping to improve the intelligence level of intranet service quality management.

[0081] S2. Obtain historical intranet performance data from the database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in the actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results;

[0082] The simulation environment is a virtual platform for simulating real-world systems, providing a safe space for experiments and verification, and is often used to test algorithms and models. The offline pre-training refers to the preliminary training of the model using an existing data set before formal training to improve the performance of the model in subsequent online training. The learning algorithm based on priority experience replay stores and reuses experience, giving priority to more important experience for training to improve learning efficiency. The adversarial network is a generative adversarial network (GAN), which consists of a generator and a discriminator. The generator attempts to generate realistic data, and the discriminator strives to distinguish between real data and generated data.

[0083] In an optional embodiment,

[0084] The historical intranet performance data is obtained from the database to build a simulation environment, and the adaptive deep reinforcement learning model is pre-trained offline. The hyperparameters of the adaptive deep reinforcement learning model are adjusted by a learning algorithm based on priority experience replay. The adaptive deep reinforcement learning model is optimized again by combining the adversarial network to generate multiple network abnormality scenarios. The optimized adaptive deep reinforcement learning model is used to continuously learn in the actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to the pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results including:

[0085] Extracting historical intranet performance data from a database, the historical intranet performance data including network throughput, delay, packet loss rate, CPU utilization and memory usage, cleaning, standardizing and feature engineering the historical intranet performance data, using principal component analysis to perform feature dimension reduction, and using a Monte Carlo method combined with variational inference to construct a statistical model for the historical intranet performance data after feature dimension reduction, simulating dynamic changes in the intranet environment, and obtaining a simulation environment;

[0086] For the pre-built adaptive deep reinforcement learning model, the simulation environment is used for offline pre-training. During the offline pre-training process, the behavior cloning technology is used to imitate the optimal decision in the historical data, a hierarchical experience replay pool is constructed, and the experience is classified and stored according to importance. A priority sampling strategy based on temporal difference error is used, and a Bayesian optimization method is used to search for the optimal configuration in the hyperparameter space. A Gaussian process regression model is used to predict the hyperparameter performance. The next set of hyperparameters to be evaluated is selected through the expected improvement criterion, and the hyperparameters with the best performance are selected as the hyperparameters of the adaptive deep reinforcement learning model;

[0087] Construct a generative adversarial network architecture, including a generator and a discriminator, use conditional generative adversarial network technology to control the properties of the generated scenario, generate a variety of network anomaly scenarios, input the generated network anomaly scenarios into the adaptive deep reinforcement learning model, use the policy gradient method to update the model parameters, optimize the adaptive deep reinforcement learning model again, and deploy the optimized adaptive deep reinforcement learning model to the actual intranet environment for continuous learning;

[0088] Deploy a distributed sensor network, use edge computing technology to perform preliminary data processing, collect and receive multi-level network status information in real time, use the ε-greedy strategy to select optimization actions based on the multi-level network status information, introduce a soft update mechanism to execute the optimization actions, update the network status and calculate multi-dimensional rewards after executing the optimization actions, use sliding window technology to comprehensively evaluate short-term and long-term optimization effects, use a multi-objective Q learning algorithm to simultaneously optimize multiple performance indicators, introduce a solution set update strategy based on dominance relations to maintain non-dominated solution sets, use a weighted Chebyshev divergence method to select the optimal solution in each iteration, generate a Pareto optimal solution set based on a non-dominated sorting genetic algorithm, and combine clustering technology for grouping to obtain the multi-objective optimization results.

[0089] The Monte Carlo method combined with variational inference combines variational inference with Monte Carlo sampling to improve the estimation accuracy of the posterior distribution, especially in complex models. The behavioral cloning technology is a method of learning by imitating the behavior of experts. It is often used in autonomous driving and robot control and relies on collected demonstration data for training. The priority sampling strategy based on temporal difference error selects samples by evaluating the temporal difference error, and prioritizes sampling of experiences that are more important for learning, thereby accelerating convergence. The expected improvement criterion is a policy update method for reinforcement learning, which adjusts the policy by calculating the expected return to achieve better performance. The ε-greedy strategy is a method that balances exploration and exploitation. The method is usually randomly selected with a probability of ε to increase the possibility of exploration and improve the learning effect. The soft update mechanism is a strategy for updating model parameters. By slowly updating the parameters of the target network, the fluctuations in the learning process are reduced. The sliding window technology is used to process time series data. By maintaining a fixed-size window to smooth the data, it is easy to analyze and predict. The multi-objective Q learning algorithm considers multiple goals at the same time in reinforcement learning, and optimizes multiple goals by adjusting the reward mechanism. The weighted Chebyshev divergence method is a method for measuring the difference between distributions. It captures the deviation between distributions by assigning different weights and is often used to evaluate and optimize generative models.

[0090] Extract historical intranet performance data from the database, including network throughput, latency, packet loss rate, CPU utilization, and memory usage. Clean, standardize, and feature engineer the historical intranet performance data. Use principal component analysis to reduce feature dimensionality. Use the Monte Carlo method combined with variational inference to build a statistical model for the historical intranet performance data after feature dimensionality reduction. Simulate the dynamic changes of the intranet environment to obtain a simulation environment. Use the simulation environment to perform offline pre-training on the pre-built adaptive deep reinforcement learning model. During the offline pre-training process, use behavioral cloning technology to imitate the optimal decision in historical data, build a hierarchical experience replay pool, and store the experience according to importance. Use a priority sampling strategy based on temporal difference error. Use the Bayesian optimization method to search for the optimal configuration in the hyperparameter space. Use the Gaussian process regression model to predict hyperparameter performance. Select the next set of hyperparameters to be evaluated through the expected improvement criterion. Select the hyperparameters with the best performance as the hyperparameters of the adaptive deep reinforcement learning model.

[0091] Construct a generative adversarial network architecture, including a generator and a discriminator, use conditional generative adversarial network technology to control the properties of the generated scenarios, generate diverse network anomaly scenarios, input the generated network anomaly scenarios into the adaptive deep reinforcement learning model, use the policy gradient method to update the model parameters, optimize the adaptive deep reinforcement learning model again, deploy the optimized adaptive deep reinforcement learning model to the actual intranet environment for continuous learning, deploy a distributed sensor network, use edge computing technology for preliminary data processing, collect and receive multi-level network status information in real time, use the ε-greedy strategy to select optimization actions based on the multi-level network status information, introduce a soft update mechanism to execute the optimization actions, update the network status and calculate multi-dimensional rewards after executing the optimization actions, use sliding window technology to comprehensively evaluate short-term and long-term optimization effects, use a multi-objective Q learning algorithm to simultaneously optimize multiple performance indicators, introduce a solution set update strategy based on dominance relations to maintain the non-dominated solution set, use the weighted Chebyshev divergence method to select the optimal solution in each iteration, generate the Pareto optimal solution set based on the non-dominated sorting genetic algorithm, and combine clustering technology for grouping to obtain multi-objective optimization results.

[0092] The intranet of a certain enterprise contains 500 network nodes, each of which generates 1,000 performance indicator data per minute, including network throughput, latency, packet loss rate, CPU utilization, and memory usage. The historical database stores performance data for the past year, totaling about 3 billion records.

[0093] Historical performance data was extracted from the database and cleaned to remove missing values ​​and outliers. Then, the data was standardized to unify different indicators to the same scale, and new features were constructed using feature engineering techniques, such as the ratio of network throughput to latency, the correlation between CPU utilization and memory usage, etc. The high-dimensional features were reduced to 20 dimensions using principal component analysis. Based on the reduced-dimensional data, a statistical model was constructed using the Monte Carlo method of variational inference. The dynamic changes of the intranet environment were simulated by sampling, and 10,000 simulation scenarios were generated.

[0094] For the pre-built adaptive deep reinforcement learning model, offline pre-training is performed in a simulation environment. The behavior cloning technology is used to screen out the 10% decisions with the best performance indicators from historical data, and the model is allowed to imitate these decisions. A three-layer experience replay pool is built, and the experience is divided into short-term, medium-term and long-term according to the time span, and stored in different pools. A priority sampling strategy based on temporal difference error is used to give priority to the experience that has a greater impact on model training. The Bayesian optimization method is used to search for the model's hyperparameters, including learning rate, discount factor, number of network layers, etc. The Gaussian process regression model is used to predict the performance of different hyperparameter combinations. The next set of hyperparameters to be evaluated is selected through the expected improvement criterion, and the optimal hyperparameter configuration is finally obtained;

[0095] Construct a generative adversarial network, including a generator and a discriminator. The generator receives random noise and conditional attributes (such as network scale, business type, etc.) as input and outputs a network abnormality scenario. The discriminator receives real scenarios and generated scenarios and outputs a score between 0 and 1, indicating the probability that the input scenario is a real scenario. Through training, the generator can generate realistic abnormal scenarios and make it difficult for the discriminator to distinguish between real scenarios and generated scenarios. The generated abnormal scenarios are input into the reinforcement learning model for training, and the policy gradient method is used to update the model parameters to improve the generalization ability and robustness of the model.

[0096] 500 sensor nodes are deployed in the actual intranet environment. Each node uses an edge computing chip for preliminary data processing and compression, and sends the data to the central server in real time. The reinforcement learning model receives network status information in a period of 30 seconds, and uses the ε-greedy strategy to select optimization actions (such as adjusting routing weights, allocating cache resources, etc.), where the initial value of ε is 0.1, which gradually decays as the training progresses. A soft update mechanism is introduced to average the current strategy and the target strategy at an update rate of 0.01 to improve the stability of the strategy. After executing the optimization action, wait for 30 seconds, collect the network status again, and calculate the improvement values ​​of multiple performance indicators as immediate rewards. At the same time, a sliding window is used to calculate the average performance improvement of the past 10 minutes as a long-term reward. A multi-objective Q learning algorithm is used to optimize the three indicators of network throughput, delay and packet loss rate at the same time. A solution set update strategy based on dominance relations is introduced to maintain a solution set containing 50 non-dominated solutions. In each iteration, the weighted Chebyshev divergence is used to select an optimal solution from the solution set as the target strategy. Finally, the non-dominated sorting genetic algorithm is used to generate the Pareto optimal solution set, and the K-means clustering algorithm is used to divide the solution set into 5 clusters, each cluster represents an optimization strategy for reference by decision makers.

[0097] In this embodiment, by cleaning, standardizing, feature engineering and dimensionality reduction of massive historical performance data, effective information in the data is extracted, and a statistical model is constructed using the Monte Carlo method of variational inference to simulate the dynamic changes of the intranet environment and generate realistic simulation scenarios, which provides a high-quality data basis for offline training of the reinforcement learning model and helps to improve the convergence speed and generalization ability of the model. In actual deployment, the ε-greedy strategy is adopted to balance exploration and utilization, and the probability of random exploration is gradually reduced as the training proceeds, so that the strategy can be adaptively optimized and improved. The multi-objective optimization method can trade off between different indicators and generate a Pareto optimal solution set, providing decision makers with a variety of optimization strategies. The reinforcement learning model can receive the network status in a cycle of seconds and quickly select optimization actions to achieve real-time response and dynamic adjustment to network anomalies. In summary, this embodiment can continuously improve the performance of the intranet through adaptive optimization strategies and multi-objective trade-offs, provide strong network support for the rapid development of enterprise business, and has broad application prospects.

[0098] In an optional embodiment,

[0099] The statistical model is constructed using the Monte Carlo method combined with variational inference as shown in the following formula:

[0100]

[0101] Among them, ELOB(q) represents the variational lower bound, K represents the number of Monte Carlo sampling, and p(x, y, z k) represents the latent variable z obtained by the kth sampling k The corresponding joint probability distribution, z k represents the hidden variable obtained by the kth sampling, q(z k |x, y) represents the latent variable z obtained by the k-th sampling of the variational distribution q(z|x, y) k The approximate posterior probability density of .

[0102] In this embodiment, the variational lower bound is optimized to make the variational distribution as close as possible to the true posterior distribution, thereby obtaining a good approximation of the latent variables. An unbiased estimate of the variational lower bound can be obtained by averaging the difference between the log joint probability corresponding to the sample and the log probability density of the variational distribution, thereby avoiding the difficulty of directly calculating the complex posterior distribution. By analyzing the distribution and changes of the latent variables, the dynamic characteristics of the intranet environment can be better understood, providing useful prior knowledge for optimizing decisions. In summary, this embodiment has good generalization and flexibility, can be applied to various intranet optimization tasks, and provides important data support and decision-making basis for intelligent network management.

[0103] S3. Based on the multi-objective optimization results, hierarchical performance indicator thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, and the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and fault points are identified and resource allocation and service configuration are adjusted in combination with the pre-set prediction model. User feedback data is periodically collected, and the weights of different indicators in the pre-set service quality evaluation model are adjusted according to the user feedback data. The reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

[0104] The hierarchical performance indicator threshold is a preset standard for evaluating the performance of each level of the system. The threshold is set to determine whether specific performance requirements are met. The hierarchical optimization strategy is a strategy that decomposes complex problems into multiple levels for optimization. It can handle problems at different levels and improve overall efficiency and manageability. The network topology is a graphical structure that describes the nodes in the network and their connection relationships, which affects the performance, reliability and data transmission efficiency of the network. The performance bottleneck refers to the key part of the system that limits the overall performance improvement, which usually needs to be optimized or improved to improve the overall effectiveness of the system. The optimal service decision sequence is a decision step generated for service optimization, which aims to improve user experience and service quality and meet user needs by continuously adjusting strategies.

[0105] In an optional embodiment,

[0106] Based on the multi-objective optimization results, hierarchical performance index thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance index threshold, a hierarchical optimization strategy is triggered, and the network topology is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and failure points are identified in combination with the pre-set prediction model, and resource allocation and service configuration are adjusted, including:

[0107] Perform cluster analysis on the multi-objective optimization results, use a clustering algorithm to divide the multi-objective optimization results into three subsets: network layer, service layer, and resource layer, apply principal component analysis to the three subsets of the network layer, service layer, and resource layer, respectively, extract the main feature vectors as the key performance indicators corresponding to each layer, use extreme value statistical theory, and establish a performance indicator probability distribution model for the network layer, service layer, and resource layer based on generalized extreme value distribution, and use a quantile regression method, combined with historical data and expert experience, to set adaptive performance indicator thresholds for the network layer, service layer, and resource layer;

[0108] Deploy a distributed monitoring agent to collect the performance parameters of the network layer, service layer and resource layer in real time, calculate the exponential moving average and standard deviation of the performance parameters of the network layer, service layer and resource layer through an anomaly detection algorithm based on a sliding window, design a multi-level trigger mechanism, and classify alarms into mild alarms, moderate alarms and severe alarms according to the relationship between the performance parameters, the exponential moving average, the standard deviation and the performance indicator threshold. Combined with a decision-making system based on a rule engine, trigger corresponding hierarchical optimization strategies according to the mild alarm, the moderate alarm and the severe alarm;

[0109] Design a network topology representation learning module based on graph neural network, encode the network topology structure into a low-dimensional vector representation, build a dynamic graph convolutional network, capture the temporal change characteristics of the network topology, fuse the output of the dynamic graph convolutional network with the traditional network state characteristics as the input of the adaptive deep reinforcement learning model, design a policy gradient algorithm based on the actor-critic architecture, in which the actor network outputs continuous network adjustment actions, the critic network evaluates the value of the action, introduces an entropy regularization term, adjusts the exploration degree of the network adjustment action, and combines the proximal policy optimization algorithm to limit the policy update step size by clipping the objective function;

[0110] Construct a multilayer perceptron, input the performance indicators of each layer, output the probability scores of potential bottlenecks and failure points, introduce the attention mechanism into the multilayer perceptron, integrate the decision tree model with the multilayer perceptron, use the Shapley additive explanatory value method to explain the output results of the integrated learning framework, use the genetic algorithm to design the resource allocation optimizer, use chromosome encoding to represent the allocation scheme of different resources, define the multi-objective fitness function, and use the multi-objective optimization process of non-dominated sorting to generate the resource allocation scheme.

[0111] The extreme value statistical theory studies the distribution characteristics of extreme events, helping to analyze and predict rare phenomena that occur under extreme conditions. The generalized extreme value distribution is a probability distribution used to model extreme value data, suitable for describing the statistical characteristics of maximum or minimum values. The quantile regression method is used to model different quantiles of the dependent variable, which captures the distribution characteristics of the data more comprehensively than the traditional regression method. The sliding window-based anomaly detection algorithm monitors abnormal changes in the data stream by maintaining a fixed-size window and identifies potential problems in a timely manner. The decision-making system based on the rule engine uses predefined rules to make automatic decisions, which can quickly respond to changes and execute The network topology representation learning module based on graph neural network captures the node relationship in the network structure through graph neural network, and improves the network analysis and prediction capabilities. The actor-critic architecture is a reinforcement learning framework, which consists of actors (responsible for strategy learning) and critics (responsible for evaluating strategies), and combines the advantages of both to achieve optimization. The proximal strategy optimization algorithm is a reinforcement learning algorithm, which ensures the stability and reliability of the learning process by limiting the amplitude of each strategy update. The Shapley additive explanation value method is used to evaluate the contribution of each feature in the model to the prediction results, provide an explanation of the importance of the features, and help understand the model decision-making process.

[0112] Use clustering algorithms, such as K-means or hierarchical clustering, to divide the multi-objective optimization results into three subsets: network layer, service layer, and resource layer. Apply principal component analysis to the three subsets, extract the main feature vectors of each layer as the corresponding key performance indicators, use extreme value statistical theory, and establish performance indicator probability distribution models for the network layer, service layer, and resource layer based on generalized extreme value distribution (GEV). Use quantile regression method, combined with historical data and expert experience, to set adaptive performance indicator thresholds for the network layer, service layer, and resource layer.

[0113] Deploy distributed monitoring agents to collect performance parameters of the network layer, service layer, and resource layer in real time. Calculate the exponential moving average (EMA) and standard deviation of performance parameters through anomaly detection algorithms based on sliding windows, design a multi-level trigger mechanism, and classify alarms into mild alarms, moderate alarms, and severe alarms based on the relationship between performance parameters, exponential moving averages, standard deviations, and performance indicator thresholds. Combined with a decision-making system based on a rule engine, trigger corresponding hierarchical optimization strategies based on different levels of alarms.

[0114] Design a network topology representation learning module based on graph neural network (GNN), encode the network topology structure into a low-dimensional vector representation, and construct a dynamic graph convolutional network to capture the temporal change characteristics of the network topology. Fusion the output of the dynamic graph convolutional network with the traditional network state features as the input of the adaptive deep reinforcement learning model, design a policy gradient algorithm based on the actor-critic architecture, the actor network outputs continuous network adjustment actions, the critic network evaluates the value of the action, introduces an entropy regularization term, adjusts the exploration degree of the network adjustment action, and combines the proximal policy optimization algorithm to limit the policy update step by clipping the objective function;

[0115] Construct a multi-layer perceptron (MLP), input the performance indicators of each layer, and output the probability distribution of potential bottlenecks and failure points. Introduce the attention mechanism in the multi-layer perceptron to highlight the impact of key performance indicators. Integrate the decision tree model with the multi-layer perceptron to form an integrated learning framework. Use the Shapley additive explanation value method to explain the output results of the integrated learning framework. Use genetic algorithms to design resource allocation optimizers, use chromosome encoding to represent the allocation scheme of different resources, define multi-objective fitness functions, consider performance indicators and resource constraints at different levels, and use the multi-objective optimization process of non-dominated sorting to generate resource allocation schemes.

[0116] For example, suppose a large Internet company needs to optimize the performance of its internal network. By collecting historical performance data of the network layer, service layer, and resource layer, the K-means clustering algorithm is used to divide the data into three subsets. The principal component analysis algorithm is applied to each subset to extract key performance indicators, such as latency and throughput of the network layer, response time and error rate of the service layer, CPU and memory utilization of the resource layer, etc. A performance indicator probability distribution model for each layer is established based on the generalized extreme value distribution, and the quantile regression method is used to set the adaptive threshold. Deploy distributed monitoring agents to collect performance parameters in real time, calculate the exponential moving average and standard deviation through the sliding window anomaly detection algorithm, generate mild, moderate and severe alarms according to the multi-level trigger mechanism, and trigger the corresponding optimization strategy;

[0117] The network topology is encoded into a low-dimensional vector based on a graph neural network, and a dynamic graph convolutional network is constructed to capture the characteristics of topological changes, which are then integrated with the traditional network state characteristics and input into the adaptive deep reinforcement learning model. The policy gradient algorithm based on the Actor-Critic architecture is used, and the proximal policy optimization algorithm is combined to limit the policy update step size, output continuous network adjustment actions, build a multi-layer perceptron model, input performance indicators of each layer, introduce an attention mechanism, and integrate it with the decision tree. The Shapley additive interpretation value method is used to explain the output of potential bottlenecks and probability distribution of fault points. A genetic algorithm is used to design a resource allocation optimizer, and the chromosome encoding is used to represent the resource allocation plan. A multi-objective fitness function is defined, and the optimal resource allocation strategy is generated through the multi-objective optimization process of non-dominated sorting to improve the performance of the entire internal network.

[0118] In this embodiment, the hierarchical optimization method can evaluate the overall performance of the intranet more comprehensively and meticulously, which is helpful to identify bottlenecks and optimization directions at different levels. In combination with distributed monitoring and sliding window anomaly detection algorithms, real-time monitoring and multi-level alarms of intranet performance are realized, potential anomalies and faults can be discovered and located in a timely manner, and topology information is integrated with traditional network status features as inputs of the adaptive deep reinforcement learning model, so as to realize dynamic optimization of network adjustment strategies and adapt to dynamic changes in the intranet environment. In the reinforcement learning model, the policy gradient algorithm of the actor-critic architecture is adopted, and the exploration degree is adjusted by introducing the entropy regularization term. At the same time, the policy update step is limited in combination with the proximal policy optimization algorithm. While improving the efficiency of policy exploration, the stability of policy update is taken into account, and the problem of excessive exploration or premature convergence is effectively avoided. In summary, this embodiment can evaluate the intranet performance from a global perspective, monitor and identify potential anomalies in real time, dynamically adjust the optimization strategy, and optimize resource allocation, so as to finally realize the continuous improvement and stable operation of the intranet performance, which has important practical significance and application value for ensuring the efficient operation of enterprise business and improving user experience and service quality.

[0119] In an optional embodiment,

[0120] Periodically collect user feedback data, adjust the weights of different indicators in the preset service quality evaluation model according to the user feedback data, update the reward function of the adaptive deep reinforcement learning model according to the adjusted service quality evaluation model and solve the optimal service decision based on the updated reward function, generate the optimal service decision sequence and execute it, including:

[0121] The Bayesian optimization method is used to optimize the service configuration, including defining the configuration parameter space, building the proxy model using Gaussian process regression, selecting the configuration parameters by maximizing the expected improvement criterion, exploring the joint configuration strategy, collecting user feedback through questionnaire surveys, user behavior log analysis and social media sentiment analysis, and using natural language processing to extract keywords and sentiment tendencies, using collaborative filtering algorithms to cluster users, and using time series analysis to process user satisfaction data, using the hierarchical analysis method to organize service quality indicators into a multi-level structure, and using the fuzzy comprehensive evaluation method to calculate the relative importance of each indicator;

[0122] The particle swarm optimization algorithm is used to adjust the indicator weights, and the learning step size is adjusted according to the effect of historical weight adjustment. The Durst-Schafer evidence theory is used to integrate expert opinions, historical data and user feedback. The reward function of the adaptive deep reinforcement learning model is updated according to the service quality evaluation model. The Thompson sampling strategy is used for exploration, and the recursive least squares temporal difference learning algorithm is used to solve the optimal service decision based on the updated reward function and the adaptive deep reinforcement learning model. A recurrent neural network is used to generate a decision sequence, and a beam search algorithm is used to generate multiple candidate decisions. The decision sequence is processed, and the implementation process of the decision sequence is controlled in combination with a finite state machine.

[0123] The Gaussian process regression is a non-parametric Bayesian regression method that models the distribution of data through Gaussian processes and is suitable for dealing with uncertainty and complexity. The collaborative filtering algorithm is a recommendation system technology that generates personalized recommendations based on similar users or projects by analyzing user behavior and preferences. The Thompson sampling strategy is an online learning algorithm that optimizes the decision-making process by balancing exploration and utilization through the use of Bayesian updates. The finite state machine is a mathematical model for modeling system state changes. By defining states and their transition rules, it is convenient to describe and analyze dynamic system behavior.

[0124] Use Bayesian optimization method to optimize service configuration, define configuration parameter space, use Gaussian process regression to build proxy model, select configuration parameters by maximizing expected improvement criterion, explore joint configuration strategy, collect user feedback through questionnaire survey, user behavior log analysis and social media sentiment analysis, use natural language processing technology to extract keywords and sentiment tendency, use collaborative filtering algorithm to cluster users, and use time series analysis to process user satisfaction data;

[0125] The service quality indicators are organized into a multi-level structure using the analytic hierarchy process, the relative importance of each indicator is calculated using the fuzzy comprehensive evaluation method, the indicator weights are adjusted using the particle swarm optimization algorithm, and the learning step size is adjusted according to the effect of historical weight adjustment. The Durst-Schafer evidence theory is used to integrate expert opinions, historical data and user feedback. The reward function of the adaptive deep reinforcement learning model is updated according to the service quality evaluation model. The Thompson sampling strategy is used for exploration. The recursive least squares temporal difference learning algorithm is used to solve the optimal service decision based on the updated reward function and the adaptive deep reinforcement learning model.

[0126] Use recurrent neural networks to generate decision sequences, use beam search algorithms to generate multiple candidate decisions, process decision sequences, and combine finite state machines to control the implementation of decision sequences;

[0127] For example, an online education platform needs to optimize its service configuration to improve user satisfaction and learning effects. Through the Bayesian optimization method, the configuration parameter space (such as video quality, course difficulty, interaction method, etc.) is defined, and the agent model is established using Gaussian process regression. The optimal configuration parameter combination is selected by maximizing the expected improvement criterion. The platform collects user feedback through questionnaires, user behavior log analysis (such as viewing time, exercise completion, etc.) and social media sentiment analysis (such as comment sentiment tendency), uses natural language processing technology to extract keywords and sentiment tendencies, and uses collaborative filtering algorithms to cluster users, identify the preferences and needs of different user groups, and uses time series analysis to process user satisfaction data to capture the dynamic trend of satisfaction.

[0128] The service quality indicators (such as video clarity, course content richness, teacher response speed, etc.) are organized into a multi-level structure using the hierarchical analysis method, and the relative importance of each indicator is calculated using the fuzzy comprehensive evaluation method. The particle swarm optimization algorithm is used to dynamically adjust the indicator weights, and the learning step size is adaptively adjusted according to the effect of historical weight adjustment. The Durst-Schafer evidence theory is used to integrate expert opinions, historical data and user feedback to form a comprehensive service quality evaluation model.

[0129] According to the service quality evaluation model, the reward function of the adaptive deep reinforcement learning model is updated to make it more in line with user preferences and platform optimization goals. The Thompson sampling strategy is adopted to balance exploration and utilization, and the recursive least squares temporal difference learning algorithm is used to solve the optimal service decision (such as recommended courses, personalized learning paths, etc.) based on the updated reward function and reinforcement learning model. A recurrent neural network is used to generate a decision sequence, and a beam search algorithm is used to generate multiple candidate decisions. The implementation process of the decision sequence is controlled by a finite state machine to ensure a smooth transition of the service optimization strategy and consistency of the user experience.

[0130] In this embodiment, the Bayesian optimization method is used to search for the optimal configuration in the high-dimensional configuration parameter space, and the Gaussian process regression is used to establish a performance proxy model, which effectively balances exploration and utilization, improves the efficiency and quality of configuration optimization, extracts keywords and sentiment tendencies through natural language processing technology, and deeply understands user needs and experience. The collaborative filtering algorithm is used to cluster users, identify the preferences of different user groups, and realize personalized services. The particle swarm optimization algorithm is used to dynamically adjust the indicator weights, and the learning step size is adaptively adjusted to achieve continuous optimization of the evaluation model. The Dester-Schafer evidence theory is used to fuse multi-source heterogeneous data, which improves the reliability and robustness of the evaluation results. In summary, this embodiment can automatically optimize service configuration, deeply understand user needs, scientifically evaluate service quality, dynamically adjust optimization strategies, and smoothly implement decisions, ultimately achieving continuous improvement of user experience and maximization of platform value. It has important reference significance for service optimization in the fields of online education, e-commerce, content recommendation, etc., and helps enterprises maintain their leading position in the fierce market competition.

[0131] Figure 2 Schematic diagram of the structure of the intranet service quality optimization system based on deep reinforcement learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0132] The first unit is used to deploy a distributed sensor network, collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into a network layer, a service layer and a resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model;

[0133] The second unit is used to obtain historical intranet performance data from a database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in an actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate a multi-objective optimization result;

[0134] The third unit is used to set hierarchical performance indicator thresholds for the network layer, service layer and resource layer respectively based on the multi-objective optimization result. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model, performance bottlenecks and fault points are identified in combination with a preset prediction model and resource allocation and service configuration are adjusted, user feedback data is periodically collected, the weights of different indicators in a preset service quality evaluation model are adjusted according to the user feedback data, the reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

[0135] According to a third aspect of the embodiments of the present invention,

[0136] An electronic device is provided, comprising:

[0137] processor;

[0138] a memory for storing processor-executable instructions;

[0139] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0140] According to a fourth aspect of the embodiments of the present invention,

[0141] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0142] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intranet service quality optimization method based on deep reinforcement learning, characterized in that: include: Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to integrate the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model; Acquire historical intranet performance data from a database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in an actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results; Based on the multi-objective optimization result, hierarchical performance indicator thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, and the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and fault points are identified and the resource allocation and service configuration are adjusted in combination with the preset prediction model. User feedback data is periodically collected, and the weights of different indicators in the preset service quality evaluation model are adjusted according to the user feedback data. The reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

2. The method according to claim 1, characterized in that Deploy a distributed sensor network to collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into network layer, service layer and resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model, including: Deploy a hierarchical distributed sensor network, which includes bottom-level sensing nodes, middle-level aggregation nodes, and top-level aggregation nodes. A time-based polling mechanism and a threshold-based active reporting mechanism are combined to collect multi-dimensional performance indicator data in the intranet in real time through the hierarchical distributed sensor network. The multi-dimensional performance indicator data is adaptively optimized for collection frequency, the collection frequency is dynamically adjusted according to the indicator change speed, and the collected multi-dimensional performance indicator data is compressed in combination with differential coding technology. Using an improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data, using an adaptive weighted moving average method to smooth the detected multi-dimensional performance indicator data, and performing trend analysis on the smoothed multi-dimensional performance indicator data based on a nonlinear trend term estimation method and an adaptive seasonal cycle detection mechanism; Constructing a hierarchical analysis model, using the hierarchical analysis method to stratify and assign weights to the processed multi-dimensional performance indicator data, dividing the multi-dimensional performance indicator data into a network layer, a service layer, and a resource layer, using the partial least squares regression method to establish a relationship model between the network layer, the service layer, and the resource layer, using an integrated learning method to construct a performance scoring model for the network layer, the integrated learning method combines random forests and gradient boosting trees, using a support vector regression model based on multi-kernel learning technology to construct a performance scoring model for the service layer, and using a long short-term memory network model based on an attention mechanism to construct a performance scoring model for the resource layer; Construct a fuzzy set based on triangular fuzzy numbers, and use an adaptive fuzzy set generation algorithm to dynamically adjust fuzzy set parameters, design a fuzzy rule base, use a data-driven approach combined with expert knowledge and machine learning technology to automatically generate and optimize fuzzy rules, and use an improved fuzzy inference system to comprehensively evaluate the performance scores of the network layer, the service layer, and the resource layer to obtain an intranet service quality score; A multi-scale state space is designed, and the principal component analysis method is used to reduce the dimension of the state space. A continuous action space including network parameter adjustment, service configuration optimization and resource allocation strategy is defined, a multi-objective reward function is constructed, and a reward allocation mechanism based on Pareto optimization is introduced to construct the adaptive deep reinforcement learning model.

3. The method according to claim 2, characterized in that Using the improved local anomaly factor algorithm to perform outlier detection on the compressed multi-dimensional performance indicator data includes: in, Represents the i-th data point P i The local anomaly factor is used to measure the data point P i The degree of abnormality, Represents data point P i With k i The distance neighborhood, that is, in the selected dynamic neighborhood size k i Next, the set of neighbor data points included, lrd(P j ) represents the data point P j The local reachability density, lrd(P i ) represents the data point P i The local reachable density of Represents neighborhood The number of data points in , that is, data point P i The neighborhood size.

4. The method according to claim 1, characterized in that: The historical intranet performance data is obtained from the database to build a simulation environment, and the adaptive deep reinforcement learning model is pre-trained offline. The hyperparameters of the adaptive deep reinforcement learning model are adjusted by a learning algorithm based on priority experience replay. The adaptive deep reinforcement learning model is optimized again by combining the adversarial network to generate multiple network abnormality scenarios. The optimized adaptive deep reinforcement learning model is used to continuously learn in the actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to the pre-set multi-objective reinforcement learning algorithm, and generate multi-objective optimization results including: Extracting historical intranet performance data from a database, the historical intranet performance data including network throughput, delay, packet loss rate, CPU utilization and memory usage, cleaning, standardizing and feature engineering the historical intranet performance data, using principal component analysis to perform feature dimension reduction, and using a Monte Carlo method combined with variational inference to construct a statistical model for the historical intranet performance data after feature dimension reduction, simulating dynamic changes in the intranet environment, and obtaining a simulation environment; For the pre-built adaptive deep reinforcement learning model, the simulation environment is used for offline pre-training. During the offline pre-training process, the behavior cloning technology is used to imitate the optimal decision in the historical data, a hierarchical experience replay pool is constructed, and the experience is classified and stored according to importance. A priority sampling strategy based on temporal difference error is used, and a Bayesian optimization method is used to search for the optimal configuration in the hyperparameter space. A Gaussian process regression model is used to predict the hyperparameter performance. The next set of hyperparameters to be evaluated is selected through the expected improvement criterion, and the hyperparameters with the best performance are selected as the hyperparameters of the adaptive deep reinforcement learning model; Construct a generative adversarial network architecture, including a generator and a discriminator, use conditional generative adversarial network technology to control the properties of the generated scenario, generate a variety of network anomaly scenarios, input the generated network anomaly scenarios into the adaptive deep reinforcement learning model, use the policy gradient method to update the model parameters, optimize the adaptive deep reinforcement learning model again, and deploy the optimized adaptive deep reinforcement learning model to the actual intranet environment for continuous learning; Deploy a distributed sensor network, use edge computing technology to perform preliminary data processing, collect and receive multi-level network status information in real time, use the ε-greedy strategy to select optimization actions based on the multi-level network status information, introduce a soft update mechanism to execute the optimization actions, update the network status and calculate multi-dimensional rewards after executing the optimization actions, use sliding window technology to comprehensively evaluate short-term and long-term optimization effects, use a multi-objective Q learning algorithm to simultaneously optimize multiple performance indicators, introduce a solution set update strategy based on dominance relations to maintain non-dominated solution sets, use a weighted Chebyshev divergence method to select the optimal solution in each iteration, generate a Pareto optimal solution set based on a non-dominated sorting genetic algorithm, and combine clustering technology for grouping to obtain the multi-objective optimization results.

5. The method according to claim 4, characterized in that The statistical model is constructed using the Monte Carlo method combined with variational inference as shown in the following formula: Among them, ELOB(q) represents the variational lower bound, K represents the number of Monte Carlo sampling, and p(x, y, z k ) represents the latent variable z obtained by the kth sampling k The corresponding joint probability distribution, z k represents the hidden variable obtained by the kth sampling, q(z k |x, y) represents the latent variable z obtained by the k-th sampling of the variational distribution q(z|x, y) k The approximate posterior probability density of .

6. The method according to claim 1, characterized in that Based on the multi-objective optimization results, hierarchical performance index thresholds are set for the network layer, service layer and resource layer respectively. If there is a hierarchical performance parameter lower than the hierarchical performance index threshold, a hierarchical optimization strategy is triggered, and the network topology is dynamically adjusted based on the output of the adaptive deep reinforcement learning model. The performance bottlenecks and failure points are identified in combination with the pre-set prediction model, and resource allocation and service configuration are adjusted, including: Perform cluster analysis on the multi-objective optimization results, use a clustering algorithm to divide the multi-objective optimization results into three subsets: network layer, service layer, and resource layer, apply principal component analysis to the three subsets of the network layer, service layer, and resource layer, respectively, extract the main feature vectors as the key performance indicators corresponding to each layer, use extreme value statistical theory, and establish a performance indicator probability distribution model for the network layer, service layer, and resource layer based on generalized extreme value distribution, and use a quantile regression method, combined with historical data and expert experience, to set adaptive performance indicator thresholds for the network layer, service layer, and resource layer; Deploy a distributed monitoring agent to collect the performance parameters of the network layer, service layer and resource layer in real time, calculate the exponential moving average and standard deviation of the performance parameters of the network layer, service layer and resource layer through an anomaly detection algorithm based on a sliding window, design a multi-level trigger mechanism, and classify alarms into mild alarms, moderate alarms and severe alarms according to the relationship between the performance parameters, the exponential moving average, the standard deviation and the performance indicator threshold. Combined with a decision-making system based on a rule engine, trigger corresponding hierarchical optimization strategies according to the mild alarm, the moderate alarm and the severe alarm; Design a network topology representation learning module based on graph neural network, encode the network topology structure into a low-dimensional vector representation, build a dynamic graph convolutional network, capture the temporal change characteristics of the network topology, fuse the output of the dynamic graph convolutional network with the traditional network state characteristics as the input of the adaptive deep reinforcement learning model, design a policy gradient algorithm based on the actor-critic architecture, in which the actor network outputs continuous network adjustment actions, the critic network evaluates the value of the action, introduces an entropy regularization term, adjusts the exploration degree of the network adjustment action, and combines the proximal policy optimization algorithm to limit the policy update step size by clipping the objective function; Construct a multilayer perceptron, input the performance indicators of each layer, output the probability scores of potential bottlenecks and failure points, introduce the attention mechanism into the multilayer perceptron, integrate the decision tree model with the multilayer perceptron, use the Shapley additive explanatory value method to explain the output results of the integrated learning framework, use the genetic algorithm to design the resource allocation optimizer, use chromosome encoding to represent the allocation scheme of different resources, define the multi-objective fitness function, and use the multi-objective optimization process of non-dominated sorting to generate the resource allocation scheme.

7. The method according to claim 1, characterized in that Periodically collect user feedback data, adjust the weights of different indicators in the preset service quality evaluation model according to the user feedback data, update the reward function of the adaptive deep reinforcement learning model according to the adjusted service quality evaluation model and solve the optimal service decision based on the updated reward function, generate the optimal service decision sequence and execute it, including: The Bayesian optimization method is used to optimize the service configuration, including defining the configuration parameter space, building the proxy model using Gaussian process regression, selecting the configuration parameters by maximizing the expected improvement criterion, exploring the joint configuration strategy, collecting user feedback through questionnaire surveys, user behavior log analysis and social media sentiment analysis, and using natural language processing to extract keywords and sentiment tendencies, using collaborative filtering algorithms to cluster users, and using time series analysis to process user satisfaction data, using the hierarchical analysis method to organize service quality indicators into a multi-level structure, and using the fuzzy comprehensive evaluation method to calculate the relative importance of each indicator; The particle swarm optimization algorithm is used to adjust the indicator weights, and the learning step size is adjusted according to the effect of historical weight adjustment. The Durst-Schafer evidence theory is used to integrate expert opinions, historical data and user feedback. The reward function of the adaptive deep reinforcement learning model is updated according to the service quality evaluation model. The Thompson sampling strategy is used for exploration, and the recursive least squares temporal difference learning algorithm is used to solve the optimal service decision based on the updated reward function and the adaptive deep reinforcement learning model. A recurrent neural network is used to generate a decision sequence, and a beam search algorithm is used to generate multiple candidate decisions. The decision sequence is processed, and the implementation process of the decision sequence is controlled in combination with a finite state machine.

8. An intranet service quality optimization system based on deep reinforcement learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to deploy a distributed sensor network, collect multi-dimensional performance indicator data in the intranet in real time, use time series analysis technology to pre-process the collected multi-dimensional performance indicator data, including outlier detection, data smoothing and trend analysis, build a hierarchical analysis model and divide the pre-processed multi-dimensional performance indicator data into a network layer, a service layer and a resource layer, use different machine learning algorithms to output the performance score of each layer, and use a pre-set fuzzy comprehensive evaluation model to synthesize the performance score of each layer, output the intranet service quality score and build an adaptive deep reinforcement learning model; The second unit is used to obtain historical intranet performance data from a database to build a simulation environment, perform offline pre-training on the adaptive deep reinforcement learning model, adjust the hyperparameters of the adaptive deep reinforcement learning model through a learning algorithm based on priority experience replay, generate multiple network abnormality scenarios in combination with an adversarial network to optimize the adaptive deep reinforcement learning model again, use the optimized adaptive deep reinforcement learning model to continuously learn in an actual intranet environment, receive the current multi-level network status, select and execute optimization actions, update the network status and determine multi-dimensional rewards, perform multi-objective optimization according to a pre-set multi-objective reinforcement learning algorithm, and generate a multi-objective optimization result; The third unit is used to set hierarchical performance indicator thresholds for the network layer, service layer and resource layer respectively based on the multi-objective optimization result. If there is a hierarchical performance parameter lower than the hierarchical performance indicator threshold, the hierarchical optimization strategy is triggered, the network topology structure is dynamically adjusted based on the output of the adaptive deep reinforcement learning model, performance bottlenecks and fault points are identified in combination with a preset prediction model and resource allocation and service configuration are adjusted, user feedback data is periodically collected, the weights of different indicators in a preset service quality evaluation model are adjusted according to the user feedback data, the reward function of the adaptive deep reinforcement learning model is updated according to the adjusted service quality evaluation model, and the optimal service decision is solved based on the updated reward function, and the optimal service decision sequence is generated and executed.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images