Big data-based AI agent design platform decision optimization method

Abstract

The invention discloses an AI agent design platform decision optimization method based on big data, and particularly relates to the field of artificial intelligence, comprising multi-modal data sensing layer construction, a streaming feature calculation engine, a dynamic index fusion center and an adaptive decision matrix. According to the method, accurate synchronous monitoring of the utilization rate of hardware resources and dynamic collaborative optimization of heterogeneous computing units are achieved, and the resource scheduling efficiency in a complex computing scene is remarkably improved; knowledge system degradation caused by long-term learning is effectively prevented, and the continuous reliability of a cognitive system is ensured. The provided multi-dimensional decision credibility verification system is fused with interpretability penetration analysis, environment coupling modeling and logic drift detection technologies, the limitation of a traditional single credibility index is broken through, the risk prediction and fault-tolerant capability of the decision process is remarkably enhanced, and a full-dimensional safety decision guarantee system is constructed for an intelligent agent.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology. More specifically, the present invention relates to a decision optimization method for an AI agent design platform based on big data. Background Art

[0002] With the rapid development of artificial intelligence technology, AI agents are increasingly widely used in complex decision-making scenarios. An agent design platform based on big data needs to comprehensively consider multi-dimensional elements such as hardware computing resources, dynamic knowledge systems, and decision logic credibility to achieve efficient and reliable decision optimization.

[0003] Existing technologies usually adopt a single-dimensional monitoring system to collect resource utilization rates, achieve cognitive iteration through a static knowledge graph update mechanism, and rely on post hoc interpretability analysis to evaluate decision quality. Traditional methods obtain the load of computing nodes through basic performance counters at the hardware layer, update the knowledge base in batches at regular intervals at the knowledge layer, and conduct risk assessment through a static rule engine or a single credibility index at the decision layer.

[0004] However, existing technologies have significant deficiencies: the lack of unified dimension modeling for cross-heterogeneous computing units in hardware resource monitoring leads to distorted resource utilization rate evaluation; the knowledge update mechanism lags behind dynamic environmental changes, easily causing conflicts between the integrity and timeliness of the knowledge system; the credibility evaluation dimension of the decision-making process is single, making it difficult to capture risks of complex environmental coupling and logical mutations, resulting in insufficient system stability in high-load scenarios. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the existing technology, the present invention provides a decision optimization method for an AI agent design platform based on big data. Through the following solutions, it solves the problems of distorted heterogeneous resource monitoring, system conflicts caused by lagging knowledge updates, single decision credibility evaluation dimension, and insufficient system stability mentioned in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solution: A decision optimization method for an AI agent design platform based on big data, including:

[0007] S1: Construction of a multi-modal data perception layer: Establish a cross-level data collection network, including deploying heterogeneous computing resource probes, implanting knowledge graph change trackers, and integrating decision path recorders, covering three levels: the hardware layer, the knowledge layer, and the decision layer, for collecting raw data of hardware performance data, knowledge evolution data, and decision credibility data;

[0008] S2: Streaming feature calculation engine: According to the raw data collected in S1, perform multi-modal streaming calculation for high-order metrics, including a distributed computing performance metric group, a knowledge evolution tracking metric group, and a decision credibility verification metric group;

[0009] S3: Dynamic Index Fusion Center: Construct three non-linearly coupled quantitative evaluation models to transform the three groups of high-order indicators calculated in S2 into comprehensive indexes with clear physical meanings, including distributed training efficiency index, knowledge evolution health index, and decision credibility index;

[0010] S4: Adaptive Decision Matrix: By constructing a three-dimensional optimization space, map the comprehensive indexes calculated in S3 into an executable set of optimization strategies, including hardware resource reallocation strategy, knowledge snapshot rollback strategy, decision pipeline fusing strategy, and cross-domain collaborative optimization strategy.

[0011] Technical Effects and Advantages of the Present Invention:

[0012] 1. By constructing a cross-level multi-modal data perception layer, the present invention realizes the three-dimensional capture of the entire life cycle of hardware efficiency, knowledge evolution, and decision-making path, significantly improves the monitoring accuracy and time synchronization of heterogeneous computing resources, transforms the original data into a group of high-order indicators with physical meanings through a streaming feature calculation engine, breaks through the limitations of traditional single-dimensional monitoring, provides a multi-modal joint optimization basis for dynamic resource scheduling, and effectively solves the problem of low collaborative efficiency caused by hardware heterogeneity in edge computing scenarios;

[0013] 2. The knowledge evolution tracking index group proposed by the present invention innovatively introduces a memory conflict coefficient and a cognitive entropy change rate, constructs a quantitative evaluation framework for a dynamic knowledge system, realizes the real-time diagnosis of version compatibility problems during the knowledge inheritance process, and ensures the balance between stability and adaptability of the knowledge system during continuous learning through the coupled analysis of concept drift sensitivity and topological density, fundamentally suppressing the "knowledge corrosion" phenomenon in long-term learning scenarios;

[0014] 3. The decision credibility verification index group designed by the present invention integrates multi-dimensional features such as interpretability penetration, environmental coupling coefficient, and trajectory curvature, establishes a dynamic decision credibility evaluation model, realizes the penetrative analysis of potential risks in complex decision-making chains by revealing the interaction laws of transparency, robustness, and environmental adaptability, significantly improves the reviewability and anti-interference ability of AI decisions in open environments, and breaks through the limitations of traditional methods for logical drift detection. Brief Description of the Drawings

[0015] Figure 1 It is a schematic diagram of the overall structure of the present invention. Detailed Embodiments

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] Reference Figure 1 A decision optimization method for an AI agent design platform based on big data, as shown in the figure, includes:

[0018] S1: Construction of the multi-modal data perception layer: Establish a cross-layer data collection network, including deploying heterogeneous computing resource probes, implanting knowledge graph change trackers, and integrating decision path recorders, covering three levels: the hardware layer, the knowledge layer, and the decision layer, for collecting raw data of hardware performance data, knowledge evolution data, and decision trust data.

[0019] In the S1, resource probes are custom-developed through the Intel PCM tool at the hardware layer to collect the utilization rate of computing units in real time; a knowledge graph difference engine based on TransE is constructed at the knowledge layer to record semantic network changes at a granularity of 0.1 second; a SHAP interpreter and a PGD adversarial attack module are deployed at the decision layer to generate a decision path topology map synchronously. The three-layer data is transmitted through a time-division multiplexing channel of Apache Kafka and encapsulated as a time-series data packet using the Protobuf protocol.

[0020] At the hardware layer, the training time consumption of each computing node with a granularity of 10 ns is captured through the kernel-level timestamp counter of the Intel PCM tool, the SM utilization rate of NVIDIA DCGM and the TPU MXU utilization rate are obtained in real time through the PCIe bandwidth monitoring module, the gradient tensors before and after AllReduce are intercepted through the hook mechanism of the NCCL communication library, and the parameter vectors of each node are obtained by monitoring the callback of PyTorch's DistributedDataParallel.

[0021] At the knowledge layer, the difference between the new and old feature responses is recorded through the feature mapping module in the TransE difference engine, the connection status of the knowledge graph nodes is scanned at a frequency of 30 fps through the dynamic subgraph matching algorithm, the SPARQL query difference analysis between the new and old knowledge bases is realized based on the CQELS engine, and the information entropy is calculated through the version snapshot service built into the graph database.

[0022] The decision-making layer generates a heatmap of feature contributions through TreeExplainer of the SHAP interpreter, generates an adversarial perturbation set with L∞ norm constraint through the PGD module, synchronously collects the original input feature matrix through the decision path recorder, and calculates the first and second derivatives of the decision function through automatic differentiation of JAX.

[0023] It should be specifically noted in this embodiment that at least 8% of the bandwidth of the PCIe Gen4 bus needs to be reserved for monitoring data transmission during actual deployment of S1.

[0024] S1 breaks through the limitation of single-dimensional observation of traditional monitoring systems. Through the collaborative deployment of heterogeneous computing probes, knowledge graph change trackers, and decision path recorders, it realizes the three-dimensional capture of data throughout the life cycle of the intelligent agent. Through the Kafka time-sharing multiplexing channel and Protobuf protocol encapsulation technology, it solves the problem of clock drift of multi-source data in distributed systems, reduces the alignment error of the timestamps of the three types of data, and meets the real-time decision-making requirements.

[0025] S2: Streaming Feature Calculation Engine: According to the original data collected by S1, it performs multi-modal streaming calculations for high-order metrics, including a distributed computing efficiency metric group, a knowledge evolution tracking metric group, and a decision credibility verification metric group.

[0026] The distributed computing efficiency metric group includes training time volatility, heterogeneous computing unit utilization rate, gradient sparsification rate, and model convergence variance. The knowledge evolution tracking metric group includes concept drift sensitivity, knowledge topology density, memory conflict coefficient, and cognitive entropy change rate. The decision credibility verification metric group includes interpretability penetration, counterfactual robustness, environmental coupling coefficient, and decision trajectory curvature.

[0027] The training time volatility is specifically expressed as: V t is the training time volatility, and the time series is the iteration time consumption of each node, and the arithmetic mean μ t is used as the reference value, and the standard deviation σ t is used to quantify the degree of dispersion. To avoid numerical inaccuracy caused by a small mean, ∈ is introduced as a smoothing factor, and finally the coefficient of variation form σ t / (μ t +∈) is used to represent the relative fluctuation, m represents the total number of nodes. The heterogeneous computing unit utilization rate is specifically expressed as: V t U h is the heterogeneous computing unit utilization rate. For GPU / TPU, a non-linear transformation is adopted, and the CPU term retains the linear proportional coefficient. The cross-architecture computing power dimension is unified through α, β, γ, and finally the product summation structure simplifies the modeling of hardware differences, N g 、Nt , N c represents the actual number of stream processors used by the GPU / TPU / CPU, C g , C t , C c represents the theoretical peak number of stream processors of the GPU / TPU / CPU, and α, β, γ are normalization constants. The gradient sparsification rate is specifically expressed as: S g is the gradient sparsification rate. The effective gradients are dynamically screened using the quantile threshold θ to avoid environmental sensitivity caused by a fixed threshold. The denominator introduces the d 0.75 exponential decay term to suppress the influence of parameter magnitude differences on sparsity evaluation. represents the gradient value of the k-th parameter, θ represents the 30% quantile threshold, d represents the number of parameters, is the indicator function. The model convergence variance is specifically expressed as: D c is the model convergence variance. The differences between nodes are calculated using the L2 distance of the parameter vector, and multiplied by the normalization factor η / (η + ||μ w ||) to eliminate the influence of the parameter's own dimension. w i represents the parameter vector of the i-th node, represents the average parameter vector, and η represents the dimension alignment constant.

[0028] The training time volatility reflects the load balancing state of the distributed system, identifies computing nodes with hardware failures or communication blockages, and provides a quantitative basis for dynamic task migration. The heterogeneous computing unit utilization rate is used to quantify the comprehensive utilization efficiency of the hybrid computing architecture and locate the problem of computing power fragmentation. The gradient sparsification rate is used to optimize the communication overhead budget of distributed training, and realizes the adaptive triggering of the gradient compression algorithm through sparse pattern recognition. The model convergence variance is used to monitor the parameter synchronization quality and identify lagging nodes in asynchronous training.

[0029] The distributed computing efficiency metric group jointly models the time stability, hardware heterogeneity, communication efficiency, and model synchronization quality of the training process, breaking through the limitation of traditional monitoring systems that only focus on the resource consumption of a single dimension. Through the coupled analysis of the gradient sparsification rate and the convergence variance, it realizes the two-way feedback control of computing load and model quality for the first time, can dynamically identify implicit performance bottlenecks caused by hardware heterogeneity, and realizes the elastic scheduling of cross-architecture computing resources while maintaining the model convergence accuracy, solving the contradiction that it is difficult to have both high hardware resource utilization and good model training effect in traditional distributed training, especially the problem of low collaborative efficiency of heterogeneous devices in edge computing scenarios.

[0030] The concept drift sensitivity is specifically expressed as: S dis the concept drift sensitivity. In the reproducing kernel Hilbert space, the old and new features x o , x n are mapped into high-dimensional vectors through the RBF kernel function κ, the modulus of their projection difference is calculated, and denominator normalization is used to eliminate the influence of dimension. Φ represents the feature mapping function, H represents the reproducing kernel Hilbert space, and κ represents the RBF kernel function ζ = 0.1 represents the kernel width parameter. The knowledge topological density is specifically expressed as: ρ k is the knowledge topological density, s ij represents the semantic similarity of nodes i and j, σ(x) represents the Sigmoid activation function, E represents the number of edges in the knowledge graph, λ = 0.01 represents the adjustment parameter. First, the continuous similarity is converted into the probability of the existence of virtual edges through the Sigmoid function σ(s ij ), and then the adjustment effect of the actual number of edges is introduced by combining the hyperbolic tangent function tanh(λE) to realize the joint modeling of semantic association and structural complexity. The memory conflict coefficient is specifically expressed as: C m is the memory conflict coefficient, represents the decision difference identifier, represents the decision output result of the new version knowledge base for the input data at time t, represents the decision output result of the old version knowledge base for the same input at the same time t, v t represents the confidence difference, ω = 0.3 is the temperature parameter, T 0.8 is the non-linear sample size compensation factor, and the exponential decay function is used to soften the binary difference signal. The interference of high-frequency small conflicts is reduced through the denominator design of T 0.8 . The cognitive entropy change rate is specifically expressed as: R e is the cognitive entropy change rate, H(G)=-∑p(c)logp(c) represents the information entropy of the knowledge graph categories, ξ is a stable constant, L is the global entropy, and l is the local entropy. The complexity of the knowledge system is measured by the information entropy H(G), and the absolute value of the entropy change between consecutive versions is calculated. Through the stable processing of cross-magnitude entropy values is realized, and ξ eliminates the mathematical anomaly of the zero-entropy scenario.

[0031] The concept drift sensitivity quantifies the response intensity of the new features, identifies potential concept drift risks, provides a trigger threshold for incremental learning, and overcomes the lag defect of the traditional sliding window detection method. The knowledge topology density is used to evaluate the structural compactness of the knowledge network, prevent the break of the inference chain caused by excessive sparsity or the storage redundancy caused by excessive density, and optimize the trigger conditions of the knowledge distillation strategy. The memory conflict coefficient is used to detect the cognitive conflicts in the knowledge iteration process, distinguish accidental errors from system-level logical contradictions, provide a priority ranking for knowledge backtracking, and improve the reliability of the continuous learning system. The cognitive entropy change rate is used to monitor the evolution speed of the knowledge system, avoid system oscillations caused by explosive updates, and provide a smooth control mechanism for cognitive evolution.

[0032] The knowledge evolution tracking index group constructs a quantitative evaluation framework for the dynamic knowledge system. Through the spatio-temporal correlation analysis of the concept drift sensitivity and the cognitive entropy change rate, it breaks through the update paradigm of the static knowledge graph. The introduction of the memory conflict coefficient first transforms the version compatibility problem in the knowledge inheritance process into a computable index, realizes the real-time health diagnosis of the knowledge system evolution process, ensures the continuous and reliable evolution of the AI cognitive system, and solves the "knowledge corrosion" phenomenon commonly existing in the long-term learning system, that is, the industry pain point problem that the integrity of the original knowledge system is damaged due to the injection of new knowledge.

[0033] The interpretive penetration is specifically expressed as: P e is the interpretive penetration, φ i represents the importance score of the i-th feature, E represents the interpretable feature subset, KL represents the KL divergence between the interpretive distribution q and the true distribution p, ν represents the sensitivity coefficient, j is the index traversing all feature dimensions, d is the total number of features, л is a constant, and for the importance score φ i of the interpretable feature ε, apply the 1.5th power to strengthen the head effect, compress the KL divergence to the [-1,1] interval by combining with the error function erf, and finally the product structure balances the explicit interpretation and the implicit consistency. The counterfactual robustness is specifically expressed as: R a is the counterfactual robustness. Under the l ∞ norm constraint, find the minimum adversarial perturbation δ, and use ‖δ‖ 0.9 as the penalty term to highlight the sensitivity to small perturbations, ι eliminates the mathematical singularity at zero perturbation, f represents the model decision function, represents the perturbation upper limit, and ι = 0.1 represents the robustness enhancement factor. The environmental coupling coefficient is specifically expressed as: C erepresents the environmental coupling coefficient, calculates the power mean of the mutual information between features to obtain the basic coupling degree, and then links it with the input matrix condition number CN(X). The tanh function is used to control the numerical explosion when the condition number is too large. MI represents the mutual information, CN(X) represents the condition number of the input matrix, ψ=0.05 represents the scaling factor, and x i 、x j represents the i-th and j-th dimension feature variables in the input feature space, d is the total number of features, and the decision trajectory curvature is specifically expressed as: κ d To determine the curvature of the decision trajectory, based on the curvature calculation formula in differential geometry, the decision path is parameterized as a time function f(t), and the velocity vector v t With the acceleration vector a t The ratio of the fork modulus length is used to calculate the instantaneous curvature, and finally the time domain average is taken. represents the decision speed vector, represents the decision acceleration vector, ρ is the regularization term, t is the time slice identifier of the decision process, and T is the total time span of the decision trajectory.

[0034] The explainability penetration is used to quantify the degree of dominance of explainable features on decision-making, identify potential risks of over-reliance on unexplainable features, and enhance the credibility and auditability of AI decisions. Counterfactual robustness is used to evaluate the model's defense capabilities against adversarial samples, locate vulnerable areas of decision boundaries, and provide gradient backpropagation optimization directions for robustness enhancement training. The environmental coupling coefficient is used to reveal the overfitting risk caused by hidden correlations between features, guide the elimination of redundancy in feature engineering, and improve the generalization performance of the model in an open environment. The decision trajectory curvature is used to capture the mutation points of decision logic, identify abnormal jumping thinking in the reasoning process, and enhance the explainability and traceability of complex decision chains.

[0035] The decision credibility verification indicator group integrates the explainability of decision logic, environmental correlation strength and decision path stability to establish a multimodal credibility assessment model. By extracting the spatiotemporal features of the decision trajectory curvature, it realizes the quantitative characterization of the dynamic credibility of the decision process for the first time. It can penetrate and analyze the potential risk transmission path in complex decision chains, and provide a self-explanatory early warning mechanism for high-risk decisions. It solves the dilemma of traditional trusted AI technology in the decision-making process of "knowing what it is but not why it is", especially the common problem that decision logic drift is difficult to detect in a dynamic and open environment.

[0036] S3: Dynamic index fusion center: Construct three nonlinear coupled quantitative evaluation models to convert the three sets of high-order indicators calculated by S2 into comprehensive indexes with clear physical meanings, including distributed training effectiveness index, knowledge evolution health index and decision credibility index.

[0037] The distributed training effectiveness index is specifically expressed as: E d is the distributed training efficiency index, σ represents the anti-zero offset, and represents the resource efficiency decay factor.

[0038] The numerator of the distributed training efficiency index compresses the magnitude of heterogeneous computing resource utilization through ln(1 + U h ), and erf(S g ) constrains the effective range of the gradient sparsification rate. The denominator constructs a joint measure of training stability, reflecting the coupling effect of time fluctuation and parameter synchronization. The exponential term suppresses the abnormal state of high resource utilization accompanied by low communication efficiency. The distributed training efficiency index is used to achieve the dynamic balance between computing resource efficiency and training process stability, and detect the symbiotic faults of hardware-level anomalies and algorithm-level instability through non-linear coupling.

[0039] The knowledge evolution health index is specifically expressed as: K h is the knowledge evolution health index, ψ represents the concept sensitivity gain coefficient, Ω, ξ represent the damping indices of conflict and entropy change, τ represents the baseline offset, and κ represents the structure deviation penalty factor.

[0040] The knowledge evolution health index constructs a "open - conservative" dual-mode evaluation of the knowledge system through tanh(ψS d ·ρ k ), balancing the absorption of new concepts and the integrity of the knowledge structure. The denominator establishes a non-linear damping mechanism for memory conflict and cognitive entropy change, and penalizes the asymmetric deviation of topological density and entropy change rate through the cosh -1 term to prevent the distortion of the knowledge system structure. The knowledge evolution health index is used to quantify the game state of stability and adaptability in the process of knowledge evolution, and identify cognitive rigidity caused by excessive conservatism or knowledge pollution caused by excessive openness.

[0041] The decision credibility index is specifically expressed as: T c is the decision credibility index, η represents the interpretability reinforcement index, υ represents the environmental sensitivity compensation term, and ζ represents the interpretability advantage gain.

[0042] The decision credibility index passes through to strengthen the dominant influence of interpretable features on decision-making, and sigmoid(R a ) normalizes the counterfactual robustness. The denominator compresses the joint risk of environmental coupling and decision curvature, ReLU(P e - C e)Build a dynamic balance to the environmental sensitivity of interpretability, inhibit high-risk decisions of "high coupling and low transparency". The decision credibility index is used to reveal the interaction law among transparency, robustness, and environmental adaptability in decision credibility, and locate the systemic risks masked by the advantages of a single dimension.

[0043] Based on the physical association characteristics among indicators, S3 designs a fusion function with domain adaptability: for the coupling relationship between hardware resources and the training process, a logarithmic-error function architecture is adopted to balance resource utilization and communication efficiency; for the knowledge evolution characteristics, the hyperbolic function combination is used to quantify the game between the openness and stability of the knowledge system; for the complexity of decision credibility, an exponential reinforcement mechanism is established to reveal the interaction among transparency, robustness, and environmental adaptability. Each formula term corresponds to a clear physical process, forming a dynamic balance evaluation system of "ability - risk".

[0044] S3 first constructs a symbolic mathematical parser to convert the formula into an abstract syntax tree, and then deploys heterogeneous computing units for real-time processing: the distributed training efficiency index is calculated in parallel on the GPU through the LLVM compilation engine to achieve a response at the microsecond level; the knowledge evolution health index integrates the LSTM prediction unit to predict the evolution trend; the decision credibility index constructs a risk propagation graph based on the Bayesian network, and the calculation results are encapsulated as a triple containing <index value, confidence level, gradient> through the gRPC service for downstream system calls. The entire process adopts a hierarchical asynchronous processing architecture, and loose-coupling communication between computing units is achieved through a message queue to ensure system stability in high-concurrency scenarios.

[0045] S4: Adaptive decision matrix: By constructing a three-dimensional optimization space, map the comprehensive index calculated by S3 into an executable set of optimization strategies, including hardware resource reallocation strategies, knowledge snapshot rollback strategies, decision pipeline fusing strategies, and cross-domain collaborative optimization strategies.

[0046] The core of the S4 decision matrix lies in solving three key contradictions: the balance between computing resource efficiency and training stability, the trade-off between knowledge update speed and system integrity, and the game between decision response speed and credibility. By defining the strategy trigger domain in the three-dimensional index space, the continuous index values are discretized into four typical system states, including resource overload state, knowledge pollution state, decision risk state, and composite anomaly state. Each state corresponds to a preset strategy combination and dynamic weight allocation rule, and the judgment logic integrates fuzzy control theory and reinforcement learning mechanism to enable the strategy selection to have environmental adaptability.

[0047] The trigger condition for the hardware resource reallocation strategy is E d <θ e , and the action to be executed is to reallocate the GPU / TPU task load based on the greedy algorithm, giving priority to ensuring the resource supply of critical computing nodes;

[0048] θ e The training efficiency threshold indicates the minimum efficiency index allowed by the distributed training system, which is determined based on the sliding percentile of the historical training task completion time;

[0049] The triggering condition of the knowledge snapshot rollback strategy is The execution action is to call the distributed version storage engine to load the latest stable knowledge snapshot and start the difference comparison and repair process;

[0050] Δ k is the knowledge health deviation tolerance, which indicates the maximum allowable deviation of the knowledge system from the steady state. Dynamic calculation, β is the task complexity coefficient, T task is the task duration, l represents the lth knowledge base update operation, l∈[1,L], L represents the maximum number of versions in the sliding time window, is the dynamic baseline value of the knowledge health index, specifically expressed as: W is the baseline calculation window size, t is the current time point, and w is the time backtracking offset;

[0051] The triggering condition of the decision pipeline fuse strategy is T c ∈Ω t ,The execution action is to switch to the lightweight verification decision process, enable the alternative model and activate the audit tracking module;

[0052] Ω t It is a credible risk domain, mathematically defined as a three-dimensional space min is the minimum value, max is the maximum value, is the critical threshold of environmental sensitivity, specifically expressed as: Q 0.95 is the 95% quantile of the mutual information, CN is the condition number, MI is the mutual information, is the path mutation detection threshold, which is specifically expressed as: n is the number of decision cycles in the sliding window, ReLU is the rectified linear unit, Var is the variance, and i represents the i-th decision cycle in the sliding time window;

[0053] The triggering condition of the cross-domain collaborative optimization strategy is the multi-index synchronization anomaly, and the execution action is to start the global resource reorganization protocol and synchronously adjust the computing topology, knowledge version and decision logic.

[0054] The S4 realizes collaborative optimization at three levels of hardware resources, knowledge system, and decision-making logic, ensuring that the system maintains stable and efficient operation under complex load fluctuations, effectively preventing the spread of cascading failures caused by single-dimensional anomalies. The problems solved include the inability of traditional methods to handle the coupling effects of cross-level metrics, the difficulty of static policy libraries to adapt to dynamic environmental changes, and the lack of ability of artificial rule engines to handle unknown anomaly patterns.

[0055] The constants used in the formulas of the present invention are determined through theoretical derivation, empirical analysis, and engineering experience, and will not be specifically described in this embodiment.

[0056] It should be specifically noted in this embodiment that the symbol system of each formula in this embodiment is a self-consistent closed system. The reuse of symbols across formulas does not represent the relevance between physical quantities, and the same-named symbols between different formulas represent different physical quantities or indexing objects.

[0057] The present invention adopts a four-layer progressive architecture to realize the decision-making optimization of the AI intelligent agent. First, a cross-level data acquisition network is constructed through the multi-modal data perception layer: Intel PCM probes are deployed at the hardware layer to capture GPU / TPU utilization and 10ns-level training time in real time; at the knowledge layer, the TransE differential engine scans the changes in the knowledge graph at a frequency of 30fps, and combines with the CQELS engine to perform SPARQL query difference analysis; at the decision layer, the SHAP interpreter and the PGD adversarial attack module are run synchronously to generate a feature heat map and an L∞ perturbation set respectively. The three-layer data is transmitted through the Apache Kafka time-sharing multiplexing channel and encapsulated into a time-series data packet using the Protobuf protocol to solve the clock drift problem in the distributed system;

[0058] Subsequently, the streaming feature calculation engine starts multi-modal calculations: calculates the training time volatility and gradient sparsification rate for hardware data, and dynamically filters effective gradients through quantile thresholds; knowledge data generates concept drift sensitivity and cognitive entropy change rate through kernel space mapping, and quantifies topological density using the Sigmoid function; decision data analyzes the interpretability penetration through KL divergence, and calculates the curvature of the decision trajectory in combination with differential geometry. All indicators form three groups of high-order indicator sets through non-linear coupling, realizing the joint modeling of computational load - knowledge evolution - decision credibility;

[0059] The dynamic exponential fusion center converts the indicators into three-dimensional comprehensive indices: the distributed training efficiency index balances resource utilization and communication efficiency through a logarithmic-error function; the knowledge evolution health index quantifies the game relationship between the openness and stability of the knowledge system using a hyperbolic function; the decision credibility index constructs an index reinforcement mechanism to reveal the interaction of transparency - robustness - environmental adaptability. The calculation process is accelerated by the GPU through the LLVM engine, and combines with the LSTM prediction unit to predict the evolution trend, and finally encapsulates and outputs as a triple of <index value, confidence, gradient>;

[0060] The final adaptive decision matrix constructs a three-dimensional optimization space: when the distributed training efficiency index is lower than the historical sliding percentile threshold, the greedy algorithm is triggered to reallocate the GPU / TPU load; when the knowledge health index deviates from the dynamic baseline, the version snapshot rollback is called; when the decision credibility index exceeds the 95% quantile of the mutual information, the lightweight decision-making process is switched, and the global collaborative optimization is started for the composite abnormal state, and the computing topology, knowledge version, and decision logic are adjusted synchronously to form a closed-loop optimization system.

[0061] Secondly: in the drawings of the disclosed embodiments of the present invention, only the structures related to the disclosed embodiments are involved, and other structures can refer to the general design. Without conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0062] Finally: The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A decision optimization method for an AI agent design platform based on big data, characterized in that, Including: S1: Construction of multi-modal data perception layer: Establish a cross-hierarchy data collection network, including deploying heterogeneous computing resource probes, implanting knowledge graph change trackers, and integrating decision path recorders, covering three levels: the hardware layer, the knowledge layer, and the decision layer, for collecting raw data of hardware performance data, knowledge evolution data, and decision trust data; S2: Streaming feature calculation engine: According to the raw data collected in S1, calculate high-order metrics through multi-modal streaming computing, including distributed computing performance metric groups, knowledge evolution tracking metric groups, and decision credibility verification metric groups; S3: Dynamic index fusion center: Construct three non-linearly coupled quantitative evaluation models to transform the three groups of high-order metrics calculated in S2 into comprehensive indexes with clear physical meanings, including distributed training performance index, knowledge evolution health index, and decision credibility index; S4: Adaptive decision matrix: By constructing a three-dimensional optimization space, map the comprehensive indexes calculated in S3 into an executable optimization strategy set, including hardware resource reallocation strategy, knowledge snapshot rollback strategy, decision pipeline fusing strategy, and cross-domain collaborative optimization strategy.

2. A decision optimization method for an AI agent design platform based on big data according to claim 1, characterized in that: In S1, a resource probe is custom-developed through the Intel PCM tool at the hardware layer to collect the utilization rate of computing units in real time; At the knowledge layer, a knowledge graph difference engine based on TransE is constructed to record semantic network changes at a granularity of 0.1 second; at the decision layer, a SHAP interpreter and a PGD adversarial attack module are deployed to generate a decision path topology map synchronously. The three-layer data is transmitted through a time-division multiplexing channel of Apache Kafka and encapsulated into time-series data packets using the Protobuf protocol.

3. A decision optimization method for an AI agent design platform based on big data according to claim 1, characterized in that: The distributed computing performance metric groups include training time volatility, heterogeneous computing unit utilization rate, gradient sparsification rate, and model convergence variance. The knowledge evolution tracking metric groups include concept drift sensitivity, knowledge topology density, memory conflict coefficient, and cognitive entropy change rate. The decision credibility verification metric groups include interpretability penetration, counterfactual robustness, environmental coupling coefficient, and decision trajectory curvature.

4. The decision optimization method for an AI intelligent agent design platform based on big data according to claim 3, characterized in that: The specific expression of the training time volatility is as follows: V t is the training time volatility, and the time series is the iteration time consumption of each node, and the arithmetic mean μ t is calculated as the reference value, and the standard deviation σ t is used to quantify the degree of dispersion. To avoid numerical inaccuracy caused by a small mean, ∈ is introduced as a smoothing factor, and finally the coefficient of variation form σ t (μ t +∈) is used to characterize the relative fluctuation. m represents the total number of nodes. The specific expression of the heterogeneous computing unit utilization rate is as follows: V t U h is the heterogeneous computing unit utilization rate. For GPU / TPU, a non-linear transformation is adopted, and the CPU term retains the linear proportional coefficient. The dimensional consistency of cross-architecture computing power is achieved through α, β, and γ. Finally, the product summation structure simplifies the hardware difference modeling. N g 、N t 、N c represent the actual number of stream processors used by GPU / TPU / CPU. C g 、C t 、C c represent the theoretical peak number of stream processors of GPU / TPU / CPU. α, β, and γ are normalization constants. The specific expression of the gradient sparsification rate is as follows: S g is the gradient sparsification rate. The quantile threshold θ is used to dynamically screen effective gradients to avoid the environmental sensitivity caused by a fixed threshold. The denominator introduces d 0.75 exponential decay term to suppress the influence of the parameter order of magnitude difference on the sparsity evaluation. represents the gradient value of the k-th parameter. represents the 30% quantile threshold, d represents the number of parameters, is the indicator function. The specific expression of the model convergence variance is as follows: D c is the model convergence variance. The difference between nodes is calculated by the L2 distance of the parameter vector, and the dimensional influence of the parameter itself is eliminated by multiplying the normalization factor η / (η + ||μ ω ||). w i represents the parameter vector of the i-th node. represents the average parameter vector, and η represents the dimensional alignment constant.

5. A decision optimization method for an AI agent design platform based on big data according to claim 3, characterized in that: The concept drift sensitivity is specifically expressed as: S d is the concept drift sensitivity. In the reproducing kernel Hilbert space, the old and new features x o , x n are mapped into high-dimensional vectors through the RBF kernel function κ, the modulus of the projection difference is calculated, and denominator normalization is used to eliminate the influence of dimension. Φ represents the feature mapping function, H represents the reproducing kernel Hilbert space, and κ represents the RBF kernel function ζ = 0.1 represents the kernel width parameter. The knowledge topological density is specifically expressed as: ρ k is the knowledge topological density, s ij represents the semantic similarity of nodes i, j, σ(x) represents the Sigmoid activation function, E represents the number of edges in the knowledge graph, λ = 0.01 represents the adjustment parameter. First, the continuous similarity is converted into the virtual edge existence probability through the Sigmoid function σ(s ij ), and then the adjustment effect of the actual number of edges is introduced by combining the hyperbolic tangent function tanh(λE) to realize the joint modeling of semantic association and structural complexity. The memory conflict coefficient is specifically expressed as: C m is the memory conflict coefficient, represents the decision difference identifier, represents the decision output result of the new version knowledge base for the input data at time t, represents the decision output result of the old version knowledge base for the same input at the same time t, v t represents the confidence difference, ω = 0.3 is the temperature parameter, T 0.8 is the non-linear sample volume compensation factor, and the exponential decay function is used to soften the binary difference signal. The interference of high-frequency small conflicts is reduced through the denominator design of T 0.8 . The cognitive entropy change rate is specifically expressed as: R e is the cognitive entropy change rate. H(G) = -∑p(c)log p(c) represents the information entropy of the knowledge graph categories, ξ is the stability constant, L is the global entropy, and l is the local entropy. The complexity of the knowledge system is measured by the information entropy H(G), and the absolute value of the entropy change between consecutive versions is calculated. Through the stable processing of cross-magnitude entropy values is realized, and ξ eliminates the mathematical anomaly of the zero-entropy scenario.

6. The decision optimization method for an AI intelligent agent design platform based on big data according to claim 3, characterized in that: The specific expression of the interpretive penetration is as follows: P e is the interpretive penetration, φ i represents the importance score of the i-th feature, E represents the interpretable feature subset, KL represents the KL divergence between the interpretive distribution q and the true distribution p, ν represents the sensitivity coefficient, j is the index traversing all feature dimensions, d is the total number of features, л is a constant, and for the importance score φ i of the interpretable feature ε, apply a 1.5th power to enhance the head effect, compress the KL divergence to the [-1,1] interval by combining with the error function erf, and finally the product structure balances the explicit interpretation and implicit consistency. The counterfactual robustness is specifically expressed as: R a is the counterfactual robustness. Under the l ∞ norm constraint, find the minimum adversarial perturbation amount δ, and use ||δ|| 0.9 as the penalty term to highlight the sensitivity to small perturbations, ι eliminates the mathematical singularity at zero perturbation, f represents the model decision function, represents the perturbation upper limit, ι = 0.1 represents the robustness enhancement factor, and the environmental coupling coefficient is specifically expressed as: C e represents the environmental coupling coefficient. Calculate the power mean of the mutual information between features to obtain the basic coupling degree, and then link it with the condition number CN(X) of the input matrix. Control the numerical explosion when the condition number is too large through the tanh function. MI represents the mutual information amount, CN(X) represents the condition number of the input matrix, ψ = 0.05 represents the scaling factor, x i 、x j represent the i-th and j-th dimensional feature variables in the input feature space, d is the total number of features, and the decision trajectory curvature is specifically expressed as: κ d is the decision trajectory curvature. Based on the curvature calculation formula in differential geometry, parameterize the decision path as a time function f(t), and obtain the instantaneous curvature through the cross modulus ratio of the velocity vector v t and the acceleration vector a t , and finally take the time domain average. represents the decision velocity vector, represents the decision acceleration vector, ρ is the regularization term, t is the time slice identifier of the decision process, and T is the total time span of the decision trajectory.

7. The decision optimization method of an AI agent design platform based on big data according to claim 1, characterized in that: The distributed training efficiency index is specifically expressed as: E d is the distributed training efficiency index, σ represents the anti-zero offset, and represents the resource efficiency decay factor; The specific expression of the knowledge evolution health index is as follows: K h is the knowledge evolution health index, ψ represents the concept sensitivity gain coefficient, Ω, ξ represent the damping indices of conflict and entropy change, τ represents the baseline offset, and κ represents the structural deviation penalty factor; The decision credibility index is specifically expressed as: T c is the decision credibility index, η represents the interpretability enhancement index, υ represents the environmental sensitivity compensation term, and ζ represents the interpretability advantage gain.

8. The decision optimization method of an AI agent design platform based on big data according to claim 1, characterized in that: The trigger condition of the hardware resource reallocation strategy is E d <θe, the action to be executed is to reallocate the GPU / TPU task load based on the greedy algorithm, and give priority to ensuring the resource supply of critical computing nodes; θ e is the training efficiency threshold, representing the minimum efficiency index allowed by the distributed training system, and is determined based on the sliding percentile of the historical training task completion time; The triggering condition of the knowledge snapshot rollback policy is The action to be executed is to call the distributed version storage engine to load the most recent stable knowledge snapshot and start the differential comparison and repair process; Δ k represents the knowledge health deviation tolerance, which indicates the maximum allowable amplitude of the deviation of the knowledge system from the steady state. It is calculated dynamically through , where β is the task complexity coefficient, T task is the task duration, l represents the l-th knowledge base update operation, l ∈ [1, L], and L represents the maximum number of versions within the sliding time window. is the dynamic baseline value of the knowledge health index, which is specifically expressed as: W is the baseline calculation window size, t is the current time point, and w is the time backtracking offset; The trigger condition for the decision pipeline fusing strategy is T c ∈Ω t , and the action to be executed is to switch to the lightweight verification decision process, enable the alternative model, and activate the audit tracking module; Ω t is a trusted risk domain, mathematically defined as a three-dimensional space min is the minimum value, max is the maximum value, is the environmental sensitivity critical threshold, specifically expressed as: Q 0.95 is the 95% quantile of the mutual information, CN is the condition number, MI is the mutual information, is the path mutation detection threshold, specifically expressed as: n is the number of decision-making cycles within the sliding window, ReLU is the rectified linear unit, Var is the variance, and i represents the i-th decision-making cycle within the sliding time window; The trigger condition of the cross-domain collaborative optimization strategy is multi-index synchronization anomaly, and the action to be executed is to start the global resource reorganization protocol to synchronously adjust the computing topology, knowledge version, and decision logic.

Images