A robust information bottleneck optimization method based on diffusion model and bayesian optimization

Abstract

The application discloses a robust information bottleneck optimization method based on a diffusion model and Bayesian optimization, which constructs an information compression, task prediction and transmission robustness joint optimization target; adopts Latin hypercube sampling to initialize the bottleneck variable, uses kernel ridge regression to establish a mutual information constraint proxy upper bound, and uses a Gaussian process model for multi-target proxy modeling; a candidate representation is generated through a conditional diffusion model, an entropy weight gradient diffusion guidance mechanism is introduced for reverse sampling, and an adaptive hybrid sampling and Pareto optimal solution screening strategy is used to switch between diffusion sampling and a genetic algorithm, and a Pareto optimal bottleneck variable set meeting compression constraints is output. Compared with the prior art, the application can realize the joint optimization of task prediction performance and transmission robustness under the condition of meeting the information compression constraint, and output a Pareto optimal bottleneck variable set, and is suitable for intelligent communication, distributed learning and complex system decision optimization scenes.

Description

Technical Field

[0001] This invention relates to the field of intelligent information processing and machine learning optimization technology, and in particular to a robust information bottleneck optimization method based on diffusion model and Bayesian optimization, which is applicable to scenarios such as information representation learning, multi-objective optimization solution and robust feature compression and transmission in complex noise environments. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep neural networks have made significant progress in many fields such as image recognition, speech processing, natural language understanding, intelligent perception, autonomous driving, and remote sensing information processing. Especially in complex perception tasks, deep models can learn high-level semantic features through large-scale sample training, thereby effectively improving task inference accuracy and environmental adaptability. However, with the continuous increase in model size, data dimensionality, and deployment scenario complexity, how to achieve efficient and stable information representation learning under conditions of limited computing resources, limited communication resources, and complex perturbations has become a key issue in the design of current intelligent systems.

[0003] In the fields of representation learning and feature compression, information bottleneck methods provide an important theoretical tool for constructing compact and efficient intermediate representations by striking a balance between input information compression and task-related information preservation. Traditional information bottleneck methods aim to maximize the relevance between representation variables and target variables, and constrain the amount of information between representation variables and input variables, thereby compressing redundant information and preserving effective task information. These methods have high application value in tasks such as classification, compressed sensing, representation learning, and communication coding.

[0004] However, in practical applications, especially in dynamic and complex environments such as satellite edge inference, wireless communication sensing collaboration, and remote intelligent sensing, systems often face problems such as channel noise, input disturbances, link instability, and resource constraints. Traditional information bottleneck methods typically assume an ideal mapping process from input to representation, making it difficult to fully characterize the stability and robustness of representation variables under noisy transmission or disturbance conditions. To address this, robust information bottleneck methods further consider the relationship between the perturbed representation and the original representation and task objective, ensuring that the compressed representation not only has good task discrimination capabilities but also maintains strong anti-interference capabilities and transmission stability in uncertain environments.

[0005] While robust information bottleneck methods are theoretically better suited to complex environments, their practical application still faces numerous challenges. First, mutual information is an implicit statistic, particularly difficult to calculate directly under high-dimensional continuous variables. It typically requires variational estimation, neural estimation, or approximate substitution methods, making the optimization process complex and unstable. Second, most existing methods employ Lagrange multiplier-based constraint optimization, unifying compression constraints, task performance, and robustness requirements into a single objective function through weighted summation. These methods often rely on the manual setting and iterative adjustment of multiple hyperparameters, making them sensitive to parameters and struggling to simultaneously achieve compressibility, inference performance, and robustness in scenarios with significant multi-objective conflicts. This can easily lead to constraints not being strictly satisfied or solutions biased towards a particular local objective.

[0006] Furthermore, in high-dimensional non-convex representation spaces, traditional gradient-based optimization methods are easily affected by initial values, target coupling relationships, and local extrema, leading to low search efficiency, poor convergence stability, and difficulty in obtaining bottleneck variables with good global performance. Especially in black-box scenarios, the objective function may not have an explicit expression, or the analytical gradient cannot be directly obtained, further limiting the applicability of traditional optimization methods. At the same time, real-world systems often focus on a set of Pareto optimal solutions that reflect the balance between different compression ratios, different robustness levels, and different task accuracy, rather than a single optimal point. This also places higher demands on the diversity preservation and distribution modeling capabilities of optimization algorithms.

[0007] Currently, some works have attempted to alleviate the aforementioned problems through variational inference, Bayesian optimization, and heuristic search. On the one hand, some methods construct variational lower bounds or mutual information estimators to approximate the solution of high-dimensional mutual information, thereby improving the trainability of the information bottleneck model. On the other hand, some methods employ Bayesian optimization or evolutionary algorithms to perform black-box search on complex objectives, reducing reliance on explicit gradient information. However, existing methods still have the following shortcomings: First, the accuracy and stability of mutual information estimation are still limited by the sample size and the form of the estimator, and are prone to introducing large variance under complex constraints. Second, there is a lack of a unified mechanism that can simultaneously consider search efficiency and solution set diversity in the multi-objective optimization process. Third, in high-dimensional complex representation spaces, although traditional heuristic methods have certain global search capabilities, the quality and distribution controllability of generated candidate solutions are insufficient, making it difficult to efficiently approximate the Pareto front of the robust information bottleneck problem.

[0008] In recent years, diffusion-generative models have demonstrated superior performance in modeling high-dimensional complex distributions. These models, through forward noise addition and backward denoising processes, can progressively learn complex sample distributions and outperform traditional generative models in terms of generation quality, training stability, and pattern coverage. Therefore, if diffusion-generative models can be introduced into the robust information bottleneck optimization process and combined with black-box optimization mechanisms and multi-objective surrogate modeling methods, it is hoped that the dependence on explicit gradients, Lagrange relaxation, and a single optimal solution search path in traditional robust information bottleneck optimization can be overcome. This would allow for the synergistic optimization of compressibility, task performance, and robustness in complex noisy environments.

[0009] Therefore, there is an urgent need to propose a new robust information bottleneck solution method to address the problems in existing technologies, such as the complexity of mutual information computation, the sensitivity of Lagrange multiplier hyperparameters, the difficulty in simultaneously satisfying multiple objective constraints, and the insufficient optimization efficiency in high-dimensional non-convex spaces. This would provide a new technical approach for efficient, stable, and interpretable information representation learning in complex dynamic environments. Summary of the Invention

[0010] The purpose of this invention is to address the problems in existing robust information bottleneck optimization techniques, such as complex mutual information computation, sensitivity to Lagrange multiplier hyperparameters, difficulty in simultaneously satisfying multiple objective constraints, and low optimization efficiency in high-dimensional non-convex spaces. This invention provides a robust information bottleneck optimization method based on diffusion models and Bayesian optimization. This method transforms the traditional robust information bottleneck optimization process, which relies on Lagrange multipliers and gradient constraints, into a generative black-box search process. By constructing a unified solution framework of "mutual information surrogate upper bound—multi-objective surrogate modeling—diffusion generative optimization—adaptive hybrid sampling," it achieves joint modeling and collaborative optimization of information compressibility, task reasoning ability, and transmission robustness. This allows for the acquisition of Pareto-optimal bottleneck variables that satisfy compression constraints under complex noise environments and high-dimensional constraints. This invention further proposes a computable surrogate upper bound method based on kernel ridge regression and mutual information constraints. It utilizes the high-dimensional distribution approximation capability of the conditional diffusion model and the high sample efficiency of multi-objective Bayesian optimization to uniformly model and optimize the robust information bottleneck problem. At the same time, through the gradient guidance mechanism based on entropy weight and the adaptive hybrid sampling mechanism combining the diffusion model and genetic algorithm, the model can dynamically balance the relationship between the compression target, the prediction target and the robust target in the high-dimensional non-convex representation space. This frees the traditional Lagrange relaxation method from the dependence on repeated hyperparameter adjustment and improves the stability, controllability and adaptability of the solution.

[0011] The specific technical solution for achieving the objective of this invention is as follows:

[0012] A robust information bottleneck optimization method based on diffusion model and Bayesian optimization, comprising the following specific steps:

[0013] Step 1: Construct a robust information bottleneck variable space

[0014] For input data With target variable respectively through encoding mapping functions Feature extraction is performed to obtain the bottleneck variable. Feature representation corresponding to the dimension; initialization of bottleneck variables Using the value space of the bottleneck variable as the optimization variable space, the Latin hypercube sampling (LHS) method is used to generate an initial candidate bottleneck variable set. By uniformly sampling the high-dimensional search space in layers, the initial candidate bottleneck variable set is made to cover the optimization variable space; the initial candidate bottleneck variable set... Introducing channel noise disturbance The initial candidate perturbation bottleneck variable set is obtained. Construct a joint optimization objective for information compression, task prediction, and transmission robustness; and adjust the initial candidate perturbation bottleneck variables. The set serves as the initial population for subsequent optimization processes;

[0015] Step 2: Construct a computable agent upper bound with mutual information constraints

[0016] A variational upper bound estimation method based on kernel ridge regression (KRR) is introduced to perform computable surrogate estimation of mutual information constraints; this is achieved by analyzing the input data. Bottleneck variables after perturbation Kernel regression modeling of the relationship is used to construct mutual information. The upper bound of the proxy is determined, and the information compression constraint is calculated and evaluated using the upper bound of the proxy.

[0017] Step 3: Constructing a multi-objective function and surrogate model

[0018] The bottleneck variable after the disturbance As functional variables, a multi-objective optimization function for information compression, task prediction, and transmission robustness is constructed. A multi-objective surrogate model is constructed using a Gaussian Process (GP) model, and a multi-objective Bayesian optimization strategy is used to evaluate candidate perturbation bottleneck variables. Through a data acquisition function, a fitness function is used to select, at a given number of evaluations, a set of perturbation bottleneck variables that satisfy the joint optimization objectives of information compression, task prediction, and transmission robustness for updating the surrogate model and generating new candidate solutions. ;

[0019] Step 4: Construct a conditional diffusion generation model, and gradually diffuse the model towards the perturbed bottleneck variable through a forward diffusion process. By adding Gaussian noise, a neural network is used to learn the probability distribution during the diffusion process, and new candidate perturbation bottleneck variables are recovered from the noise during the backdiffusion process. The new candidate perturbation bottleneck variable Constructing a new set of bottleneck variables after candidate perturbation ;

[0020] Step 5: In the reverse generation process of the conditional diffusion generation model, gradient guidance based on multi-objective entropy is introduced. By calculating the entropy weight coefficients corresponding to the joint optimization objectives of information compression, task prediction, and transmission robustness, a weighted gradient guidance term is constructed to guide the diffusion model; thus enabling the new candidate perturbation bottleneck variable set generated through diffusion sampling to be used. New candidate bottleneck variables after perturbation Guided by the weighted gradient guidance term, the algorithm iteratively updates along a direction that satisfies information compression constraints while also considering the robustness of task prediction and transmission.

[0021] Step 6: Design an adaptive hybrid sampling and Pareto optimal solution selection strategy

[0022] By calculating the hypervolum index for the new candidate perturbation set of bottleneck variables Monitoring is conducted, and the generation method of candidate perturbation bottleneck variables is switched according to the supervolume growth rate. When the supervolume growth rate is higher than a preset threshold, a conditional diffusion generation model is used to generate new candidate perturbation bottleneck variables. When the supervolume growth rate is lower than the preset threshold, a genetic algorithm (GA) is introduced to mutate and recombine the new candidate perturbation bottleneck variables. New candidate perturbation bottleneck variables that satisfy information compression constraints are then... After performing non-dominated sorting, the Pareto-optimal set of bottleneck variables that satisfies information compression constraints, task prediction performance, and transmission robustness is obtained. ,right The Pareto optimal bottleneck variables are decoded to obtain the final robust information bottleneck variable set. .

[0023] Furthermore, the initial candidate bottleneck variable set mentioned in step 1 It is represented as: ,in, Indicates the initial number of samples. Indicates the first One initial candidate bottleneck variable; for the initial candidate bottleneck variable set Introducing channel noise disturbance The initial candidate perturbation bottleneck variable set is obtained. , represented as , ,in, Indicates the first Bottleneck variables after initial candidate perturbations Indicates corresponding to the first Channel noise perturbation of the initial candidate bottleneck variables This indicates that the mean is 0 and the covariance matrix is... Gaussian distribution, Indicates the noise intensity parameter. Represents the identity matrix;

[0024] The joint optimization objective for information compression, task prediction, and transmission robustness is expressed as follows:

[0025]

[0026] in, Indicates input data, Represents the target variable. Represents the bottleneck variable. Indicates the bottleneck variable after the disturbance; It represents the mutual information between the bottleneck variable and the target variable after perturbation, and is used to measure task prediction; It represents the mutual information between the bottleneck variable and the perturbed bottleneck variable, and is used to measure transmission robustness; It represents the mutual information between the input data and the perturbed bottleneck variable, and is used to measure information compression; This represents the preset information compression constraint threshold.

[0027] Furthermore, the variational upper bound estimation method based on kernel ridge regression described in step 2 performs a computable surrogate estimation of the mutual information constraint, and its calculation process includes:

[0028] 1) For a given sample pair ,in Representing the sample size, the conditional distribution is analyzed using a kernel ridge regression model. Approximate modeling is performed, where, Indicates input data, Represents the bottleneck variable after perturbation; calculated by the kernel matrix function. This maps the input data to a high-dimensional feature space, where... and Indicates the first in the input data The and the first One sample, Indicates the kernel bandwidth parameter;

[0029] 2) Estimate the conditional mean function using the kernel ridge regression model: ,in, Indicates input data The corresponding conditional mean estimate of the bottleneck variable after perturbation. Represents the kernel matrix, kernel matrix The Line 1 Column elements are determined by the kernel matrix function. Calculations show that Represents the regularization parameter. Represents the identity matrix. Let represent the sample matrix of the bottleneck variables after perturbation, and , wherein Indicates the first A sample of bottleneck variables after perturbation, the Indicates the number of samples;

[0030] 3) Calculate the regression residuals using the leave-one-out method: And construct a residual with one left:

[0031] ,

[0032] in, Indicates the first The regression residuals corresponding to each sample Indicates the first A sample of bottleneck variables after perturbation Indicates the removal of the first After processing a sample, the kernel ridge regression model is used to process the input sample. The estimated value obtained at that location, Indicates the first The one-left residual of each sample Represents a smoothing matrix The diagonal elements, Represents a smoothing matrix The The diagonal elements; based on the left-one residual estimation, the mean and variance of the conditional distribution are obtained, the first... The mean of the conditional distribution corresponding to each sample is defined as: , wherein, the Indicates input sample The corresponding conditional distribution mean of the bottleneck variable after perturbation; the initial estimation matrix of the conditional covariance is expressed as:

[0033] ,

[0034] in, This represents the initial estimate matrix of conditional covariance. Indicates the effective degrees of freedom. Represents the matrix trace operation;

[0035] 4) Perform covariance shrinkage regularization on the initial conditional covariance estimation matrix to obtain the shrunken conditional covariance matrix: ,in, , This represents the conditional covariance matrix after shrinkage. Indicates the shrinkage coefficient. This represents the average variance level corresponding to the isotropic regularization term. express 3D identity matrix Dimensions representing bottleneck variables;

[0036] 5) Bottleneck variables after perturbation Reference edge distribution The model is based on a Gaussian distribution, and its mean is expressed as: The covariance matrix of the corresponding reference edge distribution is expressed as: ,in, This represents the mean of the bottleneck variable sample after the perturbation. This represents the covariance matrix of the reference marginal distribution. This represents the initial estimate of the sample covariance of the bottleneck variable after perturbation. This represents the shrinkage coefficient of the reference marginal distribution covariance. This represents the corresponding mean variance level; under the Gaussian approximation, the KL divergence between the conditional distribution and the reference marginal distribution is written as: By averaging the KL divergences of all samples, an upper bound estimator for mutual information surrogate is constructed:

[0037] ,

[0038] in, This represents the upper bound estimator for the mutual information agent. Indicates the first Conditional mean estimate for each sample The mean is The covariance matrix is Gaussian distribution, The mean is The covariance matrix is The Gaussian distribution; finally, using Replacement for mutual information that is difficult to calculate directly Constraints on information compression Conduct agent evaluation and screening.

[0039] Furthermore, the multi-objective optimization function described in step 3 is expressed as: ,in, , , ;in, Represents optimizing variable space One of the candidate bottleneck variables after perturbation. Represents the bottleneck variable after the disturbance Optimize the variable space; This represents a joint objective function consisting of three objective functions; This represents the objective function corresponding to task prediction. This represents the objective function corresponding to information compression. This represents the objective function corresponding to transmission robustness; This represents the mutual information between the target variable and the perturbed bottleneck variable. This represents the mutual information between the input data and the perturbed bottleneck variable. This represents the mutual information between the bottleneck variable and the perturbated bottleneck variable. This represents the preset information compression constraint threshold;

[0040] The multi-objective surrogate model is constructed using a multi-objective Gaussian process (GP) model, where a Gaussian process surrogate model is established for each objective function. The proxy representation of the objective function is as follows:

[0041]

[0042] in, Indicates the first The mean function of the objective function, Indicates the first The covariance function of each objective function. and Represents optimizing variable space Candidate bottleneck variables after perturbation; based on the observed sample set, the first Bottleneck variables for each objective function after candidate perturbation The predicted mean and predicted variance at each point are expressed as follows:

[0043]

[0044]

[0045] in, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted mean at that location, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted variance at that location Represents the input set of observed samples. Indicates the first The observation vector corresponding to each objective function Indicated by the covariance function Constructed kernel matrix, Indicates the first The observation noise variance corresponding to each objective function Represents the identity matrix;

[0046] The multi-objective Bayesian optimization strategy, based on the multi-objective Gaussian process model, predicts the mean and variance of the candidate perturbation bottleneck variables on each objective function, and combines this with the acquisition function to select the candidate perturbation bottleneck variables to be evaluated subsequently.

[0047] The fitness function used in the acquisition function is expressed as:

[0048]

[0049] in, Represents the bottleneck variable after candidate perturbation The fitness function value, Indicates the number of evaluation rounds. The set of bottleneck variables after candidate perturbation of the wheel, and Represents the set of bottleneck variables after candidate perturbation The bottleneck variable after candidate perturbation. Indicates the number of objective functions. Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function, Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function.

[0050] Furthermore, step 4 specifically includes the following steps:

[0051] 5-1: Establishing the forward diffusion process

[0052] The set of perturbed bottleneck variables obtained by filtering using the acquisition function, which satisfies the joint optimization objectives of information compression, task prediction, and transmission robustness. Training the diffusion model is performed, where... This represents the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Indicates the first A perturbated bottleneck variable that satisfies the joint optimization objectives of information compression, task prediction, and transmission robustness. This represents the number of perturbed bottleneck variables that satisfy the joint optimization objectives of information compression, task prediction, and transmission robustness; at each time step of the diffusion process. The perturbation of the bottleneck variable to achieve the joint optimization objectives of information compression, task prediction, and transmission robustness. By gradually adding Gaussian noise, it gradually evolves into a random noise distribution. The forward diffusion process can be represented as follows: ,in, The perturbated bottleneck variable represents the value at the 1st moment that satisfies the joint optimization objective of information compression, task prediction, and transmission robustness. The variable states corresponding to each time step This indicates the state of the variable corresponding to the previous time step. Indicates the first Noise scheduling parameters for each time step Represents random Gaussian noise. Indicates the diffusion time step;

[0053] 5-2: Establishing the reverse diffusion process

[0054] During the diffusion model training phase, the neural network is used to train each time step in the forward diffusion process. The added noise is predicted, and the mean squared error (MSE) loss function is minimized to enable the model to accurately predict the noise added during the diffusion process. The loss function is expressed as: ,in, This represents the mean squared error loss function. Denotes the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Time step and random Gaussian noise Expectation operation, Indicates the neural network in the first... The predicted value of noise at each time step. The parameters of the diffusion model are represented; in the backdiffusion process, the bottleneck variable is gradually recovered from the random noise after a new candidate perturbation, and its back generation process is represented as: ,in, Indicates the first Bottleneck variables after perturbation at each time step Indicates the first The candidate perturbation bottleneck variables generated at each time step Indicates the first The diffusion coefficient corresponding to each time step The cumulative term representing the diffusion coefficient, Indicates the first Noise intensity coefficient at each time step Represents the weighted gradient guidance term; when the back diffusion process is caused by the first... Iterate to the time step . After one time step, new candidate perturbation bottleneck variables are obtained. The new candidate perturbation bottleneck variable Constructing a new set of bottleneck variables after candidate perturbation .

[0055] Furthermore, the gradient guidance based on multi-objective entropy described in step 5 constructs a weighted gradient to guide the diffusion model by calculating the entropy weight coefficients corresponding to the jointly optimized objectives of information compression, task prediction, and transmission robustness; specifically, it includes the following steps:

[0056] 6-1: Calculate the information entropy of each objective function

[0057] The information entropy of each objective function is calculated based on the multi-objective optimization function. This entropy characterizes the uncertainty of each objective function on the bottleneck variable set after the current candidate perturbation. The entropy value is expressed as: ,in, Indicates the first The entropy value of the objective function. Indicates the first The objective function at the th ... Normalized distribution values ​​on the bottleneck variable after candidate perturbations Indicates the objective function number. Indicates the bottleneck variable number after candidate perturbation;

[0058] 6-2: Calculate the entropy weight coefficients of each objective function.

[0059] The corresponding weight coefficients are calculated based on the entropy values ​​of each objective function, and these weight coefficients are expressed as follows: ,in, Indicates the first The weight coefficients of each objective function. Indicates the first The entropy value of the objective function. Indicates the number of objective functions;

[0060] 6-3: Constructing a weighted gradient and introducing a backdiffusion process

[0061] The weighted gradient is constructed based on the weight coefficients, and its weighted gradient guidance term is expressed as follows: ,in, This represents the weighted gradient guidance term. Indicates the first The objective function at the th ... The candidate perturbation bottleneck variables corresponding to each time step gradient at, Indicates the first The weight coefficients of each objective function. The number of objective functions is represented; the weighted gradient guidance term is introduced into the back diffusion process of the diffusion model to adjust the generation direction of the candidate perturbation bottleneck variable, so that the generated candidate perturbation bottleneck variable is updated along the region that satisfies the information compression constraint and takes into account the robustness of task prediction and transmission.

[0062] Furthermore, step 6 specifically includes the following steps:

[0063] 7-1: Calculation of excess volume index

[0064] By calculating the hypervolume index of the Pareto front, the new candidate perturbation bottleneck variable set generated by the reverse generation process is applied. The overvolume index is expressed as follows: ,in, Representing the solution set The excess volume index, Representing the solution set One of the candidate bottleneck variables after perturbation. Representing the solution set One of the candidate bottleneck variables after perturbation The corresponding objective function value vector, Indicates a reference point. This indicates the volume calculation in the target space;

[0065] 7-2: Conditional Diffusion Generation Model Based on Supervolume Growth Rate Switching

[0066] The supervolume growth rate is calculated based on the supervolume index of adjacent iterations, and the generation method of the bottleneck variable after candidate perturbation is determined based on the supervolume growth rate; when the supervolume growth rate is higher than a preset threshold... At that time, a conditional diffusion generation model is used to generate new candidate bottleneck variables after perturbation, where, The preset threshold representing the super-volume growth rate;

[0067] 7-3. Switching Genetic Algorithms Based on Hypervolume Growth Rate

[0068] When the supervolume growth rate is lower than a preset threshold At that time, a genetic algorithm is introduced to perform a global search for candidate perturbation bottleneck variables, and new candidate perturbation bottleneck variables are generated through crossover and mutation operations;

[0069] 7-4. Selecting the Pareto-optimal set of bottleneck variables

[0070] For candidate perturbation bottleneck variables that satisfy information compression constraints Performing non-dominated sorting includes:

[0071] (1) Calculate the bottleneck variable after each candidate perturbation The corresponding objective function value vector ;

[0072] (2) Bottleneck variables after any two candidate perturbations and Perform pairwise comparisons; if the conditions are met...

[0073]

[0074] in, Indicates the first One objective function, Indicates the number of objective functions; if at least one objective function exists... , making Then it is called Dominate ;

[0075] (3) The candidate perturbation bottleneck variables that are not dominated by other candidate perturbation bottleneck variables are divided into the first non-dominated layer and used as the candidate set corresponding to the current Pareto front. After removing the first non-dominated layer, the above comparison process is repeated for the remaining candidate perturbation bottleneck variables to obtain the subsequent non-dominated layer.

[0076] (4) The set of candidate perturbed bottleneck variables that satisfy the information compression constraint in the first non-dominated layer is taken as the Pareto optimal bottleneck variable set, denoted as ,in, This represents the Pareto optimal set of bottleneck variables that satisfies information compression constraints, task prediction performance, and transmission robustness; finally, for The Pareto optimal bottleneck variables are decoded to obtain the final robust information bottleneck variable set. .

[0077] Compared with the prior art, the present invention has the following advantages:

[0078] 1) This invention transforms the robust information bottleneck problem that traditionally relies on Lagrange multipliers and single-objective relaxation into a multi-objective optimization problem under explicit constraints, thus avoiding the problems of difficult adjustment of Lagrange hyperparameters and difficulty in strictly satisfying constraints;

[0079] 2) This invention proposes a mutual information proxy upper bound estimation method based on kernel ridge regression, which enables the mutual information compression constraint to have a computable, low variance and strong stability implementation path;

[0080] 3) This invention introduces the conditional diffusion model into robust information bottleneck optimization, which can leverage the advantages of the diffusion model in modeling high-dimensional complex distributions to generate high-quality candidate representations;

[0081] 4) This invention significantly improves the globality, stability and convergence efficiency of the search process by using an entropy-weighted gradient guidance and an adaptive hybrid sampling mechanism based on a hypervolume-based diffusion model and genetic algorithm.

[0082] 5) This invention can directly output a set of Pareto optimal representation solutions that satisfy compression constraints, which makes it easier for the system to flexibly select the optimal feature encoding and transmission strategy under different noise, different time delay and different task accuracy requirements.

[0083] In summary, this invention overcomes the dependence of traditional robust information bottleneck methods on Lagrange relaxation and explicit gradient calculation paths, and constructs a unified generative optimization framework composed of "mutual information proxy upper bound, multi-objective Bayesian optimization, conditional diffusion generation, entropy-weighted gradient guidance, and adaptive hybrid sampling." This framework can balance information compressibility, task inference capability, and representation robustness under complex noisy environments and high-dimensional constraints, exhibiting high optimization efficiency, stability, and application adaptability. It can be widely applied to scenarios such as satellite edge inference, intelligent communication, distributed learning, and decision optimization in complex systems. Attached Figure Description

[0084] Figure 1 This is a schematic diagram of the robust information bottleneck variable space provided in Embodiment 1 of the present invention;

[0085] Figure 2 This is a schematic diagram of the method flow framework provided in Embodiment 1 of the present invention. Detailed Implementation

[0086] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Obviously, the listed embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0087] Example 1

[0088] The robust information bottleneck optimization method based on diffusion model and Bayesian optimization described in this invention includes the following steps:

[0089] S1. Construct a robust information bottleneck variable space:

[0090] This embodiment first processes the input data. With target variable respectively through encoding mapping functions Feature extraction is performed to obtain the bottleneck variable. Feature representation corresponding to the dimension; initialization of bottleneck variables Using the value space of the bottleneck variable as the optimization variable space, the Latin hypercube sampling method is used to generate an initial candidate bottleneck variable set. It is represented as: ,in, Indicates the initial number of samples. Indicates the first Identify initial candidate bottleneck variables; perform uniform hierarchical sampling on the high-dimensional search space to ensure that the initial candidate bottleneck variable set covers the optimization variable space; and then apply this initial candidate bottleneck variable set to... Each initial candidate bottleneck variable introduces channel noise perturbation , is represented as: ,in, Indicates the first Bottleneck variables after initial candidate perturbations Indicates the corresponding to the first Channel noise perturbation of the initial candidate bottleneck variables This indicates that the mean is 0 and the covariance matrix is... Gaussian distribution, Indicates the noise intensity parameter. Representing the identity matrix, we obtain the set of bottleneck variables after the initial candidate perturbation. The joint optimization objective for information compression, task prediction, and transmission robustness is constructed and is described as follows:

[0091]

[0092] in, Indicates input data, Represents the target variable. Represents the bottleneck variable. This represents the bottleneck variable after the perturbation. It represents the mutual information between the bottleneck variable and the target variable after perturbation, and is used to measure task prediction; This represents the mutual information between the bottleneck variable and the perturbed bottleneck variable, used to measure transmission robustness. This represents the mutual information between the input data and the perturbed bottleneck variable, used to measure information compression. This represents a preset information compression constraint threshold; the initial candidate perturbation bottleneck variable... The set serves as the initial population for subsequent optimization processes.

[0093] See Figure 1 The figure illustrates the encoding, transmission, and decoding relationship of the robust information bottleneck variable in this embodiment. In this step, this embodiment takes information compression, task prediction performance, and representation robustness as the three core objectives for subsequent optimization, and takes the space where the perturbed bottleneck variable is located as the optimization search space, providing a unified variable representation for subsequent initial sampling, mutual information constraint surrogate estimation, multi-objective surrogate modeling, and diffusion generation optimization.

[0094] S2. Construct a computable upper bound for the agent with mutual information constraints:

[0095] To constrain information compression To perform computable surrogate estimation, this embodiment introduces a variational upper bound estimation method based on kernel ridge regression. First, for any reference margin distribution... ,have:

[0096] .

[0097] because Furthermore, we obtain:

[0098] .

[0099] in, Indicates input data, This represents the bottleneck variable after the perturbation. Indicates that given input data Conditional distribution of bottleneck variables after perturbation under given conditions. This represents the marginal distribution of the bottleneck variable after the perturbation. Indicates the reference marginal distribution. Indicates the Kullback-Leibler divergence. Represents the distribution of input data The expectation operation, therefore, as long as the right-hand side can be stably estimated, the expectation can be obtained. The computable upper bound of the agent is determined through the following process:

[0100] S21. For a given sample pair ,in Representing the sample size, the conditional distribution is analyzed using a kernel ridge regression model. Approximate modeling is performed, where, Indicates input data, Represents the bottleneck variable after perturbation; calculated by the kernel matrix function. This maps the input data to a high-dimensional feature space, where... and Indicates the first in the input data The and the first One sample, Indicates the kernel bandwidth parameter;

[0101] S22. Estimate the conditional mean function using the kernel ridge regression model: ,in, Indicates input data The corresponding conditional mean estimate of the bottleneck variable after perturbation. Represents the kernel matrix, kernel matrix The Line number Column elements are determined by the kernel matrix function. Calculations show that Represents the regularization parameter. Represents the identity matrix. Let represent the sample matrix of the bottleneck variables after perturbation, and , wherein Indicates the first A sample of bottleneck variables after perturbation, the Indicates the number of samples;

[0102] S23. Calculate the regression residuals using the leave-one-out method: And construct a residual with one left:

[0103] ,

[0104] in, Indicates the first The regression residuals corresponding to each sample Indicates the first A sample of bottleneck variables after perturbation Indicates the removal of the first After processing a sample, the kernel ridge regression model is used to process the input sample. The estimated value obtained at that location, Indicates the first The one-left residual of each sample Represents a smoothing matrix The diagonal elements, Represents a smoothing matrix The The diagonal elements; based on the left-one residual estimation, the mean and variance of the conditional distribution are obtained, the first... The mean of the conditional distribution corresponding to each sample is defined as: , wherein, the Indicates input sample The corresponding conditional distribution mean of the bottleneck variable after perturbation; the initial estimation matrix of the conditional covariance is expressed as:

[0105] ,

[0106] in, This represents the initial estimate matrix of conditional covariance. Indicates the effective degrees of freedom. Represents the matrix trace operation;

[0107] S24. Perform covariance shrinkage regularization on the initial conditional covariance estimation matrix to obtain the shrunken conditional covariance matrix: ,in, , This represents the conditional covariance matrix after shrinkage. Indicates the shrinkage coefficient. This represents the average variance level corresponding to the isotropic regularization term. express 3D identity matrix Dimensions representing bottleneck variables;

[0108] S25. Bottleneck variables after perturbation Reference edge distribution The model is based on a Gaussian distribution, and its mean is expressed as: The covariance matrix of the corresponding reference edge distribution is expressed as: ,in, This represents the mean of the bottleneck variable sample after the perturbation. This represents the covariance matrix of the reference marginal distribution. This represents the initial estimate of the sample covariance of the bottleneck variable after perturbation. This represents the shrinkage coefficient of the reference marginal distribution covariance. This represents the corresponding mean variance level; under the Gaussian approximation, the KL divergence between the conditional distribution and the reference marginal distribution is written as: By averaging the KL divergences of all samples, an upper bound estimator for mutual information surrogate is constructed:

[0109] ,

[0110] in, This represents the upper bound estimator for the mutual information agent. Indicates the first Conditional mean estimate for each sample The mean is The covariance matrix is Gaussian distribution, The mean is The covariance matrix is The Gaussian distribution; finally, using Replacement for mutual information that is difficult to calculate directly Constraints on information compression Conduct agent evaluation and screening.

[0111] S3. Construction of Multi-Objective Functions and Proxy Models:

[0112] Based on the mutual information proxy upper bound estimate constructed in step S2, this embodiment will perturb the bottleneck variable. As a function variable, a multi-objective optimization function for information compression, task prediction, and transmission robustness is constructed, which can be expressed as:

[0113] ,

[0114] in, , , ;in, Indicates optimization of variable space One of the candidate bottleneck variables after perturbation. Represents the bottleneck variable after the disturbance Optimize the variable space; This represents a joint objective function consisting of three objective functions; This represents the objective function corresponding to task prediction. This represents the objective function corresponding to information compression. This represents the objective function corresponding to transmission robustness; This represents the mutual information between the target variable and the perturbed bottleneck variable. This represents the mutual information between the input data and the perturbed bottleneck variable. This represents the mutual information between the bottleneck variable and the perturbated bottleneck variable. This represents the preset information compression constraint threshold;

[0115] To improve the efficiency of black-box optimization with a limited number of evaluations, this embodiment further employs a multi-objective Gaussian process model as a multi-objective surrogate model to perform nonparametric estimation of the three objective function values. Specifically, a Gaussian process surrogate model is established for each objective function. The proxy representation of the objective function is as follows:

[0116]

[0117] in, Indicates the first The mean function of a set of objective functions. Indicates the first The covariance function of each objective function. and Indicates optimization of variable space Candidate bottleneck variables after perturbation; based on the observed sample set, the first Bottleneck variables for each objective function after candidate perturbation The predicted mean and predicted variance at each point are expressed as follows:

[0118]

[0119]

[0120] in, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted mean at that location, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted variance at that location Represents the input set of observed samples. Indicates the first The observation vector corresponding to each objective function Indicated by the covariance function Constructed kernel matrix, Indicates the first The observation noise variance corresponding to each objective function Represents the identity matrix.

[0121] By training a Gaussian process model with candidate perturbation bottleneck variables that satisfy compression constraints and their target values, the local probability distribution information of the objective function in the search space is obtained. The probability distribution information is then used to predict the potential superiority or inferiority of unevaluated candidate perturbation bottleneck variables, thereby reducing the computational overhead of directly evaluating the true objective function.

[0122] After each round of sampling and evaluation, a multi-objective Bayesian optimization strategy is used to evaluate the candidate perturbation bottleneck variables. Specifically, based on the multi-objective Gaussian process model, the predicted mean and variance of the candidate perturbation bottleneck variables on each objective function are calculated, and combined with the acquisition function, to select the candidate perturbation bottleneck variables to be evaluated subsequently. During the candidate solution screening process, a density estimation method based on offsets is used to evaluate the fitness of the solution set, and its fitness function can be expressed as:

[0123]

[0124] in, Represents the bottleneck variable after candidate perturbation The fitness function value, Indicates the number of evaluation rounds. The set of bottleneck variables after candidate perturbation of the wheel, and Represents the set of bottleneck variables after candidate perturbation The bottleneck variable after candidate perturbation. Indicates the number of objective functions. Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function, Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function.

[0125] Furthermore, in each round of optimization, this invention prioritizes retaining those that satisfy the compression constraints. The candidate solutions are selected, and the top-scoring solutions are chosen from them. The perturbated bottleneck variables that satisfy the joint optimization objectives of information compression, task prediction, and transmission robustness are obtained as a set. As the primary data source for updating the surrogate model and generating new candidate solutions, among which, This indicates the number of high-quality samples to be set.

[0126] S4. Generate a new set of candidate perturbation-adjusted bottleneck variables based on the conditional diffusion model:

[0127] Based on the multi-objective algebraic function and surrogate model constructed in step S3, this embodiment introduces a conditional diffusion generative model as a generative optimizer to efficiently search for feasible solutions in a high-dimensional, non-convex, and implicitly constrained representation space. Unlike traditional methods based on gradient descent or heuristic enumeration, the conditional diffusion model learns the distribution structure of high-quality candidate representations through a "step-by-step noise addition-step-by-step noise reduction" approach, thereby recovering and generating new perturbed bottleneck variables that satisfy multi-objective requirements from random noise. .

[0128] S41. Establish the forward diffusion process

[0129] The set of perturbed bottleneck variables obtained by using the acquisition function to meet the joint optimization objectives of information compression, task prediction, and transmission robustness. Training the diffusion model is performed, where... This represents the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Indicates the first A perturbated bottleneck variable that satisfies the joint optimization objectives of information compression, task prediction, and transmission robustness. This represents the number of bottleneck variables after perturbation that satisfy the joint optimization objectives of information compression, task prediction, and transmission robustness.

[0130] At each time step of the diffusion process The perturbation of the bottleneck variable to achieve the joint optimization objectives of information compression, task prediction, and transmission robustness. By gradually adding Gaussian noise, it gradually evolves into a random noise distribution. The forward diffusion process can be represented as follows: ,in, The perturbated bottleneck variable represents the value at the 1st moment that satisfies the joint optimization objective of information compression, task prediction, and transmission robustness. The variable states corresponding to each time step This indicates the state of the variable corresponding to the previous time step. Indicates the first Noise scheduling parameters for each time step Represents random Gaussian noise. This represents the diffusion time step. Through the above noise-adding process, a path can be constructed in the latent space that smoothly transitions from a high-quality candidate solution distribution to a simple Gaussian distribution. This facilitates the neural network in learning the implicit distribution pattern of candidate solutions. The high-quality solutions are then fed into the diffusion model, and generative optimization is performed through a noise-adding and noise-reducing loop.

[0131] S42. Establish the reverse diffusion process

[0132] During the diffusion model training phase, a neural network is used to predict the Gaussian noise injected at each diffusion time step, and the parameters are updated by minimizing the mean square error loss function. The training objective can be expressed as:

[0133] .

[0134] in, This represents the mean squared error loss function. Denotes the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Time step and random Gaussian noise Expectation operation, Indicates the neural network in the first... The predicted value of noise at each time step. This represents the parameters of the diffusion model. Through the above training process, the diffusion model can gradually grasp the evolution law of the candidate bottleneck variable distribution under different noise levels, and provide reliable noise removal capabilities for the subsequent reverse generation process.

[0135] In the backdiffusion phase, starting from samples initialized with standard Gaussian noise, the trained noise prediction network iteratively performs denoising updates, thereby gradually recovering new candidate perturbation bottleneck variables. The backdiffusion process is represented as follows:

[0136] .

[0137] in, Indicates the first Bottleneck variables after perturbation at each time step Indicates the first The candidate perturbation bottleneck variables generated at each time step Indicates the first The diffusion coefficient corresponding to each time step The cumulative term representing the diffusion coefficient, Indicates the first Noise intensity coefficient at each time step Represents the weighted gradient guidance term; when the back diffusion process is caused by the first... Iterate to the time step . After one time step, new candidate perturbation bottleneck variables are obtained. The new candidate perturbation bottleneck variable Constructing a new set of bottleneck variables after candidate perturbation .

[0138] S5. Diffusion-guided mechanism based on entropy-weighted gradient:

[0139] To enable the conditional diffusion model in step S4 to more accurately search for feasible regions that meet the multi-objective requirements during the reverse denoising generation process, this embodiment further introduces gradient guidance based on multi-objective entropy. Addressing the conflict between the information compression objective, task prediction objective, and transmission robustness objective, instead of using fixed weights for a simple linear combination of multiple objectives, the embodiment dynamically calculates the objective weights based on the uncertainty and discriminative ability of each objective at the current iteration stage, and introduces these weights into the reverse diffusion process, thereby achieving adaptive adjustment of the generation direction of candidate bottleneck variables.

[0140] Specifically, in each round of backdiffusion sampling, the information entropy of each objective is first calculated based on the distribution of the current candidate solution set on the multi-objective function. This entropy characterizes the dispersion and uncertainty of the objective in the current sample set. Its entropy value can be expressed as:

[0141] .

[0142] in, Indicates the first The entropy value corresponding to each objective function Indicates the first The first goal in The normalized probability distribution values ​​over candidate samples. The smaller the entropy value of a target, the stronger its certainty and informativeness at the current stage, and the greater its guiding role should be given in subsequent generation processes. After obtaining the entropy values ​​of each target, the corresponding entropy weight coefficients are further constructed:

[0143] .

[0144] in, Indicates the first Dynamic weights of each objective function This represents the total number of objective functions. This weighting method allows objectives with lower uncertainty and stronger discriminative power to occupy a higher proportion in the current round, without needing to manually set fixed objective weight coefficients beforehand, thus reducing the instability caused by empirical parameter tuning in traditional multi-objective optimization. Furthermore, the entropy weight coefficients are combined with the gradient direction of each objective function at the current sampling point to construct a weighted gradient guidance term:

[0145] .

[0146] in, This represents the weighted gradient guidance term. Indicates the first The objective function at the th ... The candidate perturbation bottleneck variables corresponding to each time step gradient at, Indicates the first The weight coefficients of each objective function. This represents the number of objective functions. During the backdiffusion update process in step S4, this weighted gradient term is embedded as conditional guiding information in the denoising process. This allows candidate samples to gradually migrate towards regions with better compression performance, higher task relevance, and stronger robustness while being gradually denoised and recovered. This mechanism can impart greater sampling influence to low-entropy, high-determinism objectives and improve the convergence stability, solution diversity, and consistency with the multi-objective Pareto front of the backdiffusion process.

[0147] S6. Adaptive hybrid sampling and Pareto optimal solution selection:

[0148] To further avoid a single generation strategy getting trapped in local optima in high-dimensional non-convex spaces, this embodiment proposes an adaptive hybrid sampling and Pareto optimal solution selection method based on hypervolume indices. This method dynamically selects either "conditional diffusion model sampling" or "genetic algorithm global search" as the candidate solution generation strategy by real-time monitoring of the Pareto front coverage quality of the current candidate solution set, thus balancing the ability to generate high-quality samples with the ability to escape local optima globally. Specifically, let the non-dominated solution set obtained in the current iteration be... The reference point is Then its overvolume index can be expressed as:

[0149] ,

[0150] in, Representing the solution set The excess volume index, Representing the solution set One of the candidate bottleneck variables after perturbation. Representing the solution set One of the candidate bottleneck variables after perturbation The corresponding objective function value vector, Indicates a reference point. It represents the volume in the target space.

[0151] After each iteration, the supervolume growth rate is first calculated based on the supervolume change between the current and previous iterations; when the growth rate is greater than or equal to a preset threshold... When the growth rate is below a preset threshold, it indicates that the current optimization process is still in the stage of rapidly approaching the Pareto front. At this time, the conditional diffusion model is preferentially invoked to generate new candidate bottleneck variables, so as to make full use of the diffusion model's ability to learn and generate high-quality solution distributions and accelerate the continuous refinement search of high-quality regions. When the current search process may be trapped in a local extreme region, continuing to rely on the diffusion model to generate solutions locally will not significantly improve the quality of the solution set. Therefore, the algorithm is automatically switched to a genetic algorithm to perform global mutation and recombination search of candidate solutions. Through operations such as crossover, mutation and population evolution, the search range is expanded and the ability to escape local optimal regions is improved.

[0152] After obtaining a new batch of candidate solutions, this embodiment further performs batch evaluation and feasibility screening on all candidate samples, retaining feasible solutions that satisfy the compression constraints, and merging them with the historical candidate solution set. This merged set is used to incrementally update the multi-objective Gaussian process surrogate model and hypervolume index, thereby continuously improving the quality and coverage of the solution set. After all iterations are completed, a non-dominated sorting is performed on the candidate solution set, specifically including:

[0153] (1) Calculate the bottleneck variable after each candidate perturbation The corresponding objective function value vector ;

[0154] (2) Bottleneck variables after any two candidate perturbations and Perform pairwise comparisons; if the conditions are met...

[0155]

[0156] in, Indicates the first One objective function, Indicates the number of objective functions; if at least one objective function exists... , making Then it is called Dominate ;

[0157] (3) The candidate perturbation bottleneck variables that are not dominated by other candidate perturbation bottleneck variables are divided into the first non-dominated layer and used as the candidate set corresponding to the current Pareto front. After removing the first non-dominated layer, the above comparison process is repeated for the remaining candidate perturbation bottleneck variables to obtain the subsequent non-dominated layer.

[0158] (4) The set of candidate perturbed bottleneck variables that satisfy the information compression constraint in the first non-dominated layer is taken as the Pareto optimal bottleneck variable set, denoted as ,in, This represents the Pareto optimal set of bottleneck variables that satisfies information compression constraints, task prediction performance, and transmission robustness; finally, for The Pareto optimal bottleneck variables are decoded to obtain the final robust information bottleneck variable set. .

[0159] Since the final output is a set of Pareto optimal candidate solutions, rather than a single optimal point, this invention can provide various trade-offs between different compression ratios, robustness levels, and task accuracy for practical application systems. For example, when channel noise is high or communication resources are limited, a representation scheme with a higher compression ratio and stronger robustness can be preferentially selected; while in scenarios with higher task accuracy requirements, a representation scheme with better prediction performance can be selected from the Pareto solution set. Therefore, this invention has strong scenario adaptability and practical deployment flexibility.

[0160] Through the steps S1 to S7 described above, this invention constructs a unified generative optimization framework consisting of "robust information bottleneck variable space modeling, construction of computable surrogate upper bounds for mutual information constraints, construction of multi-objective functions and surrogate models, conditional diffusion generation, multi-objective entropy weight gradient guidance, and adaptive hybrid sampling and Pareto screening".

[0161] See Figure 2 This collection of figures illustrates the overall process of the method in this embodiment, from constructing the robust information bottleneck variable space, constructing the multi-objective optimization function, generating the perturbation-based information bottleneck variable, to finally filtering and decoding the Pareto optimal bottleneck variable to obtain the final robust information bottleneck variable.

[0162] This embodiment can be applied to task-oriented communication and edge inference scenarios with noisy channels, limited resources, or environmental disturbances, such as satellite edge inference, wireless sensing collaboration, distributed intelligent sensing, and robust decision optimization for complex systems. Compared with existing robust information bottleneck optimization methods based on Lagrange multipliers and fixed weight relaxation, this invention can more stably satisfy information compression constraints, reduce dependence on manual adjustment of hyperparameters, improve search efficiency in high-dimensional non-convex spaces, and directly output a set of Pareto optimal candidate solutions that can be flexibly selected under different application conditions. Therefore, it has better stability, interpretability, and practical application adaptability.

[0163] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robust information bottleneck optimization method based on diffusion model and Bayesian optimization, characterized in that, The method includes the following specific steps: Step 1: Construct a robust information bottleneck variable space For input data With target variable respectively through encoding mapping function Feature extraction is performed to obtain the bottleneck variable. Feature representation corresponding to the dimension; initialization of bottleneck variables Using the value space of the bottleneck variable as the optimization variable space, the Latin hypercube sampling method is used to generate an initial candidate bottleneck variable set. By uniformly sampling the high-dimensional search space in layers, the initial candidate bottleneck variable set is made to cover the optimization variable space; the initial candidate bottleneck variable set... Introducing channel noise disturbance The initial candidate perturbation bottleneck variable set is obtained. Construct a joint optimization objective for information compression, task prediction, and transmission robustness; and adjust the initial candidate perturbation bottleneck variables. The set serves as the initial population for subsequent optimization processes; Step 2: Construct a computable agent upper bound with mutual information constraints A variational upper bound estimation method based on kernel ridge regression is introduced to perform computable surrogate estimation of mutual information constraints; this is achieved by analyzing the input data. Bottleneck variables after perturbation Kernel regression modeling of the relationship is used to construct mutual information. The upper bound of the proxy is determined, and the information compression constraint is calculated and evaluated using the upper bound of the proxy. Step 3: Constructing a multi-objective function and surrogate model The bottleneck variable after the perturbation As a function variable, construct a multi-objective optimization function for information compression, task prediction, and transmission robustness; A multi-objective surrogate model is constructed using a Gaussian process model, and a multi-objective Bayesian optimization strategy is employed to evaluate candidate perturbation bottleneck variables. Using a fitness function and a sampling function, a set of perturbation bottleneck variables satisfying the joint optimization objectives of information compression, task prediction, and transmission robustness is selected for updating the surrogate model and generating new candidate solutions, given a given number of evaluations. ; Step 4: Construct a conditional diffusion generation model, and gradually diffuse the model towards the perturbed bottleneck variable through a forward diffusion process. By adding Gaussian noise, a neural network is used to learn the probability distribution during the diffusion process, and new candidate perturbation bottleneck variables are recovered from the noise during the backdiffusion process. The new candidate perturbation bottleneck variable Constructing a new set of bottleneck variables after candidate perturbation ; Step 5: In the reverse generation process of the conditional diffusion generation model, gradient guidance based on multi-objective entropy is introduced. By calculating the entropy weight coefficients corresponding to the joint optimization objectives of information compression, task prediction, and transmission robustness, a weighted gradient guidance term is constructed to guide the diffusion model; thus enabling the new candidate perturbation bottleneck variable set generated through diffusion sampling to be used. New candidate bottleneck variables after perturbation Guided by the weighted gradient guidance term, the algorithm iteratively updates along a direction that satisfies information compression constraints while also considering the robustness of task prediction and transmission. Step 6: Design an adaptive hybrid sampling and Pareto optimal solution selection strategy By calculating the hypervolume index, the new candidate perturbation-induced bottleneck variable set is... Monitoring is conducted, and the generation method of candidate perturbation bottleneck variables is switched according to the supervolume growth rate. When the supervolume growth rate is higher than a preset threshold, a conditional diffusion generation model is used to generate new candidate perturbation bottleneck variables. When the supervolume growth rate is lower than the preset threshold, a genetic algorithm (GA) is introduced to mutate and recombine the new candidate perturbation bottleneck variables. New candidate perturbation bottleneck variables that satisfy information compression constraints are then... After performing non-dominated sorting, the Pareto-optimal set of bottleneck variables that satisfies information compression constraints, task prediction performance, and transmission robustness is obtained. ,right The Pareto optimal bottleneck variables are decoded to obtain the final robust information bottleneck variable set. .

2. The robust information bottleneck optimization method according to claim 1, characterized in that, The initial candidate bottleneck variable set described in step 1 It is represented as: ,in, Indicates the initial number of samples. Indicates the first One initial candidate bottleneck variable; for the initial candidate bottleneck variable set Introducing channel noise disturbance The initial candidate perturbation bottleneck variable set is obtained. , represented as , ,in, Indicates the first Bottleneck variables after initial candidate perturbations Indicates the corresponding to the first Channel noise perturbation of the initial candidate bottleneck variables This indicates that the mean is 0 and the covariance matrix is... Gaussian distribution, Indicates the noise intensity parameter. Represents the identity matrix; The joint optimization objective for information compression, task prediction, and transmission robustness is expressed as follows: in, Indicates input data, Represents the target variable. Represents the bottleneck variable. Indicates the bottleneck variable after the disturbance; It represents the mutual information between the bottleneck variable and the target variable after perturbation, and is used to measure task prediction; It represents the mutual information between the bottleneck variable and the perturbed bottleneck variable, and is used to measure transmission robustness; It represents the mutual information between the input data and the perturbed bottleneck variable, and is used to measure information compression; This represents the preset information compression constraint threshold.

3. The robust information bottleneck optimization method according to claim 1, characterized in that, Step 2 describes a variational upper bound estimation method based on kernel ridge regression, which performs a computable surrogate estimation of mutual information constraints. The calculation process includes: 1) For a given sample pair , among which Representing the sample size, the conditional distribution is analyzed using a kernel ridge regression model. Approximate modeling is performed, where, Indicates input data, Represents the bottleneck variable after perturbation; calculated by the kernel matrix function. This maps the input data to a high-dimensional feature space, where... and Indicates the first in the input data The and the first One sample, Indicates the kernel bandwidth parameter; 2) Estimate the conditional mean function using the kernel ridge regression model: ,in, Indicates input data The corresponding conditional mean estimate of the bottleneck variable after perturbation. Represents the kernel matrix, kernel matrix The Line number Column elements are determined by the kernel matrix function. Calculations show that Represents the regularization parameter. Represents the identity matrix. Let represent the sample matrix of the bottleneck variables after perturbation, and , wherein Indicates the first A sample of bottleneck variables after perturbation, the Indicates the number of samples; 3) Calculate the regression residuals using the leave-one-out method: And construct a residual with one left: ,in, Indicates the first The regression residuals corresponding to each sample Indicates the first A sample of bottleneck variables after perturbation Indicates the removal of the first After processing a sample, the kernel ridge regression model is used to process the input sample. The estimated value obtained at that location, Indicates the first The one-left residual of each sample Represents a smoothing matrix The diagonal elements, Represents a smoothing matrix The The diagonal elements; based on the left-one residual estimation, the mean and variance of the conditional distribution are obtained, the first... The mean of the conditional distribution corresponding to each sample is defined as: , wherein, the Indicates input sample The corresponding conditional distribution mean of the bottleneck variable after perturbation; the initial estimation matrix of the conditional covariance is expressed as: ,in, This represents the initial estimate matrix of conditional covariance. Indicates the effective degrees of freedom. Represents the matrix trace operation; 4) Perform covariance shrinkage regularization on the initial conditional covariance estimation matrix to obtain the shrunken conditional covariance matrix: ,in, , This represents the conditional covariance matrix after shrinkage. Indicates the shrinkage coefficient. This represents the average variance level corresponding to the isotropic regularization term. express 3D identity matrix Dimensions representing bottleneck variables; 5) Bottleneck variables after perturbation Reference edge distribution The model is based on a Gaussian distribution, and its mean is expressed as: The covariance matrix of the corresponding reference edge distribution is expressed as: ,in, This represents the mean of the bottleneck variable sample after the perturbation. This represents the covariance matrix of the reference marginal distribution. This represents the initial estimate of the sample covariance of the bottleneck variable after perturbation. This represents the shrinkage coefficient of the reference marginal distribution covariance. This represents the corresponding mean variance level; under the Gaussian approximation, the KL divergence between the conditional distribution and the reference marginal distribution is written as: By averaging the KL divergences of all samples, an upper bound estimator for mutual information surrogate is constructed: ,in, This represents the upper bound estimator for the mutual information agent. Indicates the first Conditional mean estimate for each sample The mean is The covariance matrix is Gaussian distribution, The mean is The covariance matrix is The Gaussian distribution; finally, using Replacement for mutual information that is difficult to calculate directly Constraints on information compression Conduct agent evaluation and screening.

4. The robust information bottleneck optimization method according to claim 1, characterized in that, The multi-objective optimization function described in step 3 is expressed as: ,in, , , ;in, Indicates optimization of variable space One of the candidate bottleneck variables after perturbation. Represents the bottleneck variable after the disturbance Optimize the variable space; This represents a joint objective function consisting of three objective functions; This represents the objective function corresponding to task prediction. This represents the objective function corresponding to information compression. This represents the objective function corresponding to transmission robustness; This represents the mutual information between the target variable and the perturbed bottleneck variable. This represents the mutual information between the input data and the perturbed bottleneck variable. This represents the mutual information between the bottleneck variable and the perturbated bottleneck variable. This represents the preset information compression constraint threshold; The multi-objective surrogate model is constructed using a multi-objective Gaussian process model, wherein a Gaussian process surrogate model is established for each objective function, and the... The proxy representation of the objective function is as follows: in, Indicates the first The mean function of the objective function, Indicates the first The covariance function of each objective function. and Indicates optimization of variable space Candidate bottleneck variables after perturbation; based on the observed sample set, the first Bottleneck variables for each objective function after candidate perturbation The predicted mean and predicted variance at each point are expressed as follows: in, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted mean at that location, Indicates the first Bottleneck variables for each objective function after candidate perturbation The predicted variance at that location Represents the input set of observed samples. Indicates the first The observation vector corresponding to each objective function Indicated by the covariance function Constructed kernel matrix, Indicates the first The observation noise variance corresponding to each objective function Represents the identity matrix; The multi-objective Bayesian optimization strategy, based on the multi-objective Gaussian process model, predicts the mean and variance of the candidate perturbation bottleneck variables on each objective function, and combines this with the acquisition function to select the candidate perturbation bottleneck variables to be evaluated subsequently. The fitness function used in the acquisition function is expressed as: in, Represents the bottleneck variable after candidate perturbation The fitness function value, Indicates the number of evaluation rounds. The set of bottleneck variables after candidate perturbation of the wheel, and Represents the set of bottleneck variables after candidate perturbation The bottleneck variable after candidate perturbation. Indicates the number of objective functions. Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function, Represents the bottleneck variable after candidate perturbation In the The objective value on the objective function.

5. The robust information bottleneck optimization method according to claim 1, characterized in that, Step 4 specifically includes the following steps: 5-1: Establishing the forward diffusion process The set of perturbed bottleneck variables obtained by filtering using the acquisition function, which satisfies the joint optimization objectives of information compression, task prediction, and transmission robustness. Training the diffusion model is performed, where... This represents the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Indicates the first A perturbated bottleneck variable that satisfies the joint optimization objectives of information compression, task prediction, and transmission robustness. This represents the number of perturbed bottleneck variables that satisfy the joint optimization objectives of information compression, task prediction, and transmission robustness; at each time step of the diffusion process. The perturbation of the bottleneck variable to achieve the joint optimization objectives of information compression, task prediction, and transmission robustness. By gradually adding Gaussian noise, causing it to gradually evolve into a random noise distribution, the forward diffusion process can be represented as follows: ,in, The perturbated bottleneck variable represents the value at the 1st moment that satisfies the joint optimization objective of information compression, task prediction, and transmission robustness. The variable states corresponding to each time step This indicates the state of the variable corresponding to the previous time step. Indicates the first Noise scheduling parameters for each time step Represents random Gaussian noise. Indicates the diffusion time step; 5-2: Establishing the reverse diffusion process During the diffusion model training phase, a neural network is used to train each time step in the forward diffusion process. The added noise is predicted, and the mean squared error (MSE) loss function is minimized to enable the model to accurately predict the noise added during the diffusion process. The loss function is expressed as: ,in, This represents the mean squared error loss function. Denotes the perturbed set of bottleneck variables that satisfy the joint optimization objective of information compression, task prediction, and transmission robustness. Time step and random Gaussian noise Expectation operation, Indicates the neural network in the first... The predicted value of noise at each time step. The parameters of the diffusion model are represented; in the backdiffusion process, the bottleneck variable is gradually recovered from the random noise after a new candidate perturbation, and its back generation process is represented as: ,in, Indicates the first Bottleneck variables after perturbation at each time step Indicates the first The candidate perturbation bottleneck variables generated at each time step Indicates the first The diffusion coefficient corresponding to each time step The cumulative term representing the diffusion coefficient, Indicates the first Noise intensity coefficient at each time step Represents the weighted gradient guidance term; when the back diffusion process is caused by the first... Iterate to the time step . After one time step, new candidate perturbation bottleneck variables are obtained. The new candidate perturbation bottleneck variable Constructing a new set of bottleneck variables after candidate perturbation .

6. The robust information bottleneck optimization method according to claim 1, characterized in that, Step 5, which describes gradient guidance based on multi-objective entropy, constructs a weighted gradient to guide the diffusion model by calculating the entropy weight coefficients corresponding to the jointly optimized objectives of information compression, task prediction, and transmission robustness. Specifically, it includes the following steps: 6-1: Calculate the information entropy of each objective function The information entropy of each objective function is calculated based on the multi-objective optimization function. This entropy characterizes the uncertainty of each objective function on the bottleneck variable set after the current candidate perturbation. The entropy value is expressed as: ,in, Indicates the first The entropy value of an objective function. Indicates the first The objective function at the th ... Normalized distribution values ​​on the bottleneck variable after candidate perturbations Indicates the objective function number. Indicates the bottleneck variable number after candidate perturbation; 6-2: Calculate the entropy weight coefficients of each objective function. The corresponding weight coefficients are calculated based on the entropy values ​​of each objective function, and these weight coefficients are expressed as follows: ,in, Indicates the first The weight coefficients of each objective function. Indicates the first The entropy value of an objective function. Indicates the number of objective functions; 6-3: Constructing a weighted gradient and introducing a backdiffusion process The weighted gradient is constructed based on the weight coefficients, and its weighted gradient guidance term is expressed as follows: ,in, This represents the weighted gradient guidance term. Indicates the first The objective function at the th ... The candidate perturbation bottleneck variables corresponding to each time step gradient at, Indicates the first The weight coefficients of each objective function. The number of objective functions is represented; the weighted gradient guidance term is introduced into the back diffusion process of the diffusion model to adjust the generation direction of the candidate perturbation bottleneck variable, so that the generated candidate perturbation bottleneck variable is updated along the region that satisfies the information compression constraint and takes into account the robustness of task prediction and transmission.

7. The robust information bottleneck optimization method according to claim 1, characterized in that, Step 6 specifically includes the following steps: 7-1: Calculation of excess volume index By calculating the hypervolume index of the Pareto front, the new candidate perturbation bottleneck variable set generated by the reverse generation process is applied. The overvolume index is expressed as follows: ,in, Representing the solution set The excess volume index, Representing the solution set One of the candidate bottleneck variables after perturbation. Representing the solution set One of the candidate bottleneck variables after perturbation The corresponding objective function value vector, Indicates a reference point. This indicates the volume calculation in the target space; 7-2: Conditional Diffusion Generation Model Based on Supervolume Growth Rate Switching The supervolume growth rate is calculated based on the supervolume index of adjacent iterations, and the generation method of the bottleneck variable after candidate perturbation is determined based on the supervolume growth rate; when the supervolume growth rate is higher than a preset threshold... At that time, a conditional diffusion generation model is used to generate new candidate bottleneck variables after perturbation, where, The preset threshold representing the super-volume growth rate; 7-3. Switching Genetic Algorithms Based on Hypervolume Growth Rate When the supervolume growth rate is lower than a preset threshold At that time, a genetic algorithm is introduced to perform a global search for candidate perturbation bottleneck variables, and new candidate perturbation bottleneck variables are generated through crossover and mutation operations; 7-4. Selecting the Pareto-optimal set of bottleneck variables For candidate perturbation bottleneck variables that satisfy information compression constraints Performing non-dominated sorting includes: (1) Calculate the bottleneck variable after each candidate perturbation The corresponding objective function value vector ; (2) Bottleneck variables after any two candidate perturbations and Perform pairwise comparisons; if the conditions are met... in, Indicates the first One objective function, Indicates the number of objective functions; if at least one objective function exists... , making Then it is called Dominate ; (3) The candidate perturbation bottleneck variables that are not dominated by other candidate perturbation bottleneck variables are divided into the first non-dominated layer and used as the candidate set corresponding to the current Pareto front. After removing the first non-dominated layer, the above comparison process is repeated for the remaining candidate perturbation bottleneck variables to obtain the subsequent non-dominated layer. (4) The set of candidate perturbed bottleneck variables that satisfy the information compression constraint in the first non-dominated layer is taken as the Pareto optimal bottleneck variable set, denoted as ,in, This represents the Pareto optimal set of bottleneck variables that satisfies information compression constraints, task prediction performance, and transmission robustness; finally, for The Pareto optimal bottleneck variables are decoded to obtain the final robust information bottleneck variable set. .

Images