Integrated feature selection method and product based on quantum computing and bayesian optimization

Abstract

The application provides an integrated feature selection method and product based on quantum computing and Bayesian optimization, and relates to the technical field of data processing. The original mathematical integrated model is converted into a QUBO model that can be solved by quantum computing, and the QUBO model is decomposed into a joint optimization discretization step optimization subproblem and a parameterized QUBO subproblem. In the mixed quantum-classical optimization algorithm framework, an adaptive Q learning model is designed in the upper layer, which can dynamically select the appropriate subproblem in the search process. In the lower layer, a dropout Bayesian optimization algorithm is proposed to effectively optimize the high-dimensional discretization step in each iteration. The CIM-based quantum computing method is used to efficiently solve the parameterized QUBO subproblem under the given discretization step. The method proposed in the embodiment of the application can successfully and efficiently solve the feature selection problem, obtain the selected features of the credit classification problem, and obtain the credit classification model based on the selected features.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to an integrated feature selection method and product based on quantum computing and Bayesian optimization. Background Technology

[0002] In recent years, with the deep integration of big data and artificial intelligence technologies, the explosive growth of high-dimensional data and the complex and ever-changing business needs have placed higher demands on the interpretability and generalization ability of machine learning models. In many machine learning problems, data is often composed of high-dimensional features. Directly using the original feature space as input not only leads to a decline in model generalization performance but also causes overfitting risks due to the "curse of dimensionality," reducing data interpretability. Feature selection is an effective method to solve these problems. However, traditional feature selection techniques struggle to guarantee computational efficiency when dealing with combinations of high-dimensional feature spaces due to the dimensionality explosion. Therefore, exploring new computational paradigms that surpass existing algorithms is of great significance.

[0003] Feature selection plays a crucial role in machine learning, not only reducing data dimensionality but also improving model performance. However, most existing feature selection methods focus on reducing the number of features rather than improving the accuracy of classification models. For example, Lasso regularization achieves sparsity by reducing the number of features, but on high-dimensional gene datasets, this method leads to a decrease in the accuracy of random forest classifiers. Related research has also proposed a correlation-based feature selection method that focuses on reducing the number of features rather than directly optimizing classification accuracy. Few studies extract feature subsets by evaluating the correlation between features and classification accuracy. Another related study developed a hybrid filtering model based on principal component analysis and information gain for feature selection. The information gain criterion explicitly quantifies the contribution of each feature to classification accuracy. Yet another related study proposes a novel fuzzy rough set method based on fuzzy set dispersion metrics, achieving efficient feature selection by optimizing fuzzy positive regions. Another related study proposes a method that utilizes force and torque calculations during drilling and milling processes in feature-based CAD. The system can automatically identify and select key component features to support support, positioning, and clamping operations. Related research also proposed a feature pruning method, which uses a statistical significance fusion mechanism to classify and remove redundant features by integrating the standard deviation and mean-median difference.

[0004] It can be seen that existing feature selection methods mainly focus on selecting non-redundant and discriminative features, but neglect the impact of these selected features on downstream machine learning tasks. This neglect leads to suboptimal feature reduction strategies, thereby weakening the accuracy of downstream machine learning models. Furthermore, existing feature selection algorithms struggle to maintain computational efficiency when dealing with combinatorial explosion in high-dimensional feature spaces. Determining the most critical features is computationally very expensive because finding the optimal subset of features from a high-dimensional feature space is an NP-hard problem. n A dataset with features, having at least one feature. There are several possible subsets. Even in a moderately sized feature set, exhaustive search is impractical. Summary of the Invention

[0005] This invention provides an integrated feature selection method and product based on quantum computing and Bayesian optimization. This method can simultaneously optimize feature selection and classifier parameters, and efficiently solve the feature selection problem.

[0006] The first aspect of this invention provides an integrated feature selection method based on quantum computing and Bayesian optimization, the method comprising:

[0007] Based on user feature sets and user credit classification problems, an original mathematical ensemble model for the ensemble feature selection problem is constructed. The user feature set includes: basic user features and user behavioral features.

[0008] The original mathematical integrated model is reconstructed into a discretized model, and the discretized model is transformed into a quantum computing-solvable QUBO model.

[0009] The QUBO model is decomposed into a discretization step size optimization subproblem and a parameterized QUBO subproblem under a given discretization step size. The QUBO model is solved using a hybrid quantum classical optimization algorithm framework, which includes an adaptive Q-learning surrogate module, a Dropout Bayesian hyperparameter optimization module, and a CIM-based quantum computing module. At the upper layer of this framework, the adaptive Q-learning surrogate module selects appropriate subproblems during the search process. At the lower layer, the Dropout Bayesian hyperparameter optimization module explores the discretization step size space and determines the discretization step size. The CIM-based quantum computing module solves the ensemble model for the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, obtaining selected features for the credit classification problem and a credit classification model based on these selected features.

[0010] Optionally, the original mathematical ensemble model for the integrated feature selection problem is expressed as:

[0011] First objective function:

[0012] ;

[0013] First constraint:

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] Among them, user feature set = ， ，⋯， ) , m Indicates the number of samples. n Indicates the number of features. Indicates the first j The true label of each sample, whether it belongs to an overdue user. }, A 0-1 variable, representing the first... j Estimated labels for each sample, This represents the weight vector, which serves as the parameter for the user classification model. For continuous variables, it represents the first... i The weights of each feature, A 0-1 variable, indicating whether the feature was selected. K represents the number of features selected.

[0020] Optionally, the discretized model is represented as:

[0021] Second objective function:

[0022] ;

[0023] Second constraint:

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] in, Represents a discretized index. , 0-1 variables represent discretization. Auxiliary variables generated at that time express , l <0 and l Binary variables > 0 They were encoded separately. The negative and positive components.

[0031] Optionally, based on the adaptive Q-learning agent module, appropriate sub-problems are selected during the search process, including:

[0032] Using a Q-learning-based reinforcement learning mechanism, at each decision step... t Agent observation status s t Select Action a Receive rewards r t+1 and update Q(s,a), , γ Denotes the discount factor, 0 ≤ γ ≤ 1, α The learning rate is 0 < α ≤ 1;

[0033] The states include: the search progress state, the sub-problem selection sequence state, and the model performance state. The action refers to choosing between solving the discretization step size or solving the ensemble model. The reward represents the sum of the accuracy of the credit classification model and the constraint satisfaction rate.

[0034] Optionally, based on the Dropout Bayesian hyperparameter optimization module, the discretization step space is explored to determine the discretization step size, including:

[0035] exist n+1 In the dimensional discretization step size parameter, the optimal step size is selected in each iteration. d A subset of dimensions, fixing the rest ( n + 1-d ) variables;

[0036] In each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until a candidate discretization step size is determined.

[0037] Optionally, in each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until a candidate discretization step size is determined, including:

[0038] A small number of initial sample points are randomly selected in the parameter space. The credit classification model is trained using the hyperparameter combination corresponding to the initial sample points and the corresponding prediction accuracy is recorded. The initial sample points and the corresponding prediction accuracy are used as the initial observation data to construct the observed dataset.

[0039] Based on the observed dataset, a Gaussian process proxy model is constructed;

[0040] The Gaussian process surrogate model provides the predicted mean and variance for any sample point in the entire parameter space based on the initial observation data;

[0041] Based on the prediction of the Gaussian process surrogate model, a sampling function is defined; the sampling function is optimized, and the sample point corresponding to the maximum value of the sampling function is found. This sample point is then used as the next candidate sample point to be evaluated.

[0042] The candidate sample points are evaluated using a credit classification model to obtain the true accuracy corresponding to the candidate sample points.

[0043] The candidate sample points and their corresponding true accuracy are added to the observed dataset, and the Gaussian process surrogate model is retrained using the updated observed dataset to incorporate the newly acquired information.

[0044] Repeat the steps of updating the Gaussian process surrogate model, selecting candidate sample points, and evaluating candidate sample points until the maximum number of iterations is reached, and obtain the hyperparameter combination corresponding to the optimal sample point as the discretization step size parameter.

[0045] Optionally, based on the CIM-based quantum computing module, the ensemble model is solved using the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, including:

[0046] Transform a QUBO model with a given discretization step size into an Ising model;

[0047] Map the parameters of the Ising model to the physical components of the CIM;

[0048] The physical parameters of the CIM are initialized to put the CIM in a specific initial state. The CIM evolves naturally according to its own dynamic laws. When the CIM evolves to a steady state, the state of each physical component in the CIM is measured to obtain the selected features for the credit classification problem, as well as the credit classification model based on the selected features.

[0049] A second aspect of this invention provides an integrated feature selection device based on quantum computing and Bayesian optimization, the device comprising:

[0050] The module is used to construct an original mathematical ensemble model for the integrated feature selection problem based on the user feature set and the user credit classification problem. The user feature set includes: basic user features and user behavior features.

[0051] The transformation module reconstructs the original mathematical integrated model into a discretized model, and transforms the discretized model into a quantum computing-solvable QUBO model;

[0052] The solution module decomposes the QUBO model into a discretization step-size optimization subproblem and a parameterized QUBO subproblem under a given discretization step-size. It solves the QUBO model based on a hybrid quantum classical optimization algorithm framework, which includes an adaptive Q-learning agent module, a Dropout Bayesian hyperparameter optimization module, and a CIM-based quantum computing module. At the upper layer of the hybrid quantum classical optimization algorithm framework, the adaptive Q-learning agent module selects appropriate subproblems during the search process. At the lower layer, the Dropout Bayesian hyperparameter optimization module explores the discretization step-size space and determines the discretization step-size. The CIM-based quantum computing module solves the QUBO model with respect to the discretization step-size determined by the Dropout Bayesian hyperparameter optimization module, obtaining selected features for the credit classification problem and a credit classification model based on these selected features.

[0053] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the integrated feature selection method based on quantum computing and Bayesian optimization as described in the first aspect of the present invention.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the integrated feature selection method based on quantum computing and Bayesian optimization as described in the first aspect of the present invention.

[0055] A fifth aspect of the present invention provides a computer program product, including a computer program / instructions, which are implemented by a processor as the steps in the integrated feature selection method based on quantum computing and Bayesian optimization described in the first aspect of the present invention.

[0056] The integrated feature selection method proposed in this invention can jointly optimize the parameters of feature selection and user classification models.

[0057] In this embodiment of the invention, the original mathematical ensemble model is transformed into a quantum computing-solvable QUBO model. Specifically, this embodiment proposes a discretization reformulation method that can approximate the original data model. The reconstructed discretized model can approximate the proposed original mathematical ensemble model within an arbitrarily set error tolerance ε (ε>0). This embodiment utilizes the convexity of the objective function to transform the discretized model into a quantum computing-solvable QUBO model.

[0058] In this embodiment of the invention, the QUBO model is decomposed into a joint optimization subproblem of discretization step size optimization and a parameterized QUBO subproblem. Within the hybrid quantum classical optimization algorithm framework, an adaptive Q-learning model is designed at the upper layer, capable of dynamically selecting appropriate subproblems during the search process. This adaptive selection strategy increases the probability of identifying potential subproblems that can optimize algorithm performance, thereby contributing to more efficient algorithm optimization. At the lower layer, a dropout Bayesian optimization algorithm is proposed to effectively optimize the high-dimensional discretization step size in each iteration. Furthermore, a CIM-based quantum computing method is employed to efficiently solve the parameterized QUBO subproblem under a given discretization step size. The method proposed in this embodiment of the invention can successfully and efficiently solve the feature selection problem, obtain selected features for the credit classification problem, and develop a credit classification model based on the selected features. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of the steps of the integrated feature selection method based on quantum computing and Bayesian optimization provided in the embodiments of the present invention;

[0061] Figure 2 This is a schematic diagram of the data processing flow in the integrated feature selection method based on quantum computing and Bayesian optimization provided in the embodiments of the present invention, which uses a hybrid quantum classical optimization algorithm framework.

[0062] Figure 3 This is the AUC performance of the QUBO-L method on different datasets in the integrated feature selection method based on quantum computing and Bayesian optimization provided in the embodiments of the present invention;

[0063] Figure 4 This is the AUC performance of the QUBO-U method on different datasets in the integrated feature selection method based on quantum computing and Bayesian optimization provided in the embodiments of the present invention. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] In recent years, with the continuous advancement of quantum hardware technology, quantum computing has emerged as a highly promising computing paradigm, achieving exponential speedups in factorization. Quantum approximate optimization algorithms have demonstrated outstanding performance in solving combinatorial optimization problems, both in terms of numerical efficiency and hardware implementation. Currently, several companies, including IBM, Rigetti, and Originquantum, are actively engaged in quantum computing research, aiming to advance the development of quantum programming frameworks and hardware technologies. Milestones in this research include Google's "Sycamore" and IBM's "Q System."

[0066] Research on quantum computing in feature selection has shown diverse development paths. One related study combines Vision Transformer-based feature extraction techniques with a QUBO formalized feature selection method, using quantum computing to address the problem of cross-dependencies between features. Another related study developed a quantum feature engineering framework, which, combined with the SMOTE-Tomek data balancing technique, is used for liver disease classification tasks. Yet another related study constructs a quantum-supported feature combinatorial search framework, combining the mRMR heuristic algorithm and various distance metrics. Yet another related study theoretically establishes a QUBO balance between feature independence and predictive performance, providing a new modeling perspective for credit scoring systems. Yet another related study proposes a quantum computing-based optimized load scheduling technique specifically designed for real-time IoT applications in fog computing environments. Finally, another related study addresses the instability in predictive performance caused by reliance on local optima and interval partitioning in neutral time series modeling by combining quantum optimization algorithms with neutral time series modeling methods, thereby improving the global optimum of time series prediction.

[0067] Despite significant progress in quantum computing, quantum computers are still in their early stages of development and have not yet achieved widespread commercial application. We are currently in an era known as "noisy medium-scale quantum," where the number of qubits and the fidelity of quantum gates are limited. Quantum simulators support simulations using relatively few qubits. These simulators aim to verify the feasibility of quantum algorithms in small-scale scenarios, laying the foundation for future applications of these algorithms to large-scale practical problems. Given the limitations of current quantum hardware and the number of available qubits, the development of hybrid quantum-classical algorithms has become a trend, aiming to leverage the advantages of quantum algorithms while reducing the demand for quantum resources. A hybrid quantum-classical alternating optimization algorithm has been proposed for the first time, which combines a quantum approximation optimization algorithm with an alternating direction method of multiplier frameworks. The core idea of ​​QADMM is to decompose a high-dimensional problem into multiple independent subproblems and use a polling approach, employing an evolutionary optimizer to independently optimize each subproblem.

[0068] This invention proposes an integrated feature selection method based on quantum computing. This method can simultaneously optimize feature selection and classifier parameters, efficiently solving the feature selection problem.

[0069] Specifically, embodiments of the present invention provide an integrated feature selection method based on quantum computing and Bayesian optimization. For example... Figure 1 As shown, it illustrates the flowchart of the integrated feature selection method based on quantum computing and Bayesian optimization provided by an embodiment of the present invention. The method includes the following steps:

[0070] S101. Based on the user feature set and the user credit classification problem, construct the original mathematical ensemble model for the integrated feature selection problem. The user feature set includes: user basic features and user behavior features.

[0071] In this embodiment of the invention, for logistic regression-based... The dataset is a user credit classification task using Logistic Regression (LR). Depend on m It consists of 1 sample, each sample contains 1 sample and : , where [m] = {1, ..., m}. Specifically, ) is the first in dataset D j A sample, the sample has n Dimensional features, representing the 3rd dimension j The feature vector of each sample, and That is the label of the sample.

[0072] In this embodiment of the invention, dataset D includes an existing set of user features labeled with credit tags. The user feature set includes: basic user features and user behavioral features. Basic user features include: age, gender, occupation, and place of residence, etc. User behavioral features include: user consumption behavior, user borrowing behavior, and user repayment behavior, etc. The credit tag specifically refers to whether the user is an overdue user.

[0073] In microfinance, assessing user credit classification and determining whether a user is an "overdue user" is crucial. In this embodiment of the invention, ensemble feature selection is used to simultaneously filter features relevant to user credit classification and complete the classification training process of the credit classification model. The resulting credit classification model can accurately categorize users as "overdue users," reducing the amount of loans issued to them and mitigating bad debt risk, thus lowering the company's lending risk.

[0074] The integrated feature selection method based on quantum computing and Bayesian optimization proposed in this invention can also be used to process other classification problems, obtaining corresponding optimal feature combinations and classification models. For example, it can be used to predict diabetes risk classification based on user basic features and user behavioral features. User basic features may include: age, gender, family medical history, BMI (body mass index), diastolic blood pressure, systolic blood pressure, fasting blood glucose, insulin level, triglycerides, and cholesterol, etc. User behavioral features may include: user dietary habits and user exercise habits, etc.

[0075] In this embodiment of the invention, to improve the generalization ability of the model, feature selection and classification model parameters are jointly optimized to generate the optimal K feature subsets and their corresponding classification models. Specifically, let... Indicates any Features The choice, among which =1 indicates that this feature Selected. Define weight vector. As parameters of the classification model, where Indicates the bias term. Corresponding features The weights. Let... Let be the estimated label of the j-th sample, where j ∈[m]. The goal of the ensemble model in this embodiment of the invention is to select K features from the n-dimensional feature space and find a set of parameters { : i∈[n]}, to minimize the loss function .

[0076] In this embodiment of the invention, the original mathematical ensemble model for the ensemble feature selection problem is represented as follows:

[0077] (MINLP) Objective Function:

[0078] (1)

[0079] Constraints:

[0080] (2)

[0081] (3)

[0082] (4)

[0083] (5)

[0084] (6)

[0085] Among them, user feature set = ， ，⋯， ) , m Indicates the number of samples. n Indicates the number of features. Indicates the first j The true label of each sample, whether it belongs to an overdue user. }, A 0-1 variable, representing the first... j Estimated labels for each sample, This represents the weight vector, which serves as the parameter for the user classification model. Let be a continuous variable, representing the first... i The weights of each feature A 0-1 variable, indicating whether the feature was selected. K represents the number of features selected. As auxiliary parameters, =1.

[0086] Wherein, the objective function (1) indicates that the optimization objective is to minimize the predicted value. and the true value The difference between them is measured by cross-entropy loss. Constraint (2) represents the label. The estimated value. Constraint (3) limits the number of selected features to K. Constraints (5) and (6) define the range of decision variables.

[0087] S102, the original mathematical integrated model is reconstructed into a discretized model, and the discretized model is transformed into a quantum computing-solvable QUBO model.

[0088] In this embodiment of the invention, the mixed-integer linear programming (MINLP) model proposed in step S101 cannot be directly solved by existing commercial solvers (such as Gurobi and CPLEX). To address this issue, this embodiment proposes a hybrid quantum-classical algorithm based on the Coherent Ising Machine (CIM). CIM is a quantum computing paradigm utilizing optical parameter oscillator networks, providing an innovative method for solving this type of NP-hard combinatorial optimization problem. By mapping the discrete optimization problem to dynamically controllable bistable light pulses, CIM utilizes quantum wave dynamics to escape local optimum traps. This enables more efficient feature selection when dealing with high-dimensional data challenges. Based on this, this embodiment requires reformulating the original mathematical ensemble model proposed in step S101 into two quantum computing-compatible QUBO models.

[0089] Specifically, in order to transform the (MINLP) model into a quantum-computation-solvable QUBO model, embodiments of the present invention use a discretization method to approximate the continuous variable W. For each The embodiments of the present invention utilize 2L binary variables. To represent linear combinations The details are as follows:

[0090] (7)

[0091] in ,and This discretization method represents a series of uniformly distributed points. The step size of these points is The coverage area is .in, l <0 and l Binary variables > 0 They were encoded separately. The negative and positive components. Substituting equation (7) into (2), the original mathematical integration model can be reconstructed as:

[0092]

[0093] because {0,1} and {0,1}, embodiments of the present invention introduce linear constraints. <= To further develop nonlinearity Linearization, the specific methods are as follows:

[0094]

[0095] (8)

[0096] Based on the above analysis, the original mathematical ensemble model can be reconstructed into the following discretized model:

[0097] (D-MINLP) objective function:

[0098] (9)

[0099] Constraints:

[0100] (10)

[0101] (11)

[0102] (12)

[0103] (13)

[0104] (14)

[0105] (15)

[0106] In this embodiment of the invention, the following proof is also provided to demonstrate that the discretized model can effectively approximate (MINLP) the model. The proof is as follows:

[0107] Property 1: For any given error tolerance ε>0, there exists a discretization parameter. and , making

[0108] and These are the optimal solutions for (MILNP) and (D-MILNP), respectively.

[0109] Proof: First, prove .set up It is an optimal solution of (D-MINLP). Construction , making and It can be verified. (MINLP) is feasible and satisfies Therefore, there are Next, we will prove its existence. and Such that for any given ε>0, .

[0110] set up It is the optimal solution to the (MINLP) objective function, and is defined as follows: ,in Choose discretization parameters and The selection of these parameters is to ensure Under this parameterization setting, The value of is located in the discrete set In the middle, the step size is δ i Therefore, for i ∈[n]U{0}, there exists , where I∈L, such that .

[0111] Next, construct a feasible solution. (u', ​​z') (Satisfying the conditions) z'=z* And for all i ∈[n]∪{0}, l ∈L, This structure ensures Let the cross-entropy loss function be defined as: .

[0112] because and Therefore, there is This indicates It has 1-Lipschitz continuity, that is: .

[0113] Therefore, we can obtain:

[0114]

[0115]

[0116]

[0117]

[0118]

[0119] ;

[0120] The first inequality stems from the Lipschitz continuity property of the cross-entropy loss function, while the second inequality is derived from the Cauchy-Schwarz inequality. Therefore, we conclude: .

[0121] To address the ensemble feature selection problem, this invention presents a quantum-computable QUBO model. By utilizing Taylor expansion and discretization, the equivalent (D-MILNP) model is transformed into a QUBO model. The high performance of quantum computing is then leveraged to solve the ensemble model.

[0122] The upper bound of the QUBO model is:

[0123] ;

[0124] The lower bound QUBO model is:

[0125] ;

[0126] S103, the QUBO model is decomposed into a discretization step size optimization subproblem and a parameterized QUBO subproblem under a given discretization step size. The QUBO model is solved based on a hybrid quantum classical optimization algorithm framework, which includes an adaptive Q-learning agent module, a Dropout Bayesian hyperparameter optimization module, and a CIM-based quantum computing module. At the upper layer of the hybrid quantum classical optimization algorithm framework, the adaptive Q-learning agent module selects a suitable subproblem during the search process. At the lower layer of the hybrid quantum classical optimization algorithm framework, the Dropout Bayesian hyperparameter optimization module explores the discretization step size space and determines the discretization step size. The CIM-based quantum computing module solves the QUBO model for the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, obtaining the selected features for the credit classification problem and a credit classification model based on the selected features.

[0127] For the upper and lower bound QUBO model constructed in step S102, this embodiment of the invention designs an ensemble algorithm based on Bayesian hyperparameter tuning. The hyperparameters are tuned using Bayesian methods to obtain the discretization step size, and then quantum computing is used to solve the QUBO model based on the discretization step size.

[0128] In the QUBO-reformed ensemble model, the discretization step size determines the computational accuracy of the model. Therefore, in this embodiment of the invention, the solution of the QUBO model is decomposed into a discretization step size optimization subproblem and a parameterized QUBO subproblem under a given discretization step size. Specifically, this is solved by combining quantum classical optimization algorithms.

[0129] In this embodiment of the invention, the hybrid quantum classical optimization algorithm framework is as follows: Figure 2 As shown, it illustrates the data processing flow diagram of the hybrid quantum classical optimization algorithm framework.

[0130] Specifically, an adaptive Q-learning agent module is used at the upper layer of the hybrid quantum classical optimization algorithm framework to select appropriate sub-problems during the search process. In co-evolutionary algorithms, the optimal solution given by one evolutionary algorithm is applied to assist the evolution of another algorithm. Considering that the feature selection algorithm needs to continuously decide whether to perform Bayesian hyperparameter search during the computation process, this embodiment of the invention uses a task selection mechanism based on Q-learning.

[0131] Specifically, in this embodiment of the invention, a reinforcement learning mechanism based on Q-learning is used at each decision step. t Agent observation status s t Select Action a Receive rewards r t+1 and update Q(s,a), , γ Denotes the discount factor, 0 ≤ γ ≤ 1, α The learning rate is 0 < α ≤ 1;

[0132] The states include: the search progress state, the sub-problem selection sequence state, and the model performance state. The action refers to choosing between solving the discretization step size or solving the ensemble model. The reward represents the sum of the accuracy of the credit classification model and the constraint satisfaction rate.

[0133] In this embodiment of the invention, state In order to obtain dynamic information about the solution during the solution process, in this embodiment of the invention, the state is designed as three state spaces, namely:

[0134] , The status represents the search progress and is used to track the current iteration stage of the proposed algorithm. H represents the total number of iterations.

[0135] , The state represents the subproblem selection sequence, which records the selection of the subproblem during the previous iteration. -1 represents selecting Bayesian optimization hyperparameters, and 1 represents performing quantum computation.

[0136] , This represents the model's performance status, recording the model's performance in this iteration. 0 indicates no improvement in accuracy, while 1 indicates an improvement in accuracy relative to the set threshold.

[0137] In the embodiments of the present invention, the action space of Q-learning is: In the specific execution stage of Q-learning, interaction with the environment should be carried out through actions. It represents that in each state , the action of the agent is to select which subproblem to solve, that is, to solve the discretization step or the integrated model. In addition, the embodiments of the present invention also use the ε-greedy strategy. At each decision point, a random number is generated. If the random number is greater than a certain probability ε (0 ≤ ε ≤ 1), the action will be greedily selected.

[0138] In the embodiments of the present invention, the reward function of Q-learning is: The reward function clarifies the purpose of Q-learning and at the same time represents the real-time and long-term improvement brought by the selected action to the objective function. In order to intuitively reflect the efficiency of the algorithm in solving subproblems, in the embodiments of the present invention, the reward is defined as the sum of the accuracy rate of the classification model and the constraint satisfaction rate.

[0139] In the embodiments of the present invention, considering that quantum computing is relatively expensive, if quantum computing is performed every time a set of hyperparameter discretization steps is given, a relatively high computing cost is required. Based on this, the embodiments of the present invention propose to optimize the discretization steps based on the dropout Bayesian optimization algorithm. After determining the candidate discretization steps, quantum computing is performed based on the candidate discretization steps, thereby saving the bits required for quantum computing.

[0140] Specifically, in the embodiments of the present invention, based on Q-learning, it is determined whether the discretization steps obtained by the dropout Bayesian optimization algorithm can make the model reach the optimum. Based on the reward function of Q-learning, it is determined whether to select the action to solve the discretization steps or the integrated model.

[0141] In the embodiments of the present invention, in the QUBO model, the discretization steps significantly affect the solution quality of the QUBO model. However, it is very difficult computationally to comprehensively evaluate the classifier accuracy of all possible steps to determine the optimal step. To optimize the high-dimensional discretization steps, the embodiments of the present invention propose a dropout Bayesian optimization algorithm (DBOA), which combines Gaussian process regression with the dimensional dropout mechanism. Different from the traditional Bayesian optimization method, in the embodiments of the present invention, among the (n + 1)-dimensional discretization step parameters, only a d-dimensional subset (d < n + 1) is selected for optimization in each iteration. The Bayesian algorithm proposed in the embodiments of the present invention not only maintains the theoretical accuracy but also significantly improves the computational efficiency.

[0142] In the embodiments of the present invention, based on the Dropout Bayesian hyperparameter optimization module, the discretization step space is explored to determine the discretization steps, including:

[0143] In n+1In the dimensional discretization step size parameter, the optimal step size is selected in each iteration. d A subset of dimensions, fixing the rest ( n + 1-d There are 10 variables; in each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until a candidate discretization step size is determined.

[0144] In this embodiment of the invention, during each iteration, in n+1 Optimization of step size parameters in dimensional discretization d A subset of dimensions, fixing the rest ( n + 1-d ( ) variables are used to obtain new sampling points. Based on these new sampling points, new observation data are obtained. In the iterative process, different variables are selected in each iteration. d The dimensional subset is optimized using a Bayesian algorithm, and at the end of the final iteration, a subset covering all dimensions can be obtained. n+1 Candidate discretization step size parameters for dimensional discretization step size parameters.

[0145] In this embodiment of the invention, during each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until a candidate discretization step size is determined, including:

[0146] S11, Randomly select a small number of initial sample points in the parameter space, use the hyperparameter combination corresponding to the initial sample points to train the credit classification model and record the corresponding prediction accuracy; Use the initial sample points and the corresponding prediction accuracy as initial observation data to construct the observed dataset.

[0147] S12, Based on the observed dataset, construct a Gaussian process surrogate model.

[0148] S13, the Gaussian process surrogate model provides the predicted mean and variance for any sample point in the entire parameter space based on the initial observation data;

[0149] S14, Based on the prediction of the Gaussian process surrogate model, define the acquisition function; optimize the acquisition function, find the sample point corresponding to the maximum value of the acquisition function, and use the sample point as the next candidate sample point to be evaluated;

[0150] S15, use a credit classification model to evaluate the candidate sample points and obtain the true accuracy corresponding to the candidate sample points;

[0151] S16, add the candidate sample points and the corresponding true accuracy to the observed dataset, and retrain the Gaussian process surrogate model using the updated observed dataset to incorporate the newly acquired information;

[0152] S17, repeat the steps of updating the Gaussian process surrogate model, selecting candidate sample points, and evaluating candidate sample points until the maximum number of iterations is reached, and obtain the hyperparameter combination corresponding to the optimal sample point as the discretization step size parameter.

[0153] In this embodiment of the invention, it is first necessary to determine the value range of the discretization step size parameter in each dimension in order to define the parameter space. In this embodiment of the invention, the objective function is used... The function quantifies the predictive accuracy of credit classification models by using a discretization step size. is the independent variable.

[0154] In this embodiment of the invention, a small number of initial sample points can be randomly selected from the parameter space. The credit classification model is trained using the hyperparameter combinations corresponding to the initial sample points, and the corresponding prediction accuracy is recorded. Based on the initial observation data from the 1st iteration to the tth iteration. ,in Construct a probabilistic proxy model .

[0155] In this embodiment of the invention, the original function is approximated by Gaussian process regression (GPR). The Gaussian process-upper confidence bound (GP-UCB) is used to obtain the function. Select sampling points Subsequently, the sampling points selected in the (t+1)th iteration... and its corresponding function value Add to the initial observation data. Based on the updated observation set. Select the next sampling point and repeat the process until the maximum number of iterations T is reached.

[0156] In a Gaussian process (GP), the sample of the Gaussian process is a random function. This function follows Distribution. Among them It is the mean function, and This is the covariance kernel function. Assume the prior mean function of the Gaussian process is zero, i.e. Since any Gaussian process can be decomposed into a zero-mean Gaussian process plus a definite mean, it is only necessary to use the covariance kernel function to capture the correlation between the data in order to describe a Gaussian process. The covariance kernel function is defined as a squared exponential kernel:

[0157] ;

[0158] Let S be the covariance kernel matrix:

[0159] ;

[0160] For new sampling points , Given a Gaussian process prior, F(Δt) is a random variable. According to the definition of the Gaussian prior, any marginal distribution of finite dimension is a multivariate normal distribution. Specifically, F(Δt) and The prior joint distribution is given by the following formula:

[0161] ;

[0162] in Based on the conditional properties of Gaussian processes, given the observed value F(Δt), the surrogate model in... The predicted result at time t follows a Gaussian posterior distribution:

[0163]

[0164] Where P(a|b) represents the distribution of random variable a under condition b; It is the posterior mean function; It is the posterior variance function. Using the GPR method, the dropout Bayesian optimization algorithm proposed in this invention updates the posterior distribution and generates the predicted mean in each iteration. and variance .

[0165] GP-UCB acquisition function based on The distribution of , quantifies the potential value of each candidate point δ, and is defined as . ,in It is a positive tradeoff parameter used to balance exploration and exploitation. In each iteration t, the next sampling point is selected by maximizing the GP-UCB acquisition function. ,Right now This problem is usually solved by gradient-based algorithms or heuristics.

[0166] In this embodiment of the invention, during each iteration, instead of updating all discretized step sizes in the high-dimensional space, d variables are randomly selected from the (n+1)-dimensional space, and only the selected d-dimensional subset is optimized, while the remaining (n+1-d) variables are fixed. By dynamically adjusting the optimization variables, this embodiment of the invention can efficiently explore the high-dimensional space, thereby providing a principle-based and scalable solution.

[0167] In this embodiment of the invention, the performance of the Bayesian optimization algorithm is measured by the accumulated regret value, which is calculated cumulatively over a total of T iterations, as shown in the formula: ,in This represents the optimal solution with the discretization step size. According to relevant research, if the kernel function... If the U-Lipschitz condition is satisfied for all δ, then the cumulative regret of the proposed dynamic Bayesian optimization algorithm is limited to: .

[0168] In this embodiment of the invention, the CIM-based quantum computing module solves the ensemble model using the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, including:

[0169] S21, transform the QUBO model with a given discretization step size into the Ising model;

[0170] S22, map the parameters of the Ising model to the physical components of the CIM;

[0171] S23, initialize the physical parameters of CIM to put CIM in a specific initial state. CIM evolves naturally according to its own dynamic laws. When CIM evolves to a steady state, measure the state of each physical component in CIM to obtain the selected features of the credit classification problem, as well as the credit classification model based on the selected features.

[0172] In this embodiment of the invention, a CIM-based quantum method is employed to solve the parameterized QUBO subproblem. CIM is a quantum architecture that solves the combinatorial optimization problem by optically simulating Ising spin coupling. This problem is described by minimizing the Ising Hamiltonian:

[0173] ;

[0174] Where τ is the spin vector. Indicates coupling strength. h ᵢ represents a local field. Through transformation... xᵢ = (1-τᵢ) / 2 Hamiltonians are converted to QUBO form:

[0175] ;

[0176] in,

[0177] ;

[0178] CIM solves the parameterized QUBO subproblem through an optical measurement feedback network.

[0179] In this embodiment of the invention, the structure of the quadratic matrix Q and the value of the QUBO model essentially depend on the discretization step size vector. δ Therefore, the QUBO model involved in the embodiments of the present invention can be described as follows:

[0180] ;

[0181] in,

[0182] Describe a vector, which is composed of It consists of 0-1 variables. A matrix. Q(δ) sum vector c(δ) It can be obtained from the QUBO model.

[0183] The following verification examples demonstrate the effectiveness of the integrated feature selection method based on quantum computing and Bayesian optimization provided in this invention:

[0184] Table 1 compares the QUBO-L method proposed in this invention with Lasso-LR, Correlation-LR, and PCA-LR methods. It can be seen that the QUBO-L lower bound model proposed in this invention achieves better results in accuracy and AUC when using fewer features. Furthermore, compared to other models, the QUBO-L model has a shorter running time and better performance. Figure 3 The AUC performance of the QUBO-L method on different datasets is shown. The left graph shows the AUC performance of each method on the German credit dataset, and the right graph shows the AUC performance of each method on the Wisconsin breast cancer dataset. It can be seen that this method performs better than other models in terms of AUC.

[0185] Table 1 Comparison of the solution performance of the QUBO-L method

[0186]

[0187] Table 2 shows the comparison results of the QUBO-U method with the Lasso-LR method, the correlation LR method, and the PCA-LR method. It can be seen that the QUBO-U upper bound model proposed in this embodiment of the invention has better performance in accuracy and AUC when using fewer features, and compared with other models, the QUBO-U model has shorter computation time and higher running performance. Figure 4 The AUC performance of the QUBO-U method on different datasets is shown. The left graph shows the AUC performance of each method on the German credit dataset, and the right graph shows the AUC performance of each method on the Wisconsin breast cancer dataset. It can be seen that the QUBO-U algorithm outperforms the other models in terms of AUC.

[0188] Table 2 Comparison of the solution performance of the QUBO-U method

[0189]

[0190] Based on the same inventive concept, embodiments of the present invention also provide an integrated feature selection device based on quantum computing and Bayesian optimization, the device comprising:

[0191] The module is used to construct an original mathematical ensemble model for the integrated feature selection problem based on the user feature set and the user credit classification problem. The user feature set includes: basic user features and user behavior features.

[0192] The transformation module reconstructs the original mathematical integrated model into a discretized model, and transforms the discretized model into a quantum computing-solvable QUBO model;

[0193] The solution module decomposes the QUBO model into a discretization step-size optimization subproblem and a parameterized QUBO subproblem under a given discretization step-size. It solves the QUBO model based on a hybrid quantum classical optimization algorithm framework, which includes an adaptive Q-learning agent module, a Dropout Bayesian hyperparameter optimization module, and a CIM-based quantum computing module. At the upper layer of the hybrid quantum classical optimization algorithm framework, the adaptive Q-learning agent module selects appropriate subproblems during the search process. At the lower layer, the Dropout Bayesian hyperparameter optimization module explores the discretization step-size space and determines the discretization step-size. The CIM-based quantum computing module solves the QUBO model with respect to the discretization step-size determined by the Dropout Bayesian hyperparameter optimization module, obtaining selected features for the credit classification problem and a credit classification model based on these selected features.

[0194] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the integrated feature selection method based on quantum computing and Bayesian optimization as described in any of the above embodiments.

[0195] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the integrated feature selection method based on quantum computing and Bayesian optimization described in any of the above embodiments.

[0196] Based on the same inventive concept, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the integrated feature selection method based on quantum computing and Bayesian optimization described in any of the above embodiments.

[0197] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0198] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0202] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0203] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0204] The above provides a detailed description of the integrated feature selection method based on quantum computing and Bayesian optimization provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An integrated feature selection method based on quantum computing and Bayesian optimization, characterized in that, The method includes: Based on user feature sets and user credit classification problems, an original mathematical ensemble model for the ensemble feature selection problem is constructed. The user feature set includes: basic user features and user behavioral features. The original mathematical integrated model is reconstructed into a discretized model, and the discretized model is transformed into a quantum computing-solvable QUBO model. The QUBO model is decomposed into a discretization step size optimization subproblem and a parameterized QUBO subproblem under a given discretization step size. The QUBO model is solved based on a hybrid quantum classical optimization algorithm framework, which includes an adaptive Q-learning agent module, a Dropout Bayesian hyperparameter optimization module, and a CIM-based quantum computing module. At the upper layer of the hybrid quantum classical optimization algorithm framework, the adaptive Q-learning agent module selects a suitable subproblem during the search process. At the lower layer of the hybrid quantum classical optimization algorithm framework, the Dropout Bayesian hyperparameter optimization module explores the discretization step size space and determines the discretization step size. The CIM-based quantum computing module solves the QUBO model for the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, obtaining the selected features for the credit classification problem and a credit classification model based on the selected features. The original mathematical ensemble model for the ensemble feature selection problem is expressed as follows: First objective function: ； First constraint: ； ； ； ； ； Among them, user feature set , m Indicates the number of samples. n Indicates the number of features. Indicates the first j The true label of each sample, whether it belongs to an overdue user. }, A 0-1 variable, representing the first... j Estimated labels for each sample, This represents the weight vector, which serves as the parameter for the user classification model. Indicates the bias term. Indicates the first i The weights of each feature are continuous variables. A 0-1 variable, indicating whether the feature was selected. K represents the number of features selected; The discretization model is expressed as: Second objective function: ； Second constraint: ； ； ； ； ； ； in, Represents a discretized index. , 0-1 variables represent discretization weights. Auxiliary variables generated at that time Indicates weight The generated discretization step size parameter, l <0 and l Binary variables > 0 They were encoded separately. The negative and positive components.

2. The integrated feature selection method based on quantum computing and Bayesian optimization according to claim 1, characterized in that, Based on the adaptive Q-learning agent module, appropriate sub-problems are selected during the search process, including: Using a Q-learning-based reinforcement learning mechanism, at each decision step... t Agent observation status s t Select Action a Receive rewards r t+1 and update Q(s,a), , γ Denotes the discount factor, 0 ≤ γ ≤ 1, α The learning rate is 0 < α ≤ 1; The states include: the search progress state, the sub-problem selection sequence state, and the model performance state. The action refers to choosing between solving the discretization step size or solving the ensemble model. The reward represents the sum of the accuracy of the credit classification model and the constraint satisfaction rate.

3. The integrated feature selection method based on quantum computing and Bayesian optimization according to claim 1, characterized in that, Based on the Dropout Bayesian hyperparameter optimization module, the discretization step size space is explored, and the discretization step size is determined, including: exist n+1 In the dimensional discretization step size parameter, the optimal step size is selected in each iteration. d A subset of dimensions, fixing the rest ( n + 1-d ) variables; In each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until a candidate discretization step size is determined.

4. The integrated feature selection method based on quantum computing and Bayesian optimization according to claim 3, characterized in that, In each iteration, the discretization step size parameter is optimized based on the Bayesian algorithm until candidate discretization step sizes are determined, including: A small number of initial sample points are randomly selected in the parameter space. The credit classification model is trained using the hyperparameter combination corresponding to the initial sample points and the corresponding prediction accuracy is recorded. The initial sample points and the corresponding prediction accuracy are used as the initial observation data to construct the observed dataset. Based on the observed dataset, a Gaussian process proxy model is constructed; The Gaussian process surrogate model provides the predicted mean and variance for any sample point in the entire parameter space based on the initial observation data; Based on the prediction of the Gaussian process surrogate model, a sampling function is defined; the sampling function is optimized, and the sample point corresponding to the maximum value of the sampling function is found. This sample point is then used as the next candidate sample point to be evaluated. The candidate sample points are evaluated using a credit classification model to obtain the true accuracy corresponding to the candidate sample points. The candidate sample points and their corresponding true accuracy are added to the observed dataset, and the Gaussian process surrogate model is retrained using the updated observed dataset to incorporate the newly acquired information. Repeat the steps of updating the Gaussian process surrogate model, selecting candidate sample points, and evaluating candidate sample points until the maximum number of iterations is reached, and obtain the hyperparameter combination corresponding to the optimal sample point as the discretization step size parameter.

5. The integrated feature selection method based on quantum computing and Bayesian optimization according to claim 1, characterized in that, Based on the CIM quantum computing module, the ensemble model is solved using the discretization step size determined by the Dropout Bayesian hyperparameter optimization module, including: Transform a QUBO model with a given discretization step size into an Ising model; Map the parameters of the Ising model to the physical components of the CIM; The physical parameters of the CIM are initialized to put the CIM in a specific initial state. The CIM evolves naturally according to its own dynamic laws. When the CIM evolves to a steady state, the state of each physical component in the CIM is measured to obtain the selected features for the credit classification problem, as well as the credit classification model based on the selected features.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the integrated feature selection method based on quantum computing and Bayesian optimization as described in any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the integrated feature selection method based on quantum computing and Bayesian optimization as described in any one of claims 1-5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps in the integrated feature selection method based on quantum computing and Bayesian optimization as described in any one of claims 1-5.

Images