Data Denoising Method Based on DNN Two-Layer Optimization

Abstract

The data denoising method based on DNN two-layer optimization includes the following steps: acquiring a dataset and normalizing it to obtain initial data; dividing the initial data into training, validation, and test sets using a hierarchical random algorithm; constructing a two-layer network model based on the initial data and the DNN neural network; training the two-layer network model using the training set to obtain an optimized network model; validating the optimized network model using the validation set to obtain the optimal network model; and testing the optimal network model using the test set to obtain the test results. This method solves the problem in existing technologies where a large amount of redundant and interfering information in high-dimensional data affects the accuracy and reliability of the data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine learning, and particularly relates to a data denoising method based on DNN double-layer optimization. BACKGROUND

[0002] Classification task is one of the main branches of machine learning. The classification performance of the model depends on the data features and the quality of the classifier. The performance of the classifier is positively correlated with the quality of the data. Higher quality features can often provide more effective information for the classifier, thus obtaining better classification performance. However, with the development of big data and artificial intelligence, more and more data has been generated, such as biological genetic information, protein information, signal data, etc. These data have higher and higher dimensions, and can even reach tens of thousands of dimensions. These characteristics include high-quality characteristics associated with class labels, redundant characteristics independent of class labels, and even some features containing error information. These redundant features and error information bring great challenges to the performance of the classifier. In addition, the time and hardware cost of training an efficient classifier increases exponentially with the number of features, because more floating-point operations are required to process high-dimensional features. Therefore, feature selection is very meaningful. Feature selection can select relevant or complementary features, remove redundant features, and form a new feature subset to improve the quality of the data set. Feature selection includes two alternative steps: selecting a feature subset through a search mechanism, and designing an index to evaluate the performance of the subset.

[0003] While feature selection can filter out high-quality feature subsets to reduce model time and hardware costs, the large search space of high-dimensional data presents a significant challenge to finding high-quality feature subsets. Therefore, an effective search mechanism is essential. Evolutionary computation is widely used in feature selection due to its potential global search capability. In evolutionary computation, Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) are widely applied in feature selection. Competitive Swarm Optimizer (CSO) is a variant of PSO proposed by Cheng et al. Studies have shown that CSO has better population diversity and better global search capability compared to standard PSO. Therefore, CSO is more suitable for feature selection problems with larger search spaces. Gu et al. proposed a feature selection model based on CSO and proved that its performance outperforms standard PSO. The main idea of ​​CSO is to randomly select two solutions and compare their fitness, with the worse solution learning from the better one. However, a better solution is not necessarily a feasible solution for feature selection. To address this issue, Nguyen et al. proposed a Constrained Competitive Optimizer (CCSO). The main idea of ​​CCSO is to learn from feasible solutions to avoid creating even worse solutions. Experimental results show that CCSO outperforms the standard CSO. Maleki et al. proposed a feature selection model based on a genetic algorithm and validated its effectiveness on a lung cancer dataset using a KNN classifier. Li et al. proposed a feature selection model based on a genetic algorithm (WKNNGAFS). WKNNGAFS combines a genetic algorithm and a weighted KNN (WKNN) algorithm to calculate the score for each feature. This algorithm performs well on 13 datasets. Based on the advantages of evolutionary computation algorithms in feature selection, this invention chooses a genetic algorithm as the search mechanism for feature selection.

[0004] DNNs have attracted widespread attention due to their ability to extract global information. However, the performance of DNNs largely depends on their structure, connection parameters, and activation functions. Optimizing DNNs, which involves designing network structures, connection parameters, and activation functions, is often a combinatorial optimization problem, thus posing significant challenges to designers. In recent years, evolutionary computation has been widely applied to DNN optimization due to its powerful search capabilities and the fact that it does not require differentiability of the objective function. Du et al. proposed an improved version combining Non-Dominant Ranking Adaptive Differential Evolutionary Algorithm (NSJADE) and Knee-Joint Strategy-Based NSADE (KP-NSADE) to optimize a single-hidden-layer feedforward neural network (SLFN) model for time series prediction. The error rate on the validation set, the L2 norm of the network output, and the number of nodes in the hidden layer were used as evaluation metrics for network performance. Experiments showed that NSJADE is more suitable for multimodal problems than the standard NSGAII. Bai et al. improved upon the genetic algorithm, proposing a cooperative genetic algorithm based on Extreme Learning Machine (SLI) called CGA-ELM. CGA-ELM uses the error rate on the validation set and network complexity as methods to evaluate network performance. Binary encoding is used to optimize the number of hidden layer nodes in SLFN, while real encoding is used to optimize the input weights. A pseudo-inverse method is employed to compute the network output. Research shows that CGA-ELM can guarantee the classification performance of SLFN and achieve a more compact network structure. Li et al. proposed a two-layer learning-based model to optimize DNN. The upper layer uses the binary particle swarm optimization (BPSO) algorithm to optimize the network structure, while the lower layer uses the Levenberg-Marquardt (LM) algorithm to optimize the network connection parameters. The optimization of the upper and lower layers is performed alternately to produce a high-performance, compact DNN. For DNN optimization, most inventions only design the number of network layers and the number of nodes per layer, ignoring the network connection patterns and activation functions. A reasonable topology can enable feature reuse and improve DNN performance. Therefore, the search space of this invention includes a splicing scheme that enables feature reuse.

[0005] Evolutionary computation has achieved significant success in feature selection and DNN optimization. Many feature selection inventions based on evolutionary computation often use existing classifiers such as SVM, KNN, and DNN, but they neglect the fact that the structure and parameters of the classification also need to be adjusted. Summary of the Invention

[0006] The purpose of this invention is to provide a data denoising method based on DNN dual-layer optimization, which solves the problem that a large amount of redundant and interfering information in the data affects the accuracy and reliability of the data in the prior art.

[0007] The technical solution adopted in this invention is a data denoising method based on DNN two-layer optimization, which specifically includes the following operation steps:

[0008] Step 1: Obtain the dataset and normalize it to obtain the initial data;

[0009] Step 2: Divide the initial data into training set, validation set and test set using a stratified random algorithm;

[0010] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0011] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0012] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0013] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0014] The invention is further characterized by:

[0015] In step 1, the dataset is obtained from the UCI database. The dataset contains multiple feature subsets, including biometric information, facial recognition information, speech recognition information, and medical diagnostic information.

[0016] In step 3, the two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower objective function and the Adam algorithm.

[0017] The upper-level objective function consists of three parts, the first of which is the validation set error rate:

[0018]

[0019] In equation (1), F1(Θ,Ω,W) * ) represents the validation set error rate, with values ​​in the range [0, 1]; N ve N represents the number of misclassified samples in the validation set. vs Represents the number of validation set samples; Θ represents feature selection encoding; Ω represents the network structure; W* represents the optimal network parameters;

[0020] The second part is the feature selection ratio:

[0021]

[0022] In equation (2), F2(Θ) represents the feature selection ratio, with a value of [0, 1]; Θ represents the feature selection encoding, θ i =1 indicates that the i-th feature is selected, and D represents the total length of the feature selection code;

[0023] The third part is the network complexity metric:

[0024]

[0025] In equations (3) and (4), F3(Θ, Ω) represents the network complexity index, with a value of [0, 1]; N tc N represents the total number of network connections. ac This represents the actual number of network connections. I represents the number of input nodes in the DNN; K represents the number of output nodes in the DNN; L represents the upper bound of the preset number of hidden layers in the model; δ max This represents the maximum preset number of nodes in all hidden layers;

[0026] The asymmetric competitive genetic algorithm optimizes the upper-level objective function:

[0027]

[0028] Equation (5), F u Let λ1, λ2, and λ3 represent the upper-level objective function, respectively, and let F1(Θ, Ω, W). * The contributions of F2(Θ) and F3(Θ,Ω) to the upper objective function;

[0029] The lower-level objective function is:

[0030]

[0031] In equation (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0032] Step 4 involves optimizing the training of the network model as follows: The training set is input into the two-layer network model, and the initial feature subset and initial network structure are obtained through the upper-layer objective function. The initial feature subset and initial network structure are then optimized using an asymmetric genetic algorithm. The optimized initial feature subset and initial network structure are then input into the lower-layer Adam algorithm for further optimization to obtain the network parameters. The network parameters are then input into the lower-layer objective function to determine whether the lower-layer objective function has converged. If the lower-layer objective function has converged, the optimal network parameters are obtained. If the lower-layer objective function has not converged, this step is repeated.

[0033] The formula for calculating the optimal network parameters is:

[0034] W * =arg W min{f l (Θ,Ω,W)} (7)

[0035] In equation (7), W takes the value [-1, 1]. N .

[0036] The specific steps for validating the optimal network model in step 5 are as follows:

[0037] Step 5.1: Input the validation set into the two-layer network model with optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper objective function;

[0038] Step 5.2: Input the optimized network structure and optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm has converged;

[0039] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0040] The beneficial effects of this invention are as follows: The proposed data denoising method based on DNN dual-layer optimization uses a dual-layer DNN network, which is a two-layer learning model. The upper-layer ACGA algorithm performs feature selection and constructs the DNN structure on the initial data through the upper-layer objective function; the lower-layer Adam algorithm optimizes the network parameters through the cross-entropy loss function to determine the optimal network parameters and optimal network model; a concatenation operation is introduced into the search space to achieve feature reuse; the concatenation operation can bring more complexity to the search space and generate more feasible solutions; the lower-layer Adam algorithm optimizes the network parameters through the loss function, improving the accuracy of the data processing process. Attached Figure Description

[0041] Figure 1 This is a flowchart of the data denoising method based on DNN two-layer optimization of the present invention;

[0042] Figure 2 This is a flowchart of the encoding and decoding process in the DNN-based dual-layer optimized data denoising method of this invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0044] The data denoising method based on DNN two-layer optimization of the present invention, such as Figure 1 As shown, the specific operation steps are as follows:

[0045] Step 1: Obtain the dataset and normalize it to obtain the initial data;

[0046] Step 2: Divide the initial data into training set, validation set and test set using a stratified random algorithm;

[0047] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0048] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0049] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0050] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0051] In step 1, the dataset is obtained from the UCI database. The dataset contains multiple feature subsets, including biometric information, facial recognition information, speech recognition information, and medical diagnostic information.

[0052] In step 3, the two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower objective function and the Adam algorithm.

[0053] The upper-level objective function consists of three parts, the first of which is the validation set error rate:

[0054]

[0055] In equation (1), F1(Θ,2,W*) represents the validation set error rate, with values ​​ranging from [0,1]; N ve N represents the number of misclassified samples in the validation set. vs Represents the number of validation set samples; Θ represents feature selection encoding; Ω represents the network structure; W* represents the optimal network parameters;

[0056] The second part is the feature selection ratio:

[0057]

[0058] In equation (2), F2(Θ) represents the feature selection ratio, with a value of [0, 1]; Θ represents the feature selection encoding, θ i =1 indicates that the i-th feature is selected, and D represents the total length of the feature selection code;

[0059] The third part is the network complexity metric:

[0060]

[0061] In equations (3) and (4), F3(Θ, Ω) represents the network complexity index, with a value of [0, 1]; N tc N represents the total number of network connections. ac This represents the actual number of network connections. I represents the number of input nodes in the DNN; K represents the number of output nodes in the DNN; L represents the upper bound of the preset number of hidden layers in the model; δ maxThis represents the maximum preset number of nodes in all hidden layers;

[0062] The asymmetric competitive genetic algorithm optimizes the upper-level objective function:

[0063]

[0064] Equation (5), F u Let λ1, λ2, and λ3 represent the upper-level objective function, respectively, and let F1(Θ, Ω, W). * The contributions of F2(Θ) and F3(Θ,Ω) to the upper objective function;

[0065] The lower-level objective function is:

[0066]

[0067] In equation (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0068] Step 4 involves optimizing the training of the network model as follows: The training set is input into the two-layer network model, and the initial feature subset and initial network structure are obtained through the upper-layer objective function. The initial feature subset and initial network structure are then optimized using an asymmetric genetic algorithm. The optimized initial feature subset and initial network structure are then input into the lower-layer Adam algorithm for further optimization to obtain the network parameters. The network parameters are then input into the lower-layer objective function to determine whether the lower-layer objective function has converged. If the lower-layer objective function has converged, the optimal network parameters are obtained. If the lower-layer objective function has not converged, this step is repeated.

[0069] The formula for calculating the optimal network parameters is:

[0070] W * =arg W min{f l (Θ,Ω,W)} (7)

[0071] In equation (7), W takes the value [-1, 1]. N .

[0072] The specific steps for validating the optimal network model in step 5 are as follows:

[0073] Step 5.1: Input the validation set into the two-layer network model with optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper objective function;

[0074] Step 5.2: Input the optimized network structure and optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm has converged;

[0075] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0076] Binary encoding is performed on a subset of features in the training set; the selection encoding of the feature subset is Θ = (θ1, θ2, ..., θ D ); where θ i =1 indicates that the i-th feature is selected, θ i =0 indicates that the i-th feature was not selected, and D represents the total length of the encoding;

[0077] The two-layer network model is encoded using a hybrid binary and integer encoding: the hybrid encoding is Ω=(α,β,γ,δ), where The binary code representing feature selection, α i ={0,1}, i = 1,2,...,S1; It is the binary code representing each hidden layer connection pattern, β j ={0,1}, j =1,2,…,S2; It is the binary code representing the type of activation function in each hidden layer, γ k ={0,1},k=1,2,…,S3; δ=(δ1,δ2,…δ L ) is an integer encoding representing the number of hidden layer nodes, δ l =[δ min ,δ max ], l=1,2,…,L;δ max Indicates the maximum preset number of nodes in all hidden layers; δ min L represents the minimum preset number of nodes in all hidden layers; L represents the maximum preset number of hidden layers in the model, such as... Figure 2 As shown.

[0078] Each hidden layer contains an activation function, which can be any one of ReLU, Identity, sigmoid, or tanh.

[0079] The formula for calculating the optimal network parameters in step 4 is:

[0080] W * =arg W min{f l (Θ,Ω,W)} (7)

[0081] In equation (7), W takes the value [-1, 1]. N .

[0082] The steps for determining the optimal network parameters in step 4 are as follows:

[0083] The training set is input into a two-layer network model. The initial feature subset and initial network structure are obtained through the upper-layer objective function. An asymmetric genetic algorithm is used to optimize the initial feature subset and initial network structure. The optimized initial feature subset and initial network structure are then input into the lower-layer Adam algorithm for further optimization to obtain network parameters. The network parameters are then input into the lower-layer objective function to determine whether the lower-layer objective function has converged. If the lower-layer objective function has converged, the optimal network parameters are obtained. If the lower-layer objective function has not converged, this step is repeated.

[0084] The specific steps for validating the optimal network model in step 5 are as follows:

[0085] Step 5.1: Input the validation set into the two-layer network model with optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper objective function;

[0086] Step 5.2: Input the optimized network structure and optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm has converged;

[0087] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0088] After the test is completed, the test results are decoded to obtain the final test result. The decoding process is as follows:

[0089] Binary encoding Θ=(θ1,θ2,…,θ D If θ i =1, which means the i-th feature is selected; otherwise, the i-th feature is not selected; the number of selected feature subsets is equal to It is equal to the number of input nodes I, that is For binary encoding The number of hidden layers is

[0090] Binary encoding The network connection mode is decoded as follows: β1 represents the connection relationship between the second hidden layer and the input layer; if β1 = 1, it means that the second hidden layer is connected to the input layer, otherwise it means that they are not connected; (β2,β3) represents the connection status of the third hidden layer with the input layer and the first hidden layer; if (β2,β3) = (1,1), it means that the third hidden layer is connected to the input layer and the first hidden layer; (β2,β3) = (0,0) means that the third hidden layer is not connected to the input layer and the first hidden layer; (β2,β3) = (1,0) means that the third hidden layer is connected to the input layer, but not connected to the first hidden layer; (β2,β3) = (0,1) means that the third hidden layer is connected to the first hidden layer, but not connected to the input layer; (β4,β5,β6) represents the connection status of the fourth layer with the input layer, the first layer, and the second layer. If (β4,β5,β6) = (1,1,1), it means the fourth hidden layer is connected to the input layer, the first hidden layer, and the second hidden layer; (β4,β5,β6) = (0,0,0), it means the fourth hidden layer is not connected to the input layer, the first hidden layer, or the second hidden layer; (β4,β5,β6) = (1,1,0), it means the fourth hidden layer is connected to the input layer and the first hidden layer, but not to the second hidden layer, and so on, until the connection state between the first layer and the previous layer from the first layer to the (L-2)th layer is decoded; that is:

[0091]

[0092] Binary encoding (γ1, γ2) represents the activation function used in the first hidden layer, which is decoded as... (γ3, γ4) represents the activation function used in the second hidden layer, which is decoded as... And so on; S3 = 2 × L, N ai =0, 1, 2, 3 indicate that ReLU, Identity, sigmoid, or tanh are selected as the activation functions of the hidden layer, respectively.

[0093] The asymmetric competitive genetic algorithm (ACGA) mainly consists of three parts: crossover operation, mutation operation, and elite solution selection operation, specifically:

[0094] Crossover operation: Each individual X has a relatively long encoded string with a length of D+S1+S2+S3+L; L represents the upper bound of the preset number of hidden layers in the network in the model; S1, S2, and S3 represent the encoding lengths of α, β, and γ, respectively; this invention adopts the commonly used two-point crossover method with probability p c Choose a pair of parents; each generation has a population, the previous generation of which is called the parent or paternal generation, denoted by X. i and X j Let i ≠ j, i, j = 1, 2, ..., N. PRandomly select X i and X j The two cross sites in the X i Gene substitution between two crossover sites is X j Genes between two crossover sites generate a marker U i The daughter gene.

[0095] Mutation operation based on asymmetric learning strategy and ReliefF score: Each mutated individual Vi (i = 1, 2, ..., N) P Individual E randomly selected from the elite set k (k = 1, 2, ..., 10%N) P The feature subset is learned in the first stage of mutation, and then the binary encoding is corrected using the ReliefF score. In the first stage of the mutation operation, a strategy of learning from elite individuals is adopted. For the binary encoding, U... i China is different from E k The variant sites have a higher probability of flipping. For integer encoding, U i Each mutation site has a greater probability of mutating to U i and E k A random integer between two integers. U i binary encoding and E k The difference between the binary codes is represented as V. diff :

[0096] V diff,j =|U i,j -E k,j | j=1,2,…,N B (9)

[0097] In equation (9), N B The length of the binary code is represented by N. B =D + S1 + S2 + S3, p i,j U i The probability that the j-th site is a mutation site:

[0098]

[0099] In equation (13), p a This represents the preset probability of asymmetric mutation.

[0100] V i The binary encoding is:

[0101]

[0102] In equation (11), j = 1, 2, ... N Brand(j) represents a random number in the j-th interval (0,1).

[0103] V i The integer encoding is:

[0104]

[0105] In equation (12), This represents the floor function, j = N B +1,N B +2,…N B +L, rand(j) represents any value in the j-th interval (0,1).

[0106] In the second stage of the mutation operation, only the encoding of the feature subset is manipulated, without affecting the encoding of the network structure. Some features are selectively added or removed from the encoding of each individual's feature subset based on the ReliefF score and roulette wheel method. Features with high ReliefF scores are considered highly relevant and are more likely to be added to the feature subset. The probability of each feature being added to the feature subset is calculated using the following formula:

[0107]

[0108] In equation (13), Rf j This represents the ReliefF score of the j-th feature;

[0109] If a selected feature with a high ReliefF score already exists in the feature subset, then the corresponding encoding of that feature does not require mutation. Features with low ReliefF scores have lower relevance to the label and are therefore more likely to be removed. The probability of each feature being removed is 1-p. f,j The roulette wheel method was used to remove features with low ReliefF scores.

[0110] Selection Strategies: For FS and DNN optimization based on genetic algorithms, common selection strategies are elite selection and roulette wheel selection. In roulette wheel selection, the probability of each individual being selected is equal to the ratio of its fitness value to the sum of fitness values ​​in the entire population. However, excellent individuals in each generation may be eliminated, which is unacceptable for DNN structure optimization and FS with its large search space and high computational cost. For DNN optimization, a high-fitness individual only indicates good performance on the validation set, not necessarily on the test set. Simply retaining the best individual of each generation may result in solutions that perform well on the validation set but poorly on the test set. To address these issues, this invention employs a hybrid selection strategy. Parental individuals, crossover individuals, and mutated individuals are merged, with the merged individuals ordered in ascending order of their upper-level objective values. The top 10% of individuals in the population directly enter the next generation, assuming they are equally important; the remaining individuals enter the next generation through roulette wheel selection.

[0111] Input the training set and calculate the gradients of the network parameters using the given upper-layer network structure; calculate the first moment estimate m at the current time step. t Second moment estimate v t ,m t =τ1m t-1 +(1-τ1)d k , Where: m t-1 Let m represent the first moment estimate of the (t-1)th iteration. t Let τt represent the first-order moment estimate of the t-th iteration, τ1 represent the exponential weighting parameter, and dt represent the first-order moment estimate of the t-th iteration. t-1 Let v represent the gradient of the network parameter W in the (t-1)th iteration. t-1 Let v represent the second moment estimate of the (t-1)th iteration. t τ represents the second moment estimate of the t-th iteration, and τ2 represents the exponential weighting parameter; Let W be the square of the gradient of the (t-1)th iteration; the corrected first-order momentum and second-order momentum are... and According to the equation Update network parameters, where W t W represents the network parameters at the t-th iteration. t-1 This represents the network parameters at the (t-1)th iteration.

[0112] Example 1

[0113] The data denoising method based on DNN two-layer optimization of the present invention specifically includes the following operation steps:

[0114] Step 1: Obtain biological information from the UCI database and normalize the biological information to obtain initial data;

[0115] Step 2: Divide the initial data into a training set, a validation set, and a test set in a 6:2:2 ratio;

[0116] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0117] The two-layer network model consists of an upper layer and a lower layer. The upper layer includes an upper objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower objective function and the Adam classification algorithm.

[0118] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0119] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0120] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0121] The test results were compared with the biological information processed by the Constrained Competitive Optimizer (CCSO), CUS-SPSO, bAAAs1, MSFS, and SLMEA, respectively, and the results are shown in Table 1.

[0122] Table 1 Comparison results of bio-information denoising in Example 1

[0123]

[0124]

[0125] Through in-depth analysis of the data in the table above, we found that this invention demonstrated significant feature selection capabilities in 11 comparative experiments on the bioinformatics dataset. Specifically, in 8 experiments, the network model of this invention selected fewer features than other methods, exhibiting extremely high feature selection efficiency. More remarkably, in four comparative experiments, the number of features selected by this invention was only half that of other methods, a result that strongly demonstrates the superior performance of this invention in simplifying the feature set. Furthermore, the prediction accuracy of this invention surpassed other methods in all comparative experimental groups, fully demonstrating its superior prediction capabilities. Particularly noteworthy is that in three experiments, the accuracy of this invention improved by more than 30% compared to the original method; this significant improvement further highlights the effectiveness and efficiency of this invention in the field of bioinformatics feature selection. In conclusion, this invention, by reducing the number of features and improving prediction accuracy, fully validates its value and potential in practical applications.

[0126] Example 2

[0127] The data denoising method based on DNN two-layer optimization of the present invention specifically includes the following operation steps:

[0128] Step 1: Obtain medical diagnostic information from the UCI database and normalize the medical diagnostic information to obtain initial data; the dataset contains multiple feature subsets.

[0129] Step 2: Divide the initial data into a training set, a validation set, and a test set in a 6:2:2 ratio;

[0130] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0131] The two-layer network model consists of an upper layer and a lower layer. The upper layer includes an upper objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower objective function and the Adam classification algorithm.

[0132] The upper-level objective function consists of three parts, the first of which is the validation set error rate:

[0133]

[0134] In equation (1), F1(Θ,Ω,W) * ) represents the validation set error rate, with values ​​in the range [0, 1]; N ve N represents the number of misclassified samples in the validation set. vs Represents the number of validation set samples; Θ represents feature selection encoding; Ω represents the network structure; W* represents the optimal network parameters;

[0135] The second part is the feature selection ratio:

[0136]

[0137] In equation (2), F2(Θ) represents the feature selection ratio, with a value of [0, 1]; Θ represents the feature selection encoding, θ i =1 indicates that the i-th feature is selected, and D represents the total length of the feature selection code;

[0138] The third part is the network complexity metric:

[0139]

[0140] In equations (3) and (4), F3(Θ, Ω) represents the network complexity index, with a value of [0, 1]; N tc N represents the total number of network connections. ac This represents the actual number of network connections. I represents the number of input nodes in the DNN; K represents the number of output nodes in the DNN; L represents the upper bound of the preset number of hidden layers in the model; δ maxThis represents the maximum preset number of nodes in all hidden layers;

[0141] The asymmetric competitive genetic algorithm optimizes the upper-level objective function:

[0142]

[0143] Equation (5), F u Let F1, λ2, and λ3 represent the contributions of F1(Θ,Ω,W*), F2(Θ), and F3(Θ,Ω) to the upper-level objective function, respectively, with values ​​of 0.98, 0.01, and 0.01.

[0144] The lower-level objective function is:

[0145]

[0146] In equation (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0147] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0148] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0149] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0150] The test results were compared with the medical diagnostic information after being processed by CUS-SPSO, bAAAs1, MSFS, and SaWDE, respectively, and the results are shown in Table 2.

[0151] Table 2 Comparison Results of Noise Reduction in Medical Diagnostic Information in Example 2

[0152]

[0153] In the context of medical diagnosis, accurate and efficient feature selection is crucial for achieving high-quality diagnoses. Feature selection not only improves the classification performance of a model but also reduces computational complexity and avoids overfitting. In this field, the model of this invention demonstrates significant advantages. Specifically, this invention outperforms four state-of-the-art models on seven sets of medical diagnostic datasets, and 12 comparative experiments were conducted, showing that its feature selection capability and classification performance are both outstanding.

[0154] Experimental data shows that the number of features selected in the six sets of experiments of this invention is significantly better than other comparative models, and the accuracy in the nine sets of experiments is also higher than other methods. Particularly on the PGE, Leuk, and Lung2 datasets, this invention significantly outperforms CUS-SPSO in feature selection, while its classification performance is also generally superior to existing models. For example, on the PGE dataset, the accuracy of this invention is improved by 12.64%, indicating its excellent performance in handling complex data and maintaining high accuracy with a large number of features, demonstrating its good stability and feature selection ability. Furthermore, this invention achieves varying degrees of accuracy improvement on multiple datasets, with an accuracy improvement of 34.02% compared to SaWDE on the Lung1 dataset. These results demonstrate that the model of this invention not only performs well in reducing the number of features but also simplifies the model while maintaining high classification accuracy, reducing computational burden and improving diagnostic efficiency. In summary, this invention provides an efficient and reliable solution for medical diagnostic applications by optimizing feature selection and model simplicity, contributing to higher accuracy and efficiency in practical applications.

[0155] Example 3

[0156] The data denoising method based on DNN two-layer optimization of the present invention specifically includes the following operation steps:

[0157] Step 1: Obtain face recognition information from the UCI database and normalize the face recognition information to obtain initial data;

[0158] Step 2: Divide the initial data into a training set, a validation set, and a test set in a 6:2:2 ratio;

[0159] Step 3: Construct a two-layer network model based on the DNN neural network;

[0160] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0161] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0162] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0163] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0164] The two-layer network model consists of an upper layer and a lower layer. The upper layer includes an upper objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower objective function, a loss function, and the Adam classification algorithm.

[0165] Step 4, optimizing the network model training process, involves: inputting the training set into the two-layer network model; obtaining the initial feature subset and initial network structure through the upper-layer objective function; optimizing the initial feature subset and initial network structure using an asymmetric genetic algorithm; inputting the optimized initial feature subset and initial network structure into the lower-layer Adam algorithm for further optimization to obtain network parameters; inputting the network parameters into the lower-layer objective function to determine whether the lower-layer objective function converges; if the lower-layer objective function converges, obtaining the optimal network parameters; if the lower-layer objective function does not converge, repeating this step.

[0166] The formula for calculating the optimal network parameters in the lower layer is:

[0167] W * =arg W min{f l (Θ,Ω,W)} (7)

[0168] In equation (7), W takes the value [-1, 1]. N .

[0169] The test results were compared with the face recognition information processed by CCSO, SLMEA, CUS-SPSO, bAAAs1, and SaWDE respectively, and the results are shown in Table 3.

[0170] Table 3 Comparison Results of Noise Reduction for Face Recognition Information in Example 3

[0171]

[0172]

[0173] In the field of face recognition and segmentation, the model of this invention demonstrates significant advantages based on the data in the table. Specifically, the model of this invention was compared with five state-of-the-art methods on 11 image recognition datasets, with a total of 14 comparative experiments conducted. Experimental results show that, except for slightly inferior performance compared to the CCSO method on the Wall dataset and the bAAAs1 method on the OR dataset, the accuracy of the model of this invention is superior to the comparative methods in all other experiments. In particular, the accuracy of this invention is improved by more than 30% in comparisons with the SLMEA method on the USPS dataset and the PIE10P dataset. Regarding feature selection, this invention selects fewer features than other methods in 9 sets of experiments. These conclusions fully verify the stable performance of this invention on different datasets, maintaining high accuracy with fewer features, demonstrating its excellent stability and feature selection ability. In summary, the stable performance and excellent accuracy of this invention on different datasets fully verify its powerful capabilities in image recognition and segmentation tasks, demonstrating its efficiency and reliability under high feature dimensionality. This makes the present invention an efficient and stable solution for dealing with face recognition and segmentation problems.

[0174] Example 4

[0175] The data denoising method based on DNN two-layer optimization of the present invention specifically includes the following operation steps:

[0176] Step 1: Obtain speech recognition information from the UCI database and normalize the speech recognition information to obtain initial data;

[0177] Step 2: Divide the initial data into a training set, a validation set, and a test set in a 6:2:2 ratio;

[0178] Step 3: Construct a two-layer network model based on the initial data and the DNN neural network;

[0179] The two-layer network model consists of two layers: the upper layer includes the upper objective function and the asymmetric competitive genetic algorithm; the lower layer includes the lower objective function and the Adam classification algorithm.

[0180] The upper-level objective function consists of three parts, the first of which is the validation set error rate:

[0181]

[0182] In equation (1), F1(Θ,Ω,W) * ) represents the validation set error rate, with values ​​in the range [0, 1]; N ve N represents the number of misclassified samples in the validation set. vsRepresents the number of validation set samples; Θ represents feature selection encoding; Ω represents the network structure; W* represents the optimal network parameters;

[0183] The second part is the feature selection ratio:

[0184]

[0185] In equation (2), F2(Θ) represents the feature selection ratio, with a value of [0, 1]; Θ represents the feature selection encoding, θ i =1 indicates that the i-th feature is selected, and D represents the total length of the feature selection code;

[0186] The third part is the network complexity metric:

[0187]

[0188] In equations (3) and (4), F3(Θ, Ω) represents the network complexity index, with a value of [0, 1]; N tc N represents the total number of network connections. ac This represents the actual number of network connections. I represents the number of input nodes in the DNN; K represents the number of output nodes in the DNN; L represents the upper bound of the preset number of hidden layers in the model; δ max This represents the maximum preset number of nodes in all hidden layers;

[0189] The asymmetric competitive genetic algorithm optimizes the upper-level objective function:

[0190]

[0191] Equation (5), F u Let λ1, λ2, and λ3 represent the upper-level objective function, respectively, and let F1(Θ, Ω, W). * The contributions of F2(Θ) and F3(Θ,Ω) to the upper objective function are 0.98, 0.01, and 0.01, respectively.

[0192] In the lower layer, the Adam algorithm is used to optimize the cross-entropy loss function. The objective function of the lower layer is:

[0193]

[0194] In equation (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0195] Step 4: Input the training set into the two-layer network model for training to obtain the optimized network model;

[0196] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0197] Step 6: Input the test set into the optimal network model for testing and obtain the test results.

[0198] The steps for determining the optimal network parameters in step 4 are as follows: Input the training set into the two-layer network model, obtain the initial feature subset and initial network structure through the upper-layer objective function, and optimize the initial feature subset and initial network structure using an asymmetric genetic algorithm; input the optimized initial feature subset and initial network structure into the lower-layer Adam algorithm for further optimization to obtain the network parameters; input the network parameters into the lower-layer objective function to determine whether the lower-layer objective function converges; if the lower-layer objective function converges, the optimal network parameters are obtained; if the lower-layer objective function does not converge, repeat this step.

[0199] The formula for calculating the optimal network parameters is:

[0200] W * =arg W min{f l (Θ,Ω,W)} (7);

[0201] In equation (7), W takes the value [-1, 1]. N .

[0202] The specific steps for validating the optimal network model in step 5 are as follows:

[0203] Step 5.1: Input the validation set into the two-layer network model with optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper objective function;

[0204] Step 5.2: Input the optimized network structure and optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm has converged;

[0205] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0206] The test results were compared with the face recognition information processed by CCSO, CUS-SPSO, bAAAs1, and SaWDE, respectively, resulting in Table 4:

[0207] Table 4 Comparison Results of Noise Reduction for Speech Recognition Information in Example 4

[0208]

[0209] In the field of speech recognition, based on the tabular data, the model of this invention demonstrates significant performance advantages. Specifically, the performance of this invention was compared with four state-of-the-art methods on nine text recognition datasets, with a total of 14 sets of comparative experiments. Experimental results show that although this invention is slightly inferior to the CCSO method on the Wall dataset, it outperforms the comparative methods on other datasets, with varying degrees of accuracy improvement. In particular, this invention outperforms other methods in the number of selected features. In the nine sets of experiments, this invention generally selects fewer features, while its accuracy is improved in most experiments. This indicates that this invention can maintain high classification accuracy while reducing feature dimensionality, demonstrating its excellent stability and efficient feature selection ability in text recognition tasks. These results fully verify the stable performance of this invention on different datasets, showing that the model can not only effectively reduce the number of features but also maintain high accuracy with fewer features. This characteristic makes this invention an efficient and reliable solution in the field of text recognition, highlighting its advantages in practical applications.

Claims

1. A data denoising method based on DNN double-layer optimization, characterized in that, Specifically comprising the following steps: Step 1, obtaining a data set and normalizing the data set to obtain initial data, the data set comprising biological information, face recognition information, voice recognition information and medical diagnosis information; Step 2, dividing the initial data into a training set, a validation set and a test set using a hierarchical random algorithm; Step 3, constructing a double-layer network model based on the initial data and a DNN neural network; Step 4, inputting the training set into the double-layer network model for training to obtain an optimized network model; Step 5, inputting the validation set into the optimized network model for validation to obtain an optimal network model; Step 6, inputting the test set into the optimal network model for testing to obtain a test result; The double-layer network model in step 3 is divided into an upper layer and a lower layer, the upper layer comprising an upper layer objective function and an asymmetric competitive genetic algorithm; the lower layer comprising a lower layer objective function and an Adam algorithm; The upper layer objective function is composed of three parts, the first part being the error rate of the validation set: （1）； In formula (1), denotes the validation set error rate, and takes a value in [0, 1]; denotes the number of samples with classification errors in the validation set; denotes the number of samples in the validation set; denotes feature selection encoding; Ω denotes network structure; and W* denotes optimal network parameters; The second part is the feature selection ratio: （2）； In formula (2), represents the feature selection ratio, and takes a value of [0, 1]; represents the feature selection encoding, represents that the ith feature is selected, and D represents the total length of the feature selection encoding; The third part is the network complexity index: （3）； （4）； in formula (3) and formula (4), denotes a network complexity index, and takes a value of [0, 1]; denotes a total number of network connections, denotes an actual number of network connections; denotes a number of input nodes of the DNN; denotes a number of output nodes of the DNN; denotes an upper bound of a preset number of network hidden layers in the model; denotes a preset maximum number of nodes in all hidden layers; The asymmetric competitive genetic algorithm optimizes the upper layer objective function: （5）； Formula (5), denotes an upper-layer objective function, respectively denote , , a contribution degree to the upper-layer objective function; The lower layer objective function is: （6）； In formula (6), N represents the number of training samples; O represents the network output of the training sample; and T represents the label of the training sample.

2. The DNN-based double-layer optimization data denoising method according to claim 1, wherein, The data set in step 1 is obtained from the UCI database, and the data set contains multiple feature subsets.

3. The DNN-based double-layer optimization data denoising method according to claim 2, characterized in that, The training steps of the optimized network model in step 4 are as follows: inputting the training set into the double-layer network model, obtaining an initial feature subset and an initial network structure through the upper layer objective function, optimizing the initial feature subset and the initial network structure through the asymmetric genetic algorithm; inputting the optimized initial feature subset and the initial network structure into the Adam algorithm in the lower layer for further optimization to obtain network parameters, inputting the network parameters into the lower layer objective function to determine whether the lower layer objective function converges, if the lower layer objective function converges, obtaining the optimal network parameters, if the lower layer objective function does not converge, repeating the step.

4. The DNN-based double-layer optimization data denoising method according to claim 3, characterized in that, The calculation formula of the optimal network parameters is: （7）； In formula (7), W has a value of .

5. The DNN-based double-layer optimization data denoising method according to claim 4, characterized in that, The validation steps of the optimal network model in step 5 are as follows: Step 5.1, inputting the validation set into the optimized network model, calculating the optimized network structure and the optimized feature subset through the upper layer objective function; Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges; Step 5.3, if the asymmetric genetic algorithm converges in step 5.2, obtaining the optimal network model; if the asymmetric genetic algorithm does not converge, inputting the optimized network structure and the optimized feature subset into the Adam algorithm to calculate the optimal network parameters, and repeating steps 5.1-5.3.

Images