Data denoising method based on DNN double-layer optimization

Abstract

A data denoising method based on DNN double-layer optimization specifically comprises the following operation steps: obtaining a data set and carrying out normalization processing on the data set to obtain initial data; dividing the initial data into a training set, a verification set and a test set by using a hierarchical random algorithm; constructing a double-layer network model based on the initial data and a DNN neural network; inputting the training set into a double-layer network model for training to obtain an optimized network model; inputting the verification set into the optimized network model for verification to obtain an optimal network model; inputting the test set into the optimal network model for testing to obtain a test result; the problem that in the prior art, a large amount of redundant information and interference information exist in high-dimensional data, and the accuracy and reliability of the data are affected is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a data denoising method based on DNN double-layer optimization. Background Art

[0002] Classification tasks are one of the main branches of machine learning. The classification performance of the model depends on the quality of the data features and the classifier. The performance of the classifier is positively correlated with the data quality. Higher-quality features can often provide more effective information for the classifier, thereby achieving better classification performance. However, with the development of big data and artificial intelligence, more and more data are generated, such as biological gene information, protein information, signal data, etc. The dimensions of these data are getting higher and higher, and can even reach tens of thousands of dimensions. These features include high-quality features associated with class labels, redundant features independent of class labels, and even some features containing wrong information. These redundant features and wrong information pose great challenges to the performance of the classifier. In addition, the time and hardware cost of training efficient classifiers grow exponentially with the number of features, because processing high-dimensional features requires more floating-point operations. Therefore, feature selection is very meaningful. Feature selection can select relevant or complementary features, remove redundant features, and form a new feature subset to improve the quality of the data set. Feature selection includes two alternating steps: selecting a feature subset through a search mechanism, and designing indicators to evaluate the performance of the subset.

[0003] Although feature selection can screen out high-quality feature subsets to reduce the time cost and hardware cost of the model. However, the search space of high-dimensional data is large, which brings great challenges to searching for high-quality feature subsets. Therefore, an effective search mechanism is very necessary. Evolutionary computation is widely used in feature selection due to its potential global search capability. In evolutionary computation, particle swarm optimization (PSO) and genetic algorithm (GA) have been widely used in feature selection. Competitive Swarm Optimizer (CSO) is a variant of PSO proposed by Cheng et al. Studies have shown that CSO has better population diversity and better global search capability than standard PSO. Therefore, CSO is more suitable for feature selection problems with larger search space. Gu et al. proposed a feature selection model based on CSO and proved that its performance is better than standard PSO. The main idea of ​​CSO is to randomly select two solutions and compare their fitness, and the worse solution learns from the better solution. But the better solution is not necessarily a feasible solution for feature selection. To solve this problem, Nguyen et al. proposed a constrained competitive optimizer (CCSO). The main idea of ​​CCSO is that worse solutions learn from feasible solutions to avoid creating worse solutions. Experimental results show that the experimental performance of CCSO is better than that of standard CSO. Maleki et al. proposed a feature selection model based on genetic algorithm and verified its effectiveness on lung cancer dataset through KNN classifier. Li et al. proposed a feature selection model based on genetic algorithm (WKNNGAFS). WKNNGAFS combines genetic algorithm and weighted KNN (WKNN) algorithm to calculate the score of each feature. The algorithm performs well on 13 data sets. Based on the advantages of evolutionary computing algorithms in feature selection, the present invention selects genetic algorithm as the search mechanism for feature selection.

[0004] DNN has attracted extensive attention due to its global information extraction capability. However, the performance of DNN depends largely on its structure, connection parameters and activation function. DNN optimization, which involves designing network structure, connection parameters and activation function, is usually a combinatorial optimization problem, so it brings great challenges to designers. In recent years, evolutionary computation has been widely used in DNN optimization due to its powerful search capability and the lack of differentiability of the objective function. Du et al. proposed an adaptive differential evolution algorithm combining non-dominant dominated sorting (NSJADE) and an improved version based on the knee joint strategy (KP-NSADE) to optimize the single hidden layer feedforward neural network (SLFN) model for time series prediction. The three objective functions of error rate on the validation set, L2 norm of network output and number of nodes in the hidden layer are used as evaluation indicators of network performance. Experiments show that NSJADE is more suitable for multimodal problems than the standard NSGAII. Bai et al. improved the genetic algorithm and proposed a collaborative genetic algorithm CGA-ELM based on extreme learning machine. CGA-ELM uses the error rate of the validation set and network complexity as methods to evaluate network performance. Binary coding is used to optimize the number of hidden layer nodes of SLFN, real coding is used to optimize the input weights of SLFN, and the pseudo-inverse method is used to calculate the output of the network. Studies have shown that CGA-ELM can guarantee the classification performance of SLFN and obtain a more compact network structure. Li et al. proposed a model based on double-layer learning to optimize DNN. The upper layer uses the binary particle swarm optimization algorithm (BPSO) to optimize the structure of the network, and the lower layer uses the Levenberg-Marquardt (LM) algorithm to optimize the connection parameters of the network. The optimization of the upper and lower layers is performed alternately to produce a good-performance and compact DNN. For the optimization of DNN, most inventions only design the number of network layers and the number of nodes in each layer, ignoring the connection mode and activation function of the network. A reasonable topological structure can realize feature reuse and improve the performance of DNN. Therefore, the search space of the present invention includes a splicing scheme to achieve feature reuse.

[0005] Evolutionary computation has made great achievements in the field of feature selection and DNN optimization. Many feature selection inventions based on evolutionary computation usually use existing classifiers, such as SVM, KNN, DNN, etc., but ignore the problem that the structure and parameters of the classification also need to be adjusted. Summary of the invention

[0006] The purpose of the present invention is to provide a data denoising method based on DNN double-layer optimization, which solves the problem that a large amount of redundant information and interference information in the data in the prior art affects the accuracy and reliability of the data.

[0007] The technical solution adopted by the present invention is a data denoising method based on DNN double-layer optimization, which specifically includes the following steps:

[0008] Step 1: Obtain a data set and normalize the data set to obtain initial data;

[0009] Step 2: Use the stratified random algorithm to divide the initial data into training set, validation set and test set;

[0010] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0011] Step 4: Input the training set into the two-layer network model for training to obtain an optimized network model;

[0012] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0013] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0014] The present invention is also characterized in that:

[0015] In step 1, the data set is obtained from the UCI database. The data set contains multiple feature subsets, including biological information, face recognition information, speech recognition information, and medical diagnosis information.

[0016] In step 3, the two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper layer objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower layer objective function and an Adam algorithm.

[0017] The upper objective function consists of three parts. The first part is the validation set error rate:

[0018]

[0019] In formula (1), F1(Θ,Ω,W * ) represents the validation set error rate, and its value is [0, 1]; N ve Indicates the number of samples misclassified in the validation set; N vs represents the number of samples in the validation set; Θ represents the feature selection code; Ω represents the network structure; W* represents the optimal network parameters;

[0020] The second part is the feature selection ratio:

[0021]

[0022] In formula (2), F2(θ) represents the feature selection ratio, which takes a value of [0, 1]; θ represents the feature selection code, θ i =1 means that the i-th feature is selected, and D represents the total length of the feature selection code;

[0023] The third part is the network complexity index:

[0024]

[0025] In formula (3) and formula (4), F3(Θ,Ω) represents the network complexity index, and its value is [0, 1]; N tc Represents the total number of network connections, N ac Indicates the actual number of network connections. I represents the number of input nodes of DNN; K represents the number of output nodes of DNN; L represents the upper bound of the number of network hidden layers preset in the model; δ max Indicates the preset maximum number of nodes in all hidden layers;

[0026] The asymmetric competitive genetic algorithm optimizes the upper objective function:

[0027]

[0028] Formula (5), F u represents the upper layer objective function, λ1,λ2,λ3 represent F1(Θ,Ω,W * ), F2(Θ), F3(Θ,Ω) contribution to the upper objective function;

[0029] The lower objective function is:

[0030]

[0031] In formula (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0032] The training steps of the optimized network model in step 4 are as follows: input the training set into the two-layer network model, obtain the initial feature subset and the initial network structure through the upper layer objective function, and optimize the initial feature subset and the initial network structure through the asymmetric genetic algorithm; input the optimized initial feature subset and the initial network structure into the lower layer Adam algorithm for further optimization to obtain the network parameters, and input the network parameters into the lower layer objective function to determine whether the lower layer objective function converges. If the lower layer objective function converges, the optimal network parameters are obtained. If the lower layer objective function does not converge, repeat this step.

[0033] The calculation formula for the optimal network parameters is:

[0034] W * =arg W min{f l (Θ,Ω,W)} (7)

[0035] In formula (7), W is [-1, 1] N .

[0036] The verification steps of the optimal network model in step 5 are as follows:

[0037] Step 5.1: Input the validation set into the two-layer network model with the optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper layer objective function;

[0038] Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges;

[0039] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and the optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0040] The beneficial effects of the present invention are as follows: the data denoising method based on DNN double-layer optimization proposed by the present invention, the DNN double-layer network is a double-layer learning model, the upper-layer ACGA algorithm performs feature selection on the initialization data and constructs the DNN structure through the upper-layer objective function; the lower-layer Adam algorithm optimizes the network parameters through the cross-entropy loss function to determine the optimal network parameters and the optimal network model; the splicing operation is introduced in the search space to realize feature reuse; the splicing operation can bring more complexity to the search space and generate more feasible solutions, and the lower-layer Adam algorithm optimizes the network parameters through the loss function, thereby improving the accuracy of the data processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the data denoising method based on DNN double-layer optimization of the present invention;

[0042] Figure 2 It is a flowchart of encoding and decoding in the data denoising method based on DNN double-layer optimization of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The data denoising method based on DNN double-layer optimization of the present invention is as follows: Figure 1 As shown, the specific steps include the following:

[0045] Step 1: Obtain a data set and normalize the data set to obtain initial data;

[0046] Step 2: Use the stratified random algorithm to divide the initial data into training set, validation set and test set;

[0047] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0048] Step 4: Input the training set into the two-layer network model for training to obtain an optimized network model;

[0049] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0050] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0051] In step 1, the data set is obtained from the UCI database. The data set contains multiple feature subsets, including biological information, face recognition information, speech recognition information, and medical diagnosis information.

[0052] In step 3, the two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper layer objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower layer objective function and an Adam algorithm.

[0053] The upper objective function consists of three parts. The first part is the validation set error rate:

[0054]

[0055] In formula (1), F1(θ,2,W*) represents the validation set error rate, which takes the value of [0, 1]; N ve Indicates the number of samples misclassified in the validation set; N vs represents the number of samples in the validation set; Θ represents the feature selection code; Ω represents the network structure; W* represents the optimal network parameters;

[0056] The second part is the feature selection ratio:

[0057]

[0058] In formula (2), F2(θ) represents the feature selection ratio, which takes a value of [0, 1]; θ represents the feature selection code, θ i =1 means that the i-th feature is selected, and D represents the total length of the feature selection code;

[0059] The third part is the network complexity index:

[0060]

[0061] In formula (3) and formula (4), F3(Θ,Ω) represents the network complexity index, and its value is [0, 1]; N tc Represents the total number of network connections, N ac Indicates the actual number of network connections. I represents the number of input nodes of DNN; K represents the number of output nodes of DNN; L represents the upper bound of the number of network hidden layers preset in the model; δ maxIndicates the preset maximum number of nodes in all hidden layers;

[0062] The asymmetric competitive genetic algorithm optimizes the upper objective function:

[0063]

[0064] Formula (5), F u represents the upper layer objective function, λ1,λ2,λ3 represent F1(Θ,Ω,W * ), F2(Θ), F3(Θ,Ω) contribution to the upper objective function;

[0065] The lower objective function is:

[0066]

[0067] In formula (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0068] The training steps of the optimized network model in step 4 are as follows: input the training set into the two-layer network model, obtain the initial feature subset and the initial network structure through the upper layer objective function, and optimize the initial feature subset and the initial network structure through the asymmetric genetic algorithm; input the optimized initial feature subset and the initial network structure into the lower layer Adam algorithm for further optimization to obtain the network parameters, and input the network parameters into the lower layer objective function to determine whether the lower layer objective function converges. If the lower layer objective function converges, the optimal network parameters are obtained. If the lower layer objective function does not converge, repeat this step.

[0069] The calculation formula for the optimal network parameters is:

[0070] W * =arg W min{f l (Θ,Ω,W)} (7)

[0071] In formula (7), W is [-1, 1] N .

[0072] The verification steps of the optimal network model in step 5 are as follows:

[0073] Step 5.1: Input the validation set into the two-layer network model with the optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper layer objective function;

[0074] Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges;

[0075] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and the optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0076] Binary encoding is performed on the feature subset in the training set; the feature subset selection encoding is θ = (θ1, θ2, ..., θ D ), where θ i =1 means the i-th feature is selected, θ i =0 means that the i-th feature is not selected, and D represents the total length of the code;

[0077] The two-layer network model is mixed with binary and integer encoding: the mixed encoding is Ω = (α, β, γ, δ), where represents the binary encoding of feature selection, α i ={0,1}, i=1,2,...,S1; is the binary code representing the connection mode of each hidden layer, β j ={0,1}, j=1,2,…,S2; is the binary code representing the type of activation function for each hidden layer, γ k ={0,1},k=1,2,…,S3; δ=(δ1,δ2,…δ L ) is an integer code representing the number of nodes in each hidden layer, δ l =[δ min ,δ max ],l=1,2,…,L;δ max Indicates the preset maximum number of nodes in all hidden layers; δ min represents the minimum number of nodes preset in all hidden layers; L represents the upper limit of the number of network hidden layers preset in the model, such as Figure 2 shown.

[0078] There is an activation function in each hidden layer, which can be any one of relu, identity, sigmoid or tanh.

[0079] The calculation formula for the optimal network parameters in step 4 is:

[0080] W * =arg W min{f l (Θ,Ω,W)} (7)

[0081] In formula (7), W is [-1, 1] N .

[0082] The steps to determine the optimal network parameters in step 4 are:

[0083] The training set is input into the two-layer network model, and the initial feature subset and initial network structure are obtained through the upper layer objective function. The initial feature subset and initial network structure are optimized by the asymmetric genetic algorithm. The optimized initial feature subset and initial network structure are input into the lower layer Adam algorithm for further optimization to obtain network parameters. The network parameters are input into the lower layer objective function to determine whether the lower layer objective function converges. If the lower layer objective function converges, the optimal network parameters are obtained. If the lower layer objective function does not converge, repeat this step.

[0084] The verification steps of the optimal network model in step 5 are as follows:

[0085] Step 5.1: Input the validation set into the two-layer network model with the optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper layer objective function;

[0086] Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges;

[0087] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and the optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0088] After the test is completed, the test result is decoded to obtain the test result. The decoding process is as follows:

[0089] Binary code θ = (θ1, θ2, ..., θ D ), if θ i =1, it means that the i-th feature is selected; otherwise, the i-th feature is not selected; the number of selected feature subsets is equal to It is equal to the number of input nodes I, i.e. For binary encoding The number of hidden layers is

[0090] Binary coded The network connection mode is decoded as follows: β1 represents the connection relationship between the second hidden layer and the input layer; if β1=1, it means that the second hidden layer is connected to the input layer, otherwise it means no connection; (β2,β3) represents the connection status of the third hidden layer with the input layer and the first hidden layer; if (β2,β3)=(1,1), it means that the third hidden layer is connected to the input layer and the first hidden layer; (β2,β3)=(0,0) means that the third hidden layer is not connected to the input layer and the first hidden layer; (β2,β3)=(1,0) means that the third hidden layer is connected to the input layer but not to the first hidden layer; (β2,β3)=(0,1) means that the third hidden layer is connected to the first hidden layer but not to the input layer; (β4,β5,β6) represents the connection status of the fourth layer with the input layer, the first layer, and the second layer. If (β4,β5,β6)=(1,1,1) means that the fourth hidden layer connects the input layer, the first hidden layer and the second hidden layer; (β4,β5,β6)=(0,0,0) means that the fourth hidden layer is not connected to the input layer, the first hidden layer and the second hidden layer; (β4,β5,β6)=(1,1,0) means that the fourth hidden layer connects the input layer and the first hidden layer, but not the second hidden layer, and so on, until the connection state of the first layer and the previous layer from the first layer to the (L-2) layer is decoded; that is:

[0091]

[0092] Binary coded (γ1,γ2) represents the activation function used in the first hidden layer, which is decoded as (γ3,γ4) represents the activation function used in the second hidden layer, which is decoded as And so on; S3 = 2 × L, N ai =0, 1, 2, 3 means selecting relu, identity, sigmoid or tanh as the activation function of the hidden layer respectively.

[0093] Among them, the asymmetric competitive genetic algorithm (ACGA) mainly includes three parts: crossover operation, mutation operation and elite solution selection operation, specifically:

[0094] Crossover operation: Each individual X has a longer encoding string with a length of D+S1+S2+S3+L; L represents the upper limit of the number of network hidden layers preset in the model; S1, S2, S3 represent the α, β, γ encoding lengths respectively; the present invention adopts the commonly used two-point crossover method with probability p c Choose a pair of parents; each generation has a population, the previous generation population is called the parents or parents, and is represented by X. i and X j Indicates that i≠j, i, j=1,2,…,N P, randomly select X i and X j The two crossover sites in i The gene between the two crossover sites is replaced by X j Genes between the two crossover sites in the i sub-gene.

[0095] Mutation operation based on asymmetric learning strategy and ReliefF score: Each mutant individual Vi (i = 1, 2, ..., N P ) Individual E randomly selected from the elite set k (k=1,2,…,10%N P ), and then use the ReliefF score to correct the binary encoding of the feature subset. In the first stage of the mutation operation, the strategy of new individuals learning from elite individuals is adopted. For binary encoding, U i Different from E k The mutation site has a greater flip probability. For integer encoding, U i Each mutation site has a greater probability of mutating to U i and E k A random integer between two integers U. i The binary encoding of E k The binary coded difference is represented as V diff :

[0096] V diff,j =|U i,j -E k,j | j=1,2,…,N B (9)

[0097] In formula (9), N B Indicates the length of the binary code, specifically N B =D+S1+S2+S3, p i,j Indicates U i The probability that the jth site of is a mutation site is:

[0098]

[0099] In formula (13), p a Indicates the preset probability of asymmetric mutation.

[0100] V i The binary encoding is:

[0101]

[0102] In formula (11), j = 1, 2, ... N B,rand(j) represents a random number in the jth interval (0,1).

[0103] V i The integer encoding of is:

[0104]

[0105] In formula (12), represents the rounding function, j = N B +1,N B +2,…N B +L, rand(j) represents any value in the jth interval (0,1).

[0106] In the second stage of the mutation operation, only the encoding of the feature subset is operated, not the encoding of the network structure. Some features will be selectively added and deleted in the encoding of each individual's feature subset based on the ReliefF score and the roulette wheel method. Features with high ReliefF scores are considered to be highly relevant and are more likely to be added to the feature subset. The probability of each feature being added to the feature subset is calculated using the following formula:

[0107]

[0108] In formula (13), Rf j ReliefF score of the jth feature;

[0109] If the selected feature with a high ReliefF score already exists in the feature subset, the corresponding encoding of the feature does not need to be mutated. For features with lower ReliefF scores, they are less relevant to the label, so they are more likely to be deleted. The probability of each feature being deleted is 1-p f,j . Use the roulette wheel method to remove features with low ReliefF scores.

[0110] Selection strategy: For FS and DNN optimization based on genetic algorithms, common selection strategies are elite selection strategy and roulette selection strategy. For the roulette selection strategy, the probability of each individual being selected is equal to the ratio of its fitness value to the sum of the fitness values ​​in the entire population. However, excellent individuals in each generation are likely to be eliminated, which is unacceptable for DNN structure optimization and FS with large search space and expensive computational cost. For DNN optimization, an individual with high fitness can only indicate that the individual performs well on the validation set, but not necessarily on the test set. Retaining only the best individual of each generation is likely to produce a solution that performs well on the validation set but not on the test set. In order to solve these problems, the present invention adopts a hybrid selection strategy. The parent individuals, crossover individuals and mutant individuals are merged, and the merged individuals are arranged in ascending order of their upper target values. The top 10% of individuals in the population will directly enter the next generation, assuming that they are equally important, and the remaining individuals enter the next generation population through roulette selection.

[0111] Input the training set and calculate the gradient of the network parameters through the upper layer given network structure; calculate the first moment estimate m in the current time step t and the second moment estimate v t ,m t =τ1m t-1 +(1-τ1)d k , Where: m t-1 represents the first-order moment estimate of the t-1th iteration, m t represents the first-order moment estimate of the t-th iteration, τ1 represents the exponential weighting parameter, and d t-1 represents the gradient of the network parameter W at the t-1th iteration, v t-1 represents the second-order moment estimate of the t-1th iteration, v t represents the second-order moment estimate of the t-th iteration, τ2 represents the exponential weighting parameter; represents the square of the gradient of W at the t-1th iteration; the corrected first-order momentum and second-order momentum are and According to the equation Update network parameters, where W t represents the network parameters at the tth iteration, W t-1 Represents the network parameters at the t-1th iteration.

[0112] Example 1

[0113] The data denoising method based on DNN double-layer optimization of the present invention specifically includes the following steps:

[0114] Step 1, obtain biological information from the UCI database and normalize the biological information to obtain initial data;

[0115] Step 2: Divide the initial data into training set, validation set and test set according to the ratio of 6:2:2;

[0116] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0117] The two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper layer objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower layer objective function and an Adam classification algorithm.

[0118] Step 4: input the training set into the double-layer network model for training to obtain an optimized network model;

[0119] Step 5: input the verification set into the optimized network model for verification to obtain the optimal network model;

[0120] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0121] The obtained test results were compared with the biological information processed by constrained competitive optimizer (CCSO), CUS-SPSO, bAAAs1, MSFS, and SLMEA, respectively, and Table 1 was obtained.

[0122] Table 1 Comparison results of biological information denoising in Example 1

[0123]

[0124]

[0125] Through in-depth analysis of the data in the above table, we found that in 11 groups of comparative experiments of bioinformatics data sets, the present invention showed significant feature selection capabilities. Specifically, in 8 groups of experiments, the number of features selected by the network model of the present invention was less than that of other methods, showing extremely high feature screening efficiency. Even more striking is that in four groups of comparative experiments, the number of features selected by the present invention was only half of that of other methods. This result strongly proves the excellent performance of the present invention in streamlining feature sets. Not only that, the prediction accuracy of the present invention in all comparative experimental groups surpassed other methods, fully demonstrating its superior prediction ability. It is particularly worth mentioning that in three groups of experiments, the accuracy of the present invention was improved by more than 30% compared with the original method. This significant improvement further highlights the effectiveness and efficiency of the present invention in the field of bioinformatics feature selection. In summary, the present invention fully verifies its value and potential in practical applications by reducing the number of features and improving the prediction accuracy.

[0126] Example 2

[0127] The data denoising method based on DNN double-layer optimization of the present invention specifically includes the following steps:

[0128] Step 1, obtaining medical diagnosis information from the UCI database and normalizing the medical diagnosis information to obtain initial data; wherein the data set contains multiple feature subsets;

[0129] Step 2: Divide the initial data into training set, validation set and test set according to the ratio of 6:2:2;

[0130] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0131] The two-layer network model is divided into an upper layer and a lower layer. The upper layer includes an upper layer objective function and an asymmetric competitive genetic algorithm; the lower layer includes a lower layer objective function and an Adam classification algorithm.

[0132] The upper objective function consists of three parts. The first part is the validation set error rate:

[0133]

[0134] In formula (1), F1(Θ,Ω,W * ) represents the validation set error rate, and its value is [0, 1]; N ve Indicates the number of samples misclassified in the validation set; N vs represents the number of samples in the validation set; Θ represents the feature selection code; Ω represents the network structure; W* represents the optimal network parameters;

[0135] The second part is the feature selection ratio:

[0136]

[0137] In formula (2), F2(θ) represents the feature selection ratio, which takes a value of [0, 1]; θ represents the feature selection code, θ i =1 means that the i-th feature is selected, and D represents the total length of the feature selection code;

[0138] The third part is the network complexity index:

[0139]

[0140] In formula (3) and formula (4), F3(Θ,Ω) represents the network complexity index, and its value is [0, 1]; N tc Represents the total number of network connections, N ac Indicates the actual number of network connections. I represents the number of input nodes of DNN; K represents the number of output nodes of DNN; L represents the upper bound of the number of network hidden layers preset in the model; δ maxIndicates the preset maximum number of nodes in all hidden layers;

[0141] The asymmetric competitive genetic algorithm optimizes the upper objective function:

[0142]

[0143] Formula (5), F u represents the upper objective function, λ1,λ2,λ3 represent the contribution of F1(Θ,Ω,W*), F2(Θ), F3(Θ,Ω) to the upper objective function, and their values ​​are 0.98, 0.01, 0.01 respectively;

[0144] The lower objective function is:

[0145]

[0146] In formula (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0147] Step 4: Input the training set into the two-layer network model for training to obtain an optimized network model;

[0148] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0149] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0150] The obtained test results were compared with the medical diagnosis information processed by CUS-SPSO, bAAAs1, MSFS, and SaWDE, respectively, and Table 2 was obtained.

[0151] Table 2 Comparison results of medical diagnosis information denoising in Example 2

[0152]

[0153] In the context of medical diagnosis, accurate and efficient feature selection is the key to achieving high-quality diagnosis. Feature selection can not only improve the classification performance of the model, but also reduce computational complexity and avoid overfitting. In this field, the model of the present invention shows significant advantages. Specifically, the performance of the present invention on seven sets of medical diagnosis data sets is better than that of four advanced models, and 12 sets of comparative experiments are conducted. The results show that its feature selection ability and classification performance are both outstanding.

[0154] From the experimental data, the number of features selected in the six groups of experiments of the present invention is significantly better than other comparison models, and the accuracy in the nine groups of experiments is also higher than other methods. In particular, on the PGE, Leuk and Lung2 data sets, the present invention is significantly better than CUS-SPSO in feature selection, and the classification performance is also mostly better than the existing models. For example, on the PGE data set, the accuracy of the present invention is improved by 12.64%, which shows that it performs well in processing complex data, can maintain a high accuracy under more features, and reflects its good stability and feature selection ability. In addition, the present invention achieves different degrees of accuracy improvement on multiple data sets, among which the accuracy is 34.02% higher than SaWDE on the Lung1 data set. These results show that the model of the present invention not only performs well in reducing the number of features, but also simplifies the model while maintaining a high classification accuracy, reduces the computational burden, and improves diagnostic efficiency. In general, the present invention provides an efficient and reliable solution for medical diagnosis applications by optimizing feature selection and model simplicity, which helps to achieve higher accuracy and efficiency in practical applications.

[0155] Example 3

[0156] The data denoising method based on DNN double-layer optimization of the present invention specifically includes the following steps:

[0157] Step 1: Obtain face recognition information from the UCI database and normalize the face recognition information to obtain initial data;

[0158] Step 2: Divide the initial data into training set, validation set and test set according to the ratio of 6:2:2;

[0159] Step 3: Build a two-layer network model based on the DNN neural network;

[0160] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0161] Step 4: Input the training set into the two-layer network model for training to obtain an optimized network model;

[0162] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0163] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0164] The two-layer network model is divided into two layers: the upper layer and the lower layer. The upper layer includes the upper layer objective function and the asymmetric competitive genetic algorithm; the lower layer includes the lower layer objective function, the loss function and the Adam classification algorithm.

[0165] Step 4: The training steps of the optimized network model are as follows: inputting the training set into the double-layer network model, obtaining the initial feature subset and the initial network structure through the upper-layer objective function, and optimizing the initial feature subset and the initial network structure through the asymmetric genetic algorithm; inputting the optimized initial feature subset and the initial network structure into the lower-layer Adam algorithm for further optimization to obtain network parameters, and inputting the network parameters into the lower-layer objective function to determine whether the lower-layer objective function converges. If the lower-layer objective function converges, the optimal network parameters are obtained. If the lower-layer objective function does not converge, repeat this step.

[0166] The calculation formula for the optimal network parameters in the lower layer is:

[0167] W * =arg W min{f l (Θ,Ω,W)} (7)

[0168] In formula (7), W is [-1, 1] N .

[0169] The obtained test results are compared with the face recognition information processed by CCSO, SLMEA, CUS-SPSO, bAAAs1, and SaWDE, and Table 3 is obtained.

[0170] Table 3 Comparison results of face recognition information denoising in Example 3

[0171]

[0172]

[0173] In the field of face recognition and segmentation, according to the tabular data, the model of the present invention shows significant advantages. Specifically, on 11 sets of image recognition data sets, the model of the present invention was compared with 5 advanced methods, and a total of 14 sets of comparative experiments were conducted. The experimental results show that, except for the comparison with the CCSO method on the Wall data set and the comparison with the bAAAs1 method on the OR data set, the accuracy of the model of the present invention is better than the comparative method in other experiments. In particular, in the comparison with the SLMEA method on the USPS data set and the comparison with the SLMEA method on the PIE10P data set, the accuracy of the present invention has increased by more than 30%. In terms of feature selection, the number of features selected by the present invention in 9 sets of experiments is less than that of other methods. These conclusions fully verify the stable performance of the present invention on different data sets, and can maintain a high accuracy rate in the case of fewer features, reflecting its excellent stability and feature selection ability. In summary, the stable performance and excellent accuracy of the present invention on different data sets fully verify its powerful ability in image recognition and segmentation tasks, and demonstrate its high efficiency and reliability under high feature dimensions. This makes the present invention an efficient and stable solution for dealing with face recognition and segmentation problems.

[0174] Example 4

[0175] The data denoising method based on DNN double-layer optimization of the present invention specifically includes the following steps:

[0176] Step 1, obtaining speech recognition information from the UCI database and normalizing the speech recognition information to obtain initial data;

[0177] Step 2: Divide the initial data into training set, validation set and test set according to the ratio of 6:2:2;

[0178] Step 3: Build a two-layer network model based on the initial data and the DNN neural network;

[0179] The two-layer network model is divided into two layers, the upper layer includes the upper layer objective function and the asymmetric competitive genetic algorithm; the lower layer includes the lower layer objective function and the Adam classification algorithm.

[0180] The upper objective function consists of three parts. The first part is the validation set error rate:

[0181]

[0182] In formula (1), F1(Θ,Ω,W * ) represents the validation set error rate, and its value is [0, 1]; N ve Indicates the number of samples misclassified in the validation set; N vsrepresents the number of samples in the validation set; Θ represents the feature selection code; Ω represents the network structure; W* represents the optimal network parameters;

[0183] The second part is the feature selection ratio:

[0184]

[0185] In formula (2), F2(θ) represents the feature selection ratio, which takes a value of [0, 1]; θ represents the feature selection code, θ i =1 means that the i-th feature is selected, and D represents the total length of the feature selection code;

[0186] The third part is the network complexity index:

[0187]

[0188] In formula (3) and formula (4), F3(Θ,Ω) represents the network complexity index, and its value is [0, 1]; N tc Represents the total number of network connections, N ac Indicates the actual number of network connections. I represents the number of input nodes of DNN; K represents the number of output nodes of DNN; L represents the upper bound of the number of network hidden layers preset in the model; δ max Indicates the preset maximum number of nodes in all hidden layers;

[0189] The asymmetric competitive genetic algorithm optimizes the upper objective function:

[0190]

[0191] Formula (5), F u represents the upper layer objective function, λ1,λ2,λ3 represent F1(Θ,Ω,W * ), F2(Θ), and F3(Θ,Ω) contribute to the upper layer objective function, and their contributions are 0.98, 0.01, and 0.01 respectively;

[0192] In the lower layer, the Adam algorithm is used to optimize the cross entropy loss function, and the lower layer objective function is:

[0193]

[0194] In formula (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

[0195] Step 4: Input the training set into the two-layer network model for training to obtain an optimized network model;

[0196] Step 5: Input the validation set into the optimized network model for validation to obtain the optimal network model;

[0197] Step 6: Input the test set into the optimal network model for testing to obtain the test results.

[0198] The steps for determining the optimal network parameters in step 4 are: inputting the training set into the two-layer network model, obtaining the initial feature subset and the initial network structure through the upper layer objective function, and optimizing the initial feature subset and the initial network structure through the asymmetric genetic algorithm; inputting the optimized initial feature subset and the initial network structure into the lower layer Adam algorithm for further optimization to obtain the network parameters, and inputting the network parameters into the lower layer objective function to determine whether the lower layer objective function converges. If the lower layer objective function converges, the optimal network parameters are obtained. If the lower layer objective function does not converge, repeat this step.

[0199] The calculation formula for the optimal network parameters is:

[0200] W * =arg W min{f l (Θ, Ω, W)} (7);

[0201] In formula (7), W is [-1, 1] N .

[0202] The verification steps of the optimal network model in step 5 are as follows:

[0203] Step 5.1: Input the validation set into the two-layer network model with the optimal network parameters, and obtain the optimized network structure and optimized feature subset through the upper layer objective function;

[0204] Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges;

[0205] Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and the optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

[0206] The obtained test results are compared with the face recognition information processed by CCSO, CUS-SPSO, bAAAs1, and SaWDE, and Table 4 is obtained:

[0207] Table 4 Comparison results of speech recognition information denoising in Example 4

[0208]

[0209] In the field of speech recognition, according to the tabular data, the model of the present invention shows significant performance advantages. Specifically, the present invention has compared the performance with 4 advanced methods on 9 text recognition data sets, and a total of 14 groups of comparative experiments have been conducted. The experimental results show that although the present invention is slightly inferior to the CCSO method on the Wall data set, it performs better than the comparative methods on other data sets, and the accuracy has been improved to varying degrees. In particular, the present invention is superior to other methods in the number of selected features. In the 9 groups of experiments, the number of features selected by the present invention is generally small, and its accuracy has been improved in most experiments. This shows that the present invention can still maintain a high classification accuracy while reducing the feature dimension, demonstrating its excellent stability and efficient feature selection capabilities in text recognition tasks. These results fully verify the stable performance of the present invention on different data sets, indicating that the model can not only effectively reduce the number of features, but also maintain high accuracy with fewer features. This feature makes the present invention an efficient and reliable solution in the field of text recognition, highlighting its advantages in practical applications.

Claims

1. A data denoising method based on DNN double-layer optimization, characterized in that: The specific steps include: Step 1: Obtain a data set and normalize the data set to obtain initial data; Step 2: using a stratified random algorithm to divide the initial data into a training set, a validation set, and a test set; Step 3: construct a double-layer network model based on the initial data and the DNN neural network; Step 4: input the training set into the double-layer network model for training to obtain an optimized network model; Step 5: input the verification set into the optimized network model for verification to obtain the optimal network model; Step 6: Input the test set into the optimal network model for testing to obtain the test results.

2. The data denoising method based on DNN double-layer optimization according to claim 1 is characterized in that: The data set in step 1 is obtained from the UCI database, and the data set contains multiple feature subsets, and the data set includes biological information, face recognition information, speech recognition information and medical diagnosis information.

3. The data denoising method based on DNN double-layer optimization according to claim 2 is characterized in that: The two-layer network model in step 3 is divided into an upper layer and a lower layer, wherein the upper layer includes an upper layer objective function and an asymmetric competitive genetic algorithm; and the lower layer includes a lower layer objective function and an Adam algorithm.

4. The data denoising method based on DNN double-layer optimization according to claim 3 is characterized in that: The upper objective function consists of three parts, the first part is the validation set error rate: In formula (1), F1(θ, Ω, W * ) represents the validation set error rate, and its value is [0, 1]; N ve Indicates the number of samples misclassified in the validation set; N vs represents the number of samples in the validation set; Θ represents the feature selection code; Ω represents the network structure; W* represents the optimal network parameters; The second part is the feature selection ratio: In formula (2), F2(θ) represents the feature selection ratio, which takes a value of [0, 1]; θ represents the feature selection code, θ i =1 means that the i-th feature is selected, and D represents the total length of the feature selection code; The third part is the network complexity index: In formula (3) and formula (4), F3(Θ,Ω) represents the network complexity index, and its value is [0, 1]; N tc Represents the total number of network connections, N ac represents the actual number of network connections; I represents the number of input nodes of DNN; K represents the number of output nodes of DNN; L represents the upper bound of the number of network hidden layers preset in the model; δ max Indicates the preset maximum number of nodes in all hidden layers; The asymmetric competitive genetic algorithm optimizes the upper objective function: Formula (5), F u represents the upper layer objective function, λ1,λ2,λ3 represent F1(Θ,Ω,W * ), F2(Θ), F3(Θ,Ω) contribution to the upper objective function; The lower layer objective function is: In formula (6), P represents the number of training samples; O represents the network output of the training samples; and T represents the label of the training samples.

5. The data denoising method based on DNN double-layer optimization according to claim 4 is characterized in that: The training steps of the optimized network model in step 4 are as follows: inputting the training set into the two-layer network model, obtaining the initial feature subset and the initial network structure through the upper layer objective function, and optimizing the initial feature subset and the initial network structure through the asymmetric genetic algorithm; inputting the optimized initial feature subset and the initial network structure into the lower layer Adam algorithm for further optimization to obtain network parameters, and inputting the network parameters into the lower layer objective function to determine whether the lower layer objective function converges. If the lower layer objective function converges, the optimal network parameters are obtained. If the lower layer objective function does not converge, repeat this step.

6. The data denoising method based on DNN double-layer optimization according to claim 5 is characterized in that: The calculation formula of the optimal network parameters is: W * =arg W min{f l (Θ,Ω,W)} (7); In formula (7), W is [-1, 1] N .

7. The data denoising method based on DNN double-layer optimization according to claim 6 is characterized in that: The verification steps of the optimal network model described in step 5 are: Step 5.1, input the verification set into the optimized network model, and obtain the optimized network structure and optimized feature subset through the upper layer objective function calculation; Step 5.2, inputting the optimized network structure and the optimized feature subset into the asymmetric genetic algorithm to determine whether the asymmetric genetic algorithm converges; Step 5.3: If the asymmetric genetic algorithm in step 5.2 converges, the optimal network model is obtained; if the asymmetric genetic algorithm does not converge, the optimized network structure and the optimized feature subset are input into the Adam algorithm to calculate the optimal network parameters, and steps 5.1 to 5.3 are repeated.

Images