Deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution

Abstract

The invention discloses a deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution. The method comprises the following steps: constructing a deep fuzzy rough neural network; establishing a multi-objective optimization problem for the neural network; designing a large-scale multi-objective evolution algorithm and carrying out iterative evolution on an optimization problem; and finally, substituting the optimal solution into the deep fuzzy rough neural network to obtain an optimal model. In order to solve the problem that convergence and diversity are difficult to balance, an evaluation method based on state information dynamic balance is provided, and the weight of diversity and convergence factors is dynamically adjusted to achieve balance of the diversity and convergence factors. In order to solve the problem of dimension disasters, the invention provides a substitute grouping method, which optimizes a grouping strategy by sorting decision variables according to change values and selectively applying the sorting; in order to accelerate the convergence speed, a self-adaptive factor updating strategy is provided; according to the strategy, the algorithm focuses on promoting convergence in the initial stage of iteration and pays attention to diversity exploration in the later stage of iteration.

Description

Technical Field

[0001] The present invention belongs to the technical field of large-scale multi-objective evolutionary algorithm and deep fuzzy rough neural network parameter and structure optimization, and relates to a prediction method of a deep fuzzy rough neural network based on large-scale multi-objective evolution. Background Art

[0002] As a computational model that simulates biological nervous systems, neural networks have powerful learning and adaptability, but they have limitations when processing fuzzy and incomplete information. Deep Fuzzy Rough Neural Network (DFRNN) has shown significant advantages in dealing with complex environmental data analysis, dynamic prediction, and uncertainty processing. Fuzzy logic was proposed by Zadeh to overcome the limitations of traditional logic in fuzziness and uncertainty, allowing variables to have continuous membership. Rough set theory was proposed by Pawlak, focusing on processing incompleteness and uncertainty in data sets, and mining important features in data through approximate relationships. The theoretical construction of DFRNN is based on these basic theories, aiming to integrate their respective advantages to improve the accuracy and robustness of data processing.

[0003] However, when dealing with fuzzy and uncertain data, DFRNN faces great challenges in parameter optimization due to the complex nonlinear relationships and uncertainty processing involved. In this case, the gradient descent method may have difficulty adapting to fuzziness and uncertainty, resulting in insufficient model generalization and robustness.

[0004] In order to solve the above problems, the present invention adopts a multi-objective evolutionary algorithm to perform neural evolution on DFRNN. Since the parameters and structure of DFRNN are large in scale, it is suitable to use a large-scale multi-objective evolutionary algorithm. The large-scale multi-objective evolutionary algorithm has a variety of search strategies, which can better explore the entire parameter space and help find the global optimal solution. At the same time, its multi-objective optimization capability can take into account the complexity and robustness of the network while improving the accuracy of the model, thereby further enhancing the overall performance of the model. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution. By evolving the parameters and structure of the deep fuzzy rough neural network and setting two objective functions of error and simplicity, the optimal model structure is obtained to improve the prediction performance of DFRNN.

[0006] The technical solution of the present invention:

[0007] A deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution is proposed. The method mainly consists of three parts: first, construct a deep fuzzy rough neural network; second, establish a multi-objective optimization problem for DFRNN; then, design a large-scale multi-objective evolutionary algorithm and iteratively evolve the multi-objective optimization problem; finally, bring the optimal solution into the deep fuzzy rough neural network to obtain the final optimal model.

[0008] (1) Constructing a deep fuzzy rough neural network

[0009] In view of the limitations of traditional neural networks in processing uncertain data, the present invention provides a deep fuzzy rough neural network which is mainly composed of a cascade of multiple fuzzy rough neural network modules. The output of the previous fuzzy rough neural network module serves as the input of the next fuzzy rough neural network module, forming a stacked integrated DFRNN architecture.

[0010] The fuzzy-rough neural network module uses interval type-2 fuzzy sets and has 5 layers: input layer l1, fuzzification layer l2, fuzzy rule layer l3, fuzzy rough layer l4 and output layer l5. l1 receives external input data and fuzzifies it in l2. The input of l3 comes from the output of l2, and each neuron in l3 corresponds to a different fuzzy rule. The output of l3 is input to l4 for rough processing. Finally, in l5, the fuzzy upper and lower limits of the output of l4 are averaged and the result is output.

[0011] 1) Input layer l1:

[0012] The input layer of the neural network contains three neurons, each of which receives an input data as the basis for the calculation of the subsequent layers.

[0013]

[0014] sti=1,2,3,......,I

[0015] in, and They represent the input and output of the input layer, and I is the number of neurons in the input layer. The function of the input layer is to receive data, and the input is the output.

[0016] 2) Fuzzification layer l2:

[0017] The input data is fuzzified using membership functions in the fuzzy layer. Each input data corresponds to a cluster, and each cluster contains multiple fuzzy membership neurons.

[0018]

[0019] stv=1,2,3,...,V

[0020] j=1,2,3,...,J

[0021] in, represents the input of l2, and represents the output of l2, V is the total number of clusters, J is the number of fuzzy membership neurons in each cluster, l and u represent the fuzzy lower limit and fuzzy upper limit, and Gaussian function is used as the membership function, where a and b are the membership function parameters of the fuzzy membership neurons, which are the parameters that need to be optimized.

[0022] 3) Fuzzy rule layer l3:

[0023] Different fuzzy membership neurons are combined to establish fuzzy rules. Therefore, the connection state between the fuzzification layer l2 and the fuzzy rule layer l3 needs to be optimized.

[0024]

[0025] std∈{0,1,2,...,I}

[0026] e∈{1,2,...,J}

[0027] h=1,2,3,...,H

[0028] in, and represents the input of l3, d is the dth cluster in l2, e is the eth neuron in the cluster, and represents the output of l3, and H represents the total number of fuzzy rule neurons.

[0029] 4) Fuzzy rough layer l4:

[0030]

[0031] stc=1,2,3,...,C

[0032] in, and represents the input of l4, and represents the output of l4, C represents the total number of fuzzy rough neurons, λ∈[0,1] is the fuzzy rough membership, ε∈{0,1} is the connection state from the hth neuron of the fuzzy rule layer l3 to the cth fuzzy rough neuron of the fuzzy rough layer l4, ε=1 indicates that there is a connection between the hth neuron and the cth neuron, and ε=0 indicates that there is no connection between the two.

[0033] 5) Output layer l5:

[0034]

[0035] str=1,2,3,...,R

[0036] in, and Represent the input and output of the output layer respectively. In the output layer, the upper and lower limits are averaged to obtain the final result and output it.

[0037] (2) Establishing a multi-objective optimization problem

[0038] In order to optimize the deep fuzzy rough neural network using a multi-objective optimization algorithm, the structure and parameters of the deep fuzzy rough neural network are modeled as a multi-objective optimization problem, and two objective functions, root mean square error and structural complexity, are defined.

[0039] 1) Decision variables

[0040] Introduce the variables that need to be optimized in each layer:

[0041] (①The input layer l1 has no variables that need to be optimized.

[0042] (② The fuzzification layer l2 needs to optimize the two parameters of the membership function a∈[0,120] and b∈[0.01,50]. Because the input layer l1 has three inputs, there are three fuzzy upper limits and three fuzzy lower limits in the fuzzification layer l2, so there are a total of 12 variables that need to be optimized in the fuzzification layer l2.

[0043] (③ Fuzzy rule layer l3, mainly the connection state between fuzzification layer l2 and fuzzy rule layer l3 needs to be optimized. Because there are three input data, the fuzzy rule layer l3 has a maximum of 14 fuzzy rule neurons. In order to optimize the connection state, it is sufficient to optimize the values ​​of 14 variables whose value range is {0,1}. Variables equal to 0 indicate no connection, and variables equal to 1 indicate connection.

[0044] (④ Fuzzy rough layer l4, the connection status between fuzzy rule layer l3 and fuzzy rough layer l4 needs to be optimized, because the fuzzy rule neuron in fuzzy rule layer l3 may be connected to each fuzzy rough neuron in fuzzy rough layer l4, and fuzzy rough layer l4 has three fixed neurons, so 42 variables with a value range of {0,1} are used to represent the connection status. A variable equal to 0 indicates no connection, and a variable equal to 1 indicates connection. Secondly, there are 6 fuzzy rough memberships that need to be optimized, and the value range of each fuzzy rough membership is [0,1].

[0045] (⑤There are no variables that need to be optimized in the output layer l5, but each neuron in the final output layer of the deep fuzzy rough neural network corresponds to a weight, so three weights need to be optimized, and the value range of the weights is [0,120].

[0046] Therefore, the sum of the parameters that need to be optimized in each layer is the dimension of the decision variable:

[0047]

[0048] use Represents the number of decision variables in each layer, and the dimension of decision variables is 447.

[0049] 2) Objective Function

[0050] In order to optimize the accuracy and complexity of the deep fuzzy rough neural network, the error objective function f is established error And the structural complexity objective function f complex .

[0051] The error objective function is as follows:

[0052]

[0053] Among them, f error represents the mean square error, n data Represents the size of the dataset, y i is the result obtained by DFRNN, is the tag value.

[0054] The structural complexity objective function is as follows:

[0055]

[0056] Among them, f complex Indicates the structural complexity, N R is the number of fuzzy rough neurons, N F is the number of fuzzy rule neurons, N M is the number of fuzzy membership neurons, C M is the number of non-zero elements in the connection state matrix between the fuzzy membership neurons and the fuzzy rule neurons, C F It is the number of non-zero elements in the connection state matrix between fuzzy rule neurons and fuzzy rough neurons.

[0057] (3) Large-scale multi-objective evolutionary algorithm for dynamic balance and generation gap grouping strategy

[0058] Aiming at the multi-objective neural evolution framework, a large-scale multi-objective evolutionary algorithm with dynamic balance and generation gap grouping strategy is designed. The algorithm first randomly generates an initial population P of size N, and then enters a cyclic iteration process. In each iteration, the proposed evaluation method of dynamic adjustment of state information is first used to evaluate the fitness of individuals. According to the size of individual fitness, the population is evenly divided into two groups: elite particle group p and elite particle group p. w and the particle p to be learned l Specifically, two particles are randomly selected each time, their fitness is compared, the particle with larger fitness is divided into the elite particle group, and the particle with smaller fitness is divided into the particle group that needs to be learned. After the division is completed, p l The particles in p w The corresponding particles in learn to update their own speed v and position x. Next, the proposed grouped polynomial mutation method is used to perform mutation operations to generate a mutated population Q, and the offspring and parent populations are merged P'=P∪Q to obtain a new population P' of size 2N. Subsequently, the new population P' is subjected to environmental selection to obtain a new population P of size N. new The final solution set obtained is the required solution set.

[0059] 1) Evaluation method of dynamic balance of state information

[0060] Diversity and convergence are two key performance evaluation indicators in optimization problems. Different from traditional evaluation methods, this method not only focuses on diversity or convergence, but also dynamically adjusts the evolution process based on the feedback information of individual states to take into account both diversity and convergence. Specifically, particles are divided into two states: exploration and approximation. The exploration state indicates that the particles tend to explore diversity, while the approximation state indicates that the particles perform well in terms of convergence. By adjusting the state weights, a balance between diversity and convergence is achieved, thereby improving the optimization effect.

[0061] In order to balance diversity and convergence, a weighted approach is used to combine diversity and convergence, as shown below:

[0062] fit(p i )=w Cd (p i )×Cd(p i )+w Cv (p i )×Cv(p i )

[0063] Among them, fit(p i ) represents individual p i The fitness value, Cd(p i ) and Cv(p i ) are the crowding distance and the Euclidean distance, respectively, which measure the individual pi The diversity and convergence of w Cd (p i ) and w Cv (p i ) are the corresponding weights. According to the information fed back by the individual status, the weights corresponding to different individuals are different.

[0064] The reference vector is introduced, and the individual is substituted into the objective function to calculate its corresponding target value vector. According to the angle between the reference vector and the target value vector, the candidate solution set, i.e., the population, is divided into different sub-regions, and the crowding distance is calculated in each sub-region. In order to more effectively cover the target value space, the generated reference vectors are all located in the first quadrant. Therefore, the target value needs to be converted as follows:

[0065] f'(p i )=f(p i )-f min

[0066] Among them, f(p i )=(f1(p i ),f2(p i ),...,f m (p i )) is the individual p i Each target value, f min =(minf1(p i ),minf2(p i ),…,minf m (p i )) is the individual p i The minimum value corresponding to each objective function, f'(p i ) is the target value after conversion.

[0067] represents the target value vector, Represents the reference vector. The cosine value of the vector angle is used to determine the relative size of the angle:

[0068]

[0069] cosθ i,q The larger the angle, the smaller the angle. The population is divided into k sub-regions X = {x1, x2, ..., x k}. Calculate the crowding distance of each individual in each sub-region independently:

[0070]

[0071] Among them, Cd β (p i ) represents the sum of the differences formed by the first β targets, fβ (p i+1 ) and f β (p i-1 ) are the values ​​of two adjacent points on the target β.

[0072] The convergence distance is the Euclidean distance from the individual target value to the ideal point z, and the calculation formula is as follows:

[0073]

[0074] m is the target number, the Euclidean distance is used as the denominator, and the target number is used as the numerator. The better the convergence of the individual, the better the Cv(p i ) value is larger.

[0075] At the same time, some individuals at the edge of the Pareto frontier are far from the ideal point, but are actually close to the Pareto frontier. In order to solve the problem of these edge individuals, two new distance variables d1 and d2 are designed to more accurately evaluate their positions. Assume that z1 = (0,1) and z2 = (1,0) are two different points, d1 and d2 are the distances of the target vector on the vector and vector The above problem can be solved by combining the distance from the target value to d1 and d2 to determine whether it has converged.

[0076]

[0077] Then, different weights are selected according to the conditions of different particles. Therefore, the average value of Cd, Cv, d1 and d2 is calculated. Cd 、mean Cv , Individuals allocate weights based on the size of their own attributes and the average value.

[0078] Individual p i It can be divided into the following four cases: ① Both diversity and convergence are very good, indicating that p i The performance is relatively good, and there is no need to perform unnecessary operations on diversity and convergence. Cd =1,w Cv =1; ② The convergence is not good but the diversity is good, and these individuals are punished, w Cd =1,w Cv =0.3; ③ The diversity is not good but the convergence is good. Some individuals are eliminated by punishing some individuals and rewarding some individuals in this case. C d=rand(0,1)×0.7+0.5,w Cv =1; ④ In the case of lack of diversity and convergence, w Cd =0.3, w Cv=0.3.

[0079] 2) Generation Gap-Based Grouping Mutation Method

[0080] The mutation method based on generation gap grouping first groups the decision variables according to their correlation, and puts the decision variables with relatively large mutual influence and close relationship into the same group. In the mutation process, polynomial mutation is performed on a group basis.

[0081] In terms of grouping method, the subject is sorted according to the size of the decision variable value and then grouped. If there are num decision variables, divided into G groups, then the smallest The decision variables are grouped into one group, and so on. At the same time, in the grouping process, the generation gap sorting is periodically inserted. The generation gap sorting is based on the size of the generation gap value between the offspring and the parent generation, and the change value is calculated as follows:

[0082] G_gap=Current D -History D

[0083] Among them, Current D is the decision variable value of the offspring, History D is the decision variable value of the parent generation, and G_gap is the generation gap value.

[0084] After grouping, the population P is divided into p1, p2, …, p G , randomly select a set of subpopulations for polynomial mutation.

[0085] 3) Adaptive factors in updating strategies

[0086] In order to speed up the convergence speed, an adaptive factor is introduced into the update strategy. Through the adaptive factor, the iterative process is divided into two stages: convergence exploration and diversity exploration: in the early stage of iteration, the focus is on accelerating the particle swarm to converge in the direction of convergence to improve the search efficiency. In the later stage of iteration, when the convergence speed of the particle swarm tends to be stable, the strategy focuses more on exploring diversity to avoid falling into the local optimum. The details are as follows:

[0087]

[0088] in, and represent the speed and position of elite particles respectively, and Respectively represent the speed and position of the particle to be learned, r0 and r1 are uniformly randomly distributed in the interval [0,1], T is the total number of iterations, t represents the current number of steps, η is the adaptive factor, and the change of η is a gradual decline process. When the gradient is large in the early stage, the decline is more drastic, and the gradient becomes smaller in the later stage, and the change tends to be stable. As the number of iterations increases, the value of η gradually decreases to 1, so as to ensure that in the early stage of optimization, a larger value of η encourages the algorithm to conduct more extensive exploration to discover potential high-quality solutions and avoid the algorithm from falling into the local optimum. As the search process progresses, the value of η gradually decreases, and the algorithm gradually turns to a more refined local search, thereby achieving refined optimization.

[0089] The beneficial effects of the present invention are as follows: by constructing a deep fuzzy rough neural network and establishing a multi-objective optimization problem, a large-scale multi-objective evolutionary algorithm combining dynamic balance and generation gap grouping strategy is proposed to evolve the parameters and structure of the deep fuzzy rough neural network, thereby improving the prediction accuracy of the model in the prediction task. This method solves the limitations of traditional methods in dealing with uncertain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a framework structure diagram of the present invention.

[0091] Figure 2 It is a structural diagram of the deep fuzzy rough neural network.

[0092] Figure 3 It is the structural diagram of the fuzzy rough neural network module.

[0093] Figure 4 is a schematic diagram of the methods for measuring diversity in evaluation methods.

[0094] Figure 5 It is a schematic diagram of the weight allocation strategy.

[0095] Figure 6 is the graph of the adaptive factor function. DETAILED DESCRIPTION

[0096] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0097] Example

[0098] like Figure 1 As shown in the figure, a deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution is shown in the figure. The steps are as follows:

[0099] 1. Constructing a deep fuzzy rough neural network

[0100] Aiming at the limitations of traditional neural networks in processing uncertain data, the present invention provides a deep fuzzy rough neural network. Figure 2 As shown in the figure, the network is composed of a cascade of multiple fuzzy rough neural network modules, and the output of the previous module is used as the input of the next module, forming a stacked integrated DFRNN architecture.

[0101] like Figure 3 As shown in the figure, the fuzzy rough neural network module adopts interval type-2 fuzzy sets and has 5 layers: input layer l1, fuzzification layer l2, fuzzy rule layer l3, fuzzy rough layer l4 and output layer l5. l1 receives external input data and fuzzifies it in l2. The input of l3 comes from the output of l2, and each neuron in l3 corresponds to a different fuzzy rule. The output of l3 is input to l4 for rough processing. Finally, in l5, the fuzzy upper limit and fuzzy lower limit of the output of l4 are averaged and the result is output.

[0102] 2. Establish a multi-objective optimization problem

[0103] In order to optimize the deep fuzzy rough neural network using a multi-objective optimization algorithm, the structure and parameters of the network are modeled as a multi-objective optimization problem, and two objective functions, root mean square error and structural complexity, are defined.

[0104] (1) Decision variables

[0105] Introduce the variables that need to be optimized in each layer:

[0106] 1) The input layer l1 has no variables that need to be optimized.

[0107] 2) Fuzzification of l2 requires optimization of two parameters a∈[0,120] and b∈[0.01,50]. Since l1 has three inputs, there are three fuzzy upper bounds and three fuzzy lower bounds in l2, so there are 12 variables that need to be optimized in l2.

[0108] 3) In the fuzzy rule layer l3, it is mainly the connection status between the two layers l2 and l3 that needs to be optimized. Because there are three input data, l3 has a maximum of 14 neurons. In order to optimize the connection status, it is sufficient to optimize the values ​​of 14 variables whose value range is {0,1}. If the variable is equal to 0, it means there is no connection, and if the variable is equal to 1, it means there is a connection.

[0109] 4) In the fuzzy rough layer l4, the connection state between l3 and l4 needs to be optimized, because the fuzzy rule neuron in l3 may be connected to each fuzzy rough neuron in l4, and l4 has three fixed neurons, so 42 variables with a value range of {0,1} are used to represent the connection state. A variable equal to 0 indicates no connection, and a variable equal to 1 indicates connection. Secondly, there are 6 fuzzy rough memberships that need to be optimized, and the value range of each fuzzy rough membership is [0,1].

[0110] 5) There are no variables that need to be optimized in the output layer l5 of the fuzzy rough neural network module, but each neuron in the final output layer of the deep fuzzy rough neural network corresponds to a weight, so three weights need to be optimized. The value range of the weight is [0,120].

[0111] Therefore, the sum of the parameters that need to be optimized in each layer is the dimension of the decision variable:

[0112] (Dn l1 +Dn l2 +Dn l3 +Dn l4 +Dn l5 )×6+Dn output =447

[0113] Here we use Dn lα (α=1,2,3,4,5) represents the number of decision variables in each layer, so the dimension of decision variables is 447.

[0114] (2) Objective function

[0115] In order to optimize the accuracy and complexity of the deep fuzzy rough neural network, the error objective function f is established error And the structural complexity objective function f complex .

[0116] The error objective function is as follows:

[0117]

[0118] Among them, f error represents the mean square error, n d It represents the size of the data set, y i is the result obtained by DFRNN, is the tag value.

[0119] The structural complexity objective function is as follows:

[0120]

[0121] Among them, f complex Indicates the structural complexity, N R is the number of fuzzy rough neurons, N F is the number of fuzzy rule neurons, N M is the number of fuzzy membership neurons, C M is the number of non-zero elements in the connection state matrix between the fuzzy membership neurons and the fuzzy rule neurons, C F It is the number of non-zero elements in the connection state matrix between fuzzy rule neurons and fuzzy rough neurons.

[0122] 3. Large-scale multi-objective evolutionary algorithm for dynamic balance and generation gap grouping strategy

[0123] like Figure 1 As shown in the figure, a large-scale multi-objective evolutionary algorithm with dynamic balance and generation gap grouping strategy is designed for the multi-objective neural evolution framework. The algorithm first randomly generates an initial population P of size N. Then it enters a cyclic iteration process. In each iteration, the proposed evaluation method of dynamic adjustment of state information is first used to evaluate the fitness of individuals. According to the size of individual fitness, the population is evenly divided into two groups: elite particle group p and elite particle group p. w and the particle p to be learned l Specifically, two particles are randomly selected each time, their fitness is compared, the particle with larger fitness is divided into the elite particle group, and the particle with smaller fitness is divided into the particle group that needs to be learned. After the division is completed, p l The particles in p w The corresponding particles in learn to update their own speed v and position x. Next, the proposed grouped polynomial mutation method is used to perform mutation operations to generate a mutated population Q, and the offspring and parent populations are merged P'=P∪Q to obtain a new population P' of size 2N. Subsequently, P' is subjected to environmental selection to obtain a new population P of size N. new The final solution set obtained is the required solution set.

[0124] (1) Evaluation method of dynamic balance of state information

[0125] In order to balance diversity and convergence, a weighted approach is used to combine diversity and convergence, as shown below:

[0126] fit(p i )=w Cd (p i )×Cd(p i )+w Cv (p i )×Cv(p i )

[0127] Use fit(p i ) represents individual p i The fitness value, Cd(p i ) and Cv(p i ) are respectively used to measure p i The crowding distance for diversity and the Euclidean distance for convergence, w Cd (p i ) and w Cv (p i ) are the corresponding weights. According to the information fed back by the individual status, the weights corresponding to different individuals are different.

[0128] The reference vector is introduced, and the individual is substituted into the objective function to calculate its corresponding target value vector. According to the angle between the reference vector and the target value vector, the candidate solution set, i.e., the population, is divided into different sub-regions, and the crowding distance is calculated in each sub-region. In order to more effectively cover the target value space, the generated reference vectors are all located in the first quadrant. Therefore, the target value needs to be converted as follows:

[0129] f'(p i )=f(p i )-f min

[0130] Where f(p i )=(f1(p i ),f2(p i ),…,f m (p i )) is the individual p i Each target value, f min =(minf1(p i ),minf2(p i ),…,minf m (p i )) is the individual p i The minimum target value, f'(p i ) is the converted target value. This can transfer the target value, narrow the range of the target value vector, and make the reference vector have a better effect.

[0131] represents the target value vector, Represents the reference vector. The cosine value of the vector angle is used to determine the relative size of the angle:

[0132]

[0133] cosθ i,q The larger the angle, the smaller the angle. The population is divided into k sub-regions X = {x1, x2, ..., x k}. Calculate the crowding distance of each individual in each sub-region independently:

[0134]

[0135] Cd β (p i ) represents the sum of the differences formed by the first β targets. β (p i+1 ) and f β (p i-1 ) are the values ​​of two adjacent points on the target β.

[0136] like Figure 4 As shown in the figure, in an optimization problem with 2 objective functions, v1, v2, and v3 are three reference vectors, and the origin is selected as the ideal point z = (0,0). For example, the angles between the vector from the ideal point z to the individual p1 and the reference vectors v1, v2, and v3 are θ1, θ2, and θ3, respectively. It can be seen from the figure that θ2 is the smallest, so a relationship is established between p1 and v2, and p1 is divided into the region related to the v2 vector. Then the crowding distance is calculated in each region. For example, the crowding distance of individual p2 is the perimeter of the rectangle formed by the left and right adjacent individuals p3 and p4 as vertices.

[0137] The convergence distance is the Euclidean distance from the individual target value to the ideal point z, and the calculation formula is as follows:

[0138]

[0139] m is the target number, the Euclidean distance is used as the denominator, and the target number is used as the numerator. The larger the value.

[0140] At the same time, some individuals at the edge of the Pareto frontier are far from the ideal point, but are actually close to the Pareto frontier. In order to solve the problem of these edge individuals, two new distance variables d1 and d2 are designed to more accurately evaluate their positions. Assume that z1 = (0, 1) and z2 = (1, 0) are two different points, d1 and d2 are the distances of the target vector on the vector and vector The projection length in the direction. Combining the distance from the target value to d1 and d2 to determine whether it has converged can effectively solve the above problem.

[0141]

[0142] Then, different weights are selected according to the conditions of different particles. Therefore, the average value of Cd, Cv, d1 and d2 is calculated. Cd 、mean Cv , Individuals allocate weights based on the size of their own attributes and the average value.

[0143] Individual p i It can be divided into the following four cases: ① Both diversity and convergence are very good, indicating that p i The performance is relatively good, so there is no need to perform unnecessary operations on diversity and convergence, w Cd =1,w Cv =1. ② In the case of poor convergence but good diversity, it is necessary to punish these individuals. Therefore, w Cd =1,w Cv=0.3. ③ The part with excellent convergence but lack of diversity. Because the convergence is good, although the diversity is not good, some individuals can be eliminated by punishing some individuals and rewarding some individuals in this case, so that the diversity can be optimized. Therefore, w C d=rand(0,1)×0.7+0.5,w Cv = 1. ④ In the case of lack of diversity and lack of convergence, the performance of this solution is relatively poor, so w Cd =0.3, w Cv =0.3.

[0144] like Figure 5 As shown, examples of the above four situations are listed respectively. In order to make convergence and diversity proportional to fitness, some operations are done on the Euclidean distance representing convergence. Therefore, the larger the Cd and Cv of an individual, the better its diversity and convergence. Therefore, there are the following individuals:

[0145] a) p1, p2 and p3 belong to case (1), Individuals are convergent and diversity is excellent.

[0146] b) p4 and p5 belong to case (2),

[0147] Therefore, p4 and p5 do not converge, but the distribution is uniform and the diversity is good.

[0148] c) p6, p7, and p8 belong to case (3),

[0149] These three individuals are convergent but unevenly distributed and have poor diversity.

[0150] d) p9 belongs to case (4), p9 has neither convergent distribution nor good diversity.

[0151] (2) Generation Gap-Based Grouping Mutation Method

[0152] The mutation method based on generation gap grouping first groups the decision variables according to their correlation, and puts the decision variables with relatively large mutual influence and close relationship into the same group. In the mutation process, polynomial mutation is performed on a group basis, which can reduce the mutual influence between related decision variables and effectively alleviate the adverse effects of mutation.

[0153] In terms of grouping method, in order to balance the low computational effort and the effectiveness of grouping, the subject is sorted according to the size of the decision variable value and then grouped. If there are num decision variables, divided into G groups, then the smallest The decision variables are grouped into one group, and so on. When grouping, the generation gap sorting is inserted regularly. The generation gap sorting is based on the size of the generation gap value between the offspring and the parent generation, and the change value is calculated as follows:

[0154] G_gap=Current D -History D

[0155] HereCurrent D is the decision variable value of the offspring, History D is the decision variable value of the parent generation, and G_gap is the generation gap value.

[0156] After grouping, the population P is divided into p1, p2, ..., p N , randomly select a set of subpopulations for polynomial mutation.

[0157] (3) Adaptive factors in updating strategies

[0158] In order to speed up the convergence speed, an adaptive factor is added to the update strategy. By introducing the adaptive factor, the entire optimization process is divided into two different stages: in the early iteration, the focus is on accelerating the particle swarm to converge in the direction of convergence to improve the search efficiency. In the later iteration, when the convergence speed of the particle swarm tends to be stable, the strategy focuses more on exploring diversity to avoid falling into the local optimum. The details are as follows:

[0159]

[0160] in, and represent the speed and position of elite particles respectively, and Respectively represent the speed and position of the particle to be learned, r0 and r1 are uniformly randomly distributed in the interval [0,1], T is the total number of iterations, t represents the current number of steps, η is the adaptive factor, and the change of η is a gradually decreasing process, such as Figure 6 As shown in the figure. In the early stage, when the gradient is large, the decline is more drastic, while in the later stage, the gradient becomes smaller and the change tends to be stable. As the number of iterations increases, the η value gradually decreases to 1, which ensures that in the early stage of optimization, a larger η value encourages the algorithm to conduct more extensive exploration to discover potential high-quality solutions and avoid the algorithm from falling into the local optimum. As the search process progresses, the η value gradually decreases, and the algorithm gradually turns to a more refined local search, thereby achieving refined optimization.

[0161] Crude Oil (CL=F) (2001-2006) and Global X DAX Germany ETF (DAX) (2015-2020) are used as data sets. The performance of the model DFRNN-LMOCSOFM proposed in the present invention is compared with the prediction model BiCuDNNLSTM-1dCNN based on hybrid deep learning in the stock prediction task. The results in Tables 1 and 2 show that the DFRNN-LMOCSOFM model proposed in the present invention is superior to the BiCuDNNLSTM-1dCNN model in terms of the mean absolute error MAE and root mean square error RMSE on the two data sets, showing the advantage of the model of the present invention in prediction accuracy.

[0162] Table 1 MAE results of BiCuDNNLSTM-1dCNN model and DFRNN-LMOCSOFM model on different datasets

[0163]

[0164] Table 2 RMSE results of BiCuDNNLSTM-1dCNN model and DFRNN-LMOCSOFM model on different data sets

[0165]

Claims

1. A deep fuzzy rough neural network prediction method based on large-scale multi-objective evolution, characterized in that: Here are the steps: (1) Constructing a deep fuzzy rough neural network The deep fuzzy rough neural network is mainly composed of a cascade of multiple fuzzy rough neural network modules. The output of the previous fuzzy rough neural network module is used as the input of the next fuzzy rough neural network module, forming a stacked integrated deep fuzzy rough neural network architecture. The fuzzy-rough neural network module adopts interval type-2 fuzzy sets, with a total of 5 layers: input layer l1, fuzzification layer l2, fuzzy rule layer l3, fuzzy rough layer l4 and output layer l5; the input layer l1 receives external input data and fuzzifies it in the fuzzification layer l2; the input of the fuzzy rule layer l3 comes from the output of the fuzzification layer l2, and each neuron in the fuzzy rule layer l3 corresponds to a different fuzzy rule; the output of the fuzzy rule layer l3 is input to the fuzzy rough layer l4 for rough processing; finally, in the output layer l5, the fuzzy upper limit and fuzzy lower limit output by the fuzzy rough layer l4 are averaged and the result is output; 1) Input layer l1: The input layer of the neural network contains three neurons, each of which receives an input data as the basis for the calculation of the subsequent layers; sti=1,2,3,......,I in, and They represent the input and output of the input layer respectively, and I is the number of neurons in the input layer. The function of the input layer is to receive data, and the input is the output. 2) Fuzzification layer l2: The input data is fuzzified using membership functions in the fuzzy layer. Each input data corresponds to a cluster, and each cluster contains multiple fuzzy membership neurons. stv=1,2,3,...,V j=1,2,3,...,J in, represents the input of the jth fuzzy membership neuron in the vth cluster of the fuzzification layer, and They represent the fuzzy lower limit and fuzzy upper limit of the output of the jth fuzzy membership neuron in the vth cluster of the fuzzification layer, V is the total number of clusters, J is the number of fuzzy membership neurons in each cluster, l and u represent the fuzzy lower limit and fuzzy upper limit, and Gaussian function is used as the membership function, where a and b are the membership function parameters of the fuzzy membership neuron, which are the parameters that need to be optimized; 3) Fuzzy rule layer l3: The combination of neurons with different fuzzy memberships establishes fuzzy rules; therefore, the connection state between the fuzzification layer l2 and the fuzzy rule layer l3 needs to be optimized; h=1,2,3,...,H in, and They represent the inputs of the hth fuzzy lower limit rule neuron and the hth fuzzy upper limit rule neuron in the fuzzy rule layer, d is the dth cluster in l2, e is the eth neuron in the cluster, and They represent the outputs of the hth fuzzy lower limit rule neuron and the hth fuzzy upper limit rule neuron in the fuzzy rule layer, respectively, and H represents the total number of fuzzy rule neurons; 4) Fuzzy rough layer l4: stc=1,2,3,...,C in, and They represent the inputs of the cth fuzzy lower limit rough neuron and the fuzzy upper limit rough neuron, respectively. and denote the outputs of the cth fuzzy lower limit rough neuron and the cth fuzzy upper limit rough neuron respectively, C denotes the total number of fuzzy rough neurons, λ∈[0,1] is the fuzzy rough membership of the hth neuron in the fuzzy rule layer l3 to the cth fuzzy rough neuron in the fuzzy rough layer l4, ε∈{0,1} is the connection state of the hth neuron in the fuzzy rule layer l3 to the cth fuzzy rough neuron in the fuzzy rough layer l4, ε=1 indicates that there is a connection between the hth neuron and the cth neuron, and ε=0 indicates that there is no connection between the two; 5) Output layer l5: str=1,2,3,...,R in, and Represent the input and output of the output layer respectively; in the output layer, the upper and lower limits are averaged to obtain the final result and output; (2) Establishing a multi-objective optimization problem In order to optimize the deep fuzzy rough neural network using a multi-objective optimization algorithm, the structure and parameters of the deep fuzzy rough neural network are modeled as a multi-objective optimization problem, and two objective functions, root mean square error and structural complexity, are defined. 1) Decision variables Introduce the variables that need to be optimized in each layer: ①The input layer l1 has no variables that need to be optimized; ② The fuzzification layer l2 needs to optimize the two parameters of the membership function a∈[0,120] and b∈[0.01,50]. Because the input layer l1 has three inputs, there are three fuzzy upper limits and three fuzzy lower limits in the fuzzification layer l2, so there are a total of 12 variables that need to be optimized in the fuzzification layer l2. (③ In the fuzzy rule layer l3, the connection state between the fuzzification layer l2 and the fuzzy rule layer l3 needs to be optimized. Since there are three input data, the fuzzy rule layer l3 has a maximum of 14 fuzzy rule neurons. In order to optimize the connection state, the values ​​of 14 variables with a value range of {0,1} can be optimized. A variable equal to 0 indicates no connection, and a variable equal to 1 indicates connection. ④ The connection state between the fuzzy rule layer l3 and the fuzzy rough layer l4 in the fuzzy rough layer l4 needs to be optimized, because the fuzzy rule neuron in the fuzzy rule layer l3 may be connected to each fuzzy rough neuron in the fuzzy rough layer l4, and the fuzzy rough layer l4 has three fixed neurons, so 42 variables with a value range of {0,1} are used to represent the connection state. A variable equal to 0 indicates no connection, and a variable equal to 1 indicates connection; secondly, there are 6 fuzzy rough memberships that need to be optimized, and the value range of each fuzzy rough membership is [0,1]; ⑤ There are no variables that need to be optimized in the output layer l5, but each neuron in the final output layer of the deep fuzzy rough neural network corresponds to a weight, so three weights need to be optimized; the value range of the weight is [0,120]; Therefore, the sum of the parameters that need to be optimized in each layer is the dimension of the decision variable: use Indicates the number of decision variables at each layer; the dimension of decision variables is 447; 2) Objective Function In order to optimize the accuracy and complexity of the deep fuzzy rough neural network, the error objective function f is established error And the structural complexity objective function f complex ; The error objective function is as follows: Among them, f error represents the mean square error, n data Represents the size of the dataset, y i is the result obtained by the deep fuzzy rough neural network. is the tag value; The structural complexity objective function is as follows: Among them, f complex Indicates the structural complexity, N R is the number of fuzzy rough neurons, N F is the number of fuzzy rule neurons, N M is the number of fuzzy membership neurons, C M is the number of non-zero elements in the connection state matrix between the fuzzy membership neurons and the fuzzy rule neurons, C F is the number of non-zero elements in the connection state matrix between fuzzy rule neurons and fuzzy rough neurons; (3) Large-scale multi-objective evolutionary algorithm for dynamic balance and generation gap grouping strategy Aiming at the multi-objective neural evolution framework, a large-scale multi-objective evolutionary algorithm with dynamic balance and generation gap grouping strategy is designed. The algorithm first randomly generates an initial population P of size N, and then enters a cyclic iteration process. In each iteration, the proposed evaluation method of dynamic adjustment of state information is first used to evaluate the fitness of individuals, and the population is evenly divided into two groups according to the size of individual fitness: elite particle group p and elite particle group p. w and the particle p to be learned l , specifically described as, each time two particles are randomly selected, their fitness is compared, the one with large fitness is divided into the elite particle group, and the one with small fitness is divided into the particle group that needs to be learned; after the division is completed, p l The particles in p w The corresponding particles in the learning to update their own speed v and position x; Next, the proposed grouped polynomial mutation method is used to perform mutation operations to generate a mutated population Q, and the offspring and parent populations are merged P'=P∪Q to obtain a new population P' of size 2N. Subsequently, the new population P' is subjected to environmental selection to obtain a new population P of size N. new ; The final solution set is the required solution set; 1) Evaluation method of dynamic balance of state information Diversity and convergence are two key performance evaluation indicators in optimization problems. Particles are divided into two states: exploration and approximation. The exploration state indicates that particles tend to explore diversity, while the approximation state indicates that particles perform well in terms of convergence. By adjusting the state weights, a balance between diversity and convergence can be achieved, thereby improving the optimization effect. In order to balance diversity and convergence, a weighted approach is used to combine diversity and convergence, as shown below: fit(p i )=w Cd (p i )×Cd(p i )+w Cv (p i )×Cv(p i ) Among them, fit(p i ) represents individual p i The fitness value, Cd(p i ) and Cv(p i ) are the crowding distance and the Euclidean distance, respectively, which measure the individual p i The diversity and convergence of w Cd (p i ) and w Cv (p i ) are the corresponding weights. According to the information fed back by the individual status, the weights corresponding to different individuals are different; The reference vector is introduced, and the individual is substituted into the objective function to calculate its corresponding target value vector. According to the angle between the reference vector and the target value vector, the candidate solution set, i.e., the population, is divided into different sub-regions, and the crowding distance is calculated in each sub-region. In order to more effectively cover the target value space, the generated reference vectors are all located in the first quadrant. Therefore, the target value needs to be converted as follows: f'(p i )=f(p i )-f min Among them, f(p i )=(f1(p i ),f2(p i ),...,f m (p i )) is the individual p i Each target value, f min =(minf1(p i ),minf2(p i ),...,minf m (p i )) is the individual p i The minimum value corresponding to each objective function, f'(p i ) is the target value after transformation; Use the cosine value of the vector angle to determine the relative size of the angle: in, represents the target value vector, represents the reference vector; cosθ i,q The larger the angle, the smaller the angle. The population is divided into k sub-regions X = {x1, x2, ..., x k }; Find the crowding distance of each individual in each sub-area independently: Among them, Cd β (p i ) represents the sum of the differences formed by the first β targets, f β (p i+1 ) and f β (p i-1 ) are the values ​​of two adjacent points on the target β; The convergence distance is the Euclidean distance from the individual target value to the ideal point z, and the calculation formula is as follows: Among them, m is the number of targets, the Euclidean distance is used as the denominator, and the number of targets is used as the numerator; the better the convergence of the individual, the better the Cv(p i ) value is larger; At the same time, some individuals at the edge of the Pareto frontier are far from the ideal point, but are actually close to the Pareto frontier. In order to solve the problem of these edge individuals, two new distance variables d1 and d2 are designed to more accurately evaluate their positions. Assume that z1 = (0, 1) and z2 = (1, 0) are two different points, d1 and d2 are the distances of the target vector in the vector and vector The projection length in the direction; judging whether it converges by combining the distance from the target value to d1 and d2, can solve the above problem; Then, different weights are selected according to the conditions of different particles; therefore, the average values ​​of Cd, Cv, d1 and d2 are calculated. Individuals allocate weights based on the size of their own attributes and the average value; Individual p i It can be divided into the following four cases: ① Both diversity and convergence are very good, indicating that p i The performance is relatively good, and there is no need to perform unnecessary operations on diversity and convergence. Cd =1,w Cv =1; ② The convergence is not good but the diversity is good, and these individuals are punished, w Cd =1,w Cv =0.3; ③ The diversity is not good but the convergence is good. Some individuals are eliminated by punishing some individuals and rewarding some individuals in this case. C d=rand(0,1)×0.7+0.5,w Cv =1; ④ In the case of lack of diversity and convergence, w Cd =0.3, w Cv =0.3; 2) Generation Gap-Based Grouping Mutation Method The mutation method based on generation gap grouping first groups the decision variables according to their correlation, and puts the decision variables with relatively large mutual influence and close relationship into the same group; in the mutation process, polynomial mutation is performed on the group as a unit; In terms of grouping method, the subject is sorted according to the size of the decision variable value and then grouped. If there are num decision variables, divided into G groups, then the smallest The decision variables are grouped into one group, and so on. At the same time, in the grouping process, the generation gap sorting is periodically inserted. The generation gap sorting is based on the size of the generation gap value between the offspring and the parent generation. The change value is calculated as follows: G_gap=Current D -History D Among them, Current D is the value of the decision variable of the offspring, History D is the decision variable value of the parent generation, G_gap is the generation gap value; After grouping, the population P is divided into p1, p2, ..., p G , randomly select a set of subpopulations for polynomial mutation; 3) Adaptive factors in updating strategies In order to speed up the convergence speed, an adaptive factor is introduced into the update strategy. Through the adaptive factor, the iterative process is divided into two stages: convergence exploration and diversity exploration. In the early stage of iteration, the focus is on accelerating the particle swarm to converge in the direction of convergence to improve the search efficiency. In the later stage of iteration, when the convergence speed of the particle swarm tends to be stable, the strategy focuses more on exploring diversity to avoid falling into the local optimum. The details are as follows: in, and represent the speed and position of elite particles respectively, and Respectively represent the speed and position of the particle to be learned, r0 and r1 are uniformly randomly distributed in the interval [0,1], T is the total number of iterations, t represents the current number of steps, η is the adaptive factor, and the change of η is a gradual decrease. When the gradient is large in the early stage, the decrease is more drastic, and the gradient becomes smaller in the later stage, and the change tends to be stable; with the increase of the number of iterations, the value of η gradually decreases to 1, thus ensuring that in the early stage of optimization, a larger value of η encourages the algorithm to conduct more extensive exploration to discover potential high-quality solutions and avoid the algorithm from falling into the local optimum; with the advancement of the search process, the value of η gradually decreases, and the algorithm gradually turns to a more refined local search, thereby achieving refined optimization.

Images