Fuzzy cognitive map learning method based on hybrid optimization
By combining the mixed optimization method of particle swarm optimization and gradient descent, the problem of insufficient speed and accuracy in fuzzy cognitive graph learning is solved, and rapid convergence and more accurate weight learning without relying on the initial solution are achieved.
Patent Information
- Application Number
- CN202411066905.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-22
AI Technical Summary
The existing fuzzy cognitive graph learning algorithms have shortcomings in speed and accuracy. The mathematical calculation-based methods rely on initial solutions and convergence speed is slow, while the group optimization method requires high computing power and has limited improvement in effect.
Combining the particle swarm optimization algorithm and gradient descent method, we quickly enter the global optimal interval of weights through particle swarm optimization, and later we turn to the gradient descent method to accurately find the optimization direction, forming a mixed optimization fuzzy cognitive graph learning method.
Fast convergence without relying on the initial solution and more accurate fuzzy cognitive graph weight learning are achieved, improving learning speed and accuracy.
Smart Images

Figure CN119002266A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of optimization control, and relates to a learning method of a fuzzy cognitive graph weight matrix based on a combination of a swarm intelligence optimization algorithm and a gradient descent method, and more specifically, to a fuzzy cognitive graph learning method based on hybrid optimization. Background Art
[0002] Fuzzy cognitive graphs are composed of concept nodes and weight matrices. As they combine fuzzy set theory and cognitive graph theory, they have been widely used in industrial control, multi-agent systems and other fields. When establishing a fuzzy cognitive graph model, the weights of the fuzzy cognitive graphs directly given by humans have high objectivity and accuracy requirements for prior knowledge. Therefore, learning the weights of fuzzy cognitive graphs from data has become an important research direction. Existing fuzzy cognitive graph learning algorithms can be divided into two categories: one is the learning method based on mathematical calculation, including simple differential Hebbian learning algorithm, gradient descent method, etc.; the other is the learning method based on population optimization, such as genetic algorithm, particle swarm algorithm, etc. Both types of algorithms have their own advantages and limitations: the learning method based on mathematical calculation has the advantages of fast speed and high accuracy, but requires the objective function to be continuous and differentiable, and the learning results are highly dependent on the selection of the initial solution; while the optimization algorithm based on the population does not require the objective function to be differentiable and does not depend on the selection of the initial solution, but it has high requirements on computing power and the improvement of the optimization effect is limited. Therefore, the fuzzy cognitive graph learning method based on hybrid optimization utilizes the respective advantages of the two types of algorithms. In the early stage of algorithm iteration, a large population is optimized through group optimization to quickly enter the global optimal interval of weights, avoiding the influence of random initial solutions on gradient calculations. In the later stage, the gradient descent method is used to accurately find the optimization direction by calculating the gradient, so that the algorithm converges faster. Summary of the invention
[0003] The technical problem to be solved by the present invention is how to learn the weight matrix of the fuzzy cognitive map more accurately and quickly through data. In order to solve the above problems, the present invention provides a fuzzy cognitive map learning method based on hybrid optimization, which combines the particle swarm optimization algorithm and the gradient descent method to give full play to the advantages of the two algorithms, avoid their respective limitations, and improve the speed and accuracy of weight learning.
[0004] A fuzzy cognitive graph learning method based on hybrid optimization includes the following steps:
[0005] Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The sum of squares of the error between the actual output and the expected output is calculated as the objective function;
[0006] Design the objective function of the particle swarm optimization algorithm;
[0007] Step 2: According to the number of non-zero values in the weight matrix, set the dimension of the individual particles, and correspond the non-zero values in the weight matrix to the individual dimensions one by one to complete the individual encoding. At the same time, set the number of individuals in the population and the maximum number of iterations to initialize the population.
[0008] Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm;
[0009] Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum;
[0010] Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm is reached or the change of the swarm optimality is less than the set threshold. If so, proceed to step 6, otherwise proceed to step 3;
[0011] Step 6: Stop particle swarm optimization, output the swarm optimal value as the initial solution vector of the gradient descent method, and set the maximum iteration step of the gradient descent method;
[0012] Step 7: Calculate the gradient of the objective function for each weight according to the definition of the derivative;
[0013] Step 8: Substitute the gradient and learning rate, update each weight according to the gradient descent update formula, and recalculate the objective function based on the weight;
[0014] Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method is reached or the change of the objective function is less than the set threshold. If so, proceed to step 10, otherwise proceed to step 7;
[0015] Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded to form the weight matrix of the final fuzzy cognitive map.
[0016] Beneficial effects of the present invention:
[0017] The present invention can accurately and quickly learn the fuzzy cognitive graph weight matrix from the data. Compared with a single learning algorithm, the learning method based on hybrid optimization has the advantages of being independent of the quality of the initial solution, fast convergence speed in the later stage, and more accurate learning results. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flow chart of a fuzzy cognitive graph learning method based on hybrid optimization of the present invention;
[0019] Figure 2 This is an example diagram of a fuzzy cognitive graph corresponding to a classic industrial control system of the present invention;
[0020] Figure 3It is the weight matrix of the present invention and the individual coding diagram of the corresponding particle swarm optimization algorithm. DETAILED DESCRIPTION
[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments so that those skilled in the art can better understand the present invention and implement it, but the examples are not intended to limit the present invention.
[0022] like Figures 1 to 3 As shown, a fuzzy cognitive graph learning method based on hybrid optimization includes the following steps:
[0023] Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The node value update formula is as follows:
[0024]
[0025] Among them, C i (t) is node C i The value at time t, C ji represents the element in the jth row and ith column of the weight matrix. The function f(x) is the sigmoid normalization function, and the expression is as follows:
[0026]
[0027] Calculate the sum of squares of the error between the actual output and the expected output as the objective function: For a set of input data of the fuzzy cognitive map, let its expected output be S = (S1, S2, ..., S8) and the actual output be C = (C1, C2, ..., C8). For N sets of training data, the objective function is designed as follows:
[0028]
[0029] Among them, S ki represents the ideal value of the i-th node in the k-th group of data, C ki Represents the actual value of the i-th node in the k-th group of data. The optimization process reduces the error between the actual value and the ideal value by minimizing the objective function, and finally obtains the ideal weight matrix.
[0030] Step 2: Design the individual encoding method of the particle swarm optimization algorithm: The dimension of the individual particle is set to the number of non-zero values in the weight matrix, and each dimension corresponds to the non-zero weight in the weight matrix in turn, that is, each particle individual has 13 dimensions, such as Figure 3 As shown. Assume that there are m particles in the population and the maximum number of iterations is pso_iter max , the position and velocity vector of particle i are as follows:
[0031] X i =(x i1 ,x i2 ,...,x id ),i=1,2,...,m
[0032] V i =(v i1 ,v i2 ,...,v id ),i=1,2,...,m
[0033] Among them, x ij 、v ij The vector X i 、V i The j-th dimension of the particle, d is the maximum dimension of the particle, which is 13 in this example. The initialization method is as follows:
[0034]
[0035] Where rand(-1,1) represents a random value between [-1,1]. Record the position of each particle as the individual optimal position:
[0036] Pbest i =(Pbest i1 ,Pbest i2 ,...,Pbest id ),i=1,2,...,m
[0037] Substitute the weight matrix corresponding to each particle into the objective function calculation formula in turn, and find the particle that minimizes the objective function as the optimal position of the entire population:
[0038] Gbest=(Gbest1,Gbest2,...,Gbest d )
[0039] Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm:
[0040] V i t+1 =w·V i t +c1·r1·(Pbest i t -X i t )+c2·r2·(Gbest t -X i t )
[0041] X it+1 =X i t +V i t
[0042] Among them, V i t , X i t They represent the velocity and position of particle i at the tth iteration step respectively, w is the inertia constant, usually takes the value of 1, c1 and c2 are learning factors, usually takes the value of 2, and r1 and r2 are random values between [0,1].
[0043] Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum.
[0044] Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm pso_iter is reached max Or the change of the group optimality is smaller than the set threshold (|Gbest t -Gbest t-1 |<α), where α is the set threshold. If yes, go to step 6, otherwise go to step 3.
[0045] Step 6: Stop particle swarm optimization, output the swarm optimal solution as the initial solution vector of the gradient descent method, and set the maximum iteration step gda_iter of the gradient descent method max .
[0046] Step 7: Calculate the gradient of the objective function with respect to each weight Since the fuzzy cognitive graph undergoes multiple inferences and is normalized after each inference, the mathematical expression of the final objective function for each weight is very complex, and the gradient is difficult to solve by the derivative formula. Therefore, the gradient is calculated according to the definition of the derivative, and the formula is as follows:
[0047]
[0048] Among them, δ is actually calculated as 10 -8 .
[0049] Step 8: Update the weights according to the following formula:
[0050]
[0051] w ji (t+1)=w ji (t)+Δw ji
[0052] Where η is the learning rate, which is set to 0.01. Substitute the new weights and recalculate the objective function.
[0053] Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method gda_iter is reached max Or the change in the objective function is less than the set threshold (|f(W) t -f(W) t-1 |<β), where β is the set threshold. If yes, go to step 10, otherwise go to step 7.
[0054] Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded: each dimension of the weight vector is assigned to the weight matrix in turn, and the remaining positions of the matrix are all 0 to form the weight matrix of the final fuzzy cognitive map.
Claims
1. A fuzzy cognitive graph learning method based on hybrid optimization, characterized in that: It includes the following steps: Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The node value update formula is as follows: Among them, C i (t) is node C i The value at time t, C ji represents the element in the jth row and ith column of the weight matrix. The function f(x) is the sigmoid normalization function, and the expression is as follows: Calculate the sum of squares of the error between the actual output and the expected output as the objective function: For a set of input data of the fuzzy cognitive map, let its expected output be S = (S1, S2, ..., S8) and the actual output be C = (C1, C2, ..., C8). For N sets of training data, the objective function is designed as follows: Among them, S ki represents the ideal value of the i-th node in the k-th group of data, C ki represents the actual value of the i-th node in the k-th group of data. The optimization process reduces the error between the actual value and the ideal value by minimizing the objective function, and finally obtains the ideal weight matrix; Step 2: Design the individual encoding method of the particle swarm optimization algorithm: the dimension of the individual particle is set to the number of non-zero values in the weight matrix, and each dimension corresponds to the non-zero weight in the weight matrix in turn, that is, each particle individual has 13 dimensions; suppose there are m particles in the population, and the maximum number of iterations of the population is pso_iter max , the position and velocity vector of particle i are as follows: X i =(x i1 ,x i2 ,...,x id ),i=1,2,...,m V i =(v i1 ,v i2 ,...,v id ),i=1,2,...,m Among them, x ij 、v ij The vector X i 、V i The j-th dimension of the particle, d is the maximum dimension of the particle, which is 13 in this example. The initialization method is as follows: Where rand(-1,1) represents a random value between [-1,1]; the position of each particle is recorded as the individual optimal position: Pbest i =(Pbest i1 ,Pbest i2 ,...,Pbest id ),i=1,2,...,m Substitute the weight matrix corresponding to each particle into the objective function calculation formula in turn, and find the particle that minimizes the objective function as the optimal position of the entire population: Gbest=(Gbest1,Gbest2,...,Gbest d ) Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm: V i t+1 =w·V i t +c1·r1·(Pbest i t -X i t )+c2·r2·(Gbest t -X i t ) X i t+1 =X i t +V i t Among them, V i t , X i t They represent the velocity and position of particle i at the tth iteration step, w is the inertia constant, usually 1, c1 and c2 are learning factors, usually 2, and r1 and r2 are random values between [0,1]; Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum; Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm pso_iter is reached max Or the change of the group optimality is smaller than the set threshold (|Gbest t -Gbest t-1 |<α), where α is the set threshold. If yes, go to step 6, otherwise go to step 3; Step 6: Stop particle swarm optimization, output the swarm optimal solution as the initial solution vector of the gradient descent method, and set the maximum iteration step gda_iter of the gradient descent method max ; Step 7: Calculate the gradient of the objective function with respect to each weight Since the fuzzy cognitive graph undergoes multiple inferences and is normalized after each inference, the mathematical expression of the final objective function for each weight is very complex, and the gradient is difficult to solve by the derivative formula. Therefore, the gradient is calculated according to the definition of the derivative, and the formula is as follows: Among them, δ is actually calculated as 10 -8 ; Step 8: Update the weights according to the following formula: w ji (t+1)=w ji (t)+Δw ji Where η is the learning rate, which is set to 0.01; substitute the new weights to recalculate the objective function; Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method gda_iter is reached max Or the change in the objective function is less than the set threshold (|f(W) t -f(W) t-1 |<β), where β is the set threshold; if so, proceed to step 10, otherwise proceed to step 7; Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded: each dimension of the weight vector is assigned to the weight matrix in turn, and the remaining positions of the matrix are all 0 to form the weight matrix of the final fuzzy cognitive map.
Citation Information
Cited By
Method and system for globally calibrating multi-channel directional diagram distinction degree of plate-shaped array antenna
CN121441424A
Plate array antenna multi-channel pattern resolution global calibration method and system
CN121441424B
