Fuzzy cognitive map learning method based on hybrid optimization

By combining the mixed optimization method of particle swarm optimization and gradient descent, the problem of insufficient speed and accuracy in fuzzy cognitive graph learning is solved, and rapid convergence and more accurate weight learning without relying on the initial solution are achieved.

CN119002266APending Publication Date: 2024-11-22NORTHWEST ELECTROMECHANICAL ENG RES INST
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411066905.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing fuzzy cognitive graph learning algorithms have shortcomings in speed and accuracy. The mathematical calculation-based methods rely on initial solutions and convergence speed is slow, while the group optimization method requires high computing power and has limited improvement in effect.

Method used

Combining the particle swarm optimization algorithm and gradient descent method, we quickly enter the global optimal interval of weights through particle swarm optimization, and later we turn to the gradient descent method to accurately find the optimization direction, forming a mixed optimization fuzzy cognitive graph learning method.

Benefits of technology

Fast convergence without relying on the initial solution and more accurate fuzzy cognitive graph weight learning are achieved, improving learning speed and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002266A_ABST
    Figure CN119002266A_ABST
Patent Text Reader

Abstract

The invention discloses a fuzzy cognitive map learning method based on hybrid optimization, which comprises the following steps of: converting a fuzzy cognitive map weight matrix into particle individuals through coding, setting a reasonable target function, quickly minimizing the target function through particle swarm iteration to find an optimal interval of the weight matrix, and obtaining a fuzzy cognitive map; and when the maximum iteration step number of particle swarm optimization is achieved or the optimization effect is no longer obvious, a gradient descent method is switched to, an optimization direction is accurately found by defining a calculation gradient, and finally rapid convergence is performed to find a global optimal solution. According to the method, the defects that the later effect of particle swarm optimization is not obvious and the optimization result of a gradient descent method depends on the quality of an initial solution are overcome, and the weight matrix of the fuzzy cognitive map can be quickly and accurately obtained based on historical data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of optimization control, and relates to a learning method of a fuzzy cognitive graph weight matrix based on a combination of a swarm intelligence optimization algorithm and a gradient descent method, and more specifically, to a fuzzy cognitive graph learning method based on hybrid optimization. Background Art

[0002] Fuzzy cognitive graphs are composed of concept nodes and weight matrices. As they combine fuzzy set theory and cognitive graph theory, they have been widely used in industrial control, multi-agent systems and other fields. When establishing a fuzzy cognitive graph model, the weights of the fuzzy cognitive graphs directly given by humans have high objectivity and accuracy requirements for prior knowledge. Therefore, learning the weights of fuzzy cognitive graphs from data has become an important research direction. Existing fuzzy cognitive graph learning algorithms can be divided into two categories: one is the learning method based on mathematical calculation, including simple differential Hebbian learning algorithm, gradient descent method, etc.; the other is the learning method based on population optimization, such as genetic algorithm, particle swarm algorithm, etc. Both types of algorithms have their own advantages and limitations: the learning method based on mathematical calculation has the advantages of fast speed and high accuracy, but requires the objective function to be continuous and differentiable, and the learning results are highly dependent on the selection of the initial solution; while the optimization algorithm based on the population does not require the objective function to be differentiable and does not depend on the selection of the initial solution, but it has high requirements on computing power and the improvement of the optimization effect is limited. Therefore, the fuzzy cognitive graph learning method based on hybrid optimization utilizes the respective advantages of the two types of algorithms. In the early stage of algorithm iteration, a large population is optimized through group optimization to quickly enter the global optimal interval of weights, avoiding the influence of random initial solutions on gradient calculations. In the later stage, the gradient descent method is used to accurately find the optimization direction by calculating the gradient, so that the algorithm converges faster. Summary of the invention

[0003] The technical problem to be solved by the present invention is how to learn the weight matrix of the fuzzy cognitive map more accurately and quickly through data. In order to solve the above problems, the present invention provides a fuzzy cognitive map learning method based on hybrid optimization, which combines the particle swarm optimization algorithm and the gradient descent method to give full play to the advantages of the two algorithms, avoid their respective limitations, and improve the speed and accuracy of weight learning.

[0004] A fuzzy cognitive graph learning method based on hybrid optimization includes the following steps:

[0005] Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The sum of squares of the error between the actual output and the expected output is calculated as the objective function;

[0006] Design the objective function of the particle swarm optimization algorithm;

[0007] Step 2: According to the number of non-zero values ​​in the weight matrix, set the dimension of the individual particles, and correspond the non-zero values ​​in the weight matrix to the individual dimensions one by one to complete the individual encoding. At the same time, set the number of individuals in the population and the maximum number of iterations to initialize the population.

[0008] Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm;

[0009] Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum;

[0010] Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm is reached or the change of the swarm optimality is less than the set threshold. If so, proceed to step 6, otherwise proceed to step 3;

[0011] Step 6: Stop particle swarm optimization, output the swarm optimal value as the initial solution vector of the gradient descent method, and set the maximum iteration step of the gradient descent method;

[0012] Step 7: Calculate the gradient of the objective function for each weight according to the definition of the derivative;

[0013] Step 8: Substitute the gradient and learning rate, update each weight according to the gradient descent update formula, and recalculate the objective function based on the weight;

[0014] Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method is reached or the change of the objective function is less than the set threshold. If so, proceed to step 10, otherwise proceed to step 7;

[0015] Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded to form the weight matrix of the final fuzzy cognitive map.

[0016] Beneficial effects of the present invention:

[0017] The present invention can accurately and quickly learn the fuzzy cognitive graph weight matrix from the data. Compared with a single learning algorithm, the learning method based on hybrid optimization has the advantages of being independent of the quality of the initial solution, fast convergence speed in the later stage, and more accurate learning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flow chart of a fuzzy cognitive graph learning method based on hybrid optimization of the present invention;

[0019] Figure 2 This is an example diagram of a fuzzy cognitive graph corresponding to a classic industrial control system of the present invention;

[0020] Figure 3It is the weight matrix of the present invention and the individual coding diagram of the corresponding particle swarm optimization algorithm. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments so that those skilled in the art can better understand the present invention and implement it, but the examples are not intended to limit the present invention.

[0022] like Figures 1 to 3 As shown, a fuzzy cognitive graph learning method based on hybrid optimization includes the following steps:

[0023] Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The node value update formula is as follows:

[0024]

[0025] Among them, C i (t) is node C i The value at time t, C ji represents the element in the jth row and ith column of the weight matrix. The function f(x) is the sigmoid normalization function, and the expression is as follows:

[0026]

[0027] Calculate the sum of squares of the error between the actual output and the expected output as the objective function: For a set of input data of the fuzzy cognitive map, let its expected output be S = (S1, S2, ..., S8) and the actual output be C = (C1, C2, ..., C8). For N sets of training data, the objective function is designed as follows:

[0028]

[0029] Among them, S ki represents the ideal value of the i-th node in the k-th group of data, C ki Represents the actual value of the i-th node in the k-th group of data. The optimization process reduces the error between the actual value and the ideal value by minimizing the objective function, and finally obtains the ideal weight matrix.

[0030] Step 2: Design the individual encoding method of the particle swarm optimization algorithm: The dimension of the individual particle is set to the number of non-zero values ​​in the weight matrix, and each dimension corresponds to the non-zero weight in the weight matrix in turn, that is, each particle individual has 13 dimensions, such as Figure 3 As shown. Assume that there are m particles in the population and the maximum number of iterations is pso_iter max , the position and velocity vector of particle i are as follows:

[0031] X i =(x i1 ,x i2 ,...,x id ),i=1,2,...,m

[0032] V i =(v i1 ,v i2 ,...,v id ),i=1,2,...,m

[0033] Among them, x ij 、v ij The vector X i 、V i The j-th dimension of the particle, d is the maximum dimension of the particle, which is 13 in this example. The initialization method is as follows:

[0034]

[0035] Where rand(-1,1) represents a random value between [-1,1]. Record the position of each particle as the individual optimal position:

[0036] Pbest i =(Pbest i1 ,Pbest i2 ,...,Pbest id ),i=1,2,...,m

[0037] Substitute the weight matrix corresponding to each particle into the objective function calculation formula in turn, and find the particle that minimizes the objective function as the optimal position of the entire population:

[0038] Gbest=(Gbest1,Gbest2,...,Gbest d )

[0039] Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm:

[0040] V i t+1 =w·V i t +c1·r1·(Pbest i t -X i t )+c2·r2·(Gbest t -X i t )

[0041] X it+1 =X i t +V i t

[0042] Among them, V i t , X i t They represent the velocity and position of particle i at the tth iteration step respectively, w is the inertia constant, usually takes the value of 1, c1 and c2 are learning factors, usually takes the value of 2, and r1 and r2 are random values ​​between [0,1].

[0043] Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum.

[0044] Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm pso_iter is reached max Or the change of the group optimality is smaller than the set threshold (|Gbest t -Gbest t-1 |<α), where α is the set threshold. If yes, go to step 6, otherwise go to step 3.

[0045] Step 6: Stop particle swarm optimization, output the swarm optimal solution as the initial solution vector of the gradient descent method, and set the maximum iteration step gda_iter of the gradient descent method max .

[0046] Step 7: Calculate the gradient of the objective function with respect to each weight Since the fuzzy cognitive graph undergoes multiple inferences and is normalized after each inference, the mathematical expression of the final objective function for each weight is very complex, and the gradient is difficult to solve by the derivative formula. Therefore, the gradient is calculated according to the definition of the derivative, and the formula is as follows:

[0047]

[0048] Among them, δ is actually calculated as 10 -8 .

[0049] Step 8: Update the weights according to the following formula:

[0050]

[0051] w ji (t+1)=w ji (t)+Δw ji

[0052] Where η is the learning rate, which is set to 0.01. Substitute the new weights and recalculate the objective function.

[0053] Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method gda_iter is reached max Or the change in the objective function is less than the set threshold (|f(W) t -f(W) t-1 |<β), where β is the set threshold. If yes, go to step 10, otherwise go to step 7.

[0054] Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded: each dimension of the weight vector is assigned to the weight matrix in turn, and the remaining positions of the matrix are all 0 to form the weight matrix of the final fuzzy cognitive map.

Claims

1. A fuzzy cognitive graph learning method based on hybrid optimization, characterized in that: It includes the following steps: Step 1: The value of the fuzzy cognitive map node after inference to the stable state according to the ideal weight is taken as the expected output, and the value of the node after inference to the stable state according to the current actual weight is taken as the actual output. The node value update formula is as follows: Among them, C i (t) is node C i The value at time t, C ji represents the element in the jth row and ith column of the weight matrix. The function f(x) is the sigmoid normalization function, and the expression is as follows: Calculate the sum of squares of the error between the actual output and the expected output as the objective function: For a set of input data of the fuzzy cognitive map, let its expected output be S = (S1, S2, ..., S8) and the actual output be C = (C1, C2, ..., C8). For N sets of training data, the objective function is designed as follows: Among them, S ki represents the ideal value of the i-th node in the k-th group of data, C ki represents the actual value of the i-th node in the k-th group of data. The optimization process reduces the error between the actual value and the ideal value by minimizing the objective function, and finally obtains the ideal weight matrix; Step 2: Design the individual encoding method of the particle swarm optimization algorithm: the dimension of the individual particle is set to the number of non-zero values ​​in the weight matrix, and each dimension corresponds to the non-zero weight in the weight matrix in turn, that is, each particle individual has 13 dimensions; suppose there are m particles in the population, and the maximum number of iterations of the population is pso_iter max , the position and velocity vector of particle i are as follows: X i =(x i1 ,x i2 ,...,x id ),i=1,2,...,m V i =(v i1 ,v i2 ,...,v id ),i=1,2,...,m Among them, x ij 、v ij The vector X i 、V i The j-th dimension of the particle, d is the maximum dimension of the particle, which is 13 in this example. The initialization method is as follows: Where rand(-1,1) represents a random value between [-1,1]; the position of each particle is recorded as the individual optimal position: Pbest i =(Pbest i1 ,Pbest i2 ,...,Pbest id ),i=1,2,...,m Substitute the weight matrix corresponding to each particle into the objective function calculation formula in turn, and find the particle that minimizes the objective function as the optimal position of the entire population: Gbest=(Gbest1,Gbest2,...,Gbest d ) Step 3: Under the guidance of individual optimality and group optimality, update the entire population according to the update formula of the particle swarm algorithm: V i t+1 =w·V i t +c1·r1·(Pbest i t -X i t )+c2·r2·(Gbest t -X i t ) X i t+1 =X i t +V i t Among them, V i t , X i t They represent the velocity and position of particle i at the tth iteration step, w is the inertia constant, usually 1, c1 and c2 are learning factors, usually 2, and r1 and r2 are random values ​​between [0,1]; Step 4: Substitute the weight matrix corresponding to the updated individual position to recalculate the objective function and update the individual optimum and group optimum; Step 5: Determine whether the stopping condition is met: the maximum number of iterations of the particle swarm pso_iter is reached max Or the change of the group optimality is smaller than the set threshold (|Gbest t -Gbest t-1 |<α), where α is the set threshold. If yes, go to step 6, otherwise go to step 3; Step 6: Stop particle swarm optimization, output the swarm optimal solution as the initial solution vector of the gradient descent method, and set the maximum iteration step gda_iter of the gradient descent method max ; Step 7: Calculate the gradient of the objective function with respect to each weight Since the fuzzy cognitive graph undergoes multiple inferences and is normalized after each inference, the mathematical expression of the final objective function for each weight is very complex, and the gradient is difficult to solve by the derivative formula. Therefore, the gradient is calculated according to the definition of the derivative, and the formula is as follows: Among them, δ is actually calculated as 10 -8 ; Step 8: Update the weights according to the following formula: w ji (t+1)=w ji (t)+Δw ji Where η is the learning rate, which is set to 0.01; substitute the new weights to recalculate the objective function; Step 9: Determine whether the stopping condition is met: the maximum number of iterations of the gradient descent method gda_iter is reached max Or the change in the objective function is less than the set threshold (|f(W) t -f(W) t-1 |<β), where β is the set threshold; if so, proceed to step 10, otherwise proceed to step 7; Step 10: According to the individual coding order of the particle swarm, the current weight vector is de-coded: each dimension of the weight vector is assigned to the weight matrix in turn, and the remaining positions of the matrix are all 0 to form the weight matrix of the final fuzzy cognitive map.

Citation Information

Cited By

  • Method and system for globally calibrating multi-channel directional diagram distinction degree of plate-shaped array antenna

    CN121441424A

  • Plate array antenna multi-channel pattern resolution global calibration method and system

    CN121441424B