Multi-task TSK fuzzy system modeling method based on multi-objective optimization

Abstract

The invention belongs to the field of intelligent computing, and particularly relates to a multi-task TSK fuzzy system modeling method based on multi-objective optimization. According to the method, a multi-task fuzzy system learning task is converted into a corresponding multi-objective optimization problem in the first stage, in addition, in the first stage, modeling is carried out by utilizing independent sample information of each task, and the classification performance is enhanced by effectively utilizing related information among the tasks. In the second stage, a new multi-target multi-task fuzzy system training method is designed in combination with the particle swarm optimization algorithm, and the learning performance of the method is more efficient while the target of multi-task learning is converted into searching for the Pareto optimal solution.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent computing, and in particular relates to a multi-task TSK fuzzy system modeling method based on multi-objective optimization. Technical Background

[0002] In the real world, many complex systems often need to handle multiple interrelated yet independent tasks simultaneously. This type of problem is known as multi-task learning. For example, in smart manufacturing, systems need to simultaneously predict product quality, energy consumption, and equipment lifespan; or in medical diagnosis, the risk of multiple diseases needs to be assessed simultaneously.

[0003] To efficiently solve complex multi-task problems, multi-task learning methods have made significant progress in recent years. However, current multi-task algorithms suffer from two issues. First, they all share a common assumption: that all tasks have equal priority. However, in real-world scenarios, the importance of each task in most multi-task problems varies. For example, in intelligent driving systems, vehicle control can be divided into multiple tasks, such as obstacle avoidance, path planning, and comfort control. However, the safety-related obstacle avoidance task clearly has a higher priority than the comfort control task. Another example is that in medical decision-making systems, multiple indicators, such as treatment efficacy and side effects, often need to be considered simultaneously. However, the importance of these indicators may vary for different patients. In these cases, traditional multi-task methods cannot effectively balance the weights between tasks, resulting in suboptimal model performance. Second, with the widespread application of machine learning in recent years, the issue of machine learning interpretability has received increasing attention. However, these algorithms focus too much on algorithm performance and neglect model interpretability.

[0004] In order to deal with the challenges brought by inconsistent task priorities in multi-task learning, people have conducted some explorations, and these explorations can be divided into the following four categories. 1. Using a fixed weight method to weight each task, this approach ignores the dynamic relationship and mutual influence between tasks. 2. Using expert knowledge to manually sort and weight different tasks. This type of processing method relies too much on expert experience and is difficult to cope with large-scale complex systems. 3. Using serial optimization technology, first optimize high-priority tasks, and then optimize secondary priority tasks on this basis. This type of method is prone to falling into local optimality and it is difficult to ensure the balance between tasks. 4. Using a hierarchical learning framework, the multi-task problem is decomposed into multiple sub-problems and solved in sequence. This type of method can better handle the hierarchical relationship between tasks, but often ignores the possible synergistic effects between tasks. Therefore, the priority balance problem in multi-task learning still faces major challenges.

[0005] To address people's demand for explainability, there are currently two mainstream strategies. The first category uses relatively simple, essentially explainable models to deal with the problem, such as linear models, tree-based models, and rule-based models. The second category adopts post-interpretation methods, using methods such as visualization and example explanation to explain the model's decision-making process while ensuring model performance. The TSK fuzzy system has attracted widespread attention due to its rule-based interpretability and strong data-driven learning capabilities. It has also made significant progress in the multi-task field in recent years. However, the current TSK-based multi-task model still cannot effectively deal with the problem of inconsistent task priorities. Therefore, how to establish a TSK fuzzy system model that can simultaneously take into account the optimization of multiple objectives and has high transparency to solve multi-task problems remains a challenging task. Summary of the Invention

[0006] In view of the above shortcomings of the prior art, the present invention provides a multi-task TSK fuzzy system modeling method based on multi-objective optimization (MOO MT-TSK-FS). In the first stage, the method converts the multi-task fuzzy system learning task into a corresponding multi-objective optimization problem. In addition, in the first stage, the method uses the independent sample information of each task for modeling and effectively uses the relevant information between tasks to enhance the classification performance. In the second stage, the present invention combines the particle swarm optimization algorithm to design a new multi-objective multi-task fuzzy system training method, which converts the goal of multi-task learning into finding the Pareto optimal solution, while making the learning performance of this method more efficient.

[0007] The technical solutions of the present invention are as follows:

[0008] A multi-task TSK fuzzy system modeling method based on multi-objective optimization includes the following steps:

[0009] Step 1: Determine the number of tasks K of the multi-task data used for training, the number of fuzzy rules M corresponding to each task, and set the regularization parameter τ k and the equilibrium parameter λ;

[0010] Step 2: Use FCM clustering algorithm to obtain the antecedent parameters of the fuzzy system;

[0011] Step 3: Construct the objective formula of the multi-task TSK fuzzy system for multi-objective optimization;

[0012] 3.1 Define the objective function of each task k as follows:

[0013]

[0014] in,

[0015]

[0016] Among them, P g0 and θ k Both are consequent parameter matrices, the former represents inter-task related information, the latter represents task independent information, ξ i,k is the slack variable, ε k is the marginal value.

[0017] 3.2 Constructing the overall multi-objective optimization problem:

[0018]

[0019]

[0020] 3.3 Integrating constraints into the objective function through penalty function method:

[0021]

[0022] Among them, W k =P g0 +θ k , μ is the penalty factor, which increases with the number of iterations to gradually strengthen the constraints.

[0023] Step 4: Solve the constructed multi-objective optimization problem through the multi-objective particle swarm optimization algorithm;

[0024] 4.1 Initialize the particle swarm and randomly generate particles satisfying ξ k >0 and ε k >0 particle position and velocity vectors;

[0025] 4.2 Encode the decision variables as particle position vectors:

[0026] x i =P g0 ,θ1,…,θ K ,ξ1,ε1,…,ξ K ,ε K (7)

[0027] 4.3 Calculate multi-objective vectors to evaluate particle fitness:

[0028]

[0029] 4.4 During the iteration process, the particle's velocity and position are updated according to the following formula:

[0030]

[0031] 4.5 pairs i,k and ε k Perform projection processing to force it to remain positive;

[0032] 4.6 Maintain an external archive A, save the Pareto frontier solution set through non-dominated sorting and crowding distance strategies, and each stored solution contains complete matrix and scalar parameters;

[0033]

[0034] Step 5: Select the optimal trade-off solution from the external archive to obtain the final classification result.

[0035] 5.1 Obtaining the Optimal Postprocessor Parameters

[0036] 5.2 Substitute the obtained optimal consequent parameters into the symbolic function to obtain the classification result:

[0037]

[0038] The advantages of the present invention include the following:

[0039] 1) This paper integrates the theory and methods of Pareto optimization with fuzzy system modeling. Through this combination, it successfully constructs a unified framework that can handle multiple conflicting learning tasks simultaneously, thereby significantly expanding the potential areas of multi-task fuzzy learning in practical applications.

[0040] 2) The present invention introduces a multi-objective particle swarm optimization algorithm into the proposed method, which enables the method to have good global search capabilities and fast convergence speed, can effectively find the optimal solution set in a complex multi-objective search space, and improves the ability to handle conflicts between tasks;

[0041] 3) A large number of experiments have verified the effectiveness of the proposed method on artificial and real datasets. Compared with traditional methods, the proposed method has good classification performance and robustness, as well as good transparency and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall framework diagram of the algorithm of the present invention.

[0043] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be described in detail below with reference to the accompanying drawings and embodiments:

[0045] like Figure 1-2As shown in the figure, the present invention implements a multi-task TSK fuzzy system modeling method based on multi-objective optimization. The method includes two stages: constructing the objective function and implementing multi-objective optimization. In the first stage of the present invention, independent sample information of each task is used for modeling, and the correlation information between tasks is effectively utilized to enhance classification performance. In the second stage, the goal of multi-task learning is transformed into finding a Pareto optimal solution, making the learning performance of this method more efficient.

[0046] Table 1 Statistics of the dataset

[0047]

[0048] Example 1

[0049] A multi-task TSK fuzzy system modeling method based on multi-objective optimization includes the following steps:

[0050] Step 1: Determine the number of tasks K of the multi-task data used for training, the number of fuzzy rules M corresponding to each task, and set the regularization parameter τ k and the equilibrium parameter λ;

[0051] Step 2: Use FCM clustering algorithm to obtain the antecedent parameters of the fuzzy system;

[0052] Step 3: Construct the objective formula of the multi-task TSK fuzzy system for multi-objective optimization;

[0053] Step 4: Solve the constructed multi-objective optimization problem through the multi-objective particle swarm optimization algorithm;

[0054] Step 5: Get the final classification results.

[0055] In Example 1, the present invention uses four public multi-task data sets for model construction and evaluation. The specific data set information is shown in Table 1.

[0056] Tables 2-7 summarize the classification accuracy of the present invention and four single-task and two multi-task classification algorithms on four data sets. By observing Tables 2-7, we can draw the following conclusions:

[0057] (1) Single-task classifiers generally perform worse than multi-task classifiers when dealing with multi-task problems, especially when there are conflicts between tasks, the performance of single-task classifiers is significantly limited. (2) When the task labeling risk is low, ordinary multi-task learning methods perform better, while when the labeling risk is high, the advantages of methods based on multi-objective optimization are more obvious. For example, on image datasets, when the labeling risk is 5%, the performance of methods such as MT-TSK-FC is relatively close to that of MOO MT-TSK-FS, but when the labeling risk increases to 30%, the performance difference between this type of algorithm and MOO MT-TSK-FS increases significantly. (3) Different algorithms perform differently on different datasets. For example, on different center data of the ABIDE dataset, the optimal indicators of different algorithms are different, but overall, MOO MT-TSK-FS obtains the most optimal indicators. The performance differences between the algorithms are more obvious on simple binary classification datasets (such as Two Moon), and the differences are even more significant on complex multi-classification datasets (such as Image dataset). (4) In the multi-task labeling risk scenario, the performance of all algorithms is improved compared to the single-task labeling risk scenario, which shows that the mutual collaboration between tasks helps to improve the classification performance. (5) It can be seen that because MOO MT-TSK-FS uses the Pareto optimization method to deal with conflicts between tasks, its performance is better than other algorithms in most cases. In addition, because MOO MT-TSK-FS integrates the advantages of multi-objective optimization and fuzzy systems, it achieves balance and collaboration between tasks. Therefore, even when the labeling risk ratio is large, MOO MT-TSK-FS still performs well.

[0058] Table 2 Performance of the proposed method and the baseline classifier on the TwoMoon dataset in a single-task labeling risk scenario with a labeling risk of 30%

[0059]

[0060] Table 3 Performance of the proposed method and the baseline classifier on the TwoMoon dataset in a multi-task labeling risk scenario with a labeling risk of 30%

[0061]

[0062] Table 4 Classification performance of the proposed method and the comparison method on the Landmine dataset

[0063]

[0064] Table 5 Classification performance of the proposed method and the comparison method on the ABIDE dataset

[0065]

[0066] Table 6 Performance of the proposed method and the baseline method on the Image dataset in single-task labeling risk scenarios with different labeling risks

[0067]

[0068] Table 7 Performance of the proposed method and baseline classifier on image datasets in multi-task labeling risk scenarios with different labeling risks

[0069]

Claims

1. A multi-task TSK fuzzy system modeling method based on multi-objective optimization, characterized in that: The following steps are involved: Step 1: Determine the number of tasks K of the multi-task data used for training, the number of fuzzy rules M corresponding to each task, and set the regularization parameter τ k and the equilibrium parameter λ; Step 2: Use FCM clustering algorithm to obtain the antecedent parameters of the fuzzy system; Step 3: Construct the objective formula of the multi-task TSK fuzzy system for multi-objective optimization; 3.1 Define the objective function of each task k as follows: in, 3.2 Constructing the overall multi-objective optimization problem: Step 4: Solve the constructed multi-objective optimization problem through the multi-objective particle swarm optimization algorithm; 4.1 Initialize the particle swarm and randomly generate particles satisfying ξ k >0 and ε k >0 particle position and velocity vectors; 4.2 Encode the decision variables as particle position vectors: x i =P g0 ,θ1,…,θ K ,ξ1,ε1,…,ξ K ,he K (6) 4.3 Calculate multi-objective vectors to evaluate particle fitness: 4.4 During the iteration process, the particle's velocity and position are updated according to the following formula: 4.5 pairs i,k and ε k Perform projection processing to force it to remain positive; 4.6 Maintain an external archive A and save the Pareto frontier solution set using the non-dominated sorting and crowding distance strategies; Step 5: Select the optimal trade-off solution from the external archive to obtain the final classification result; 5.1 Obtaining the Optimal Postprocessor Parameters 5.2 Substitute the obtained optimal consequent parameters into the symbolic function to obtain the classification result:

2. A multi-task TSK fuzzy system modeling method based on multi-objective optimization according to claim 1, characterized in that: In step 3.2, the overall multi-objective optimization problem is to integrate the constraints into the objective function through the penalty function method: Among them, W k =P g0 +θ k , μ is the penalty factor, which increases with the number of iterations to gradually strengthen the constraints.

3. A multi-task TSK fuzzy system modeling method based on multi-objective optimization according to claim 1 or 2, characterized in that: In step 4.6, the storage format of external file A is: Each stored solution contains the complete matrix and scalar parameters.

4. A multi-task TSK fuzzy system modeling method based on multi-objective optimization according to claim 1 or 2, characterized in that: The classification results in the fifth step are specifically:

5. The multi-task TSK fuzzy system modeling method based on multi-objective optimization according to claim 3 is characterized in that: The classification results in the fifth step are specifically:

Images