Particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight
Through the particle swarm optimization algorithm that dynamically adjusts the inertial weight, the problems of high time cost and local optimal solutions in SVM parameter optimization are solved, and more efficient and stable parameter optimization effects are achieved.
Patent Information
- Application Number
- CN202410744109.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-07
- Publication Date
- 2024-11-08
AI Technical Summary
When optimizing error penalty parameters and kernel function parameters in support vector machines (SVMs), the existing technology has problems such as high time cost, low efficiency and local optimal solutions, and the inertial weight adjustment is not dynamic enough, resulting in poor stability.
Using a particle swarm optimization algorithm that dynamically adjusts the inertial weight, the particle swarm is initialized, the fitness value is calculated, the individual and global optimal positions are updated, and the inertial weight is adjusted according to the inertial factor formula to ensure that particles effectively get rid of local optimal solutions during the iteration process. .
It improves the efficiency and accuracy of SVM parameter optimization, reduces the occurrence of local optimal solutions, and improves the stability and optimization performance of the algorithm.
Smart Images

Figure CN118917348A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight. Background Art
[0002] In order to improve the performance and generalization ability of SVM in classification tasks, it is important to accurately select the error penalty parameter C and the kernel function parameter γ. Traditionally, the grid search algorithm is widely used in the optimization process of these two parameters, and the optimal parameters are determined by traversing the search and performing multiple verifications. This method can achieve relatively satisfactory optimization results, but its significant time cost and low efficiency have resulted in poor results in practical applications.
[0003] Genetic algorithms are based on the principles of natural selection and genetics, and optimize parameters by simulating the "survival of the fittest" principle in the process of biological evolution. Through crossover and mutation operations, the optimal solution is searched within a certain range, making the optimization process more efficient, and ultimately obtaining the best parameters through multiple generations of evolution. In contrast, the particle swarm optimization algorithm guides the search process through information sharing between individuals, with clearer goals and higher search efficiency, thus finding the optimal solution in a shorter time.
[0004] However, most of them fail to completely change the trend of reducing inertia weight, and may still cause the algorithm to fall into local optimum. At the same time, the existing method of optimizing inertia factor weight may lead to poor stability. Summary of the invention
[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a particle swarm optimization support vector machine algorithm for dynamically adjusting the inertia weight.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] The present invention provides a particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight, which specifically includes the following steps:
[0008] S1. Initialization: Generate an initial particle swarm and assign a random initial position and velocity to each particle;
[0009] S2. Calculate fitness: calculate the fitness value for each particle;
[0010] S3. Update individual best: For each particle, compare its current fitness value with its historical best fitness value, individual best, denoted as pbest; if the current position has a higher fitness value, update the current position to the new pbest;
[0011] S4. Update global optimum: for each particle, compare its fitness value with the historical best fitness value of the entire particle group, the global optimum, denoted as gbest; if the current position has a higher fitness value, update the current position to the new gbest;
[0012] When calculating the speed and position of the next iteration, for the inertia factor ω, the specific formula of its inertia weight is:
[0013]
[0014] Where x=t / T, k is a random number between 0 and 1, and its specific value is adjusted and determined according to the actual situation; rand() represents a random number generation function; the following conditions must be met:
[0015]
[0016] Where C represents an integer value greater than 1, iter represents the number of iterations, and j is a parameter between 0 and 1, whose specific value depends on the actual situation; N refers to the total number of iterations, ncc[i] represents the number of times the position of the i-th particle has not been updated during the continuous iteration process, n is a set value based on the actual situation, and pre[i] is used to indicate whether the particle has adopted this strategy in the previous iteration;
[0017] S5. Termination condition: If the preset termination condition is reached, such as the maximum number of iterations or the fitness threshold, the algorithm stops; otherwise, return to step S2 to continue execution.
[0018] As a preferred technical solution of the present invention, in step S4, the speed and position of the next iteration are calculated according to formula (1) and formula (2):
[0019] v i+1 =ω*v i +c 1 *rand()*(pbest i -x i )+c 2 *rand()*(gbest i -x i ) (1)
[0020] x i+1 =x i +v i (2)
[0021] Where i = 1, 2, ..., n, n represents the total number of particle swarms; v i Represents the current particle speed, v i+1Indicates the particle speed of the next iteration; the function rand() is used to generate a random number between 0 and 1, x i Represents the current position coordinates of the particle. Learning factor c 1 and c 2 Usually assigned a fixed value; if during the iteration v i Exceeding the allowable value V max , then v i Set to V max ; pbesti represents the best position reached by an individual particle, while gbesti represents the global optimal position reached by the entire particle swarm.
[0022] Compared with the prior art, the present invention has the following beneficial effects:
[0023] The present invention uses an improved PSO (WPSO) algorithm to more efficiently determine the error penalty parameters and kernel function parameters that can optimize the SVM classification effect; the improved PSO algorithm tends to fall into local optimal solutions in the later stages of the algorithm and the later stability problems caused by other improved PSO algorithms, aiming to achieve better optimization performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0025] Figure 1 It is the flow chart of the WPSO-SVM algorithm of the present invention. Figure 2 It is a visual display of different evaluation criteria of SVM, PSO-SVM, and WPSO-SVM algorithms on different data sets. DETAILED DESCRIPTION
[0026] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0027] Example 1
[0028] like Figure 1 As shown, the present invention provides a particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight, which specifically includes the following steps:
[0029] S1. Initialization: Generate an initial particle swarm and assign a random initial position and velocity to each particle;
[0030] S2. Calculate fitness: calculate the fitness value for each particle;
[0031] S3. Update individual best: For each particle, compare its current fitness value with its historical best fitness value, individual best, denoted as pbest; if the current position has a higher fitness value, update the current position to the new pbest;
[0032] S4. Update global optimum: For each particle, compare its fitness value with the historical best fitness value of the entire particle group, the global optimum, denoted as gbest; if the current position has a higher fitness value, update the current position to the new gbest; calculate the speed and position of the next iteration according to formula (1) and formula (2):
[0033] v +1 =ω*v i +c 1 *rand()*(pbest i -x i )+c 2 *rand()*(gbest i -x i ) (1)
[0034] x i+1 =X i +v i (2)
[0035] Where i = 1, 2, ..., n, n represents the total number of particle swarms; v i Represents the current particle speed, v i+1 Indicates the particle speed of the next iteration; the function rand() is used to generate a random number between 0 and 1, x i Represents the current position coordinates of the particle. Learning factor c 1 and c 2 Usually assigned a fixed value; if during the iteration v i Exceeding the allowable value V max , then v i Set to V max ; pbesti represents the best position reached by an individual particle, while gbesti represents the global optimal position reached by the entire particle group; and for the inertia factor ω, the following formula is satisfied:
[0036]
[0037] Among them, x = t / T, k is a random number between 0 and 1, and its specific value is adjusted and determined according to the actual situation; rand() represents a random number generation function; it will change during the optimization process of the particle swarm optimization algorithm. If the number of iterations of a particle is lower than a certain threshold, and the particle has not moved in the local optimal position for n times, it will be increased accordingly, thereby increasing the inertia weight and the perturbation amplitude of the particle, so that the particle has a greater possibility of escaping the local optimum; the following conditions must be met:
[0038]
[0039] Where C represents an integer value greater than 1, iter represents the number of iterations, and j is a parameter between 0 and 1, whose specific value depends on the actual situation; N refers to the total number of iterations, ncc[i] represents the number of times the position of the i-th particle has not been updated during the continuous iteration process, n is a set value based on the actual situation, and pre[i] is used to indicate whether the particle has adopted this strategy in the previous iteration;
[0040] S5. Termination condition: If the preset termination condition is reached, such as the maximum number of iterations or the fitness threshold, the algorithm stops; otherwise, return to step S2 to continue execution.
[0041] The present invention is based on the method flow, and tests the data set for weight formulas (3) and (4), and constructs an experimental flow;
[0042] In order to evaluate the performance of the WPSO algorithm, seven datasets including Vehicle, Wine, Iris, Heart, Phishing, CMC and Rice were selected from the standard UCI dataset for testing. The UCI dataset (University of California, Irvine Machine Learning Repository) is a public database created by the machine learning group of the University of California, Irvine. It aims to provide convenient data resources for machine learning and data mining research, so that researchers can test and compare various machine learning algorithms and methods. No additional feature extraction was performed on the above seven datasets. A detailed description of each dataset is shown in Table 1.
[0043]
[0044] Table 1 Dataset feature types
[0045] WPSO will be combined with SVM to form a WPSO-SVM model. This model uses the WPSO algorithm to optimize the parameters of SVM. In order to verify the wide applicability of the WPSO algorithm, the data set selected in this study not only includes the emotional speech selection data set, but also extends to seven data sets in the standard UCI data set: Vehicle, Wine, Iris, Heart, Phishing, CMC and Rice. At the same time, to comprehensively evaluate the impact of the WPSO algorithm and the introduction of state features, this experiment set up a control group as follows:
[0046] We selected the improved algorithms for the inertia factor in the particle swarm optimization algorithm published in recent years, namely DWPSO, SWISPSO, AIWPSO, IDWPSO, and DSOPSO. The above algorithms are all public algorithm processes, and then combined with SVM to train and predict the above eight data sets. In this experiment, we mainly evaluate the accuracy and stability of the algorithm on different data sets, and the stability will be reflected by the variance of the accuracy.
[0047] Other types of improved algorithms are selected for comparison with the proposed WPSO algorithm. The selected algorithms include GA, sparrow search algorithm, whale optimization algorithm, and sine-cosine algorithm. The advantages and disadvantages of different algorithms are compared by evaluating the accuracy of the above algorithms on eight data sets.
[0048] The proposed WPSO-SVM algorithm was subjected to multi-dimensional experiments to evaluate the improvement effect. Different evaluation indicators such as accuracy, precision, recall and F1 score were used to show the comparison results. The experimental results are calculated based on the average of multiple experiments to calculate each indicator.
[0049] The WPSO-SVM algorithm is compared with the PSO-SVM algorithm which is a combination of the traditional PSO algorithm and SVM. The results are shown in Table 2, which contains a comparison between the accuracy, precision, recall and F1 data.
[0050]
[0051]
[0052] Table 2 Comparison of methods under the dataset
[0053] The results show that on multiple datasets, the WPSO-SVM algorithm outperforms the PSO-SVM algorithm in terms of accuracy, precision, recall, and F1 score. In particular, on the Wine, Iris, and Heart datasets, the accuracy of WPSO-SVM is 2.77%, 1.89%, and 1.85% higher than that of PSO-SVM, respectively. On the Heart dataset, the recall rate of WPSO-SVM reaches 99.68%, which is significantly higher than that of PSO-SVM. On the Iris and Heart datasets, the F1 scores of WPSO-SVM reach 98.62% and 94.29%, respectively, which also exceeds the performance of SVM and PSO-SVM. Figure 2 This is a visual display of SVM, PSO-SVM, and WPSO-SVM algorithms on different data sets for different evaluation criteria. From the displayed results, we can see that the WPSO-SVM algorithm has better results in most data sets.
[0054] Table 3 shows the experimental results of the accuracy and stability comparison of the DWPSO algorithm and other PSO improved algorithms.
[0055]
[0056]
[0057] As can be seen from the table, the support vector machine using the WPSO optimization strategy shows better classification accuracy and lower variance on most data sets. For example, on the Vehicle data set, the WPSO algorithm achieved a classification accuracy of 81.89%, the highest among all algorithms, and its variance was 0.26, the lowest among all algorithms, with good stability. In addition, the WPSO algorithm has the highest classification accuracy on the Iris, Heart, and CMC data sets. At the same time, except for the higher variance on the Iris and Heart data sets than other algorithms, the variance in the remaining six data sets is the lowest, that is, compared with other algorithms, it has better stability.
[0058] Table 4 is a comparison of the accuracy of GA, SSA, WOA, SCA algorithms and the proposed WPSO algorithm. It can be seen from the table that except for the Vehicle and CMC datasets, the accuracy of WPSO is slightly worse than that of other algorithms. On the other five datasets, WPSO has better accuracy performance.
[0059]
[0060] Table 4 Comparison of accuracy under different algorithms
[0061] In summary, the algorithm constructed using the process of this method has higher accuracy, adaptability, speed, update degree and computational efficiency, especially in the face of dynamically adjusted processes, which can more easily adapt to the model calculation of the data set.
[0062] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight, characterized in that: The specific steps include: S1. Initialization: Generate an initial particle swarm and assign a random initial position and velocity to each particle; S2. Calculate fitness: calculate the fitness value for each particle; S3. Update individual best: For each particle, compare its current fitness value with its historical best fitness value, individual best, denoted as pbest; if the current position has a higher fitness value, update the current position to the new pbest; S4. Update global optimum: for each particle, compare its fitness value with the historical best fitness value of the entire particle group, the global optimum, denoted as gbest; if the current position has a higher fitness value, update the current position to the new gbest; When calculating the speed and position of the next iteration, for the inertia factor ω, the specific formula of its inertia weight is: Where x = t / T, k is a random number between 0 and 1, and its specific value is adjusted and determined according to the actual situation; the following conditions must be met: Where C represents an integer value greater than 1, iter represents the number of iterations, and j is a parameter between 0 and 1, whose specific value depends on the actual situation; N refers to the total number of iterations, ncc[i] represents the number of times the position of the i-th particle has not been updated during the continuous iteration process, n is a set value based on the actual situation, and pre[i] is used to indicate whether the particle has adopted this strategy in the previous iteration; S5. Termination condition: If the preset termination condition is reached, such as the maximum number of iterations or the fitness threshold, the algorithm stops; otherwise, return to step S2 to continue execution.
2. The particle swarm optimization support vector machine algorithm for dynamically adjusting inertia weight according to claim 1 is characterized in that: In step S4, the speed and position of the next iteration are calculated according to formula (1) and formula (2): V i+1 =ω*v i +c1*rand()*(pbest i -x i )+c2*rand()*(gbest i -x i ) (1) x i+1 =x i +v i (2) Where i = 1, 2, ..., n, n represents the total number of particle swarms; v i Represents the current particle speed, v i+1 Indicates the particle speed of the next iteration; the function rand() is used to generate a random number between 0 and 1, x i Represents the current position coordinates of the particle. The learning factors c1 and c2 are usually assigned fixed values; if v i Exceeding the allowable value V max , then v i Set to V max ; pbesti represents the best position reached by an individual particle, while gbesti represents the global optimal position reached by the entire particle swarm.
Citation Information
Cited By
DDoS detection method for optimizing convolutional neural network based on particle swarm optimization
CN120017361A
