A feature selection method and device based on memristive firefly algorithm

Abstract

A feature selection method and device based on a memristor firefly algorithm belongs to the field of artificial intelligence technology. The method comprises the following steps: improving the traditional firefly algorithm, designing an improved firefly distance representation method and discretization method; constructing a memristor cross array and peripheral control circuits according to the required number of firefly individuals and the dimension of unscreened features, and setting the read and write voltages; using computer simulation technology to simulate the designed memristor cross array to verify the effectiveness of the design scheme; designing the encoding method of the features to be screened on fireflies; writing fireflies into the memristor cross array, iterating the algorithm, and outputting the optimal feature subset when the termination condition is reached. The present invention can avoid the problems of insufficient attraction between individuals and low search breadth in the traditional firefly algorithm when facing high-dimensional optimization problems, and can achieve efficient parallelization and scalability. The output feature subset reduces the feature dimension while improving the model recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a feature selection method and device based on a memristive firefly algorithm to reduce the dimension of training data for deep neural networks and other artificial intelligence models. Background Art

[0002] Feature selection is a crucial step in machine learning and data mining, which involves selecting the most relevant and informative feature subsets from a large number of available features. This process is crucial to improving the predictive accuracy of the model, reducing overfitting, reducing computational costs, enhancing the interpretability of the model, and meeting specific application requirements. Feature selection is particularly important when dealing with high-dimensional datasets because it can effectively handle situations where the number of features far exceeds the number of samples, thereby improving the generalization ability and overall performance of the model. Through feature selection, noise and irrelevant features can be removed, the complexity of the dataset can be reduced, data quality can be improved, and high-quality data can be ensured to be used by the model.

[0003] The Firefly Algorithm (FA) is a swarm intelligence optimization algorithm that simulates the flashing behavior of fireflies in nature. It was proposed by Xin-She Yang in 2008 and is used to solve global optimization problems. In the algorithm, each firefly represents a potential solution, and the brightness of the firefly represents the quality of the solution. Fireflies attract each other to find the optimal solution, and brighter fireflies attract other fireflies with lower brightness. The core of the firefly algorithm includes two main steps: brightness update and position update. In the brightness update stage, the algorithm determines the direction of movement of the fireflies based on the brightness difference between the fireflies; in the position update stage, the algorithm updates the position of the fireflies based on the direction of movement of the fireflies and the step size factor. The algorithm repeats these two stages and eventually finds the optimal solution to the problem.

[0004] However, the traditional firefly algorithm also has some problems. First, the firefly algorithm may oscillate during the iteration process, especially when the solution space is large or the problem is complex, which may affect the stability of the algorithm in converging to the optimal solution. Second, although the firefly algorithm usually has a fast convergence speed, in some cases, especially for complex problems with multiple local optimal solutions, the algorithm may converge to the local optimal solution prematurely instead of the global optimal solution. To this end, we propose a feature selection method for the memristive firefly algorithm.

[0005] Memristor is a new type of basic circuit element that can memorize the amount of charge or magnetic flux passing through the original. It is non-volatile and can be used as a memory. Memristors are usually integrated on a large scale in the form of a memristor crossbar array. The fully interconnected topology of the memristor crossbar array enables it to efficiently perform parallel matrix-vector addition operations, which can achieve the fusion of storage and computing, help solve the bottleneck problem in the traditional von Neumann architecture, and improve computing efficiency. The energy required to switch states is very low, which helps to reduce the power consumption of the entire system. Summary of the invention

[0006] The purpose of the present invention is to provide a feature selection method and device based on the memristive firefly algorithm, which can reduce the disadvantages of the traditional firefly algorithm and achieve efficient parallelization and scalability. The screened features can improve the model recognition accuracy while reducing the data dimension.

[0007] The present invention relates to a feature selection method and device based on a memristor firefly algorithm, and searches for an optimal feature subset through a swarm intelligence algorithm. The swarm intelligence algorithm is an optimization algorithm that simulates the behavior of biological groups in nature. It simulates the collective behavior of social organisms such as ants and bees, and uses local interactions between simple individuals to generate complex global intelligent behaviors. This algorithm does not require central control and has the characteristics of self-organization, scalability and strong robustness. It is widely used in optimization problem solving, robot collaborative operations, data mining and machine learning.

[0008] In order to achieve the above object, the technical solution of the present invention is as follows:

[0009] A feature selection method based on a memristive firefly algorithm comprises the following steps:

[0010] S101: Establish an improved firefly algorithm; specifically, use cosine similarity to improve the representation method of the attraction between fireflies, and improve the representation method of the individual moving step of fireflies based on the discretization method of the moving step of two adjacent generations;

[0011] S102: Obtain a data set, randomly initialize a firefly population, and binarize the firefly population according to the feature dimension of the data set; a feature encoding method is designed here, and "1" and "0" in the fireflies represent the use and non-use of a certain feature.

[0012] S103: construct a memristor crossbar array to represent individual fireflies; specifically, construct a memristor crossbar array according to the required number of fireflies and unscreened data feature dimensions, design a read voltage, a write voltage and a matching peripheral control circuit that conform to the physical characteristics of the memristor, and design a voltage generator with random polarity and amplitude required by the firefly algorithm;

[0013] S104: Initialize the memristor crossbar array and set all memristors to 0; then randomly initialize the firefly population. Firefly If the bit is 1, the Line Set the memristor of the column to 1; set the number of iterations;

[0014] S105: start iteration; input a reading voltage into one row to read a firefly; input voltages with the same amplitude and opposite polarity into two different rows for a period of time to achieve parallel multiplication and subtraction; a random voltage source generates a voltage with random polarity and random duration, and applies it to a fixed resistor; obtain the output result of the final charge meter to calculate the individual moving step of the firefly;

[0015] In the tth iteration, the feature with a median value of 1 in the i-th firefly is extracted as the feature subset , evaluate the recognition accuracy of the feature subset as the brightness of the firefly; design the iteration rule of the firefly algorithm, that is, when the recognition accuracy of the new feature subset is not less than that of the old feature subset, and the size of the new feature subset is smaller than that of the old feature subset, adopt the new feature subset;

[0016] S106: When the set number of iterations is reached, the final firefly population is output.

[0017] The improved firefly algorithm is specifically implemented by the following steps:

[0018] S201: Randomly initialize the population ,in It's a population of fireflies. Is in the population fireflies; set the brightness function, the brightness of each firefly individual ,in is the brightness vector of the population, is the brightness of each firefly; sets the number of iterations;

[0019] S202: Start iteration; if brightness , then firefly individual i moves to firefly individual j, and firefly individual The attraction between Represented by cosine similarity:

[0020] ;

[0021] in Firefly individual The spatial coordinates Dimensional component, Firefly individual The spatial coordinates Dimensional component, is the dimension of the solution space;

[0022] The distance an individual moves follows the following formula:

[0023] ;

[0024] ;

[0025] ;

[0026] in, , It is a manually set parameter. is a random number between 0 and 1. is the transfer function, which represents the individual The spatial coordinates The dimension component is Iterations and The absolute value of the change in the iteration, is the discretized function, Indicates negation; The output result of the charge meter of the memristor crossbar array;

[0027] S203: If the individual is the brightest individual, and its moving distance follows the following formula:

[0028] ;

[0029] ;

[0030] ;

[0031] S204: If the maximum number of iterations has been reached, proceed to S205; if not, return to S202;

[0032] S205: Output the final population.

[0033] Assume that the number of firefly populations is P, the characteristic dimension of the data is N, and construct a memristor crossbar array with P rows and N columns, including P voltage sources, memristors, N fixed resistors and N voltage sources with random polarity and amplitude; specifically, a voltage source is provided at one end of each row, and a fixed resistor and a voltage source with random polarity and amplitude are sequentially connected in series at the tail end of each column.

[0034] In each iteration, the iterative update process of the memristor crossbar array is as follows:

[0035] S401: Read the voltage Applied in row i, the final ammeter reading represents the individual and read it in the same way ;

[0036] S402: Apply voltages with the same amplitude and opposite polarity to the i-th row and the j-th row , , the duration is s; ,

[0037] S403: While performing S402, a random voltage source generates a random polarity, a random duration, and an amplitude. Voltage;

[0038] S404: The final output of the charge meter indicates , which is used to calculate the distance moved by an individual.

[0039] The recognition accuracy of the evaluation feature subset is achieved by the following method:

[0040] S501: Subset the features The recognition accuracy As a firefly Brightness;

[0041] S502: Search and iterate to output new feature subsets through the memristive firefly algorithm , and do the following:

[0042] S5021: Subset the features Use it as a training set to train the classifier and evaluate the recognition accuracy of the classifier on the classification task ;

[0043] S5022: If and , then retain the feature subset And enter the next iteration, otherwise:

[0044] S5023: Restore feature subset And enter the next iteration.

[0045] The present invention also provides a feature selection device based on a memristor firefly algorithm, comprising a feature selection circuit, a processor, a memory and a computer program stored in the memory; the processor executes the computer program to implement the method as described above; the feature selection circuit is a memristor cross array, specifically, the memristor cross array is specifically P rows*N columns, including P voltage sources, memristors, N fixed resistors and N voltage sources with random polarity and amplitude; specifically, a voltage source is set at one end of each row, and a fixed resistor and a voltage source with random polarity and amplitude are connected in series at the tail end of each column in sequence; when the method is implemented, the improved firefly algorithm completes the update of the firefly position by updating the memristor cross array.

[0046] The beneficial effects of the present invention are as follows:

[0047] The present invention adopts the firefly algorithm, which is a heuristic search algorithm. Its goal is to search for a feature subset that can reduce feature unreading while not reducing the model recognition accuracy, or even improve the model recognition accuracy. The firefly algorithm includes two stages: brightness update and position update. In the brightness update stage, the algorithm determines the movement direction of the fireflies according to the brightness difference between the fireflies; in the position update stage, the algorithm updates the position of the fireflies according to the movement direction and step factor of the fireflies. The algorithm repeats these two stages and finally finds the optimal solution to the problem.

[0048] The advantage of this algorithm is that by simulating the attraction behavior between fireflies, it can effectively explore the solution space and find the global optimal solution, which is particularly suitable for the optimization of multi-peak functions. And by introducing randomness, the firefly algorithm can effectively avoid falling into the local optimal solution and improve the overall optimization effect.

[0049] The present invention combines the firefly algorithm with a memristor, making full use of the advantages of the memristor architecture, such as high device density, low power consumption, high parallelism, and in-memory computing, to improve the algorithm execution efficiency, reduce system power consumption, and realize the hardware deployment of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0051] Figure 1 is a flow chart of feature selection based on the memristive firefly algorithm of the present invention;

[0052] Figure 2 is a flow chart of the improved firefly algorithm of the present invention;

[0053] Figure 3 is a schematic diagram of a memristor crossbar array of the present invention;

[0054] Figure 4 is a flow chart of the memristor crossbar array read and write operations of the present invention;

[0055] Figure 5 It is a schematic diagram of the feature screening step of the present invention. DETAILED DESCRIPTION

[0056] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific examples. The specific implementation of the present invention is not limited to the following examples, and appropriate changes and modifications can be made according to actual needs without affecting the essence and scope of the present invention.

[0057] Figure 1 : is a flow chart of a feature selection method based on a memristive firefly algorithm of the present invention. As shown in the figure, the method comprises the following steps:

[0058] S101: Design an improved firefly algorithm, which can improve the convergence speed of the algorithm by optimizing the representation and discretization method of the attraction between fireflies; specifically, use cosine similarity to improve the representation method of the attraction between fireflies, and design a discretization method based on the moving step length of two adjacent generations; Figure 2 As shown, it is implemented by the following steps:

[0059] S201: Randomly initialize the population ,in It's a population of fireflies. Is in the population fireflies; set the brightness function, the brightness of each firefly ,in is the brightness vector of the population, is the brightness of each firefly, which is determined by the recognition accuracy of the feature subset;

[0060] S202: If the brightness , then individual i moves to individual j, and individual The attraction between Represented by cosine similarity:

[0061]

[0062] in Is an individual The spatial coordinates Dimensional component, Is an individual The spatial coordinates Dimensional component, is the dimension of the solution space.

[0063] The distance an individual moves follows the following formula:

[0064]

[0065]

[0066]

[0067] in It is In the iteration The spatial coordinates Dimensional component, , It is a manually set parameter. is a random number between 0 and 1. is the transfer function, which represents the individual The spatial coordinates The dimension component is Iterations and The absolute value of the change in the iteration, is the discretized function, Indicates negation;

[0068] S203: If the individual is the brightest individual, and its moving distance follows the following formula:

[0069]

[0070]

[0071]

[0072] S204: If the algorithm has reached the maximum number of iterations at this time, then enter S205; if not, then return to S202;

[0073] S205: Output the final population.

[0074] S102: Obtain a data set, randomly initialize a firefly population, and binarize the firefly population according to the feature dimension of the data set; a feature encoding method is designed here, and "1" and "0" in the fireflies represent the use and non-use of a certain feature.

[0075] S103: Construct a memristor crossbar array to represent individual fireflies and implement various operations in the algorithm; specifically, construct a memristor crossbar array based on the required number of fireflies and the unscreened data feature dimensions, design read voltages, write voltages and supporting peripheral control circuits that conform to the physical characteristics of the memristor, and design a voltage generator with random polarity and amplitude required by the firefly algorithm; use computer simulation technology to simulate the designed memristor crossbar array that runs the improved firefly algorithm, and verify the effectiveness of batch reading, batch writing, parallel addition and random voltage generator of the memristor.

[0076] Specifically, Figure 3 As shown, the memristor crossbar array constructed in this embodiment is P rows*N columns, specifically including P voltage sources, memristors, N fixed resistors, and N voltage sources with random polarity and amplitude; the circles with "-+" are voltage sources; the circles with "M" are memristors, with a total of A memristor, is the size of the population, is the characteristic dimension of the firefly; the circle with crossed arrows is a voltage source with random polarity and amplitude; the rectangle marked with "R" is 100 Fixed value resistor.

[0077] S104: Initialize the memristor crossbar array and set all memristors to 0; then randomly initialize the firefly population. Firefly If the bit is 1, the Line Set the memristor of the column to 1; set the number of iterations;

[0078] S105: Start iteration. Each time the algorithm iterates, the memristor cross array is updated to update the position of the fireflies. Specifically, a read voltage is input into one row to read a firefly. Voltages with the same amplitude and opposite polarity are input into two different rows for a period of time to achieve parallel multiplication and subtraction. A random voltage source generates a voltage with random polarity and random duration, and applies it to a fixed resistor. The output result of the final charge meter is obtained to calculate the individual moving step of the fireflies.

[0079] In the tth iteration, the feature with a median value of 1 in the i-th firefly is extracted as the feature subset ,Evaluate the recognition accuracy of a subset of features, as the brightness of fireflies;

[0080] Specifically, a feature subset is selected based on fireflies. For example, there are 8 features in total. Firefly If it is "10010011", the 1st, 4th, 7th and 8th features are extracted as feature subsets. , evaluate the recognition accuracy of feature subsets (in the Firefly for ), update the position of fireflies in the memristor crossbar array by parallel addition;

[0081] S106: When the set number of iterations is reached, the final firefly population is output.

[0082] like Figure 4 As shown, it is a flowchart of the read and write operation of the memristor crossbar array of the present invention. In each iteration, the update of the position of the fireflies in the memristor crossbar array is obtained by the following steps:

[0083] S401: Read the voltage Applied in row i, the final ammeter reading represents the individual and read it in the same way ;

[0084] S402: Set the operation voltage Applied to row i and row j respectively, the duration is s;

[0085] S403: While performing S402, a random voltage source generates a random polarity, a random duration between 1ns and 1000ns, and an amplitude of Voltage;

[0086] S404: The final output of the charge meter indicates , which is used to calculate the distance moved by an individual.

[0087] like Figure 5 As shown, the recognition accuracy of the feature subset is evaluated by the following method:

[0088] S501: Subset the features The recognition accuracy As a firefly Brightness;

[0089] S502: Search and iterate to output new feature subsets through the memristive firefly algorithm , and do the following:

[0090] S5021: Subset the features Use the data as a training set to train the classifier (classify the input information according to the previous learning process) and evaluate the accuracy of the classifier on the classification task ;

[0091] S5022: If and , then retain the feature subset And enter the next iteration, otherwise:

[0092] S5023: Restore feature subset And enter the next iteration;

[0093] S503: The maximum number of iterations is reached, and the iteration is stopped;

[0094] S105: The iteration ends and the feature subset with the highest recognition accuracy is output.

[0095] This embodiment also provides a feature selection device based on a memristor firefly algorithm, including a feature selection circuit, a processor, a memory, and a computer program stored in the memory; the processor executes the computer program to implement an improved firefly algorithm; the feature selection circuit is the above-mentioned memristor cross array, specifically, the improved firefly algorithm completes the update of the firefly position by updating the memristor cross array.

[0096] In practical applications, taking the recommendation system as an example, the present invention improves the prediction accuracy of the model by identifying the features most relevant to user behavior, thereby improving the quality of personalized recommendations and key business indicators such as click-through rate and conversion rate. In addition, feature screening helps to simplify the complexity of the model, reduce computing resource consumption, and shorten training and prediction time. It can also handle high-cardinality features (product ID, user ID, etc.) and data sparsity problems, improve the interpretability of the algorithm, and optimize the performance of real-time recommendation systems. The application of the present invention can span multiple industries and provide intelligent and personalized services for recommendation systems in multiple fields such as news, business, and entertainment. At the same time, in the context of increasing attention to user privacy protection, feature screening can reasonably utilize data while protecting privacy and build an effective recommendation model.

[0097] The sonar dataset was used to perform feature selection to verify the method of the present invention. The sonar dataset used contained a total of 208 data items, each of which contained 60 features. Here, this embodiment designed a firefly population with a length of 60 binary digits and a total of 50 fireflies. And accordingly designed the following Figure 3 , a memristor cross array of 3000 memristors with 50 rows and 60 columns. Then iterate according to the above method. Table 1 shows the recognition accuracy of Logistic regression after feature selection of the sonar data set using the method of the present invention and other methods (including variance threshold method and chi-square test method). It can be seen from the table that the method of the present invention improves the recognition accuracy by 7.82% while using 32 fewer features.

[0098] Table 1

[0099] method Number of features Accuracy (%) No feature selection 60 76.85 Variance Threshold Method 33 68.29 Chi-square test 31 78.36 Memristive Firefly Algorithm 28 84.67

[0100] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A feature selection method based on a memristor firefly algorithm, characterized in that: The following steps are involved: S101: Establish an improved firefly algorithm; specifically, use cosine similarity to improve the representation method of the attraction between fireflies, and improve the representation method of the individual moving step of fireflies based on the discretization method of the moving step of two adjacent generations; S102: Obtain a data set, randomly initialize a firefly population, and binarize the firefly population according to the characteristic dimension of the data set; S103: Construct a memristor crossbar array to represent individual fireflies; Specifically, a memristor crossbar array is constructed according to the required number of fireflies and the unscreened data feature dimensions, a read voltage, a write voltage and a matching peripheral control circuit that conform to the physical characteristics of the memristor are designed, and a voltage generator with random polarity and amplitude required by the firefly algorithm is designed; S104: Initialize the memristor crossbar array and set all memristors to 0; then randomly initialize the firefly population. Firefly If the bit is 1, the Line The memristor of the column is set to 1; Set the number of iterations; S105: start iteration; A reading voltage is applied to one row to read a firefly; voltages of the same magnitude and opposite polarity are applied to two different rows for a period of time to achieve parallel multiplication and subtraction; A random voltage source generates a voltage with random polarity and random duration and applies it to a fixed resistor; the output result of the final charge meter is obtained to calculate the individual moving step length of the firefly; In the tth iteration, the feature with a median value of 1 in the i-th firefly is extracted as the feature subset , evaluate the recognition accuracy of the feature subset as the brightness of the firefly; design the iteration rule of the firefly algorithm, that is, when the recognition accuracy of the new feature subset is not less than that of the old feature subset, and the size of the new feature subset is smaller than that of the old feature subset, adopt the new feature subset; S106: When the set number of iterations is reached, the final firefly population is output.

2. A feature selection method based on a memristor firefly algorithm according to claim 1, characterized in that: The improved firefly algorithm is specifically implemented by the following steps: S201: Randomly initialize the population ,in It's a population of fireflies. Is in the population a firefly; Set the brightness function, the brightness of each firefly individual ,in is the brightness vector of the population, is the brightness of each firefly; sets the number of iterations; S202: Start iteration; if brightness , then firefly individual i moves to firefly individual j, and firefly individual The attraction between Represented by cosine similarity: ； in Firefly individual The spatial coordinates Dimensional component, Firefly individual The spatial coordinates Dimensional component, is the dimension of the solution space; The distance an individual moves follows the following formula: ； ； ； in, , It is a manually set parameter. is a random number between 0 and 1. is the transfer function, which represents the individual The spatial coordinates The dimension component is Iterations and The absolute value of the change in the iteration, is the discretization function, Indicates negation; The output result of the charge meter of the memristor crossbar array; S203: If the individual is the brightest individual, and its moving distance follows the following formula: ； ； ； S204: If the maximum number of iterations has been reached, proceed to S205; if not, return to S202; S205: Output the final population.

3. A feature selection method based on memristor firefly algorithm according to claim 2, characterized in that: Assume that the number of firefly populations is P, the characteristic dimension of the data is N, and construct a memristor crossbar array with P rows and N columns, including P voltage sources, memristors, N fixed resistors and N voltage sources with random polarity and amplitude; specifically, a voltage source is provided at one end of each row, and a fixed resistor and a voltage source with random polarity and amplitude are sequentially connected in series at the tail end of each column.

4. A feature selection method based on memristor firefly algorithm according to claim 3, characterized in that: In each iteration, the iterative update process of the memristor crossbar array is as follows: S401: Read the voltage Applied in row i, the final ammeter reading represents the individual and read it in the same way ; S402: Apply voltages with the same amplitude and opposite polarity to the i-th row and the j-th row , , the duration is s; , ; S403: While performing S402, a random voltage source generates a random polarity, a random duration, and an amplitude. Voltage; S404: The final output of the charge meter indicates , which is used to calculate the distance moved by an individual.

5. A feature selection method based on memristive firefly algorithm according to claim 4, characterized in that: The recognition accuracy of the evaluation feature subset is achieved by the following method: S501: Subset the features The recognition accuracy As a firefly Brightness; S502: Search and iterate to output new feature subsets through the memristive firefly algorithm , and do the following: S5021: Subset the features Use it as a training set to train the classifier and evaluate the recognition accuracy of the classifier on the classification task ; S5022: If and , then retain the feature subset And enter the next iteration, otherwise: S5023: Restore feature subset And enter the next iteration.

6. A feature selection method based on memristive firefly algorithm according to claim 5, characterized in that: The read voltage , the fixed resistance is 100 .

7. A feature selection device based on a memristor firefly algorithm, characterized in that: The method comprises a feature selection circuit, a processor, a memory and a computer program stored in the memory; the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6; the feature selection circuit is a memristor cross array, specifically, the memristor cross array is specifically P rows*N columns, including P voltage sources, memristors, N fixed resistors and N voltage sources with random polarity and amplitude; specifically, a voltage source is set at one end of each row, and a fixed resistor and a voltage source with random polarity and amplitude are connected in series at the tail end of each column in sequence; when the method is implemented, the improved firefly algorithm completes the update of the firefly position by updating the memristor cross array.

Images