A collaborative evolution multi-component spectral feature selection method based on deep reinforcement learning

Abstract

The embodiment of the application discloses a collaborative evolution multi-component spectral feature selection method based on deep reinforcement learning, which comprises collecting and collecting spectral signals and component concentration values in samples, encoding and normalizing individuals in a field spectrum database SDate, calculating feature selection rates of all individuals in the field spectrum database SDate, establishing a DQN training model, and initializing a deep neural network Net and an empty experience pool M. In the iterative process of the training set, the ratio of variables in each interval in the population to the total variables is counted, the initial main population and multiple auxiliary populations are crossed and trained and updated DQN to output the final population, and the final population constructs a mixed multi-component spectral quantitative analysis model of water, fat and protein. The deep reinforcement learning model trained in the early stage of evolution recommends the auxiliary population to evolve collaboratively with the main population, so as to quickly obtain the optimal spectral feature subset.

Description

Technical Field

[0001] This application relates to the field of spectral feature variable selection in multi-component quantitative spectral analysis, and in particular to a co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning. Background Technology

[0002] In recent years, with the rapid development of chemometrics and instrument technology, quantitative spectral analysis, with its rapid and non-destructive characteristics, has been widely used in many fields. For example, in the food industry, it is used to analyze the content of food components (sugar, fat, and protein) and their additives to ensure food quality and safety; in the research and development and production of pharmaceuticals, it is used to determine drug purity to ensure consistency and safety. The theoretical basis of quantitative spectral analysis is the Lambert-Beer law, which states that there is a certain linear relationship between the absorbance of a substance and its concentration under ideal conditions. Regression algorithms can be used to construct component concentration prediction models to predict the concentration of the analyte. Partial linear squares (PLS) is the most commonly used regression modeling method in quantitative spectral analysis. PLS uses full-wavelength spectral data to establish a component concentration prediction model. However, the spectral data collected in practical applications contains absorbance data of hundreds or thousands of wavelength variables. Problems such as low spectral absorption intensity, collinearity, and cross-sensitivity result in a large number of redundant and interfering wavelength variables in the spectral data. Effective feature variable selection methods can improve the predictive ability of the model and reduce its complexity.

[0003] Due to the enormous dimensionality of spectral wavelength variables and the limited number of effective variables, the spectral feature selection problem can be defined as a sparse, large-scale, multi-objective optimization problem. Current methods for solving this problem can be broadly categorized into three types:

[0004] The first category is based on decision variable grouping methods. The main idea is to divide the decision variables into different groups and optimize them using a divide-and-conquer approach through independent subpopulations. However, the variable grouping techniques used can significantly impact their performance. The second category is based on problem reconstruction methods. The main idea is to reduce the decision space by using dimensionality reduction techniques on the decision variables or transforming complex, large-scale problems into more manageable smaller-scale problems. This transformation allows optimization algorithms to search the solution space more efficiently, thus finding the Pareto optimal solution. The third category uses special descendant generation strategies, such as designing more sophisticated regeneration operators or probabilistic models, to improve search efficiency.

[0005] Evolutionary multi-objective optimization methods have demonstrated excellent performance in the field of feature selection. However, in the problem of multi-component spectral feature selection, the complex relationships between components and the uncertainty in the distribution and quantity of wavelength features make it challenging to solve the problem quickly and efficiently. Summary of the Invention

[0006] This invention addresses some shortcomings in existing technologies by providing a co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning. The algorithm initializes with a main population and three auxiliary populations. Multiple auxiliary populations containing single-component information co-evolve with the main population containing multi-component information, thereby better exploring the search space. During the evolutionary process, a deep reinforcement learning model (deep Q network, DQN) trained in the early stages of evolution recommends the optimal co-evolutionary strategy for the current population state, i.e., recommending one auxiliary population to co-evolve with the main population, aiming to quickly obtain the optimal subset of spectral features.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning, characterized by comprising:

[0009] Step 1: Data Collection

[0010] The spectral signals and component concentration values ​​in the samples are collected and compiled into a field spectral database SDate, which includes multiple wavelength variables;

[0011] Step 2: Encoding Initialization

[0012] Individuals in the field spectral database SDate are encoded and normalized. The processed field spectral database SDate is then divided into a training set and a validation set according to a preset ratio.

[0013] The training set is used to generate the initial main population P through initialization. 主 and multiple auxiliary populations P1, P2...P K ;

[0014] Step 3: Obtain the target function

[0015] Calculate the feature selection ratio of all individuals in the field spectral database SDate, and use the prediction model to calculate the prediction accuracy of the field spectral database SDate based on the feature selection ratio;

[0016] Step 4: DQN Initialization

[0017] Establish a DQN training model and initialize a deep neural network Net and an empty experience pool M. During the iteration of the training set, count the ratio of variables in each interval to the total variables in the population.

[0018] Step 5: DQN Initialization

[0019] The initial principal population P 主 and multiple auxiliary populations P1, P2...P K Mating is carried out to obtain offspring populations, which are then optimized using DQN to obtain a new generation of offspring populations;

[0020] Step Six: Sample Collection and Training of DQN

[0021] DQN adds the samples obtained in step four to the experience pool M to train DQN;

[0022] Step 7: Late-stage population evolution guided by DQN

[0023] Repeat step five to evolve the population, complete the training and update of the DQN, and then use the updated DQN to reward and evaluate the experience pool M to output the final population P. 主 Through the final population P 主 A multi-component spectroscopic quantitative analysis model for water, fat, and protein mixtures was constructed.

[0024] Optionally, in some embodiments of this application, step two includes:

[0025] Step 2.1: Initialize Individual Encoding Method

[0026] The code for the i-th individual in the field spectral database SDate is: Individuals are encoded using binary;

[0027] Step 2.2: Data Partitioning

[0028] The dataset is normalized, and the database SDate is divided into a training set and a validation set according to a preset ratio R:1. The training set is used to train each individual in the population, and the validation set is used to verify the performance of the spectral quantitative analysis model built under each feature subset.

[0029] Step 2.3: Initialize parameter settings and population

[0030] The population size N and the number of maximum function evaluations FE max Initialization is performed, and an initial principal population P is generated from the training set using a random initialization method. 主 And auxiliary populations P1, P2...P K .

[0031] Optionally, in some embodiments of this application, step three includes:

[0032] Step 3.1: Obtain the feature selection ratio

[0033] Calculate the trait selection ratio for all individuals in each population, where the selected trait ratio is determined by... Obtain, D select D and D represent the number of selected characteristic wavelengths and the total number of characteristic wavelengths, respectively;

[0034] Step 3.2: Obtain model accuracy

[0035] The spectral feature selection method calculates the prediction accuracy f2(x) = RMSECV of the PLS model for all individuals in each population by establishing a multi-component content prediction model;

[0036]

[0037] In the formula, n is the total number of training samples; y i It is the measured value of the i-th sample; It is the predicted output value of the i-th sample during the cross-validation process.

[0038] Optionally, in some embodiments of this application, step four includes setting state settings, which include the average sparsity of the main population, the standard deviation of individual sparsity, the proportion of variables in multiple important intervals, and the current evaluation count relative to the maximum evaluation count FE. max The proportion;

[0039] The average sparsity of the main population is ξ. t The dominant population is represented as:

[0040]

[0041] The average sparsity is expressed as:

[0042]

[0043] in This represents the j-th wavelength variable bit of the i-th individual, and N is the number of choices for the individual;

[0044] The standard deviation τ of individual sparsity t Let the standard deviation of the sparsity sum of all N individuals be:

[0045]

[0046] In the formula, It is the sum of all variable bits of the i-th individual, which represents the sparsity sum of the i-th individual;

[0047] The proportion of variables in multiple important intervals is represented by the weight vector of the PLS model built in the solution evaluation stage.

[0048] Optionally, in some embodiments of this application, step four includes setting an action setting and a reward setting. The action setting initializes the action, and the action is set to 'a'. t , so as to evolve the population;

[0049] In the reward settings, let the initial reward value be r. t =0, reward r t The reward is set to reflect the improvement in HV value of the current population relative to the previous generation. HV t and HV t-1 These represent the HV values ​​of the current generation and the previous generation of the dominant population, respectively.

[0050] Optionally, in some embodiments of this application, in step five, the initial main population P is respectively... 主 and multiple auxiliary populations P1, P2...P K Select the parent population containing individuals for mating pool selection;

[0051] Obtain parent individuals a1 = [x1, x2, ... x2] from the parent population. 100 ] and a2 = [x1, x2, ... x 100 b1 = [x1, x2, ... x 100 ] and b2 = [x1, x2, ... x 100 c1 = [x1, x2, ... x 100 c2 = [x1, x2, ... x2] 100 ], d1=[x1,x2,...x 100 ] and d2 = [x1, x2, ... x 100 The parent individuals are then subjected to crossover and mutation operations to obtain offspring o1 = [x1, x2, ... x]. 100 f1 = [x1, x2, ... x 100 ], g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ];

[0052] Repeat the above process 99 times, each time obtaining two parents from the remaining parent individuals to produce one offspring, ultimately obtaining the offspring population Off. 主 =[o1,o2,...o 100 Off1 = [f1, f2, ... f 100 Off2 = [g1, g2, ... g] 100 ] and Off3 = [h1,h2,...h 100 ].

[0053] Optionally, in some embodiments of this application, step five further includes the auxiliary population and the main population co-evolving by sharing offspring information:

[0054] First, off the offspring population of the cooperative population. help With the parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P based on fitness values ​​via a binary tournament. 主 '∪Off 主 ∪Off help Select N superior individuals as the main population P of the new generation. 主 new Off the offspring population of the main population 主 With the parent population P of the co-population help 'and its offspring population Off help Merging, based on fitness values, is performed in a binary tournament from the merged population P. help '∪Off help ∪Off 主 Select N superior individuals as the next generation subpopulation P. help new Offset the offspring populations of the other two populations other1 and Off other2 Each of its parent populations P other1 'and P other2 Merge populations P based on their fitness values ​​through a binary tournament. other1 '∪Off other1 and P other2 '∪Off other2 N superior individuals are selected from each of the two groups as the next generation of subpopulation P. other1 new and P other2 new .

[0055] Optionally, in some embodiments of this application, in step six:

[0056] DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the state of the current population, S2 is the state of the next generation population, and a1 is the current action initialization;

[0057] When the current population evaluation number FE S Percentage of the most reviews Meanwhile, the newly generated population is continuously assisted in completing the next generation of evolution with the main population, and samples from the s-th generation are collected. s =(ss ,a s ,r s ,s s+1 Add the samples to the experience pool M to complete the collection of training samples;

[0058] Samples are extracted from the experience pool M and standardized to initialize the training parameters and neural network weights θ of the DQN network;

[0059] For a given sample e t =(s t ,a t ,r t ,s t+1 The predicted value r of the sample is obtained through forward propagation. t The prediction error is calculated, and the network weights θ are adjusted along the negative gradient direction of the prediction error. When the maximum number of training iterations of 100 is reached, the DQN training is complete.

[0060] Optionally, in some embodiments of this application, in step seven:

[0061] DQN training updates are performed using the L formula:

[0062]

[0063] q t =r t +γmax a∈A Net(s t+1 a)

[0064] In the formula, T represents all samples in the experience pool M, and Net(.) represents the state s. t The following actions a t The predicted reward value r t ', q t This represents the expected reward output value, which is determined by the current reward r. t and the maximum reward value obtained by taking the next step. a∈A Net(s t+1 a) Composition, where γ is the discount factor.

[0065] Compared with the prior art, the beneficial effects of this invention are as follows:

[0066] 1. This invention defines the wavelength variable selection problem in multi-component spectral quantitative analysis as a large-scale sparse multi-objective optimization problem. By combining evolutionary multi-objective optimization algorithm with deep reinforcement learning, the optimal feature subset can be effectively selected, thereby significantly improving the prediction accuracy of the model while reducing the model complexity. This method comprehensively considers the complexity of multi-component spectral features, making the model perform better when processing real data.

[0067] 2. This invention achieves better exploration of the search space and approximation of the Pareto optimal solution by realizing the co-evolution of the main population and the auxiliary population. This multi-population co-evolution mechanism can effectively improve the performance of the feature selection process and enhance the overall optimization capability. In addition, the DQN trained in the early stage of evolution can dynamically adjust the selection strategy of the auxiliary population in different generations, realize effective dynamic co-evolution of the population, and ensure the flexibility of the algorithm.

[0068] 3. In this invention, DQN will be updated periodically with new samples collected during the evolution process, further realizing the effective synergy of information from different components, thereby ensuring the accuracy of multi-component spectral feature selection;

[0069] 4. Traditional feature selection methods often ignore the complex correlations between multi-component spectral features. This invention, through the DQN intelligent recommendation co-evolution strategy, can make full use of the commonalities and uniqueness between different features, thereby improving the efficiency and accuracy of feature selection.

[0070] 5. This feature selection method can be widely applied to various fields that require spectral feature selection, such as food, agriculture, chemistry, and medicine. Its high efficiency and accuracy make it highly valuable in practical applications. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This is a flowchart illustrating the co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning provided in this application embodiment. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this application. It is understood that the accompanying drawings are provided for reference and illustration only, and are not intended to limit this application. The connection relationships shown in the accompanying drawings are only for clear description and do not limit the connection method.

[0074] Specifically, such as Figure 1As shown in the embodiments of this application, a co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning is provided.

[0075] In this embodiment, the invention is described in more detail using three-component meat spectral data containing moisture, fat, and protein as an example. Figure 1 As shown, the specific steps are as follows:

[0076] Step 1: Data Collection

[0077] First, spectral signals and component concentration values ​​of meat samples were collected using a Tecator Infratec food and feed analyzer, forming the field spectral database SDate. The SDate database includes 100 wavelength variables.

[0078] Step 2: Individual Coding and Initialization

[0079] Step 2.1: Initialize Individual Encoding Method

[0080] The code for the i-th individual is A binary individual encoding method is used, where each individual element is either 1 or 0. A value of '1' indicates that the corresponding feature variable is selected, and a value of '0' indicates that the corresponding wavelength is not selected. Each individual represents a feature selection scheme.

[0081] Step 2.2: Data Partitioning

[0082] The meat dataset was normalized, and the database SDate was divided into a training set and a validation set according to a preset ratio of R:1, with R value set to 4. The training set was used to train each individual in the population, i.e., the selected feature subset, while the validation set was used to verify the performance of the spectral quantitative analysis model built under each feature subset, using a five-fold cross-validation method.

[0083] Step 2.3: Initialize parameter settings and population

[0084] The initial population size N and the maximum number of function evaluations FEmax were set to 100 and 10000, respectively, and the initial main population P was generated using a random initialization method. 主 Auxiliary populations P1, P2, and P3. 主 This is used to solve the problem of selecting spectral features from a mixed multi-component system of water, fat, and protein. P1, P2, and P3 are used to solve the problems of selecting spectral features from water, fat, and protein, respectively.

[0085] Step 3: Obtain the objective function value

[0086] Step 3.1: Obtain the feature selection ratio

[0087] Calculate the trait selection ratio for all individuals in each population, where the selected trait ratio is determined by... Obtain, D select D and 100 represent the number of selected characteristic wavelengths and the total number of characteristic wavelengths, respectively.

[0088] Step 3.2: Obtain model accuracy

[0089] The partial least squares (PLS) method was used to establish a prediction model for the multi-component content of meat. The prediction accuracy of the model was represented by the root mean square error of five-fold cross-validation (RMSECV) and used as one of the fitness functions. The prediction accuracy f2(x) = RMSECV of the PLS model for all individuals in each population was calculated. The calculation of RMSECV is shown in equation (1):

[0090]

[0091] In the formula, n is the total number of training samples; y i It is the measured value of the i-th sample, which is specifically the concentration of the spectral component and is a measured value; It is the predicted output value of the i-th sample during the cross-validation process, specifically the predicted value of the spectral component concentration.

[0092] Step 4: DQN Initialization

[0093] Step 4.1: Initial Setup

[0094] To adaptively select the optimal cooperative population based on the current population state, a DQN mapping model between the population state and the cooperative strategy needs to be established. First, a deep neural network Net is randomly initialized, and an empty experience pool M is initialized to store the DQN training samples.

[0095] Step 4.2: Status Setting

[0096] The state of the main population in the t-th iteration ξ t ,τ t , ... μ t These are the average sparsity of the main population, the standard deviation of individual sparsity, the proportion of variables in the 10 normalized significance intervals, and the current evaluation frequency relative to the maximum evaluation frequency (FE). max The proportion. First, calculate the average sparsity ξ of the main population. t The main population is represented as Average sparsity in This represents the j-th wavelength variable bit of the i-th individual.

[0097] Next, calculate the standard deviation τ of the sparsity of individuals in the main population.t The standard deviation τ of individual sparsity t Let the standard deviation of the sparsity sum of all N individuals be:

[0098]

[0099] In the formula, It is the sum of all variable bits of the i-th individual, which represents the sparsity sum of the i-th individual.

[0100] Then, the weight vector of the PLS model built in the solution evaluation phase is used to represent the importance of each variable. The proportion of each variable in each importance interval (divided into 10 equal-length intervals within [0, 1]) to the total number of variables D is calculated. The proportions of these 10 variables are expressed as follows:

[0101] Finally, calculate the current algebraic evaluation number FE. t The proportion of the maximum number of evaluations (FEmax)

[0102] Step 4.3: Action Setup

[0103] Set the initial action to 1, action a t =1 / 2 / 3, representing the selection of water, fat, and protein as single-component auxiliary populations to assist the main population in completing the evolution of generation t;

[0104] The purpose of setting actions is to obtain the first sample. A sample contains the current state, action, and reward obtained by the action, which is used to train DQN in each iteration.

[0105] Step 4.4: Reward Settings

[0106] Initial reward value r t =0, reward r t The reward is set to reflect the improvement in HV value of the current population relative to the previous generation. HV t and HV t-1 These represent the HV values ​​of the main population in the current generation and the previous generation, respectively.

[0107] Step 5: Evolution in the early stages of population development

[0108] Step 5.1: For population P respectively 主 P1, P2, P3 select a parent population P' containing 200 individuals. 主 P1', P2', P3' are used to select mating pools;

[0109] Step 5.2: From the parent population P' 主In P1', P2', and P3', obtain any two parent individuals a1 = [x1, x2, ... x2] from the parent individuals respectively. 100 ] and a2 = [x1, x2, ... x 100 b1 = [x1, x2, ... x 100 ] and b2 = [x1, x2, ... x 100 c1 = [x1, x2, ... x 100 c2 = [x1, x2, ... x2] 100 ], d1=[x1,x2,...x 100 ] and d2 = [x1, x2, ... x 100 ], and perform crossover and mutation operations respectively to obtain offspring o1 = [x1, x2, ... x 100 f1 = [x1, x2, ... x 100 ], g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ].

[0110] Step 5.3: Repeat step 5.299 times, each time obtaining two parents from the remaining parents to produce one offspring, ultimately obtaining the offspring population Off. 主 =[o1,o2,...o 100 Off1 = [f1, f2, ... f 100 Off2 = [g1, g2, ... g] 100 ] and Off3 = [h1,h2,...h 100 ];

[0111] Step 5.4: DQN performs a random action, randomly selecting an auxiliary population P. help Optimize in coordination with the main population, as detailed in step 5.5;

[0112] Step 5.5: Auxiliary population and main population co-evolve by sharing offspring information: First, the offspring population Off of the auxiliary population selected by DQN, i.e., the co-evolutionary population... help With the parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P based on fitness values ​​via a binary tournament. 主 '∪Off 主 ∪Off help Select N superior individuals as the main population P of the new generation. 主 new Then the offspring population of the main population Off 主 With the parent population P of the co-population help'and its offspring population Off help Merging, based on fitness values, is performed in a binary tournament from the merged population P. help '∪Off help ∪Off 主 Select N superior individuals as the next generation subpopulation P. help new Finally, the offspring populations of the remaining two populations were removed. other1 and Off other2 Each of its parent populations P other1 'and P other2 Merge populations P based on their fitness values ​​through a binary tournament. other1 '∪Off other1 and P other2 '∪Off other2 N superior individuals are selected from each of the two groups as the next generation of subpopulation P. other1 new and P other2 new .

[0113] Step 6: Sample collection and training of DQN

[0114] Step 6.1: According to steps 4.2-4.4, DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the current state of the population, S2 is the state of the next generation, and a1 is the initialization of the current action;

[0115] Step 6.2: When Then, repeat steps 5.1-5.4 for the resulting new generation population to complete the evolution of the next generation, and collect the s-th generation sample e. s =(s s ,a s ,r s ,s s+1 Add the samples to the experience pool M to complete the collection of training samples;

[0116] Step 6.3: Extract samples from the experience pool M and standardize them, initializing the training parameters of the DQN network: neural network weights θ (randomly initialized), maximum training iterations v = 3, number of hidden layers s = 3, number of neurons w = 10, and the DQN learning rate lr = 0.001. Then, for a given sample e... t =(s t ,a t ,r t ,s t+1 The predicted value r of the sample is obtained through forward propagation. tThis process calculates the prediction error and adjusts the network weights θ along the negative gradient of the prediction error. Finally, DQN training is complete when the maximum number of training iterations (100) is reached.

[0117] Step 7: Late-stage population evolution guided by DQN

[0118] Step 7.1: Repeat steps 5.1-5.4 to perform mating pool selection and generate offspring;

[0119] Step 7.2: Generate a random number rand∈[0,1]. Random selection strategy Jump to step 7.3. If DQN obtains the current population state s t And using this as input, we obtain the reward values ​​for different collaborative selection strategies. For strategy a t Its reward value is r t Among all strategies, the collaborative selection strategy with the maximum reward value is represented as: Choose the strategy with the maximum reward value Guide the co-evolutionary process, as detailed in step 7.3;

[0120] Step 7.3: Action or The representative auxiliary co-evolutionary population and its offspring are denoted as P. help and Off help Repeat step 5.4 to complete the co-evolution process between the auxiliary population and the main population, generate a new population and obtain new samples to add to the experience pool M;

[0121] Step 8: Update DQN

[0122] Step 8.1: Repeat step 7 to complete one generation of population evolution. Update DQN every 10 evolutions. See step 8.2 for details on DQN updates.

[0123] Step 8.2: Select the 10 newest samples from the experience pool M and update DQN according to equation (3):

[0124]

[0125] q t =r t +γmax a∈A Net(s t+1 ,a) (4)

[0126] In equation (3), T represents all samples in the experience pool M, and Net(.) represents the state s. t The following actions a t The predicted reward value rt ', q t This represents the expected reward output value, which is determined by the current reward r. t and the maximum reward value obtained by taking the next step. a∈A Net(s t+1 a) Composition, where γ∈[0,1] is the discount factor;

[0127] Step 8.3: When the maximum function evaluation reaches 10000 times, output the final population P. 主 Population P 主 The individuals in the model represent the selected subset of multi-component spectral features, which can be used to construct a mixed multi-component spectral quantitative analysis model for water, fat, and protein.

[0128] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning, characterized in that, include: Step 1: Data Collection The spectral signals and component concentration values ​​in the samples are collected and compiled into a field spectral database SDate, which includes multiple wavelength variables; Step 2: Encoding Initialization Individuals in the field spectral database SDate are encoded and normalized. The processed field spectral database SDate is then divided into a training set and a validation set according to a preset ratio. The training set is used to generate the initial main population P through initialization. 主 and multiple auxiliary populations P1, P2...P K ; Step 3: Obtain the target function Calculate the feature selection ratio of all individuals in the field spectral database SDate, and use the prediction model to calculate the prediction accuracy of the field spectral database SDate based on the feature selection ratio; Step 4: DQN Initialization Establish a DQN training model and initialize a deep neural network Net and an empty experience pool M. During the iteration of the training set, count the ratio of variables in each interval to the total variables in the population. Step 5: DQN Initialization The initial principal population P 主 and multiple auxiliary populations P1, P2...P K Mating is carried out to obtain offspring populations, which are then optimized using DQN to obtain a new generation of offspring populations; Step Six: Sample Collection and Training of DQN DQN adds the samples obtained in step four to the experience pool M to train DQN; Step 7: Late-stage population evolution guided by DQN Repeat step five to evolve the population, complete the training and update of the DQN, and then use the updated DQN to reward and evaluate the experience pool M to output the final population P. 主 Through the final population P 主 A multi-component spectroscopic quantitative analysis model for water, fat, and protein mixtures was constructed.

2. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, Step two includes: Step 2.1: Initialize Individual Encoding Method The code for the i-th individual in the field spectral database SDate is: Individuals are encoded using binary; Step 2.2: Data Partitioning The dataset is normalized, and the database SDate is divided into a training set and a validation set according to a preset ratio R:

1. The training set is used to train each individual in the population, and the validation set is used to verify the performance of the spectral quantitative analysis model built under each feature subset. Step 2.3: Initialize parameter settings and population The population size N and the number of maximum function evaluations FE max Initialization is performed, and an initial principal population P is generated from the training set using a random initialization method. 主 And auxiliary populations P1, P2...P K .

3. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, Step three includes: Step 3.1: Obtain the feature selection ratio Calculate the trait selection ratio for all individuals in each population, where the selected trait ratio is determined by... Obtain, D select D and D represent the number of selected characteristic wavelengths and the total number of characteristic wavelengths, respectively; Step 3.2: Obtain model accuracy The spectral feature selection method calculates the prediction accuracy f2(x) = RMSECV of the PLS model for all individuals in each population by establishing a multi-component content prediction model; In the formula, n is the total number of training samples; y i It is the measured value of the i-th sample; It is the predicted output value of the i-th sample during the cross-validation process.

4. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, In step four, state settings are set, including the average sparsity of the main population, the standard deviation of individual sparsity, the proportion of variables in multiple important intervals, and the current evaluation count relative to the maximum evaluation count (FE). max The proportion; The average sparsity of the main population is ξ. t The dominant population is represented as: The average sparsity is expressed as: Where x i j This represents the j-th wavelength variable bit of the i-th individual, and N is the number of choices for the individual; The standard deviation τ of individual sparsity t Let the standard deviation of the sparsity sum of all N individuals be: In the formula, It is the sum of all variable bits of the i-th individual, which represents the sparsity sum of the i-th individual; The proportion of variables in multiple important intervals is represented by the weight vector of the PLS model built in the solution evaluation stage.

5. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, In step four, action settings and reward settings are configured. The action settings initialize the action, and the action is set to 'a'. t , so as to evolve the population; In the reward settings, let the initial reward value be r. t =0, reward r t The reward is set to reflect the improvement in HV value of the current population relative to the previous generation. HV t and HV t-1 These represent the HV values ​​of the current generation and the previous generation of the dominant population, respectively.

6. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, In step five, the initial main population P is respectively... 主 and multiple auxiliary populations P1, P2...P K Select the parent population containing individuals for mating pool selection; Obtain parent individuals a1 = [x1, x2, ... x2] from the parent population. 100 ] and a2 = [x1, x2, ... x 100 b1 = [x1, x2, ... x 100 ] and b2 = [x1, x2, ... x 100 c1 = [x1, x2, ... x 100 c2 = [x1, x2, ... x2] 100 ], d1=[x1,x2,...x 100 ] and d2=[x1,x2,...x 100 The parent individuals are then subjected to crossover and mutation operations to obtain offspring o1 = [x1, x2, ... x]. 100 f1 = [x1, x2, ... x 100 ], g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ]; Repeat the above process 99 times, each time obtaining two parents from the remaining parent individuals to produce one offspring, ultimately obtaining the offspring population Off. 主 =[o1,o2,...o 100 Off1 = [f1, f2, ... f 100 Off2 = [g1, g2, ... g] 100 ] and Off3 = [h1,h2,...h 100 ].

7. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 6, characterized in that, Step five also includes the auxiliary population and the main population co-evolving by sharing offspring information: First, off the offspring population of the cooperative population. help With the parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P based on fitness values ​​through a binary tournament. 主 '∪Off 主 ∪Off help Select N superior individuals as the main population P of the new generation. 主 new ; Off the offspring population of the main population 主 With the parent population P of the co-population help 'and its offspring population Off help Merging, based on fitness values, is performed in a binary tournament from the merged population P. help '∪Off help ∪Off 主 Select N superior individuals as the next generation subpopulation P. help new ; Off the offspring populations of the other two populations other1 and Off other2 Each of its parent populations P other1 'and P other2 Merge populations P based on their fitness values ​​through a binary tournament. other1 '∪Off other1 and P other2 '∪Off other2 N superior individuals are selected from each of the two groups as the next generation of subpopulation P. other1 new and P other2 new .

8. The method for selecting multi-component spectral features based on deep reinforcement learning according to claim 7, characterized in that, In step six: DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the state of the current population, S2 is the state of the next generation population, and a1 is the current action initialization; When the current population evaluation number FE S Percentage of the most reviews Meanwhile, the newly generated population is continuously assisted in completing the next generation of evolution with the main population, and samples from the s-th generation are collected. s =(s s ,a s ,r s ,s s+1 Add the samples to the experience pool M to complete the collection of training samples; Samples are extracted from the experience pool M and standardized to initialize the training parameters and neural network weights θ of the DQN network; For a given sample e t =(s t ,a t ,r t ,s t+1 The predicted value r of the sample is obtained through forward propagation. t The prediction error is calculated, and the network weights θ are adjusted along the negative gradient direction of the prediction error. When the maximum number of training iterations of 100 is reached, the DQN training is complete.

9. The method for co-evolutionary multi-component spectral feature selection based on deep reinforcement learning according to claim 1, characterized in that, In step seven: DQN training updates are performed using the L formula: q t =r t +γmax a∈A Net(s t+1 ,a) In the formula, T represents all samples in the experience pool M, and Net(.) represents the state s. t The following actions a t Predicted reward value r t ', q t This represents the expected reward output value, which is determined by the current reward r. t and the maximum reward value obtained by taking the next step. a∈ A Net(s t+1 a) Composition, where γ is the discount factor.

Images