Coevolution multi-component spectral feature selection method based on deep reinforcement learning

Abstract

The embodiment of the invention discloses a coevolution multi-component spectral feature selection method based on deep reinforcement learning, and the method comprises the steps: collecting spectral signals and component concentration values in a sample, carrying out the coding of individuals in a field spectral database SDate, and carrying out the normalization processing, calculating the feature selection ratio of all individuals in the field spectrum database SDate, establishing a DQN training model, initializing a deep neural network Net and an empty experience pool M, performing statistics on the ratio of the variable of each interval in the population to the total variable in the iteration process of the training set, and calculating the feature selection ratio of all the individuals in the field spectrum database SDate; mating the initial main population with the auxiliary populations, training and updating the DQN to output a final population, constructing a moisture, fat and protein mixed multi-component spectrum quantitative analysis model by the final population, and recommending co-evolution of the auxiliary populations and the main population through a deep reinforcement learning model trained in an evolution early stage to obtain a multi-component spectrum quantitative analysis model; the optimal spectral feature subset is expected to be quickly obtained.

Description

Technical Field

[0001] The present application relates to the field of spectral feature variable selection in multi-component spectral quantitative analysis, and in particular to a co-evolutionary multi-component spectral feature selection method based on deep reinforcement learning. Background Art

[0002] In recent years, with the rapid development of chemometrics and instrument technology, spectral quantitative analysis has been widely used in many fields due to its rapid and non-destructive characteristics. For example, it is used in the food industry to analyze the content of food ingredients (sugar, fat and protein) and their additives to ensure the quality and safety of food; it is used to determine the purity of drugs in the development and production of drugs to ensure the consistency and safety of drugs. The theoretical basis of spectral quantitative analysis is the Lambert-Beer law, which shows that there is a certain linear relationship between the absorbance of a substance and its concentration under ideal conditions. The regression algorithm can be used to construct a component concentration prediction model to achieve the concentration prediction of the substance to be measured. Partial least squares (PLS) is the most commonly used regression modeling method in spectral quantitative analysis. PLS uses full-wavelength spectral data to establish a component concentration prediction model. However, the spectral data collected in actual applications contains hundreds of absorbance data of wavelength variables. Problems such as low spectral absorption intensity, collinearity and cross-sensitivity result in a large number of redundant and interfering wavelength variables in the spectral data. Effective feature variable selection methods can improve the prediction ability of the model and reduce the complexity of the model.

[0003] Due to the huge dimension of spectral wavelength variables and the small number of effective variables, the spectral feature selection problem can be defined as a sparse large-scale multi-objective optimization problem. The current methods for solving this problem can be roughly divided into three categories:

[0004] The first category is based on the grouping of decision variables. The main idea is to divide the decision variables into different groups and optimize them in a divide-and-conquer manner through independent subpopulations. However, the variable grouping techniques used may greatly affect their performance; the second category is based on problem reconstruction methods. The main idea is to reduce the decision space by using dimensionality reduction techniques on decision variables or converting complex large-scale problems into more tractable small-scale problems. This conversion can enable the optimization algorithm to search the solution space more efficiently, thereby finding the Pareto optimal solution; the third category is to use special offspring generation strategies, such as designing more sophisticated regeneration operators or probability models to improve search efficiency.

[0005] Evolutionary multi-objective optimization methods have shown excellent performance in the field of feature selection. In the problem of multi-component spectral feature selection, due to the complex correlation between components and the uncertainty of wavelength features in distribution and quantity, how to quickly and efficiently solve the problem of multi-component spectral feature selection becomes a challenge. Summary of the invention

[0006] In view of some deficiencies in the prior art, the present invention provides a method for selecting multi-component spectral features through co-evolution based on deep reinforcement learning: the algorithm is initialized to generate a main population and three auxiliary populations, and multiple auxiliary populations containing single-component information co-evolve with the main population containing multi-component information, so as to better explore the search space. During the evolution process, the deep reinforcement learning model (deep Q network, DQN) trained in the early stage of evolution recommends the optimal collaborative strategy under the current population state, that is, recommends an auxiliary population to co-evolve with the main population, in order to quickly obtain the optimal spectral feature subset.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme:

[0008] A method for selecting multi-component spectral features by co-evolution based on deep reinforcement learning, characterized by comprising:

[0009] Step 1: Data Collection

[0010] The spectral signals and component concentration values ​​in the sample are collected and acquired to form a field spectral database Sdate, wherein the field spectral database Sdate includes a plurality of wavelength variables;

[0011] Step 2: Encoding initialization

[0012] Encoding and normalizing the individuals in the field spectrum database Sdate, and dividing the processed field spectrum database Sdate into a training set and a validation set according to a preset ratio;

[0013] The training set is initialized to generate an initial main population P 主 and multiple auxiliary populations P1, P2…P K ;

[0014] Step 3: Obtain the target function

[0015] Calculating the feature selection ratios of all individuals in the field spectrum database Sdate, and calculating the field spectrum database Sdate through a prediction model according to the feature selection ratios to obtain the model prediction accuracy;

[0016] Step 4: DQN initialization

[0017] A DQN training model is established, and a deep neural network Net and an empty experience pool M are initialized. During the iteration of the training set, the ratio of the variables in each interval of the population to the total variables is counted;

[0018] Step 5: DQN initialization

[0019] The initial main population P 主 and multiple auxiliary populations P1, P2…P K Perform mating to obtain offspring populations, and optimize through DQN to obtain a new generation of sub-populations;

[0020] Step 6: DQN sample collection and training

[0021] DQN adds the samples obtained in step 4 to the experience pool M to train DQN;

[0022] Step 7: Late evolution of population under the guidance of DQN

[0023] Repeat step 5 to evolve the population, complete the training update of DQN, and use the updated DQN to reward and evaluate the experience pool M to output the final population P 主 , through the final population P 主 Construct a multi-component spectral quantitative analysis model for mixed water, fat and protein.

[0024] Optionally, in some embodiments of the present application, the step 2 includes:

[0025] Step 2.1: Initialize individual encoding method

[0026] The code of the i-th individual in the field spectrum database S Date is Individuals were coded in binary;

[0027] Step 2.2: Data Partitioning

[0028] The data set is normalized, and the database Sdate is divided into a training set and a validation set according to a preset ratio R:1. The training set is used to train each individual in the population, and the validation set is used to verify the performance of the spectral quantitative analysis model established under each feature subset;

[0029] Step 2.3: Initialize parameter settings and population

[0030] The number of individuals N and the maximum number of function evaluations FE max Initialize, and use random initialization method to generate initial main population P 主 and auxiliary populations P1, P2…P K .

[0031] Optionally, in some embodiments of the present application, the step three includes:

[0032] Step 3.1: Get the ratio of feature selection

[0033] Calculate the feature selection ratio for all individuals in each population. The ratio of the selected features is given by Get, D select and D are the number of selected characteristic wavelengths and the total number of characteristic wavelengths respectively;

[0034] Step 3.2: Get model accuracy

[0035] The spectral feature selection method calculates the prediction accuracy of the PLS model built for all individuals in each population through the established multi-component content prediction model f2(x)=RMSECV;

[0036]

[0037] Where n is the total number of training samples; y i is the measured value of the i-th sample; is the predicted output value of the i-th sample during the cross-validation process.

[0038] Optionally, in some embodiments of the present application, in step 4, a state setting is set, and the state setting includes the average sparsity of the main population, the standard deviation of the individual sparsity, the proportion of variables in multiple important intervals, and the current evaluation times accounted for the maximum evaluation times FE max proportion;

[0039] Among them, the average sparsity of the main population is ξ t , the main population is expressed as:

[0040]

[0041] The average sparsity is expressed as:

[0042]

[0043] in represents the jth wavelength variable bit of the ith individual, and N is the number of individuals selected;

[0044] The standard deviation of individual sparsity τ t is the standard deviation of the sum of all N individual sparsities:

[0045]

[0046] In the formula, is the sum of all variable bits of the ith individual, which means the sparsity sum of the ith individual;

[0047] The proportion of variables in multiple important intervals is represented by the weight vector of the PLS model built in the solution evaluation phase.

[0048] Optionally, in some embodiments of the present application, in step 4, an action setting and a reward setting are set, and the action setting initializes the action, and the action setting is a t , so as to evolve the population;

[0049] In the reward setting, the initial reward value r is set t =0, reward r t Set to the improvement of the HV value of the current population relative to the previous generation, reward Among them, HV t and HV t-1 Represent the HV values ​​of the main population of the current generation and the previous generation respectively.

[0050] Optionally, in some embodiments of the present application, in step 5, the initial main population P 主 and multiple auxiliary populations P1, P2…P K The parent population containing individuals is selected for mating pool selection;

[0051] Get the parent individuals a1=[x1,x2,...x 100 ] and a2=[x1,x2,...x 100 ],b1=[x1,x2,...x 100 ] and b2=[x1,x2,...x 100 ],c1=[x1,x2,...x 100 ] and c2=[x1,x2,...x 100 ],d1=[x1,x2,...x 100 ] and d2=[x1,x2,...x 100 ], and perform crossover and mutation operations on the parent individuals to obtain the offspring o1=[x1,x2,...x 100 ], f1=[x1,x2,...x 100 ],g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ];

[0052] Repeat the above process 99 times, each time obtaining two parents from the remaining parent individuals to produce one offspring, and finally obtain the offspring population Off 主 =[o1,o2,...o 100 ],Off1=[f1,f2,...f 100 ],Off2=[g1,g2,...g 100 ] and Off3=[h1,h2,...h 100 ].

[0053] Optionally, in some embodiments of the present application, the step 5 further includes the auxiliary population and the main population co-evolving by sharing offspring information:

[0054] First, the offspring population of the cooperative population is Off help The parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P according to the fitness value through a binary tournament 主 '∪Off 主 ∪Off help Select N better individuals as the main population P of the new generation 主 new ; Off the offspring population of the main population 主 The parent population P of the cooperative population help ' and its offspring population Off help Merge, according to the fitness value through the binary tournament from the merged population P help '∪Off help ∪Off 主 Select N better individuals as the new generation of subpopulation P help new ; Off the offspring populations of the remaining two populations other1 and Off other2 and their parent population P other1 ' and P other2 'Merge, according to the fitness value through the binary tournament from the merged population P other1 '∪Off other1 and P other2 '∪Off other2 Select N better individuals as the new generation of sub-population P other1 new and P other2 new .

[0055] Optionally, in some embodiments of the present application, in step six:

[0056] DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the state of the current population, S2 is the state of the next generation population, and a1 is the current action initialization;

[0057] When the current population evaluation times FE S The proportion of the largest number of evaluations When the new generation of population is generated, the auxiliary population and the main population will continue to complete the evolution of the next generation, and collect the sth generation sample e s =(ss ,a s ,r s ,s s+1 ) is added to the experience pool M to complete the collection of training samples;

[0058] Extract samples from the experience pool M and standardize them, initialize the training parameters of the DQN network and the neural network weights θ;

[0059] For a given sample e t =(s t ,a t ,r t ,s t+1 ), and obtain the predicted value r of the sample through forward propagation t ', thereby calculating the prediction error and adjusting the network weight θ along the negative gradient direction of the prediction error; when the maximum number of training times reaches 100, DQN training is completed.

[0060] Optionally, in some embodiments of the present application, in step seven:

[0061] DQN training updates are updated using the L formula:

[0062]

[0063] q t =r t +γmax a∈A Net(s t+1 ,a)

[0064] Where T represents all samples in the experience pool M, and Net(.) represents the state s t Action taken under t The predicted reward value r t ',q t Represents the expected reward output value, which is composed of the current reward r t and the maximum reward value max obtained by taking the next action a∈A Net(s t+1 ,a) composition, γ is the discount factor.

[0065] Compared with the prior art, the beneficial effects of the present invention are:

[0066] 1. The present invention defines the wavelength variable selection problem in multi-component spectral quantitative analysis as a large-scale sparse multi-objective optimization problem. By adopting a method combining an evolutionary multi-objective optimization algorithm with deep reinforcement learning, the optimal feature subset can be effectively selected, thereby significantly improving the prediction accuracy of the model while reducing the model complexity. This method comprehensively considers the complexity of multi-component spectral features, making the model perform better when processing actual data;

[0067] 2. The present invention achieves the co-evolution of the main population and the auxiliary population to better explore the search space and approach the Pareto optimal solution. This multi-population coordination mechanism can effectively improve the performance of the feature selection process and enhance the overall optimization capability. In addition, the DQN trained in the early stage of evolution can dynamically adjust the selection strategy of the auxiliary population in different evolutionary generations, achieve effective dynamic co-evolution of the population, and ensure the flexibility of the algorithm.

[0068] 3. In the present invention, DQN will be regularly updated with new samples collected during the evolution process, further achieving effective coordination of information of different components, thereby ensuring the accuracy of multi-component spectral feature selection;

[0069] 4. Traditional feature selection methods often ignore the complex correlations between multi-component spectral features. The present invention uses the DQN intelligent recommendation co-evolution strategy to make full use of the commonalities and uniqueness between different features, thereby improving the efficiency and accuracy of feature selection;

[0070] 5. This feature selection method can be widely used in various fields that require spectral feature selection, such as food, agriculture, chemistry, medicine, etc. Its high efficiency and accuracy make it very valuable in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 A schematic flow chart of a method for collaborative evolutionary multi-component spectral feature selection based on deep reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present application. It is understood that the drawings are only provided for reference and illustration, and are not used to limit the present application. The connection relationship shown in the drawings is only for the convenience of clear description and does not limit the connection method.

[0074] Specifically, Figure 1As shown, an embodiment of the present application provides a method for collaborative evolutionary multi-component spectral feature selection based on deep reinforcement learning.

[0075] In this embodiment, the present invention is described in more detail by taking the spectral data of meat containing three components, namely water, fat and protein, as an example. Figure 1 As shown, proceed as follows:

[0076] Step 1: Data Collection

[0077] First, the spectral signals and component concentration values ​​of meat samples were collected by Tecator Infratec food and feed analyzer to form a field spectral database SDate. The database SDate includes 100 wavelength variables;

[0078] Step 2: Individual coding and initialization

[0079] Step 2.1: Initialize individual encoding method

[0080] The code of the i-th individual is Using binary individual coding, the elements in each individual are 1 or 0. The value '1' indicates that the corresponding feature variable is selected, and the value '0' indicates that the corresponding wavelength is not selected. Each individual represents a feature selection scheme;

[0081] Step 2.2: Data Partitioning

[0082] The meat data set was normalized, and the database SDate was divided into a training set and a validation set according to the preset ratio R:1, and the R value was set to 4. The training set was used to train each individual in the population, that is, the selected feature subset, and the validation set was used to verify the performance of the spectral quantitative analysis model built under each feature subset, using a five-fold cross-validation method;

[0083] Step 2.3: Initialize parameter settings and population

[0084] The number of individuals in the initial population N and the maximum number of function evaluations FEmax are set to 100 and 10000 respectively, and the initial main population P is generated by random initialization method. 主 , auxiliary populations P1, P2 and P3. 主 It is used to solve the problem of selecting spectral features of mixed multi-components of water, fat and protein. P1, P2 and P3 are used to solve the problem of selecting spectral features of water, fat and protein respectively;

[0085] Step 3: Get the objective function value

[0086] Step 3.1: Get the ratio of feature selection

[0087] Calculate the feature selection ratio for all individuals in each population. The ratio of the selected features is given by Get, D select and D are the selected characteristic wavelength number and the total characteristic wavelength number 100 respectively;

[0088] Step 3.2: Get model accuracy

[0089] The spectral feature selection method uses partial least squares to establish a meat multi-component content prediction model. The model prediction accuracy is represented by the five-fold cross-validation root mean square error RMSECV, which is used as one of the fitness functions; the prediction accuracy of the PLS model built for all individuals in each population is calculated as f2(x)=RMSECV. The calculation of RMSECV is shown in formula (1):

[0090]

[0091] Where n is the total number of training samples; y i is the measured value of the i-th sample, which is the concentration of the spectral component and is the actual measured value; It is the predicted output value of the i-th sample in the cross-validation process, specifically the predicted value of the spectral component concentration.

[0092] Step 4: DQN Initialization

[0093] Step 4.1: Initial Setup

[0094] In order to adaptively select the optimal cooperative population according to the current population state, it is necessary to establish a DQN mapping model between the population state and the cooperative strategy. First, a deep neural network Net is randomly initialized, and an empty experience pool M is initialized to store DQN training samples;

[0095] Step 4.2: State Setting

[0096] The state of the main population at the tth iteration ξ t ,τ t , ... μ t They are the average sparsity of the main population, the standard deviation of individual sparsity, the proportion of variables in the 10 normalized important intervals, and the current evaluation times to the maximum evaluation times FE max First, calculate the average sparsity ξ of the main population. t , the main population is represented by Average Sparsity in Represents the jth wavelength variable bit of the ith individual.

[0097] Secondly, the standard deviation τ of individual sparseness in the main population is calculatedt , the standard deviation of individual sparsity τ t is the standard deviation of the sum of all N individual sparsities:

[0098]

[0099] In the formula, is the sum of all variable bits of the i-th individual, that is, it represents the sparsity sum of the i-th individual.

[0100] Then, the weight vector of the PLS model built in the solution evaluation stage is used to represent the importance of each variable, and the ratio of the variables in each importance interval (10 equal-length intervals are divided in sequence within [0, 1]) to the total number of variables D is counted. The proportions of the 10 variables are expressed as

[0101] Finally, calculate the current algebraic evaluation number FE t The proportion of the maximum number of evaluations FEmax

[0102] Step 4.3: Action Settings

[0103] Set the initial action to 1, action a t =1 / 2 / 3, respectively, indicating that the water, fat, and protein single-component auxiliary populations are selected to cooperate with the main population to complete the evolution of the tth generation;

[0104] The action is set mainly to obtain the first sample. A sample contains the current state, action, and reward obtained by the action, which is used to train DQN in each iteration.

[0105] Step 4.4: Reward Setup

[0106] Initial reward value r t =0, reward r t Set to the improvement of the HV value of the current population relative to the previous generation, reward Among them, HV t and HV t-1 Represent the HV values ​​of the main population of the current generation and the previous generation respectively.

[0107] Step 5: Early stage population evolution

[0108] Step 5.1: For each population P 主 ,P1,P2,P3 select the parent population P' containing 200 individuals 主 ,P1',P2',P3' perform mating pool selection;

[0109] Step 5.2: From the parent population P' 主, P1', P2', P3' respectively obtain any two parent individuals in the parent individuals a1=[x1, x2, ...x 100 ] and a2=[x1,x2,...x 100 ],b1=[x1,x2,...x 100 ] and b2=[x1,x2,...x 100 ],c1=[x1,x2,...x 100 ] and c2=[x1,x2,...x 100 ],d1=[x1,x2,...x 100 ] and d2=[x1,x2,...x 100 ], and perform crossover and mutation operations respectively, and obtain the offspring o1=[x1,x2,...x 100 ], f1=[x1,x2,...x 100 ],g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ].

[0110] Step 5.3: Repeat step 5.299 times, each time obtaining two parents from the remaining parent individuals to produce one offspring, and finally obtaining the offspring population Off 主 =[o1,o2,...o 100 ],Off1=[f1,f2,...f 100 ],Off2=[g1,g2,...g 100 ] and Off3=[h1,h2,...h 100 ];

[0111] Step 5.4: DQN performs a random action and randomly selects an auxiliary population P help Cooperate with the main population to perform optimization, as in step 5.5;

[0112] Step 5.5: The auxiliary population and the main population co-evolve by sharing offspring information: First, the auxiliary population selected by DQN, i.e., the offspring population of the collaborative population, is Off help The parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P according to the fitness value through a binary tournament 主 '∪Off 主 ∪Off help Select N better individuals as the main population P of the new generation 主 new ; Then the offspring population of the main population is Off 主 The parent population P of the cooperative population help' and its offspring population Off help Merge, according to the fitness value through the binary tournament from the merged population P help '∪Off help ∪Off 主 Select N better individuals as the new generation of subpopulation P help new ; Finally, the offspring populations of the remaining two populations are Off other1 and Off other2 and their parent population P other1 ' and P other2 'Merge, according to the fitness value through the binary tournament from the merged population P other1 '∪Off other1 and P other2 '∪Off other2 Select N better individuals as the new generation of sub-population P other1 new and P other2 new .

[0113] Step 6: DQN sample collection and training

[0114] Step 6.1: According to steps 4.2-4.4, DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the state of the current population, S2 is the state of the next generation, and a1 is the initialization of the current action;

[0115] Step 6.2: When When the new generation population is generated, steps 5.1-5.4 are repeated continuously to complete the evolution of the next generation, and the sth generation sample e is collected. s =(s s ,a s ,r s ,s s+1 ) is added to the experience pool M to complete the collection of training samples;

[0116] Step 6.3: Extract samples from the experience pool M and standardize them, initialize the training parameters of the DQN network: neural network weight θ (randomly initialized), maximum number of training times v = 3, number of hidden layers s = 3, number of neurons w = 10, and DQN learning rate lr = 0.001. Then, for a given sample e t =(s t ,a t ,r t ,s t+1 ), and obtain the predicted value r of the sample through forward propagation t', thereby calculating the prediction error and adjusting the network weight θ along the negative gradient direction of the prediction error. Finally, when the maximum number of training times reaches 100, DQN training is completed;

[0117] Step 7: Late evolution of population under the guidance of DQN

[0118] Step 7.1: Repeat steps 5.1-5.4 to perform mating pool selection and generate offspring;

[0119] Step 7.2: Generate a random number rand∈[0,1], if Random Selection Strategy Skip to step 7.3. If DQN obtains the current population state s t And use it as input to get the reward value of different collaborative selection strategies. t , whose reward value is r t Among all the strategies, the collaborative selection strategy with the maximum reward value is expressed as Choose the strategy with the largest reward value Guide the co-evolution process, the specific operation is as in step 7.3;

[0120] Step 7.3: Action or The auxiliary co-evolution population represented by it and its offspring are denoted as P help and Off help , repeat step 5.4 to complete the co-evolution process of the auxiliary population and the main population, generate a new population and obtain new samples to add to the experience pool M;

[0121] Step 8. Update DQN

[0122] Step 8.1: Repeat step 7 to complete one generation of population evolution. After every 10 evolutions, update the DQN. The specific DQN update is as shown in step 8.2;

[0123] Step 8.2: Select the latest 10 samples in the experience pool M and update DQN according to formula (3):

[0124]

[0125] q t =r t +γmax a∈A Net(s t+1 ,a) (4)

[0126] In formula (3), T represents all samples in the experience pool M, and Net(.) represents the state s t Action taken under t The predicted reward value rt ',q t Represents the expected reward output value, which is composed of the current reward r t and the maximum reward value max obtained by taking the next action a∈A Net(s t+1 ,a) composition, γ∈[0,1] is the discount factor;

[0127] Step 8.3: When the maximum number of function evaluations reaches 10,000, output the final population P 主 Population P 主 The individuals in represent the selected multi-component spectral feature subsets, which can be used to construct a mixed multi-component spectral quantitative analysis model for water, fat, and protein.

[0128] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A collaborative evolutionary multi-component spectral feature selection method based on deep reinforcement learning, characterized in that: include: Step 1: Data Collection The spectral signals and component concentration values ​​in the sample are collected and acquired to form a field spectral database Sdate, wherein the field spectral database Sdate includes a plurality of wavelength variables; Step 2: Encoding initialization Encoding and normalizing the individuals in the field spectrum database Sdate, and dividing the processed field spectrum database Sdate into a training set and a validation set according to a preset ratio; The training set is initialized to generate an initial main population P 主 and multiple auxiliary populations P1, P2…P K ; Step 3: Obtain the target function Calculating the feature selection ratios of all individuals in the field spectrum database Sdate, and calculating the field spectrum database Sdate through a prediction model according to the feature selection ratios to obtain the model prediction accuracy; Step 4: DQN initialization Establish a DQN training model, and initialize a deep neural network Net and an empty experience pool M. During the iteration of the training set, count the ratio of variables in each interval of the population to the total variables; Step 5: DQN initialization The initial main population P 主 and multiple auxiliary populations P1, P2…P K Perform mating to obtain offspring populations, and optimize through DQN to obtain a new generation of sub-populations; Step 6: DQN sample collection and training DQN adds the samples obtained in step 4 to the experience pool M to train DQN; Step 7: Late evolution of population under the guidance of DQN Repeat step 5 to evolve the population, complete the training update of DQN, and use the updated DQN to reward and evaluate the experience pool M to output the final population P 主 , through the final population P 主 Construct a multi-component spectral quantitative analysis model for mixed water, fat and protein.

2. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: The step 2 includes: Step 2.1: Initialize individual encoding method The code of the i-th individual in the field spectrum database S Date is Individuals were coded in binary; Step 2.2: Data Partitioning The data set is normalized, and the database Sdate is divided into a training set and a validation set according to a preset ratio R:

1. The training set is used to train each individual in the population, and the validation set is used to verify the performance of the spectral quantitative analysis model established under each feature subset; Step 2.3: Initialize parameter settings and population The number of individuals N and the maximum number of function evaluations FE max Initialize, and use random initialization method to generate initial main population P 主 and auxiliary populations P1, P2…P K .

3. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: The step three includes: Step 3.1: Get the ratio of feature selection Calculate the feature selection ratio for all individuals in each population. The ratio of the selected features is given by Get, D select and D are the number of selected characteristic wavelengths and the total number of characteristic wavelengths respectively; Step 3.2: Get model accuracy The spectral feature selection method calculates the prediction accuracy of the PLS model built for all individuals in each population through the established multi-component content prediction model f2(x)=RMSECV; Where n is the total number of training samples; y i is the measured value of the i-th sample; is the predicted output value of the i-th sample during the cross-validation process.

4. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: In step 4, a state setting is set, which includes the average sparsity of the main population, the standard deviation of the individual sparsity, the proportion of variables in multiple important intervals, and the current evaluation times as a percentage of the maximum evaluation times FE. max proportion; Among them, the average sparsity of the main population is ξ t , the main population is expressed as: The average sparsity is expressed as: where x i j represents the jth wavelength variable bit of the ith individual, and N is the number of individuals selected; The standard deviation of individual sparsity τ t is the standard deviation of the sum of all N individual sparsities: In the formula, is the sum of all variable bits of the ith individual, which means the sparsity sum of the ith individual; The proportion of variables in multiple important intervals is represented by the weight vector of the PLS model built in the solution evaluation phase.

5. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: In the step 4, an action setting and a reward setting are set, in which the action is initialized and the action is set to a t , so as to evolve the population; In the reward setting, the initial reward value r is set t =0, reward r t Set to the improvement of the HV value of the current population relative to the previous generation, reward Among them, HV t and HV t-1 Represent the HV values ​​of the main population of the current generation and the previous generation respectively.

6. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: In step 5, the initial main population P 主 and multiple auxiliary populations P1, P2…P K The parent population containing individuals is selected for mating pool selection; Get the parent individuals a1=[x1,x2,...x 100 ] and a2=[x1,x2,...x 100 ],b1=[x1,x2,...x 100 ] and b2=[x1,x2,...x 100 ],c1=[x1,x2,...x 100 ] and c2=[x1,x2,...x 100 ],d1=[x1,x2,...x 100 ] and d2=[x1,x2,...x 100 ], and perform crossover and mutation operations on the parent individuals to obtain the offspring o1=[x1,x2,...x 100 ], f1=[x1,x2,...x 100 ],g1=[x1,x2,...x 100 ] and h1=[x1,x2,...x 100 ]; Repeat the above process 99 times, each time obtaining two parents from the remaining parent individuals to produce one offspring, and finally obtain the offspring population Off 主 =[o1,o2,...o 100 ],Off1=[f1,f2,...f 100 ],Off2=[g1,g2,...g 100 ] and Off3=[h1,h2,...h 100 ].

7. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 6, characterized in that: The step 5 also includes the auxiliary population and the main population co-evolving by sharing offspring information: First, the offspring population of the cooperative population is Off help The parent population P of the main population 主 and its offspring population Off 主 Merge and select from the merged population P according to the fitness value through a binary tournament 主 '∪Off 主 ∪Off help Select N better individuals as the main population P of the new generation 主 new ; Set the offspring population of the main population to Off 主 The parent population P of the cooperative population help ' and its offspring population Off help Merge, according to the fitness value through the binary tournament from the merged population P help '∪Off help ∪Off 主 Select N better individuals as the new generation of subpopulation P help new ; Set the offspring populations of the remaining two populations to Off other1 and Off other2 Respectively with their parent population P other1 ' and P other2 'Merge, according to the fitness value through the binary tournament from the merged population P other1 '∪Off other1 and P other2 '∪Off other2 Select N better individuals as the new generation of sub-population P other1 new and P other2 new .

8. The method for selecting multi-component spectral features based on deep reinforcement learning according to claim 7, characterized in that: In step six: DQN obtains the first sample e1 = (s1, a1, r1, s2) and adds sample e1 to the experience pool M; where S1 is the state of the current population, S2 is the state of the next generation population, and a1 is the current action initialization; When the current population evaluation times FE S The proportion of the largest number of evaluations When the new generation of population is generated, the auxiliary population and the main population will continue to complete the evolution of the next generation, and collect the sth generation sample e s =(s s ,a s ,r s ,s s+1 ) is added to the experience pool M to complete the collection of training samples; Extract samples from the experience pool M and standardize them, initialize the training parameters of the DQN network and the neural network weights θ; For a given sample e t =(s t ,a t ,r t ,s t+1 ), and obtain the predicted value r of the sample through forward propagation t ', thereby calculating the prediction error and adjusting the network weight θ along the negative gradient direction of the prediction error; when the maximum number of training times reaches 100, DQN training is completed.

9. The method for selecting multi-component spectral features based on deep reinforcement learning based on collaborative evolution according to claim 1, characterized in that: In the step seven: DQN training updates are updated using the L formula: q t =r t +γmax a∈A Net(s t+1 ,a) Where T represents all samples in the experience pool M, and Net(.) represents the state s t Action taken under t The predicted reward value r t ',q t Represents the expected reward output value, which is composed of the current reward r t and the maximum reward value max obtained by taking the next action a∈ A Net(s t+1 ,a), γ is the discount factor.

Images