Spectral feature screening method and system for reinforcement learning driving sled dog optimization

Abstract

The invention discloses a spectral feature screening method and system for reinforcement learning driving sled dog optimization, and relates to the technical field of spectral analysis, and the method comprises the following steps: S1, obtaining a to-be-screened photoacoustic spectrum data set, and carrying out feature coding to obtain a current sled dog population; s2, determining key parameters; s3, acquiring a current population state based on the current sled dog population; s4, selecting a current action according to the current population state based on reinforcement learning; s5, updating the key parameter based on the current action to obtain a current key parameter; s6, updating the position of the current sled dog population based on the current key parameters to obtain a new population; and S7, judging whether a termination condition is met or not, if so, outputting the new population as the optimal screening feature, otherwise, returning to S3 based on the new population until the optimal screening feature is obtained. And the accuracy and efficiency of photoacoustic spectrum feature screening are improved.

Description

Technical Field

[0001] This invention relates to the field of spectral analysis technology, and more specifically to a method and system for screening spectral features of sled dogs driven by reinforcement learning. Background Technology

[0002] Currently, photoacoustic spectroscopy is based on the photoacoustic effect. When a substance absorbs light radiation, its internal energy changes, and some of the energy is released as heat, leading to a local increase in temperature and volume expansion, thereby generating pressure waves, i.e., photoacoustic signals. The characteristics of these photoacoustic signals (such as frequency, amplitude, and phase) are closely related to the optical absorption characteristics of the substance. Feature selection is a crucial step in photoacoustic spectroscopy data analysis.

[0003] Traditional feature selection methods mainly rely on statistical methods such as principal component analysis (PCA) and correlation coefficient methods. These methods can identify features related to the target variable to some extent, but they have limitations: while PCA can reduce dimensionality, the resulting principal components are linear combinations of the original variables, which may be difficult to interpret physically; the correlation coefficient method can only measure the linear relationship between features and the target, and is insufficient for handling nonlinear relationships. Genetic algorithms and particle swarm optimization (PSO) are also widely used for photoacoustic spectroscopy feature selection. These algorithms search for optimal feature subsets by simulating biological evolution or group behavior; however, genetic algorithms are prone to getting trapped in local optima, especially in high-dimensional and complex data spaces like photoacoustic spectroscopy, where their search efficiency may be low; PSO is highly sensitive to parameters, and improper parameter settings can lead to performance degradation. SDO is an emerging swarm intelligence optimization algorithm that simulates the foraging behavior of sled dogs working together to pull a sled. This algorithm has good global search capabilities and fast convergence, but the parameters of existing sled dog optimization algorithms (such as weights ω, learning factors C1 and C2) are manually preset fixed parameters, relying on initial parameters, making them prone to premature convergence, thus affecting the accuracy of feature selection.

[0004] Therefore, improving the accuracy and efficiency of photoacoustic spectral feature screening is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for spectral feature selection driven by reinforcement learning and sled dog optimization. Combining the dynamic decision-making advantages of reinforcement learning and the efficient search capability of sled dog optimization algorithm, it overcomes the shortcomings of traditional feature selection methods and optimization algorithms, and improves the accuracy and efficiency of photoacoustic spectral feature selection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A reinforcement learning-driven spectral feature selection method for sled dog optimization includes:

[0008] S1 acquires the photoacoustic spectrum dataset to be screened and performs feature encoding to obtain the current sled dog population;

[0009] S2 determines the key parameters;

[0010] S3 obtains the current population status based on the current sled dog population;

[0011] S4 selects the current action based on the current population state using reinforcement learning;

[0012] S5 updates the key parameters based on the current action to obtain the current key parameters;

[0013] S6 updates the location of the current sled dog population based on the current key parameters to obtain a new population;

[0014] S7 determines whether the termination condition is met. If so, the new population is output as the optimal screening feature. Otherwise, S3 is returned based on the new population until the optimal screening feature is obtained.

[0015] In one embodiment, the current sled dog population is obtained as follows:

[0016] All data features are obtained based on the photoacoustic spectrum dataset to be screened;

[0017] Encode each of the aforementioned data features separately to obtain multiple encoded features;

[0018] Each of the encoded features corresponds to a sled dog position;

[0019] The current sled dog population is formed based on all the locations of the sled dogs.

[0020] In one embodiment, the key parameters specifically include: the leader dog's influence weight, the first learning factor, and the second learning factor;

[0021] The current population state includes: number of iterations, population diversity, change in optimal fitness, and convergence speed.

[0022] In one embodiment, obtaining the current key parameters specifically includes:

[0023] The updated leadership dog influence weight is obtained by increasing or decreasing the first preset value based on the leadership dog influence weight.

[0024] The first update factor is obtained by increasing or decreasing the second set value based on the first learning factor;

[0025] The second update factor is obtained by increasing or decreasing the third set value based on the second learning factor;

[0026] Based on the comparison between the updated leader dog influence weight, the first update factor, and the second update factor and the leader dog influence weight, the first learning factor, and the second learning factor respectively, the better parameters are retained to form the current key parameters;

[0027] If the updated parameters are better than the original parameters, then the updated parameters will replace the original parameters; otherwise, the original parameters will remain unchanged.

[0028] In one embodiment, the position update in S6 specifically involves:

[0029] The number of dogs participating in sled pulling is determined based on the current population size;

[0030] The first speed during normal travel is obtained based on the number of dogs and the current key parameters.

[0031] The current position of the sled dog is updated based on the first speed to obtain the first updated position;

[0032] The second updated position after the obstacle avoidance operation is obtained based on the first updated position;

[0033] Based on the second updated position, obtain the third updated position when the direction is lost;

[0034] The fourth update position is obtained based on the third update position using a training mechanism;

[0035] The new population is obtained by retiring sled dogs based on the sled dogs corresponding to the fourth updated position.

[0036] In one embodiment, the method for obtaining the fourth update position is as follows:

[0037] Calculate the corresponding fitness based on the third update position;

[0038] The third update position with a fitness value less than a threshold is selected and updated through training to obtain the training update position;

[0039] The third update position, which satisfies the threshold, and the training update position together constitute the fourth update position.

[0040] In one embodiment, the fitness calculation formula is:

[0041]

[0042] Among them, Fitness (P ij ) represents fitness, α and β both represent weights, and Accuracy(M,P) ij ) indicates that the update position P is used.ij The prediction accuracy obtained on the validation set after training the classification model M, where n represents the number of sled dogs, ||P ij ||0 represents the feature subset P ij The number of features contained therein.

[0043] 8. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 7, characterized in that, in S7, S3 is returned based on the new population until the selected features are obtained, specifically including:

[0044] The fitness of the new population is calculated by replacing the current sled dog population with the new population.

[0045] Calculate the reward function based on the fitness of the new population;

[0046] The Q-value is updated based on the reward function value to obtain the updated Q-value;

[0047] Based on the updated Q value, obtain the updated population state to replace the current population state and continue to execute S4 and subsequent steps until the optimal screening feature is obtained.

[0048] In one embodiment, the Q-value update formula is:

[0049] Q(s t+1 ,a t+1 )=Q(s t ,a t )+γ[r t +maxQ(s t+1 ,a)-Q(s t ,a t )];

[0050] Where Q(s) t+1 ,a t+1 ) indicates updating the Q value, Q(s) t ,a t ) represents the Q-value at the current time, γ represents the learning rate, and r t This represents the reward function value, maxQ(s) t+1 ,a t ) represents the maximum estimated value of the state at the next moment.

[0051] A reinforcement learning-driven spectral feature selection system for sled dog optimization includes: a data encoding module, a parameter determination module, a current state acquisition module, a current action acquisition module, a parameter update module, a position update module, and an iterative output module;

[0052] The data encoding module is used to acquire the photoacoustic spectrum dataset to be screened and perform feature encoding to obtain the current sled dog population;

[0053] The parameter determination module is used to determine key parameters;

[0054] The current status acquisition module is used to acquire the current population status based on the current sled dog population;

[0055] The current action acquisition module is used to select the current action based on the current population state using reinforcement learning;

[0056] The parameter update module is used to update the key parameters based on the current action to obtain the current key parameters;

[0057] The location update module is used to update the location of the current sled dog population based on the current key parameters to obtain a new population.

[0058] The iterative output module is used to determine whether the termination condition is met. If so, the new population is output as the optimal screening feature; otherwise, S3 is returned based on the new population until the optimal screening feature is obtained.

[0059] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for spectral feature screening in reinforcement learning-driven sled dog optimization, which has the following beneficial effects:

[0060] 1. This invention introduces reinforcement learning (RL) to adjust parameters in real time. In each iteration of the sled dog algorithm, reinforcement learning makes decisions and adjusts key parameters (leader dog influence weight and learning factor) based on the real-time state of the current population, thereby achieving dynamic adaptive optimization. The algorithm adjusts parameters in real time based on population diversity, convergence speed and other states, thereby improving the convergence speed and optimization accuracy of the algorithm.

[0061] 2. This invention, through the deep integration of reinforcement learning and the sled dog algorithm, enables SDO to evolve from a static parameter optimization algorithm into an intelligent optimization system that can adapt to the optimization process and make autonomous decisions. It solves the three major pain points of traditional swarm intelligence algorithms in high-dimensional feature selection scenarios: premature convergence, low search efficiency, and reliance on human experience. Ultimately, it achieves a practical breakthrough with fewer features, higher accuracy, and faster convergence.

[0062] 3. This invention solves the problems of traditional filtering methods (such as correlation coefficients) ignoring model feedback, easily selecting redundant features, and encapsulation methods lacking sparsity constraints by setting a multi-objective fitness function. By adjusting the weights α and β, it can adapt to different scenario requirements to balance the accuracy of the algorithm and the sparsity of features. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0064] Figure 1 The flowchart of a reinforcement learning-driven spectral feature selection method for sled dogs is provided by the present invention.

[0065] Figure 2 The flowchart of the current sled dog population acquisition method provided by the present invention.

[0066] Figure 3 This is a schematic diagram of a reinforcement learning-driven spectral feature screening system for sled dogs, provided by the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Traditional spectral feature screening methods are mainly based on statistical methods, which are simple in principle, intuitive, and efficient. However, traditional statistical methods also have some limitations:

[0069] (1) Poor ability to handle nonlinear relationships: For example, the correlation coefficient method can only measure the linear relationship between features and targets, but it is difficult to effectively handle the nonlinear relationships in photoacoustic spectral data. The generation and propagation of photoacoustic signals may involve complex nonlinear physical processes, resulting in a non-simple linear correlation between features and target component concentrations. In this case, the traditional correlation coefficient method may not be able to accurately screen out features that truly have important information.

[0070] (2) Lack of specificity for specific physical properties: For example, principal component analysis (PCA) is a commonly used method for dimensionality reduction and feature extraction. It generates principal components by linearly combining the original features, but principal components are often complex linear combinations of the original variables, making it difficult to assign clear physical meaning. In photoacoustic spectroscopy analysis, researchers usually hope that the selected features can correspond to specific physical properties of the photoacoustic signal (such as specific absorption peaks corresponding to specific material components), but the principal components generated by PCA are difficult to meet this requirement.

[0071] (3) Over-reliance on experience: Some traditional methods (such as threshold screening) require setting thresholds manually to determine which features are important. This method often relies on the experience of researchers and lacks objective standards. Different researchers may choose different thresholds, leading to inconsistencies in screening results. Especially when faced with complex and diverse photoacoustic spectral data, it is difficult to determine a suitable threshold to accurately screen out effective features.

[0072] (4) Poor stability of feature selection: When existing methods perform feature selection on different datasets or different subsets of the same dataset, they may obtain feature subsets with large differences. This reflects the lack of stability of the feature selection results, making it difficult to determine a stable and reliable set of features for the construction of photoacoustic spectroscopy analysis models in practical applications, thus affecting the generalization ability and stability of the models.

[0073] Based on the aforementioned problems of existing methods, this invention deeply integrates reinforcement learning with the sled dog algorithm, enabling the sled dog algorithm to evolve from a static parameter optimization algorithm into an intelligent optimization algorithm that can adapt to the optimization process and make autonomous decisions. This solves the three major pain points of traditional swarm intelligence algorithms in high-dimensional feature selection scenarios: premature convergence, low search efficiency, and reliance on human experience. Ultimately, it achieves a practical breakthrough with fewer features, higher accuracy, and faster convergence.

[0074] Reinforcement learning (RL) is a machine learning method that learns optimal policies through the interaction between an agent and its environment. In photoacoustic spectral feature selection, the feature selection process can be viewed as a decision-making process, and reinforcement learning can dynamically adjust the feature selection strategy based on feedback from photoacoustic spectral data.

[0075] The Sled Dog Algorithm (SDO) is an emerging swarm intelligence optimization algorithm that simulates the foraging behavior of sled dogs working together to pull a sled. This algorithm possesses excellent global search capabilities and fast convergence. Combining reinforcement learning with the Sled Dog Algorithm for photoacoustic spectral feature selection aims to leverage the dynamic decision-making advantages of reinforcement learning and the efficient search capabilities of the Sled Dog Algorithm to overcome the shortcomings of traditional feature selection methods and optimization algorithms. This will improve the accuracy and efficiency of photoacoustic spectral feature selection, better meeting the needs of photoacoustic spectroscopy in complex real-world applications.

[0076] To address the challenges of high dimensionality, redundancy, and noise among features in photoacoustic spectral data, which necessitate efficient feature selection to improve model performance, this invention employs Reinforcement Learning (RL) to improve the SDO model and utilizes the improved model for feature selection in photoacoustic spectroscopy. For example... Figure 1 As shown in the figure, this invention discloses a spectral feature selection method for reinforcement learning-driven sled dog optimization, comprising the following steps:

[0077] S1 acquires the photoacoustic spectrum dataset to be screened and performs feature encoding to obtain the current sled dog population.

[0078] The photoacoustic spectroscopy dataset to be screened contains m samples and n features;

[0079] Initialize individual sled dogs:

[0080] P = lb + rand·(ub - lb);

[0081] Where ub and lb represent the upper and lower limits of each dimension variable, respectively, and rand represents a random number between [0,1].

[0082] Furthermore, such as Figure 2 As shown, the current method for obtaining the sled dog population is as follows:

[0083] All data features are obtained based on the photoacoustic spectrum dataset to be screened;

[0084] Encode each data feature separately to obtain multiple encoded features;

[0085] Each encoded feature corresponds to a sled dog position;

[0086] The current sled dog population is composed of all the sled dog locations.

[0087] Furthermore, the corresponding positions for the sled dogs are as follows:

[0088] P i =[p i,1 ,p i,2 ,p i,3 ,...,p i,n ];

[0089] Where, p i,n Let P represent the nth encoded feature. i Each encoded feature takes a value of 0 or 1, when p ij =1 indicates that the j-th encoding feature is selected.

[0090] S2 determines the key parameters.

[0091] The following sensitive parameters affecting the global search in the sled dog algorithm are selected as key parameters: the leader dog influence weight ω, the first learning factor c1, and the second learning factor c2.

[0092] S3 obtains the current population status based on the current sled dog population.

[0093] The state space can include the features of the candidate solutions corresponding to each sled dog in the current sled dog population. Therefore, the state space of RL, i.e., the current population state, is defined as s. t :

[0094] s t =(t,Diversity(P) t ), ΔFitness best Convergence);

[0095] Where t represents the current iteration number, Diversity(P) t ) represents population diversity, △Fitness best This represents the change in optimal fitness, and Convergence represents the convergence rate.

[0096] S4 is based on reinforcement learning and selects the current action according to the current population state.

[0097] The action space is defined as the different ways to adjust the key parameters of the sled dog optimization algorithm. For example, adjusting the weight parameter for updating the position of the lead dog by increasing or decreasing it by a certain value, or changing the distance decay coefficient in the ordinary sled dog following strategy, each adjustable parameter and its possible adjustment range constitute a part of the action space.

[0098] Furthermore, the current action is represented as at:

[0099] a t =argmaxQ(s t ,a)a∈A;

[0100] Where 'a' represents the action to optimize the key parameters, and 'A' represents the updated current key parameters.

[0101] S5 updates the key parameters based on the current action to obtain the current key parameters.

[0102] The specific key parameters obtained include:

[0103] The updated leadership dog influence weight is obtained by increasing or decreasing the first set value △ω based on the leadership dog influence weight ω.

[0104] The first update factor is obtained by increasing or decreasing the second set value Δc1 based on the first learning factor c1;

[0105] The second update factor is obtained by increasing or decreasing the third set value Δc2 based on the second learning factor c2.

[0106] Based on the comparison of the updated leader dog influence weight, the first update factor and the second update factor with the leader dog influence weight ω, the first learning factor c1 and the second learning factor c2 respectively, the better parameters are retained to form the current key parameter A;

[0107] If the updated parameters are better than the original parameters, then the updated parameters will replace the original parameters; otherwise, the original parameters will remain unchanged.

[0108] Furthermore, the critical parameter A is:

[0109] A={Incω, Decω, Incc1, Decc1, Incc2, Decc2, Keep};

[0110] The specific updates to various parameters in the action space are as follows:

[0111] Incω=ω+△ω;

[0112] Decω = ω - △ω;

[0113] Incc1 = c1 + △c1;

[0114] Decc1 = c1 - △c1;

[0115] Incc2 = c2 + △c2;

[0116] Decc2 = c2 - △c2;

[0117] Keep: Remain unchanged;

[0118] Where Inc and Dec represent increasing and decreasing the corresponding parameter values, Incω or Decω represents updating the leader dog influence weight, Incc1 or Decc1 represents the first update factor, and Incc2 or Decc2 represents the second update factor.

[0119] Furthermore, based on the above formula, the relevant key parameters are perturbed and updated. If the updated parameters are better than the original parameters, the updated parameters replace the corresponding original parameters; otherwise, the original parameters remain unchanged.

[0120] S6 updates the location of the current sled dog population based on the current key parameters to obtain a new population.

[0121] The location update is as follows:

[0122] The number of dogs to participate in sled pulling is determined based on the current population size;

[0123] The initial speed during normal travel is determined based on the number of dogs and current key parameters.

[0124] The sled dog's current position is updated based on the first velocity to obtain the first updated position;

[0125] Obtain the second updated position after the obstacle avoidance operation based on the first updated position;

[0126] Obtain the third updated position when lost, based on the second updated position;

[0127] The fourth update position is obtained based on the third update position using a training mechanism.

[0128] Retirement screening was conducted based on the sled dogs corresponding to the fourth updated position to obtain a new population.

[0129] Furthermore, not all individuals in the entire sled dog group participate in every "sled pulling" task (corresponding to the search operation in the algorithm); the number of dogs N1 participating in sled pulling is:

[0130] N1 = 0.7·N + k1·2 k2 ;

[0131] Where N represents the population size, k1 represents a random variable between [-1, 1], and k2 represents a random variable between [1, 2].

[0132] Furthermore, during normal travel: In the simulated sled dog journey, they proceed in two columns. The lead dog not only continuously receives and executes commands from its owner but also needs to handle unexpected events based on its training experience. The sled dog's speed update rule varies depending on its position, and the initial speed V during normal travel is obtained based on the number of dogs N1 and the current key parameters. i1 :

[0133]

[0134] Where, ω RL V represents the updated leader dog influence weight after RL control. i0 Indicates the initial velocity. and Let r1, r2, and r3 represent the first and second updated learning factors after RL control, respectively, where r1, r2, and r3 are all random numbers between [0,1]. Gi P represents the best historical position of the i-th sled dog. Z P represents a position randomly selected from four optimal positions. i-2 P represents the position of the (i-2)th sled dog. i+2 This represents the position of the (i+2)th sled dog, where l takes the value 0 or 1, and P... t P represents another sled dog in the same group. r It refers to a sled pulled by sled dogs;

[0135] P Z1 P represents the best position found so far. Z2 P Z3 and P Z4These represent the second, third, and fourth best positions found so far;

[0136] First update position P i1 For: P i1 =P i +V i1 .

[0137] Furthermore, obstacle avoidance: During the journey, if obstacles are encountered, the sled dogs will perform obstacle avoidance maneuvers under the owner's command and guidance from daily training, obtaining a second updated position P after the obstacle avoidance maneuver based on the first updated position. i2 :

[0138]

[0139] Where r4 and r5 both represent random numbers between [0,1], P N This represents the individual with the worst fitness, where k takes the value of -1 or 1, and p1 and p2 are the relevant parameters of the algorithm.

[0140] Furthermore, disorientation: When the sled team loses its way during the journey, the sled dogs will randomly explore the surrounding area under the guidance of their owners. The corresponding third update location P. i3 for:

[0141]

[0142] Where r6 and r7 both represent random numbers between [0,1], ξ represents a number randomly generated following a standard normal distribution, and T represents the maximum number of iterations.

[0143] Furthermore, the principle of sled dog training is as follows: for sled dogs that do not meet the requirements (e.g., have poor fitness values), the algorithm improves their abilities through specific training mechanisms.

[0144] The fourth update location is obtained as follows:

[0145] Calculate the corresponding fitness based on the third update position;

[0146] The third update position with a fitness value less than the threshold is selected and updated through training to obtain the training update position;

[0147] The fourth update position is formed by combining the third update position that meets the threshold and the training update position.

[0148] Furthermore, training updates position P ix for:

[0149] P ix =P i3 +r8·F·(r9·D1·X1+r 10·D2·X2+r 11 ·D3·X3);

[0150]

[0151]

[0152] X1 = P Z -P better ;

[0153]

[0154] X3 = P better -P worse ;

[0155] Among them, r8, r9, r 10 and r 11 Both represent random numbers between [0,1], P better This represents an individual randomly selected from the sled dogs that completed the task; Dim represents the overall dimension of the sled dog population; j represents the population under different dimensions; P worse The first and second sled dogs are not selected to perform the task. P1 and P2 represent the first and second sled dogs, respectively.

[0156] Furthermore, the fitness calculation formula is as follows:

[0157]

[0158] Among them, Fitness (P ij ) represents fitness, α and β both represent weights, and Accuracy(M,P) ij ) indicates that the update position P is used. ij The prediction accuracy obtained on the validation set after training the classification model M, where n represents the number of sled dogs, ||P ij ||0 represents the feature subset P ij The number of features contained therein.

[0159] Furthermore, the principle behind sled dog retirement is as follows: Each time a sled dog pulls a sled, it inflicts some form of injury on its body. As the number of times they pull the sled increases, the likelihood of injury rises dramatically. Once injured, a sled dog struggles to complete its tasks. Considering the sled dog's physical condition, it will have to retire. With the retirement of existing sled dogs, owners select and train new sled dogs to join the sled dog pack. Since injuries to sled dogs are random, a random replacement mechanism is used to simulate the retirement process. This is reflected in a ranking process from best to worst; the higher the ranking, the better the sled dog, and the less likely it is to retire. Conversely, the higher the likelihood of it being eliminated.

[0160] S7 determines whether the termination condition is met. If so, the new population is output as the optimal screening feature. Otherwise, S7 returns to S3 based on the new population until the optimal screening feature is obtained.

[0161] Based on the new population, return to S3 until the selection features are obtained, specifically including:

[0162] The fitness of the new population is calculated by replacing the current sled dog population with a new population.

[0163] Calculate the reward function based on the fitness of the new population;

[0164] The updated Q-value is obtained by updating the Q-value based on the reward function value.

[0165] Based on the updated Q-value, obtain the updated population state, replace the current population state, and continue to execute S4 and subsequent steps until the optimal screening feature is obtained.

[0166] Furthermore, the reward function r t Specifically:

[0167] r t =Fitness best (t)-Fitness best (t-1)+λ·Diversity(P t );

[0168] Among them, Fitness best (t) represents the optimal fitness in the t-th iteration. best (t-1) represents the optimal fitness in the (t-1)th iteration, and λ represents the diversity weight coefficient.

[0169] Furthermore, construct the state-action value table Q(s) t ,a t The system performs reinforcement learning training and dynamically adjusts the parameters, updating the Q-value through temporal difference (TD) learning. The Q-value update formula is:

[0170] Q(s t+1 ,a t+1 )=Q(s t ,a t )+γ[r t +maxQ(s t+1 ,a)-Q(s t ,a t )];

[0171] Where Q(s) t+1 ,a t+1) indicates updating the Q value, Q(s) t ,a t ) represents the Q-value at the current time, γ represents the learning rate, and r t This represents the reward function value, maxQ(s) t+1 ,a t ) represents the maximum estimated value of the state at the next moment.

[0172] Furthermore, in this embodiment, the termination condition is to meet the maximum number of iterations. After the termination condition is met, the optimal screening features of the individual are output, the final model is trained based on the optimal screening features, and the performance is evaluated on the test set.

[0173] Furthermore, the applicable field of the method of the present invention is:

[0174] In the field of food safety: it can screen out the photoacoustic spectral characteristics of harmful substances in food (such as excessive food additives, illegal additives, pesticide residues, veterinary drug residues, etc.); compared with traditional detection methods, it can achieve real-time and rapid screening, and improve the efficiency of food safety supervision.

[0175] Water pollution detection: It can screen out the photoacoustic spectral characteristics of heavy metal ions (such as mercury, lead, cadmium, etc.) and organic pollutants (such as polycyclic aromatic hydrocarbons, pesticide residues, etc.) in water; compared with traditional water sample chemical analysis methods, this invention can quickly and in real time monitor water bodies, promptly detect pollution events, and provide timely decision-making basis for environmental protection departments.

[0176] In the medical field: photoacoustic spectroscopy can be used to detect specific markers in blood or other biological tissues.

[0177] Example 2

[0178] Based on the same inventive concept, such as Figure 3 As shown, the present invention also provides a spectral feature screening system for sled dog optimization driven by reinforcement learning, comprising: a data encoding module, a parameter determination module, a current state acquisition module, a current action acquisition module, a parameter update module, a position update module, and an iterative output module;

[0179] The data encoding module is used to acquire the photoacoustic spectrum dataset to be screened and perform feature encoding to obtain the current sled dog population.

[0180] The parameter determination module is used to determine key parameters;

[0181] The current status acquisition module is used to obtain the current population status based on the current sled dog population.

[0182] The current action acquisition module is used to select the current action based on the current population state using reinforcement learning;

[0183] The parameter update module is used to update key parameters based on the current action to obtain the current key parameters;

[0184] The location update module is used to update the location of the current sled dog population based on the current key parameters, thus obtaining a new population.

[0185] The iterative output module is used to determine whether the termination condition is met. If so, the new population is output as the optimal screening feature; otherwise, S3 is returned based on the new population until the optimal screening feature is obtained.

[0186] Furthermore, in this embodiment, the functional implementation methods of each functional module correspond one-to-one with the methods described above, and will not be repeated here.

[0187] Example 3

[0188] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0189] Memory, used to store computer programs;

[0190] When the processor executes a program stored in memory, it is able to implement a reinforcement learning-driven spectral feature screening method for sled dogs, as described in Example 1.

[0191] The electronic device may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions in the memory to execute a reinforcement learning-driven sled dog optimization spectral feature screening method as described in Embodiment 1.

[0192] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0194] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for spectral feature selection in reinforcement learning-driven sled dog optimization, characterized in that, include: S1 acquires the photoacoustic spectrum dataset to be screened and performs feature encoding to obtain the current sled dog population; S2 determines the key parameters; S3 obtains the current population status based on the current sled dog population; S4 selects the current action based on the current population state using reinforcement learning; S5 updates the key parameters based on the current action to obtain the current key parameters; S6 updates the location of the current sled dog population based on the current key parameters to obtain a new population; S7 determines whether the termination condition is met. If so, the new population is output as the optimal screening feature. Otherwise, S7 returns to S3 based on the new population until the optimal screening feature is obtained.

2. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 1, characterized in that, The current method for obtaining sled dog populations is as follows: All data features are obtained based on the photoacoustic spectrum dataset to be screened; Encode each of the aforementioned data features separately to obtain multiple encoded features; Each of the encoded features corresponds to a sled dog position; The current sled dog population is formed based on all the locations of the sled dogs.

3. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 1, characterized in that, The key parameters specifically include: the leader dog's influence weight, the first learning factor, and the second learning factor. The current population state includes: number of iterations, population diversity, change in optimal fitness, and convergence speed.

4. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 3, characterized in that, The specific key parameters obtained include: The updated leadership dog influence weight is obtained by increasing or decreasing the first preset value based on the leadership dog influence weight. The first update factor is obtained by increasing or decreasing the second set value based on the first learning factor; The second update factor is obtained by increasing or decreasing the third set value based on the second learning factor; Based on the updated leader dog influence weight, the first update factor and the second update factor are compared with the leader dog influence weight, the first learning factor and the second learning factor respectively, and the better parameters are retained to form the current key parameters; If the updated parameters are better than the original parameters, then the updated parameters will replace the original parameters; otherwise, the original parameters will remain unchanged.

5. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 1, characterized in that, The specific position update in S6 is as follows: The number of dogs participating in sled pulling is determined based on the current population size; The first speed during normal travel is obtained based on the number of dogs and the current key parameters. The current position of the sled dog is updated based on the first speed to obtain the first updated position; The second updated position after the obstacle avoidance operation is obtained based on the first updated position; Based on the second updated position, obtain the third updated position when the direction is lost; The fourth update position is obtained based on the third update position using a training mechanism; The new population is obtained by retiring sled dogs based on the sled dogs corresponding to the fourth updated position.

6. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 5, characterized in that, The method for obtaining the fourth update position is as follows: Calculate the corresponding fitness based on the third update position; The third update position with a fitness value less than a threshold is selected and updated through training to obtain the training update position; The third update position, which satisfies the threshold, and the training update position together constitute the fourth update position.

7. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 6, characterized in that, The fitness calculation formula is as follows: Among them, Fitness (P ij ) represents fitness, α and β both represent weights, and Accuracy(M,P) ij ) indicates that the update position P is used. ij The prediction accuracy obtained on the validation set after training the classification model M, where n represents the number of sled dogs, ||P ij ||0 represents the feature subset P ij The number of features contained therein.

8. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 7, characterized in that, In S7, S3 is returned based on the new population until the screening characteristics are obtained, specifically including: The fitness of the new population is calculated by replacing the current sled dog population with the new population. Calculate the reward function based on the fitness of the new population; The Q-value is updated based on the reward function value to obtain the updated Q-value; Based on the updated Q value, obtain the updated population state to replace the current population state and continue to execute S4 and subsequent steps until the optimal screening feature is obtained.

9. The spectral feature selection method for reinforcement learning-driven sled dog optimization according to claim 8, characterized in that, The Q-value update formula is as follows: Q(s t+1 ,to t+1 )=Q(s t ,to t )+γ[r t +maxQ(s t+1 ,a)-Q(s t ,to t )]; Where Q(s) t+1 ,a t+1 ) indicates updating the Q value, Q(s) t ,a t ) represents the Q-value at the current time, γ represents the learning rate, and r t This represents the reward function value, maxQ(s) t+1 ,a t ) represents the maximum estimated value of the state at the next moment.

10. A spectral feature selection system for reinforcement learning-driven sled dog optimization, used to execute the spectral feature selection method for reinforcement learning-driven sled dog optimization as described in any one of claims 1-9, characterized in that, include: The module includes a data encoding module, a parameter determination module, a current state acquisition module, a current action acquisition module, a parameter update module, a position update module, and an iterative output module. The data encoding module is used to acquire the photoacoustic spectrum dataset to be screened and perform feature encoding to obtain the current sled dog population; The parameter determination module is used to determine key parameters; The current status acquisition module is used to acquire the current population status based on the current sled dog population; The current action acquisition module is used to select the current action based on the current population state using reinforcement learning; The parameter update module is used to update the key parameters based on the current action to obtain the current key parameters; The location update module is used to update the location of the current sled dog population based on the current key parameters to obtain a new population. The iterative output module is used to determine whether the termination condition is met. If so, the new population is output as the optimal screening feature; otherwise, S3 is returned based on the new population until the optimal screening feature is obtained.

Images