Multi-agent hyperspectral band selection method based on fusion state and optimal experience pool

Abstract

The invention discloses a multi-agent hyperspectral band selection method based on a fusion state and a superior experience pool, and belongs to the field of remote sensing image processing. After a hyperspectral image is obtained, a sample set of the hyperspectral image is generated, and the sample set is divided into a training sample set and a test sample set; a multi-agent waveband selection model is constructed according to the training sample set, the state representation method comprises meta description statistics, an auto-encoder and a graph convolutional network, and the reward allocation strategy comprises an adaptive allocation strategy based on waveband importance; inputting the training sample set into a multi-agent waveband selection model for model training to obtain a trained multi-agent feature selection model, and optimizing a memory sample of an experience pool for model training by adopting a specific sample oriented experience pool optimization method based on a Gaussian mixture model; and inputting the test sample set into the trained multi-agent feature selection model for band selection, outputting a hyperspectral image band set, and classifying the hyperspectral image band set to obtain a classification result.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing image processing, and in particular relates to a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool. Background Art

[0002] In the field of remote sensing image processing technology, with the rapid development of hyperspectral remote sensing technology, the need to use hyperspectral image data to accurately identify and classify complex ground object information has become more urgent. However, hyperspectral image data has high dimensionality and redundancy, which makes it difficult to effectively utilize all band information. Therefore, how to extract representative bands from high-dimensional data to improve data processing efficiency and classification accuracy has become a key issue in the current field of remote sensing image processing.

[0003] Current hyperspectral image band selection methods based on reinforcement learning have demonstrated their unique advantages in strategy learning and sequential decision-making. These methods use the framework of reinforcement learning to enable the algorithm to learn how to select the optimal band combination in interaction with the environment. Through the reward feedback mechanism, the reinforcement learning model can identify and strengthen those band selection strategies that select better performance. This method can not only find the most representative and discriminative band combination in complex multi-band information, but also adaptively adjust the strategy to cope with environmental changes.

[0004] Beihang University proposed a hyperspectral image band selection method based on an unsupervised reinforcement learning band selection model in its patent application "A hyperspectral image band selection method based on reinforcement learning" (patent application number: 202111268014.8, application publication number: CN 113936219 A). This method uses the trial-and-error learning and long-term benefit considerations of reinforcement learning to solve the problems of limited spatial search and decision optimization in existing band selection methods. However, this method still has shortcomings. A significant defect is that the state representation is not comprehensive, which directly affects the effective interaction between the agent and the environment, making it difficult for the agent to accurately evaluate the current state of the environment and thus unable to make the best decision. At the same time, due to the exploration mechanism of reinforcement learning and the randomness of the initial parameters of the algorithm, not all samples in the experience pool are optimal. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems of being unable to effectively represent the environmental state, being unable to fully capture the complex relationship between bands and the deep characteristics of hyperspectral data, and uneven reward distribution in multi-agent systems. A multi-agent hyperspectral band selection method based on fusion state and optimal experience pool is proposed.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme: In the first aspect, a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool includes the following steps: After acquiring the hyperspectral image, a sample set of the hyperspectral image is generated, and the sample set of the hyperspectral image is divided into a training sample set and a test sample set; A multi-agent band selection model is constructed according to the training sample set. The state representation method of the multi-agent band selection model includes a meta-descriptive statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolutional network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. The training sample set is input into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-oriented experience pool optimization method based on a Gaussian mixture model; The test sample set is input into the trained multi-agent feature selection model for band selection, and a hyperspectral image band set is output. The hyperspectral image band set is classified to obtain the classification result.

[0007] Furthermore, the agents of the multi-agent band selection model correspond to the bands in the training sample set, the agents of the multi-agent band selection model make selection decisions on features, the actions of the multi-agent band selection model include band selection or band deselection, the environment of the multi-agent band selection model is a classification network, and the classification network adopts random forest.

[0008] Furthermore, the meta description statistical method of the feature subspace is specifically: Extracting descriptive statistics from the selected data matrix to obtain a descriptive statistics matrix, wherein the descriptive statistics include standard deviation, minimum value, maximum value, first quartile, second quartile and third quartile; extracting descriptive statistics of each row from the descriptive statistics matrix to obtain a meta-descriptive statistics matrix, and connecting each column in the meta-descriptive statistics matrix to obtain a first state vector; The selected data matrix is ​​a data matrix composed of data samples and the number of bands selected by the agent in the action selection step; The feature subspace depth representation method based on the autoencoder is specifically as follows: Using a first autoencoder to convert each feature column in the selected data matrix into a first length latent vector to obtain a latent matrix, using a second autoencoder to map each row of the latent matrix to a second length latent vector to obtain a static encoding matrix, and connecting each column in the static encoding matrix to obtain a second state vector; The graph convolutional network method based on dynamic graph is specifically as follows: The selected data matrix is ​​converted into a feature correlation graph, the adjacency matrix of the feature correlation graph is solved, the potential representation of the feature is extracted from the adjacency matrix using a graph convolutional network, the potential representation of each feature is averaged and converted into a third state representation vector, and the graph convolutional network is trained using an unsupervised mathematical description method; The state representation of the multi-agent band selection model is the sum of a first state vector, a second state vector and a third state representation vector.

[0009] Furthermore, the reward includes an overall reward and an individual agent reward. The overall reward is the classification accuracy obtained by classifying the band subsets formed after all agents take actions. The individual agent reward includes a shared reward and a competitive reward. The distribution strategy of the competitive reward adopts an adaptive distribution strategy based on band importance.

[0010] Furthermore, the individual agent reward is the sum of the shared reward and the competitive reward; The shared reward is the ratio of the difference between the classification accuracy of the current round and the classification accuracy of the previous round to the total number of agents that make selection actions; The calculation steps of the competitive reward include: iterating each band in the descriptive statistics matrix, randomly rearranging features in each iteration to generate a new data set, calculating the band importance score of each band according to the performance score of the new data set in the classification task, converting the band importance score into a competitive reward allocation contribution weight through normalization, and multiplying the competitive reward allocation contribution weight by the overall reward to obtain the competitive reward.

[0011] Furthermore, the calculation formula of the band importance is as follows:

[0012] in, is the importance score of the band, For the band, Score the model's raw performance on the descriptive statistics matrix, is the average performance score of the model on all new datasets, is the total number of iterations, k is the current iteration number, Score the performance of new datasets on classification tasks; The calculation formula for the competition reward allocation contribution weight is as follows:

[0013] in, Assign contribution weights to competitive rewards, is the importance score of the band, For the band, is the sum of all band importance scores, is the total number of iterations, k is the current iteration number, and N is the total number of agents that make selection actions.

[0014] Furthermore, the specific sample-oriented experience pool optimization method based on the Gaussian mixture model comprises the following steps: Collect a memory sample set, wherein a memory sample in the memory sample set includes an action, a state, and an individual agent reward, assign all memory samples in the memory sample set to an unselected group, reallocate the memory samples to a selected group based on whether the memory samples are selected, select a first preset ratio of high-quality samples in the selected group, use the selected high-quality samples to train two Gaussian mixture models using an expectation maximization algorithm, use the trained Gaussian mixture models to generate simulated samples, use the simulated samples to replace part of the memory samples in the unselected group according to a second preset ratio, and combine the high-quality samples of the selected group and the replaced samples of the unselected group into a new high-quality data set; The sum of the first preset ratio and the second preset ratio is 1; Use new high-quality datasets to update the agent's selection strategy.

[0015] Secondly, the multi-agent hyperspectral band selection system based on fusion state and optimal experience pool includes: A sample set acquisition module is used to generate a sample set of the hyperspectral image after acquiring the hyperspectral image, and divide the sample set of the hyperspectral image into a training sample set and a test sample set; Constructing a band selection model module, which is used to construct a multi-agent band selection model according to a training sample set. The state representation method of the multi-agent band selection model includes a meta-description statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolution network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. A training band selection model module is used to input the training sample set into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-guided experience pool optimization method based on a Gaussian mixture model; The output band module is used to input the test sample set into the trained multi-agent feature selection model for band selection, output the hyperspectral image band set, classify the hyperspectral image band set, and obtain the classification result.

[0016] In a third aspect, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool when executing the computer program.

[0017] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, provides the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool.

[0018] Compared with the prior art, the present invention has the following beneficial technical effects: The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool proposed in the present invention adopts the meta-description statistics of feature subspace to describe the environment more comprehensively through statistical methods; the intrinsic connection between states is learned in the latent space by adopting the deep representation of feature subspace based on autoencoder. This state intrinsic structure captured in the low-dimensional latent space helps the reinforcement learning model to better generalize to new and unseen states, thereby improving the learning efficiency and the robustness of the strategy. In addition, by representing the state as a vector in the latent space, the autoencoder provides a more flexible and rich way to represent the strategy for the reinforcement learning algorithm. In this low-dimensional latent space, it is easier for the algorithm to find the optimization strategy because the relationship between states is effectively encoded and simplified; the graph convolutional network based on dynamic graph considers not only the individual representation of features, but also the correlation between features. There are many advantages in using GCN (Graph Convolutional Network) to represent the state of reinforcement learning. GCN can operate directly on the graph structure, so as to naturally capture the complex relationship between state components. In addition, GCN extracts features by learning the local neighborhood pattern of nodes, thereby achieving generalization of unseen states. Compared with the existing methods, the specific sample-guided experience pool optimization method based on the Gaussian mixture model can generate more high-quality samples based on the original memory sample set for agent strategy update, so that each agent's strategy update is updated in the optimal direction. The present invention solves the problem that the existing technology cannot effectively represent the environmental state, cannot fully capture the complex relationship between bands and the deep characteristics of hyperspectral data, so that the learning algorithm cannot obtain enough information, resulting in reduced decision-making accuracy; it solves the problem of low decision-making quality and efficiency of the entire system caused by uneven reward distribution in the multi-agent system.

[0019] Furthermore, the reward design method can balance the competition and cooperation among multiple agents. By evaluating the contribution of each band to the final decision result, the reward distribution mechanism is dynamically adjusted to ensure that the agents corresponding to the bands that contribute more to the overall goal can receive more rewards. This adaptive reward distribution mechanism not only encourages agents to find and focus on more important bands, but also promotes reasonable competition and effective cooperation among agents, thereby improving the decision-making quality and efficiency of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings: Figure 1 The present invention is a flowchart of a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool.

[0021] Figure 2 This is a structural diagram of the multi-agent hyperspectral band selection system based on fusion state and optimal experience pool of the present invention.

[0022] Figure 3 This is an electronic device diagram of the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool of the present invention.

[0023] Figure 4 The present invention is a flowchart of a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to an embodiment of the present invention.

[0024] Figure 5 This is a classification result diagram obtained in Example 5 of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] Embodiment 1 See also Figure 1 , a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool includes the following steps: After acquiring the hyperspectral image, a sample set of the hyperspectral image is generated, and the sample set of the hyperspectral image is divided into a training sample set and a test sample set; A multi-agent band selection model is constructed according to the training sample set. The state representation method of the multi-agent band selection model includes a meta-descriptive statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolutional network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. The training sample set is input into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-oriented experience pool optimization method based on a Gaussian mixture model; The test sample set is input into the trained multi-agent feature selection model for band selection, and a hyperspectral image band set is output. The hyperspectral image band set is classified to obtain the classification result.

[0027] The state representation method of the multi-agent band selection model of this embodiment includes a meta-description statistical method of feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolution network method based on a dynamic graph, which can capture the features of hyperspectral images from different angles and levels, thereby improving the accuracy and comprehensiveness of band selection; a reward mechanism based on a band importance adaptive allocation strategy can dynamically adjust rewards according to the importance of different bands in the classification task, so that the agent is more focused on selecting those bands that have a significant impact on the classification results; a specific sample-oriented experience pool optimization method based on a Gaussian mixture model can be used to effectively manage and utilize sample data in the training process, and by optimizing the storage and sampling strategy of the experience pool, useful features can be learned faster, improving the efficiency and stability of model training. The multi-agent band selection model of this embodiment completes the task together through the collaboration of multiple agents, can make full use of the advantages of different agents, improve the efficiency and accuracy of band selection, and at the same time, it also has certain robustness and fault tolerance, and can cope with complex and changing scenes.

[0028] The agents of the multi-agent band selection model correspond to the bands in the training sample set. The agents of the multi-agent band selection model make selection decisions on features. The actions of the multi-agent band selection model include band selection or band deselection. The environment of the multi-agent band selection model is a classification network, and the classification network adopts random forest.

[0029] In this embodiment, each intelligent agent corresponds to one band and can make refined selections and decisions for each band. This one-to-one correspondence enables the intelligent agent to more accurately evaluate the importance of each band and thus make more reasonable choices. It can process selection tasks for multiple bands in parallel, greatly improving the efficiency of band selection. At the same time, since each agent runs independently, it has certain fault tolerance and robustness and can cope with complex and changing scenarios. The random forest used in this embodiment is an integrated learning method. By constructing multiple decision trees and integrating their prediction results, the accuracy and generalization ability of the model are improved. In the multi-agent band selection model, the classification network adopts random forest, which can make full use of its diversity advantages and improve the accuracy and stability of band selection. Since each agent can make selection decisions on features (i.e., bands), it can flexibly adjust feature subsets to adapt to different classification tasks and scenarios, and can better cope with complex hyperspectral image data and improve classification performance. The band selection task is completed through the autonomous learning and collaboration of the agent. The agent can continuously adjust its selection strategy according to the feedback of the classification network, thereby optimizing the training process. The autonomous learning mechanism improves the accuracy and efficiency of band selection. The agents can share experience and information, and complete tasks together through collaboration, which can accelerate the training process and improve the convergence speed and performance of the model. At the same time, the collaboration between agents also promotes the sharing and transfer of knowledge, and can better utilize historical data and experience.

[0030] The meta-descriptive statistical method of feature subspace is as follows: Extracting descriptive statistics from the selected data matrix to obtain a descriptive statistics matrix, the descriptive statistics include standard deviation, minimum value, maximum value, first quartile, second quartile and third quartile; extracting descriptive statistics of each row from the descriptive statistics matrix to obtain a meta-descriptive statistics matrix, connecting each column in the meta-descriptive statistics matrix to obtain a first state vector; The selected data matrix is ​​a data matrix consisting of data samples and the number of bands selected by the agent in the action selection step; The feature subspace deep representation method based on the autoencoder is as follows: Using a first autoencoder to convert each feature column in the selected data matrix into a first length latent vector to obtain a latent matrix, using a second autoencoder to map each row of the latent matrix to a second length latent vector to obtain a static encoding matrix, and connecting each column in the static encoding matrix to obtain a second state vector; The graph convolutional network method based on dynamic graph is as follows: The selected data matrix is ​​converted into a feature correlation graph, the adjacency matrix of the feature correlation graph is solved, the potential representation of the feature is extracted using a graph convolutional network from the adjacency matrix, the potential representation of each feature is averaged and converted into a third state representation vector, and the graph convolutional network is trained using an unsupervised mathematical description method; The state representation of the multi-agent band selection model is the sum of the first state vector, the second state vector and the third state representation vector.

[0031] This embodiment can comprehensively reflect the distribution characteristics and statistical laws of the band data by extracting descriptive statistical data of the selected data matrix, so that the intelligent agent can more accurately evaluate the importance of the band and make more reasonable choices. The autoencoder of this embodiment can convert high-dimensional data into low-dimensional latent vectors while retaining the main features of the data. Through the continuous mapping of the first autoencoder and the second autoencoder, a more abstract and robust band feature representation can be obtained. This deep representation method helps the intelligent agent capture key information in a complex feature space and improves the accuracy of band selection. The graph convolutional network of this embodiment can process graph structured data. By extracting the adjacency matrix of the feature correlation graph and applying the graph convolutional network, the potential relationship and dependency between the bands can be captured. This relationship representation helps the intelligent agent understand the interaction between the bands and make more comprehensive and coordinated choices.

[0032] This embodiment combines the meta-description statistics method, the autoencoder deep representation method and the graph convolution network method to form a richer band feature representation. This fusion representation method can combine the advantages of different methods and improve the generalization ability and robustness of the model. Even in the face of new or unseen data, the model can make reasonable band selections. The graph convolution network in this embodiment is trained using an unsupervised mathematical description method and can learn feature representations from unlabeled data. The unsupervised learning method helps the model adapt to different data sets and tasks and improves the flexibility and robustness of band selection. This embodiment adds the first state vector, the second state vector and the third state representation vector to obtain a concise and effective state representation. This state representation method helps the intelligent agent understand the environmental state faster, thereby accelerating the training process and convergence speed. Multiple feature representation methods provide rich band feature information, which can be used as a guide for the intelligent agent to select actions. Through the continuous attempts and learning of the intelligent agent during the training process, the band selection strategy can be gradually optimized to improve the performance and efficiency of the model.

[0033] The rewards include overall rewards and individual agent rewards. The overall reward is the classification accuracy obtained by classifying the band subsets formed after all agents take actions. The individual agent rewards include shared rewards and competitive rewards. The distribution strategy of competitive rewards adopts an adaptive distribution strategy based on band importance.

[0034] In this embodiment, the overall reward encourages the formation of a collaborative relationship between intelligent agents and jointly optimizes the selection of band subsets to improve the overall classification performance. Only by working together can the intelligent agents maximize the overall reward. The shared reward encourages the intelligent agents to contribute in the team, while the competitive reward stimulates the competition between intelligent agents through an adaptive allocation strategy based on the importance of the bands. The competitive reward based on the adaptive allocation strategy of the importance of the bands can motivate the intelligent agents to make a more accurate assessment of the importance of the bands; the intelligent agents will identify those bands that have an important impact on the classification performance and give priority to these bands, thereby improving the accuracy and efficiency of the band selection. Under the guidance of the reward mechanism, the intelligent agents can The agent can continuously learn and optimize its own selection strategy. The intelligent agent will adjust its behavior according to the feedback of the reward and gradually form a more effective band selection strategy, thereby improving the performance of the model. Due to the competitive relationship between the agents, each agent will strive to find different bands that contribute to the classification performance. This diverse feature representation helps to enhance the robustness and adaptability of the model, and can better cope with complex and changing data sets and tasks. Even if an agent's choice is wrong or biased, other agents can still make up for this defect through collaboration and competition. Fault tolerance enables the model to run more stably and adapt to different application scenarios.

[0035] The reward for an individual agent is the sum of the shared reward and the competitive reward; The shared reward is the ratio of the difference between the classification accuracy of the current round and the classification accuracy of the previous round to the total number of agents that made the selected action; The calculation steps of the competitive reward include: iterating each band in the descriptive statistics matrix, randomly rearranging the features in each iteration to generate a new data set, calculating the band importance score of each band according to the performance score of the new data set in the classification task, converting the band importance score into a competitive reward allocation contribution weight through normalization, and multiplying the competitive reward allocation contribution weight by the overall reward to obtain the competitive reward.

[0036] The shared reward in this embodiment is calculated based on the ratio of the improvement in classification accuracy in the current round to the total number of agents that make selection actions. This reward mechanism encourages agents to form collaborative relationships and jointly improve classification performance. Agents will realize that only when the entire team (i.e., all agents) work together can the shared reward be maximized. Therefore, agents tend to choose those bands that are beneficial to the entire team, thereby enhancing team cohesion. The calculation of the competitive reward in this embodiment takes into account the importance score of each band and converts it into a competitive reward allocation contribution weight through normalization. This reward mechanism stimulates the competitive relationship between agents. Each agent will strive to find and give priority to those bands that have an important impact on classification performance in order to obtain higher competitive rewards. In the calculation step of the competitive reward, by iterating each band in the descriptive statistics matrix and randomly rearranging the features to generate a new data set, the importance of each band can be more accurately evaluated. This evaluation method helps agents identify those bands that play a key role in classification performance, thereby improving the accuracy of band selection. The agent will continuously adjust its behavior strategy based on the feedback of the reward. Through the guidance of shared rewards and competitive rewards, the agent will gradually learn how to select bands more effectively to improve classification performance. This adaptive learning ability enables the agent to continuously optimize its selection strategy, thereby improving the efficiency of band selection. Due to the existence of competitive rewards, the agent will strive to find different bands that contribute to classification performance. This diversified feature selection helps to enhance the robustness and stability of the model, enabling it to better cope with complex and changing data sets and tasks.

[0037] The calculation formula for band importance is as follows:

[0038] in, is the importance score of the band, For the band, Score the model's raw performance on the descriptive statistics matrix, is the average performance score of the model on all new datasets, is the total number of iterations, k is the current iteration number, Score the performance of new datasets on classification tasks; The calculation formula for the contribution weight of competition reward distribution is as follows:

[0039] in, Assign contribution weights to competitive rewards, is the importance score of the band, For the band, is the sum of all band importance scores, is the total number of iterations, k is the current iteration number, and N is the total number of agents that make selection actions.

[0040] The specific sample-oriented experience pool optimization method based on Gaussian mixture model includes the following steps: Collect a memory sample set, where a memory sample in the memory sample set includes an action, a state, and an individual agent reward, assign all memory samples in the memory sample set to an unselected group, reallocate the memory samples to a selected group based on whether they are selected, select a first preset ratio of high-quality samples in the selected group, use the selected high-quality samples to train two Gaussian mixture models using an expectation maximization algorithm, use the trained Gaussian mixture models to generate simulated samples, use the simulated samples to replace part of the memory samples in the unselected group according to a second preset ratio, and combine the high-quality samples of the selected group and the replaced samples of the unselected group into a new high-quality data set; The sum of the first preset ratio and the second preset ratio is 1; Use new high-quality datasets to update the agent's selection strategy.

[0041] This embodiment uses a specific sample-guided experience pool optimization method based on a Gaussian mixture model to select high-quality samples for training, thereby effectively avoiding the interference of low-quality or redundant samples on model training. The first preset proportion of high-quality samples in the selected group ensures the representativeness and accuracy of the training data. The trained Gaussian mixture model is used to generate simulated samples. These simulated samples can reflect the distribution characteristics of real data. Some memory samples in the unselected group are replaced by a second preset proportion, which further improves the quality and diversity of the data, allowing the algorithm to use past experience multiple times. The generation and replacement of simulated samples improves the efficiency of data utilization, and the new high-quality data set is used to train the intelligent body. Selecting a strategy for updating can ensure the accuracy and effectiveness of the strategy. Since the quality of training data is improved, the updated strategy can better adapt to complex environments. The simulated samples generated by the Gaussian mixture model can increase the response strategies of the intelligent agent when facing different situations and improve the generalization ability of the intelligent agent in unknown or complex environments. By optimizing the data distribution in the experience pool, the utilization and representativeness of the training data are improved, the training process of the model is accelerated, and the training efficiency is improved. Since the Gaussian mixture model can handle complex data distribution, this method can maintain good stability during the training process, reduce fluctuations and uncertainties during the training process, and improve the robustness of the model.

[0042] Embodiment 2 See also Figure 2 , a multi-agent hyperspectral band selection system based on fusion state and optimal experience pool includes the following steps: A sample set acquisition module is used to generate a sample set of the hyperspectral image after acquiring the hyperspectral image, and divide the sample set of the hyperspectral image into a training sample set and a test sample set; Constructing a band selection model module, which is used to construct a multi-agent band selection model according to a training sample set. The state representation method of the multi-agent band selection model includes a meta-description statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolution network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. A training band selection model module is used to input the training sample set into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-guided experience pool optimization method based on a Gaussian mixture model; The output band module is used to input the test sample set into the trained multi-agent feature selection model for band selection, output the hyperspectral image band set, classify the hyperspectral image band set, and obtain the classification result.

[0043] The band set selected by the system of this embodiment can more accurately reflect the characteristics and category information of the hyperspectral image, which helps to improve the accuracy and robustness of the classification algorithm, making the classification results more reliable and stable. By reducing unnecessary bands, the computational complexity of the subsequent classification algorithm can be reduced and the processing speed can be improved. This is of great significance for the processing and analysis of large-scale hyperspectral image data, and is not only suitable for the band selection task of hyperspectral images, but can also be extended to other similar feature selection and data dimensionality reduction tasks.

[0044] Embodiment 3 See also Figure 3 , an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool is implemented.

[0045] Embodiment 4 A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a multi-agent hyperspectral band selection method based on a fusion state and an optimal experience pool is provided.

[0046] Embodiment 5 This embodiment provides a multi-agent hyperspectral band selection method based on fusion state and optimal experience pool. The specific process is described in detail below: S1. Input hyperspectral image.

[0047] In this embodiment, a hyperspectral image with a size of 610×340×103 and containing a true label vector is input, and the hyperspectral image contains 103 bands.

[0048] S2. Generate a sample set.

[0049] First, the image is expanded by 25×25, and a 25×25 pixel spatial window is defined with each pixel in the input hyperspectral image as the center. All pixels in each spatial window are combined into a data cube. All data cubes are combined into a sample set of the hyperspectral image.

[0050] S3. Generate training sample set and test sample set.

[0051] In the sample set of hyperspectral images, 5% of the samples are randomly selected from each category to form the training sample set of hyperspectral images, and the remaining 95% of the samples are used to form the test sample set of hyperspectral images.

[0052] S4. Construct a multi-agent band selection model.

[0053] The multi-agent model consists of the following parts: agent, action, environment, state, and reward, which will be introduced below. S401 Agent: Assume that there are bands, defined Agents For an agent, it is designed to make selection decisions on the corresponding features.

[0054] S402 action: For Agents, Actions Indicates A band is selected, Indicates bands are deselected.

[0055] S403 Environment: The environment is a downstream classification network that uses random forest as a classifier. The classification accuracy of the random forest and the classification importance of each band are used to calculate and further influence the reward function to obtain the classification accuracy evaluation of each feature subset.

[0056] S404 state: Let the training set be In the process of generating features, it is expanded into a two-dimensional image according to the band dimension Input, including data samples and bands. For the In the action selection step, the agent makes a selection action (i.e. ) The number of bands selected, defines is the selected data matrix .

[0057] The following three different strategies are used to enhance the agent’s ability to understand the environment: 1. Meta-descriptive statistical method of feature subspace: From the selected data matrix Extract descriptive statistics including standard deviation, minimum, maximum, and first quartile , second quartile , third quartile Specifically, extract Seven descriptive statistics for each feature (column) in , resulting in a dataset of size Descriptive statistics matrix of .

[0058] From the descriptive statistics matrix Extract seven descriptive statistics for each row and obtain a matrix of size Meta-descriptive statistics matrix of Finally, Each column in is linked into a state vector of fixed length 49 .

[0059] 2. Feature subspace deep representation method based on autoencoder: right For each feature (column) in , apply the autoencoder to convert each feature column into a column of length Therefore, we get a latent vector of dimension The potential matrix .However, cannot represent a state, because its size is not fixed but varies with the selected number of features. It changes constantly during different explorations.

[0060] Applying another autoencoder will Each row is mapped to a length and obtain a latent vector of fixed dimension The static encoding matrix Finally, Each column in is concatenated into a fixed length The state vector .

[0061] 3. Graph Convolutional Network Method Based on Dynamic Graph First, Convert to feature correlation graph , and then find the graph The adjacency matrix of , and then apply a graph convolutional network to extract node embeddings

[0062] in ,make Indicates potential representation, is the layer number. In order to avoid pre-training the network, the following unsupervised mathematical description method is used to calculate the graph

[0063] in , is the identity matrix, yes The diagonal node degree matrix of Afterwards, the potential representation of each feature is averaged and converted to a length of The state representation vector .

[0064] S405 Reward: After all agents take action, the band subsets formed are sent to the downstream random forest network for classification to obtain the classification accuracy ACC, which is used as the overall reward. After obtaining the overall reward, the rewards of individual agents are obtained in the following way: 1. Calculate shared rewards.

[0065] Shared rewards are based on the improvement of overall classification accuracy It is calculated as the difference between the ACC of the current round and the ACC of the previous round. This difference represents the positive contribution of the collective action of the agents to the system performance.

[0066]

[0067] 2. Calculate competition rewards.

[0068] Calculate the contribution of each band for each band through the Permutation Importance strategy For each band , calculate its importance score by following the steps below : For the dataset Each band in ,conduct ( ) times of repeated experiments, each iteration In the example above, we randomly rearrange the features. , generate a new dataset ; Calculate the new data set Performance scores on downstream classification tasks ; Band Importance score The calculation formula is as follows:

[0069] in, Is the model in the original dataset The raw performance score (i.e., ACC) on is the average performance score of the model on all new datasets.

[0070] In order to Converted into weights that can be directly used for reward distribution. Normalize Normalized weights It can be calculated by the following formula:

[0071] in, It is a band The importance score of is the total number of agents that make the selected action, is the sum of the importance scores of all bands. After this processing, the agent corresponding to each band will obtain a weight value indicating its relative contribution.

[0072] Multiply the normalized weights by the overall reward classification accuracy ACC to get the final reward value for each agent. This process can be expressed by the following formula:

[0073] 3. Final reward value It is divided into two parts: competitive reward and shared reward. The calculation process is as follows:

[0074] in, To share the rewards, Rewards for competition.

[0075] S5. Training a multi-agent selection model S501, input the training data in step 3 into the multi-agent model, and obtain the data matrix after all agents make selection actions .

[0076] S502, using the three-state design method in step 4, the obtained Add to get the status representation .

[0077] S503, Data Matrix It is sent to the classification network of You, and random forest is used as the classifier to obtain the classification accuracy ACC.

[0078] S504, using the reward calculation method in step 4 to obtain the reward for this round, and Stored in the experience pool as a memory sample.

[0079] S505, the obtained As the input for a new round, it is input into the multi-agent model and steps 1 to 4 are repeated until the number of samples in the experience pool reaches the requirement.

[0080] S506, using a specific sample-oriented experience pool optimization method based on a Gaussian mixture model to optimize the memory samples in the experience pool, and at the same time update the selection strategy of the intelligent agent. The specific process is as follows: Collect the memory sample set As input, these samples are first divided into two groups: and Initially, all samples are assigned to group, and then based on whether the sample is selected to decide whether to reallocate it to Next, select The proportion of high-quality samples in the group is , and use these samples to train two GMM (Gaussian Mixture Model) generative models, which are trained by the Expectation Maximization (EM) algorithm. Then, use these trained GMM models to generate simulated samples to replace A part of the samples in the replacement ratio is Finally, the simulated memory samples in the selected and unselected groups are combined into a new high-quality dataset, and the agent’s strategy is updated using this new dataset.

[0081] The steps of the expectation-maximization algorithm are as follows: Initialization: Randomly select the parameters of the Gaussian distribution (mean , the covariance matrix ) and the mixing coefficient of the distribution (weight ).

[0082] E step (Expectation step): Calculate the responsibility (posterior probability) of each sample in the sample set belonging to each Gaussian distribution. Indicates The samples come from The probability of a distribution:

[0083] in, It is a sample. is the probability density function of the Gaussian distribution, is the number of distributions.

[0084] M step (Maximization step): Use the responsibility to re-estimate the parameters and mixing coefficients of each Gaussian distribution: Update mean :

[0085] Update the covariance matrix :

[0086] Update the mixing coefficients :

[0087] in Represents the total degree of responsibility, which is equivalent to being assigned to the The effective number of data points for a distribution.

[0088] Check convergence: If the change in the parameter is less than a certain threshold, or the preset number of iterations is reached, the algorithm ends. Otherwise, return to step E to continue iterating.

[0089] S6. Hyperspectral image band selection and classification results The test sample set is input into the trained multi-agent feature selection model, and the selected band set is output. The output band set is sent to the random forest classifier to obtain the classification result.

[0090] The effect of this embodiment is further described below in conjunction with simulation experiments: Simulation experiment conditions: The hardware platform of the simulation experiment is: the processor is Intel i7 7820X CPU, the main frequency is 3.6GHz, and the memory is 128GB.

[0091] The software platform for the simulation experiment is: Windows 11 operating system and Python 3.8 The input image used in the simulation experiment is the hyperspectral image of Pavia University. The hyperspectral data originates from an urban area in Pavia, a northern Italian city. It contains 103 spectral bands and 9 types of ground objects. The image format is mat.

[0092] The simulation content and its results are analyzed as follows: For the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool, the name Multi-Agent Reinforcement Learning Band Selection (MARLBS) is used as a substitute in the following text. In order to prove the effectiveness of the MARLBS algorithm, deep reinforcement learning band selection Reinforcement Learning Band Selection (RLSBS), single-agent band selection method Single-Agent Reinforcement Learning Feature Selection (SARLFS), maximum relevance and minimum redundancy Max-Relevance and Min-Redundancy (mRMR), and least absolute shrinkage and selection operator (LASSO) were selected for comparison. For the band selection results of the algorithm, the random forest classifier was uniformly used for classification. The proposed MARLBS algorithm was developed and implemented using the Pytorch framework.

[0093] In the experiments, for all deep networks, the mini-batch size is set to 32 and the Adam optimizer is used with a learning rate of 0.01. For all experience replays, the storage size is set to 200. The Q network in the MARLBS method is set to a two-layer ReLU network with 64 and 8 nodes in the first and second layers, respectively. In GMM sampling, the high-quality ratio is 0.20 for the University of Pavia dataset and 0.30 for the University of Pavia dataset. The GCN method is used as the representation learning algorithm in the experiment. Its network is a two-layer ReLU with 128 and 32 nodes in the first and second layers, respectively. The prediction classifier used is a random forest with 100 decision trees.

[0094] Combine the following Figure 5 The simulation diagram and Table 1 further describe the effects of this embodiment.

[0095] Figure 5 (a) is a pseudo-color image composed of the 90th, 29th and 37th bands of the 103 bands of the hyperspectral image selected for the simulation experiment of this embodiment. Figure 5 (b) is a real object distribution map of the Pavia data set of hyperspectral images selected for the simulation experiment of this embodiment, and the size of the distribution map is 610×340 pixels. Figure 5(c) shows the result of classifying the hyperspectral image of the Pavia dataset using the existing Least Absolute Value Convergence and Selection Algorithm (LASSO) method. Figure 5 (d) shows the classification result of the hyperspectral image of the Pavia dataset using the existing maximum relevance and minimum redundancy (mRMR) method. Figure 5 (e) shows the result of classifying the hyperspectral image of the Pavia dataset using the existing single agent band selection method (SARLFS). Figure 5 (f) is a graph showing the result of classifying the hyperspectral image of the Pavia dataset using the method of this embodiment.

[0096] Three evaluation indicators are used: classification accuracy of each category, overall accuracy OA, and average accuracy AA to evaluate the classification results of the four methods respectively.

[0097] The following formula is used to calculate the total accuracy OA, average accuracy AA, and classification accuracy of 16 types of objects. All calculation results are tabulated in Table 1.

[0098]

[0099]

[0100]

[0101] Table 1. Quantitative analysis of the classification results of this embodiment and various prior arts in simulation experiments

[0102] The MARLBS algorithm showed high classification accuracy in most categories, especially in categories 5 and 6, with accuracy rates of 99.8% ± 0.2 and 81.8% ± 6.1, respectively, which are significantly higher than other algorithms. In addition, MARLBS also performed best in overall accuracy (OA), average accuracy (AA) and Kappa coefficient, which were 76.7% ± 0.9, 79.8% ± 2.2 and 71.4% ± 1.9, respectively. This shows that MARLBS can more effectively handle the characteristics of PU datasets and achieve better band selection and classification results through multi-agent cooperation and learning. The MARLBS (Multi-agent Reinforcement Learning Band Selection) algorithm showed high classification accuracy in most categories, especially in categories 5 and 6, with accuracy rates of 99.8% ± 0.2 and 81.8% ± 6.1, respectively, which are significantly higher than other algorithms. In addition, MARLBS also performs best in overall accuracy (OA), average accuracy (AA) and Kappa coefficient, which are 76.7%±0.9, 79.8%±2.2 and 71.4%±1.9 respectively. This shows that MARLBS can more effectively handle the characteristics of PU datasets and achieve better band selection and classification results through multi-agent cooperation and learning.

[0103] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A multi-agent hyperspectral band selection method based on fusion state and optimal experience pool, characterized in that: The following steps are involved: After acquiring the hyperspectral image, a sample set of the hyperspectral image is generated, and the sample set of the hyperspectral image is divided into a training sample set and a test sample set; A multi-agent band selection model is constructed according to the training sample set. The state representation method of the multi-agent band selection model includes a meta-descriptive statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolutional network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. The training sample set is input into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-oriented experience pool optimization method based on a Gaussian mixture model; The test sample set is input into the trained multi-agent feature selection model for band selection, and a hyperspectral image band set is output. The hyperspectral image band set is classified to obtain the classification result.

2. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 1 is characterized in that: The agents of the multi-agent band selection model correspond to the bands in the training sample set, the agents of the multi-agent band selection model make selection decisions on features, the actions of the multi-agent band selection model include band selection or band deselection, the environment of the multi-agent band selection model is a classification network, and the classification network adopts random forest.

3. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 1 is characterized in that: The meta-description statistical method of the feature subspace is specifically: Extracting descriptive statistics from the selected data matrix to obtain a descriptive statistics matrix, wherein the descriptive statistics include standard deviation, minimum value, maximum value, first quartile, second quartile and third quartile; extracting descriptive statistics of each row from the descriptive statistics matrix to obtain a meta-descriptive statistics matrix, and connecting each column in the meta-descriptive statistics matrix to obtain a first state vector; The selected data matrix is ​​a data matrix composed of data samples and the number of bands selected by the agent in the action selection step; The feature subspace depth representation method based on the autoencoder is specifically as follows: Using a first autoencoder to convert each feature column in the selected data matrix into a first length latent vector to obtain a latent matrix, using a second autoencoder to map each row of the latent matrix to a second length latent vector to obtain a static encoding matrix, and connecting each column in the static encoding matrix to obtain a second state vector; The graph convolutional network method based on dynamic graph is specifically as follows: The selected data matrix is ​​converted into a feature correlation graph, the adjacency matrix of the feature correlation graph is solved, the potential representation of the feature is extracted from the adjacency matrix using a graph convolutional network, the potential representation of each feature is averaged and converted into a third state representation vector, and the graph convolutional network is trained using an unsupervised mathematical description method; The state representation of the multi-agent band selection model is the sum of a first state vector, a second state vector and a third state representation vector.

4. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 3 is characterized in that: The rewards include overall rewards and individual agent rewards. The overall reward is the classification accuracy obtained by classifying the band subsets formed after all agents take actions. The individual agent rewards include shared rewards and competitive rewards. The distribution strategy of the competitive rewards adopts an adaptive distribution strategy based on band importance.

5. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 4 is characterized in that: The individual agent reward is the sum of the shared reward and the competitive reward; The shared reward is the ratio of the difference between the classification accuracy of the current round and the classification accuracy of the previous round to the total number of agents that make selection actions; The calculation steps of the competitive reward include: iterating each band in the descriptive statistics matrix, randomly rearranging features in each iteration to generate a new data set, calculating the band importance score of each band according to the performance score of the new data set in the classification task, converting the band importance score into a competitive reward allocation contribution weight through normalization, and multiplying the competitive reward allocation contribution weight by the overall reward to obtain the competitive reward.

6. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 5 is characterized in that: The calculation formula for the band importance is as follows: in, is the importance score of the band, For the band, Score the model's raw performance on the descriptive statistics matrix, is the average performance score of the model on all new datasets, is the total number of iterations, k is the current iteration number, Score the performance of new datasets on classification tasks; The calculation formula for the competition reward allocation contribution weight is as follows: in, Assign contribution weights to competitive rewards, is the importance score of the band, For the band, is the sum of all band importance scores, is the total number of iterations, k is the current iteration number, and N is the total number of agents that make selection actions.

7. The multi-agent hyperspectral band selection method based on fusion state and optimal experience pool according to claim 5 is characterized in that: The specific sample-oriented experience pool optimization method based on the Gaussian mixture model comprises the following steps: Collect a memory sample set, wherein a memory sample in the memory sample set includes an action, a state, and an individual agent reward, assign all memory samples in the memory sample set to an unselected group, reallocate the memory samples to a selected group based on whether the memory samples are selected, select a first preset ratio of high-quality samples in the selected group, use the selected high-quality samples to train two Gaussian mixture models using an expectation maximization algorithm, use the trained Gaussian mixture models to generate simulated samples, use the simulated samples to replace part of the memory samples in the unselected group according to a second preset ratio, and combine the high-quality samples of the selected group and the replaced samples of the unselected group into a new high-quality data set; The sum of the first preset ratio and the second preset ratio is 1; Use new high-quality datasets to update the agent's selection strategy.

8. A multi-agent hyperspectral band selection system based on fusion state and optimal experience pool, characterized in that: include: A sample set acquisition module is used to generate a sample set of the hyperspectral image after acquiring the hyperspectral image, and divide the sample set of the hyperspectral image into a training sample set and a test sample set; Constructing a band selection model module, which is used to construct a multi-agent band selection model according to a training sample set. The state representation method of the multi-agent band selection model includes a meta-description statistics method of a feature subspace, a feature subspace deep representation method based on an autoencoder, and a graph convolution network method based on a dynamic graph. The reward allocation strategy of the multi-agent band selection model includes an adaptive allocation strategy based on band importance. A training band selection model module is used to input the training sample set into the multi-agent band selection model for model training to obtain a trained multi-agent feature selection model, wherein the memory samples of the experience pool of the model training are optimized using a specific sample-guided experience pool optimization method based on a Gaussian mixture model; The output band module is used to input the test sample set into the trained multi-agent feature selection model for band selection, output the hyperspectral image band set, classify the hyperspectral image band set, and obtain the classification result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the multi-agent hyperspectral band selection method based on fusion state and optimal experience pool as described in any one of claims 1 to 7.

Images