Hyperspectral open set field generalization method based on meta reinforcement learning

Abstract

The invention belongs to the technical field of pattern recognition, and discloses a hyperspectral open set field generalization method based on meta reinforcement learning. Firstly, in order to realize robust open set recognition strategy learning, background pixels are used for simulating a virtual unknown class, and a scene simulation mechanism of meta-learning is used for simulating an open set recognition scene in a source domain; in the scene simulation process, an open set recognition task is modeled as an internal Markov decision process, and an open set recognition strategy is iteratively optimized by using feedback learning of reinforcement learning and a strategy optimization mechanism. Secondly, in order to realize robust cross-domain knowledge migration from a source domain to a domain without a target, cross-task knowledge is summarized and modeled into an external Markov decision process, and an invariant risk reward mechanism is designed to encourage an agent to capture cross-task shared knowledge in different open set recognition tasks. Mama is introduced as a memory strategy network, and cross-task shared knowledge induction is realized by using the strong context compression capability of the Mama.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to a hyperspectral open set domain generalization method based on meta-reinforcement learning. Background Art

[0002] Domain Adaptation: Existing deep learning methods optimize models through training data in order to achieve good generalization performance on test data. However, the effectiveness of these methods usually relies on the assumption that the training data and test data are identically distributed. In practical applications, training data usually comes from source domains with different distributions. This cross-domain distribution difference significantly affects the effect of cross-domain knowledge transfer, resulting in unsatisfactory generalization performance of models with good performance in the source domain on the target domain. To address this challenge, domain adaptation methods have emerged. The core idea is to promote knowledge transfer by aligning the data distribution of the source domain and the target domain. Although such methods have made significant progress in performance, they still have limitations. First, domain adaptation usually depends on target domain data, which causes the model to be optimized for a specific target domain, limiting its wide application in real-world scenarios. In addition, such methods usually require the simultaneous storage of source and target domain data, which increases the memory requirements of computing devices, making them difficult to apply in scenarios with limited resources.

[0003] Domain generalization: Domain generalization methods aim to train models using data from one or more source domains so that they can be generalized to any unknown target domain. This method is more suitable for practical application scenarios than domain adaptation, because target domain data is usually difficult to obtain or even unknown when the model is deployed. However, most of the existing hyperspectral domain generalization methods are limited to applications under closed set conditions, that is, it is assumed that the source domain and the target domain have the same category distribution. This assumption ignores the open set situation in real scenarios, that is, the categories of the source domain and the target domain may be inconsistent. In practical applications, due to the differences in the acquisition areas of hyperspectral images, the category distributions of various fields are often not exactly the same, which limits the applicability of existing domain generalization methods in open environments. More seriously, under closed set conditions, all objects are classified as known categories, which may cause potential safety hazards.

[0004] Open set recognition: Under the traditional closed set assumption, training and test data share the same categories and feature space, so existing deep learning methods have made significant progress in many tasks. However, classification tasks in reality are usually open, that is, categories that have not been seen in training may appear in the test phase. In this case, if the model cannot correctly identify the unseen classes, it may mistakenly classify them as known classes, which in turn causes safety issues. This problem is particularly prominent in safety-critical fields such as unmanned driving, fault diagnosis, and medical diagnosis. To this end, open set recognition methods have emerged, whose goal is not only to accurately classify known classes in training data, but also to effectively identify unknown classes that have not been seen in training. However, most of the existing open set recognition methods assume that the target domain is known, which limits their application in dynamic open environments. When the model needs to process data from unseen domains after deployment, the existing open set recognition may face challenges.

[0005] Recently, some researchers have begun to explore solutions based on domain generalization. However, these methods only consider domain generalization under closed set conditions, that is, assuming that the source domain and target domain categories are consistent. They ignore open set domain generalization where the source domain and target domain categories are inconsistent. In fact, in real open environments, the categories of different hyperspectral images are often not exactly the same due to differences in the acquisition areas, which limits the application of existing domain generalization methods in real open environments. Worse still, under closed set conditions, all objects are classified into known categories, which may lead to safety issues.

[0006] Therefore, developing a hyperspectral open set domain generalization framework for open environments so that the model can recognize unknown categories in unseen target domains is the key to improving the security and applicability of hyperspectral image classification models in open environments. Summary of the invention

[0007] Purpose of the invention: Domain generalization aims to train a hyperspectral image classification model with strong generalization ability using data from the source domain when the target domain is invisible.

[0008] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is: a hyperspectral open set domain generalization method based on meta-reinforcement learning, comprising the following steps:

[0009] Step 1: Generate a virtual unknown class based on the background pixels of the source domain hyperspectral image, sample a set of hyperspectral image blocks from the set of virtual unknown classes as virtual unknown class samples; sample known class samples from the known class set in the source domain hyperspectral image; then construct G open set recognition tasks based on the virtual unknown class samples and known class samples; the G open set recognition tasks together constitute a cross-task shared knowledge induction task;

[0010] Step 2: Model the open set recognition task as an internal Markov decision process and customize the reinforcement learning environment for the task. Learn the optimal open set recognition strategy through the feedback interaction process between the agent and the customized environment.

[0011] Step 3: Model the cross-task shared knowledge induction task as an external Markov decision process, customize the reinforcement learning environment for the task, and use the invariant risk reward to drive the agent to learn the optimal cross-task shared knowledge induction strategy from different open set recognition tasks to achieve cross-task shared knowledge induction;

[0012] By utilizing the open set recognition strategy and cross-task shared knowledge induction strategy learned in the source domain hyperspectral images, the open set recognition task of the target domain hyperspectral images is completed to achieve open domain generalization.

[0013] Furthermore, in step 2, the open set recognition task is modeled as an internal Markov decision process, and a reinforcement learning environment is customized for the open set recognition task, wherein the reinforcement learning environment includes a state transfer function, an action, and a reward of the internal Markov decision process; specifically, the following includes:

[0014] Step 2.1, process the HSI into several image blocks, which are also called high-dimensional image blocks because they have a higher dimension;

[0015] Step 2.2, through the state-aware network, the high-dimensional image block is mapped to the low-dimensional space-spectrum feature space to obtain the space-spectrum feature z, which is expressed as:

[0016]

[0017] Among them, φ(·) and denote the spatial encoder and spectral encoder respectively.

[0018] Step 2.3, at the first time step, initialize a set of category prediction vectors p0, concatenate the category prediction vectors p0 and the empty spectrum feature z as the initial state, the initial state of the internal Markov decision process It is expressed as:

[0019]

[0020] Among them, CONCAT(·) is a concatenation operation.

[0021] Finally, the state transition function that defines the internal Markov decision process is expressed as:

[0022]

[0023] in, Predict the action for the class of the agent at the t-1th time step in the internal Markov decision process, represents the state observed by the agent at the tth time step in the internal Markov decision process;

[0024] Define the agent in the internal Markov decision process as the internal agent;

[0025] Step 2.4, customize the action design and reward design of the internal Markov decision process;

[0026] The action design is as follows: In the internal Markov decision process, the internal agent takes the category prediction action according to the observed current time step state, and predicts the probability that the sample belongs to the unknown class and the known class, which is expressed as:

[0027]

[0028] Among them, APN(·) is the action policy network, which consists of fully connected layers.

[0029] The rewards are designed to:

[0030]

[0031] Among them, C is the number of known classes, y c For labels, is the category prediction of the internal agent in the t-th time step of the internal Markov decision process that the current sample belongs to the c-th category, and λ is the trade-off coefficient used to adjust the value of the reward.

[0032] Furthermore, in step 2, the optimal open set recognition strategy is learned through the feedback interaction process between the agent and the customized reinforcement learning environment, specifically: the interactive feedback process of the internal agent in the internal Markov decision process is repeated until a predefined number of interactions is reached, and the internal Markov decision process interaction trajectory of the internal agent and the customized reinforcement learning environment is obtained; based on the internal Markov decision process interaction trajectory, the policy gradient loss and the value function loss are calculated to optimize the open set recognition strategy of the internal Markov decision process;

[0033] The policy gradient loss function of the internal Markov decision process is expressed as:

[0034]

[0035] in,

[0036] Represents the internal state s I The distribution at time step t, γ is the discount factor, is a parameterized policy network that receives s I As input, it outputs the category prediction action with the maximum expected return in the current state according to the learned strategy. I , is the advantage function, which is used to evaluate the current policy network in a given state s I The output action is: Indicates the expectation of optimization.

[0037] Value loss in an internal Markov decision process It is expressed as:

[0038]

[0039] Where V(·) is the state value function, which represents the expected return of following the current strategy under the input state. t is the reward obtained by the internal agent at the tth time step, and T is the predefined number of interactions in the internal Markov decision process.

[0040] Furthermore, in step 3, the cross-task shared knowledge induction task is modeled as an external Markov decision process, and a reinforcement learning environment is customized for the task, wherein the reinforcement learning environment includes the state and state transfer function, action and reward of the external Markov decision process; specifically,

[0041] The state of the external Markov decision process is: the state set of each time step in the internal Markov decision process is passed as the state to the agent in the external Markov decision process as the state of the agent in the external Markov decision process;

[0042] The agent in the external Markov decision process is denoted as the external agent;

[0043] The state transfer function of the external Markov decision process refers to: at each time step, the state of the external agent is converted into the state set of each time step in the next internal Markov decision process.

[0044] Furthermore, the action design refers to: based on the memory strategy module of Mamba, the current state of the external agent in the external Markov decision process is used as the input of the memory strategy module of Mamba in each time step, and the cross-task knowledge induction action of the current time step is output.

[0045] Consider the following continuous-time state-space model:

[0046] h'(g)=Ah(g)+Bx(g)

[0047] y(g)=Ch(g)+Dx(g)

[0048] Among them, x(g) is the input sequence, h'(g) is the derivative of the hidden state, A is the state matrix, B is the input mapping matrix, C is the output mapping matrix, and D is the direct transfer matrix.

[0049] Using zero-order hold, the continuous parameter is converted into a discrete parameter by introducing a time scale parameter △:

[0050]

[0051] in, and are the discrete parameters corresponding to A, B and C in the system. Therefore, the following state space model can be obtained:

[0052]

[0053] Among them, h g is a memory vector that encodes contextual history information, x g is the input, y g is the output.

[0054] In the present invention, the memory strategy module based on Mamba stores the current state of the external agent in the external Markov decision process in each time step. As input, output the cross-task knowledge induction action of the current time step Therefore, the above state space model can be rewritten as:

[0055]

[0056] h g is the memory vector, Cross-task knowledge induction action at the current time step.

[0057] Furthermore, a selection mechanism is used to obtain B, C and △, which are dynamically parameterized according to the input changes:

[0058] B=f B (x g )

[0059] C = f C (x g )

[0060] △ = f △ (x g )

[0061] Among them, f B (·),f C (·) and f △ (·) are all parameterized neural networks.

[0062] Furthermore, the reward of the external Markov decision process is designed as a constant risk reward It is expressed as:

[0063]

[0064] in, Indicates that the external agent takes the knowledge induction action to obtain the memory state at the gth time step After that, the internal agent is based on the memory state Take the open set recognition reward corresponding to the category prediction action.

[0065] Represents the internal agent based on the state Take the open set recognition reward corresponding to the category prediction action. β is an adjustable regularization coefficient.

[0066] Furthermore, we use rewards to drive external agents to learn the optimal cross-task shared knowledge induction strategy from different open set recognition tasks, specifically:

[0067] Repeat the process of interactive feedback of the agent in the external Markov decision process until the predefined number of interactions is reached, and obtain the interaction trajectory of the external Markov decision process between the agent and the customized environment. Finally, the policy gradient loss and value function loss are calculated based on the interaction trajectory to optimize the cross-task knowledge induction strategy of the external Markov decision process.

[0068] Policy Gradient Loss Function for External Markov Decision Process Expressed as:

[0069]

[0070] in,

[0071] Represents the external state s O The distribution at time step g;

[0072] is a parameterized policy network that receives s O As input, it outputs the cross-task knowledge induction action a with the maximum expected return in the current state according to the learned strategy O ;

[0073] is the advantage function, which is used to evaluate the current policy network in a given state s O The following output action.

[0074] Furthermore, the value loss of the external Markov decision process is expressed as:

[0075]

[0076] V(·) is the state value function, which represents the expected return of following the current policy under the input state;

[0077] R g is the reward obtained by the external agent at the gth time step, and G is the predefined number of interactions in the external Markov decision process.

[0078] Beneficial effects: The present invention integrates meta-learning, reinforcement learning, and invariant risk minimization into a unified framework. On the one hand, in order to achieve robust open set recognition strategy learning, background pixels are innovatively used as virtual unknown classes in the model training process, and it is intended to achieve open set recognition scenario simulation containing unknown classes under the condition of using only source domain data. In addition, inspired by the human feedback learning mechanism, the present invention models the open set recognition task as an internal Markov decision process, and iteratively corrects the recognition results through feedback interaction between the agent and the customized environment to ensure that the learned open set recognition strategy has robust generalization. On the other hand, in order to promote the knowledge transfer from the source domain to the unseen domain, it is intended to model the cross-task knowledge induction task as an external Markov decision process, and design invariant risks to reward and encourage the agent to summarize generalizable cross-task invariant knowledge from different open set recognition tasks, thereby improving its generalization performance in new unseen open set recognition tasks. The present invention accurately identifies unknown classes in unseen domains under the condition that only source domain data is available, solving the problems of difficult open set recognition strategy learning and unseen domain knowledge transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a principle block diagram of the method of the present invention. DETAILED DESCRIPTION

[0080] The present invention is further described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] The present invention develops a method for exploring hyperspectral open set domain generalization based on meta-reinforcement learning. First, an attempt is made to mine the potential rich information in background pixels and use the background pixels to construct virtual unknown classes, thereby simulating the open set recognition task in the source domain and providing a data basis for open set recognition strategy learning; secondly, the open set recognition task is modeled as an internal Markov decision process, and the feedback learning mechanism of reinforcement learning is used to iteratively correct the recognition result, and its strategy optimization mechanism is used to achieve optimal strategy learning; then, the cross-task knowledge induction task is modeled as an external Markov decision process, and an invariant risk reward is designed to encourage the agent to summarize generalizable cross-task invariant knowledge from different open set recognition tasks, thereby improving its generalization performance in new and unseen open set recognition tasks; finally, Mamba is introduced as a memory strategy network, and its powerful context compression capability is used to achieve cross-task shared knowledge induction; specifically, the present invention provides a hyperspectral open set domain generalization method based on meta-reinforcement learning, such as Figure 1 As shown, the following steps are included:

[0082] Step 1: Meta-task construction.

[0083] Firstly, the background pixels of the source domain hyperspectral image are regarded as a set of virtual unknown classes, and a hyperspectral image block is sampled from the set of virtual unknown classes as a sample of the virtual unknown class to construct an open set recognition task.

[0084] Repeat the above process until G open set recognition tasks are obtained;

[0085] The G open set recognition tasks are taken as a task set to construct a cross-task shared knowledge induction task;

[0086] The meta-task refers to an open set recognition task and a cross-task shared knowledge induction task.

[0087] Step 2: Model the open set recognition task as an internal Markov decision process, and customize the reinforcement learning environment for the open set recognition task. Through the feedback interaction process between the intelligent agent and the customized reinforcement learning environment, learn the optimal open set recognition strategy.

[0088] The first part is customizing states and state transition functions.

[0089] Firstly, considering the continuity of similar pixels in the hyperspectral image, the HSI is processed into several image blocks.

[0090] Then, considering that the image patch has a high dimensionality, it may increase the complexity of the agent's policy learning. The state-aware network is used to map the high-dimensional image patch to a low-dimensional spatial-spectral feature space. The state-aware network includes a spectral encoder and a spatial encoder, which are used to capture the spectral context association between HSI spectral bands and the spatial context association between spatial pixels, respectively. The spatial-spectral feature z can be expressed as:

[0091]

[0092] Among them, φ(·) and represent the spatial encoder and spectral encoder respectively.

[0093] Next, at the first time step, a set of category prediction vectors p0 are initialized, and the empty spectrum feature z and the category prediction vector p0 are concatenated as the initial state.

[0094] Initial state of the internal Markov decision process It can be expressed as:

[0095]

[0096] Among them, CONCAT(·) is a concatenation operation.

[0097] Finally, the state transition rule of the internal Markov decision process is defined as follows to describe the impact of the agent's current time step action on the future state. The state transition function of the internal Markov decision process can be expressed as:

[0098]

[0099] in, The class prediction action of the agent in the internal Markov decision process at the t-1th time step. The agent in the internal Markov decision process is also called the internal agent; at each time step, the internal agent observes the spatial-spectral features of the sample in turn and predicts the class probability of the sample based on the historical prediction information.

[0100] The second part is to customize the design of actions and rewards in the internal Markov decision process. The first is action design. In the internal Markov decision process, the agent takes a category prediction action based on the observed current time step state to predict the probability of the sample belonging to the unknown class and the known class, which can be expressed as:

[0101]

[0102] Among them, APN(·) is the action policy network, which consists of fully connected layers.

[0103] The second is reward design. In this invention, an open set recognition reward is set to encourage the agent to predict the sample category close to the true label, so that it can learn the optimal open set recognition strategy that can minimize the difference between the category prediction action distribution and the true label distribution during the training process.

[0104] The rewards are designed to:

[0105]

[0106] Among them, C is the number of known classes, y c For labels, is the category prediction of the internal agent in the t-th time step of the internal Markov decision process that the current sample belongs to the c-th category, and λ is the trade-off coefficient used to adjust the value of the reward.

[0107] The third part is strategy optimization. Repeat the process of interactive feedback of the agent in the internal Markov decision process until the predefined number of interactions is reached, and obtain the interaction trajectory of the internal Markov decision process between the agent and the customized environment. Finally, the policy gradient loss and value function loss are calculated based on the interaction trajectory of the internal Markov decision process to optimize the open set recognition strategy of the internal Markov decision process. The policy gradient loss function of the internal Markov decision process can be expressed as:

[0108]

[0109] in, Represents the internal state s I The distribution at time step t, γ is the discount factor. is a parameterized policy network that receives s I As input, it outputs the category prediction action with the maximum expected return in the current state according to the learned strategy. I . is the advantage function, which is used to evaluate the current policy network in a given state s I The output action can provide how much additional reward compared to the expected average level. Through the subtraction operation, it highlights the advantages of specific actions over the average policy performance, thereby guiding the policy network to optimize the learning process; represents the expectation of optimization, Indicates state s I The state distribution ρ guided by the policy π p I Medium sampling, a I ~p θ I Indicates action a I According to the policy π with parameter θ, in a given state s I Downsampling.

[0110] The value function loss of the internal Markov decision process can be expressed as:

[0111]

[0112] Where V(·) is the state value function, which represents the expected return of following the current strategy under the input state. t is the reward obtained by the internal agent at the tth time step, T is the predefined number of interactions in the internal Markov decision process, and γ is the discount factor, which is the same as γ in the policy gradient loss function.

[0113] Step 3: Model the cross-task shared knowledge induction task as an external Markov decision process, and customize the reinforcement learning environment for the task. Use the invariant risk reward to drive the agent to learn the optimal cross-task shared knowledge induction strategy from different open set recognition tasks to achieve cross-task shared knowledge induction.

[0114] In the first part, we customize the states and state transition functions of the external Markov decision process.

[0115] First, considering that the knowledge induction strategy learning process of the cross-task shared knowledge induction agent depends on the accurate perception of each open set recognition task, the state set of each time step in the internal Markov decision process is passed to the external agent as the state. At each time step, the state of the external agent is converted into the state set of each time step in the next internal Markov decision process.

[0116] The second part,customizes actions and rewards for custom external Markov decision processes.

[0117] First of all, in action design, the present invention introduces an additional memory mechanism to enable the agent to integrate the current state and historical memory information when taking actions. By compressing and updating the historical context online, the historical memory information is explicitly integrated into the decision-making process of the agent, thereby effectively improving its generalization ability for new tasks that have not been seen. Specifically, the Mamba-based memory strategy module takes the current state of the external agent in the external Markov decision process as input in each time step, outputs the cross-task knowledge induction action of the current time step, and encodes the task-related historical experience information into the state of the internal Markov decision process agent.

[0118] Specifically, consider the following continuous-time state-space model:

[0119] h'(g)=Ah(g)+Bx(g)

[0120] y(g)=Ch(g)+Dx(g)

[0121] Among them, x(g) is the input sequence, h'(g) is the derivative of the hidden state, A is the state matrix, B is the input mapping matrix, C is the output mapping matrix, and D is the direct transfer matrix.

[0122] In order to make the above state space model compatible with the deep learning framework, the continuous parameters are converted into discrete parameters by introducing the time scale parameter △ using zero-order hold:

[0123]

[0124] in, and are the discrete parameters corresponding to A, B and C in the system. Therefore, the following state space model can be obtained:

[0125]

[0126] Among them, h g is a memory vector that encodes contextual history information, x g is the input, y g is the output. It should be noted that D can be ignored as a residual. It is worth noting that in order to enhance the model’s ability to capture key information from historical memory while filtering irrelevant information, a selection mechanism is used when obtaining B, C and △, that is, they are not fixed, but dynamically parameterized according to the changes in the input:

[0127] B=f B (x g )

[0128] C = f C (x g )

[0129] △ = f △ (x g )

[0130] Among them, f B (·),f C (·) and f △ (·) are all parameterized neural networks.

[0131] In the present invention, the memory strategy module based on Mamba stores the current state of the external agent in the external Markov decision process in each time step. As input, output the cross-task knowledge induction action of the current time step The historical experience information related to the task is encoded into the state of the internal Markov decision process agent. Therefore, the state space model above can be rewritten as:

[0132]

[0133] h g is the memory vector, Cross-task knowledge induction action at the current time step

[0134] The second is reward design.

[0135] In order to ensure the cross-task knowledge induction ability of the agent, a constant risk reward based on the minimization of constant risk is designed. Under the incentive of this reward, the agent needs to maintain stable performance in a series of changing internal Markov decision processes. The invariance principle is adopted, which believes that certain causal relationships remain unchanged in all internal Markov decision processes, and these invariant causal relationships constitute the core of cross-task shared knowledge induction. The constant risk reward measures the consistency of the agent's behavioral strategy with the known invariant causal structure when exploring a new environment. A high constant risk reward indicates that the agent can effectively identify and apply these invariant factors, thereby obtaining higher rewards. This design not only encourages the agent to explore and utilize invariant causal relationships, but also enables it to quickly adapt and show excellent generalization performance when facing unknown environments.

[0136] Constant risk reward It can be expressed as:

[0137]

[0138] in, The agent representing the external Markov decision process takes the knowledge induction action at the gth time step to obtain the memory state After that, the agent of the internal Markov decision process is based on the memory state Take the open set recognition reward corresponding to the category prediction action.

[0139] The agent representing the internal Markov decision process is based on the state Take the open set recognition reward corresponding to the category prediction action. β is an adjustable regularization coefficient.

[0140] In the constant risk reward, Encourage the agent to learn the optimal cross-task knowledge induction strategy that maximizes the average reward of each internal Markov decision process, It is expected that the open set recognition strategy learned by the agent will perform close to the optimal open set recognition strategy for each internal Markov decision process.

[0141] In this way, the invariant risk reward can take into account the generalization ability of the learned open set recognition strategy to different Markov decision processes and the adaptability to specific Markov decision processes, preventing the performance of the open set recognition strategy from deteriorating on the open set recognition task due to excessive pursuit of invariance.

[0142] The third part is strategy optimization. Repeat the process of interactive feedback of the agent in the external Markov decision process until the predefined number of interactions is reached, and obtain the interaction trajectory of the external Markov decision process between the agent and the customized environment. Finally, the cross-task knowledge induction strategy of the external Markov decision process is optimized by calculating the policy gradient loss and value function loss based on the interaction trajectory.

[0143] Policy Gradient Loss Function for External Markov Decision Process It can be expressed as:

[0144]

[0145] in,

[0146] Represents the external state s O The distribution at time step g. is a parameterized policy network that receives s O As input, it outputs the cross-task knowledge induction action a with the maximum expected return in the current state according to the learned strategy O . is the advantage function, which is used to evaluate the current policy network in a given state s O The output action shows how much additional reward can be provided compared to the expected average level. Through the subtraction operation, it highlights the advantages of specific actions over the average policy performance, thereby guiding the policy network to optimize the learning process.

[0147] The value function loss of the external Markov decision process can be expressed as:

[0148]

[0149] Step 4: Open set domain generalization. First, the open set recognition process of a group of samples in the target domain hyperspectral image is regarded as an internal Markov decision process. In the internal Markov decision process, the agent interacts with the customized environment through feedback according to the learned strategy to generate a state set of each internal Markov decision process. Then, the state set of these internal Markov decision processes is used as the state of the external Markov decision process and input into the Mamba-based memory strategy module. Based on the learned memory induction strategy, cross-task shared knowledge is captured from the state set of these internal Markov decision processes and knowledge induction actions are taken to obtain a memory state set. Finally, the agent in the internal Markov decision process predicts the category of the sample based on the memory state through feedback interaction with the customized environment to obtain the final open set classification prediction.

[0150] In summary, the present invention has developed a hyperspectral open set domain generalization method based on meta-reinforcement learning, which solves the problems of open set recognition strategy learning difficulty and unseen domain knowledge transfer difficulty faced by the hyperspectral open set domain generalization task. Specifically, firstly, based on the scenario simulation mechanism of meta-learning, background pixels are innovatively used to simulate unknown classes, and the strategy optimization mechanism of reinforcement learning is used to achieve the optimal open set recognition strategy learning. It should be pointed out that the introduction of reinforcement learning feedback learning mechanism breaks the existing open set recognition method trial and error. Figure 1 The inherent paradigm of step-by-step recognition of unknown classes can identify subtle differences between unknown and known classes by iteratively correcting the prediction results. Secondly, the cross-task knowledge attribution task is modeled as a Markov decision process, which enables the agent to discover cross-task shared knowledge in different open set recognition tasks under the drive of constant risk reward, promoting knowledge transfer from the source domain to the unseen target domain.

Claims

1. A hyperspectral open set domain generalization method, characterized in that: The steps include: Step 1: Generate a virtual unknown class based on the background pixels of the source domain hyperspectral image, and sample a set of hyperspectral image patches from the set of virtual unknown classes as virtual unknown class samples; Samples of known classes are obtained by sampling from a set of known classes in a source domain hyperspectral image; Then, G open set recognition tasks are constructed based on virtual unknown class samples and known class samples; G open set recognition tasks together constitute the cross-task shared knowledge induction task; Step 2: Model the open set recognition task as an internal Markov decision process and customize a reinforcement learning environment for the open set recognition task. Through the feedback interaction process between the internal agent and the customized reinforcement learning environment, learn the optimal open set recognition strategy. Step 3: Model the cross-task shared knowledge induction task as an external Markov decision process and customize the reinforcement learning environment for this task, using rewards to drive the external agent to learn the optimal cross-task shared knowledge induction strategy from different open set recognition tasks.

2. A hyperspectral open set domain generalization method according to claim 1, characterized in that: In step 2, the open set recognition task is modeled as an internal Markov decision process, and a reinforcement learning environment is customized for the open set recognition task, wherein the reinforcement learning environment includes the state transfer function, action and reward of the internal Markov decision process; specifically, it includes: Step 2.1, process HSI into several image blocks; Step 2.2, through the state-aware network, the high-dimensional image block is mapped to the low-dimensional space-spectrum feature space to obtain the space-spectrum feature z, which is expressed as: Among them, φ(·) and denote the spatial encoder and spectral encoder respectively; Step 2.3, at the first time step, initialize a set of category prediction vectors p0, concatenate the category prediction vectors p0 and the empty spectrum feature z as the initial state, the initial state of the internal Markov decision process It is expressed as: Among them, CONCAT(·) is a concatenation operation; Finally, the state transition function that defines the internal Markov decision process is expressed as: in, Predict the action for the class of the agent at the t-1th time step in the internal Markov decision process, represents the state observed by the agent at the tth time step in the internal Markov decision process; Define the agent in the internal Markov decision process as the internal agent; Step 2.4, customize the actions and rewards of the internal Markov decision process; The action design is as follows: In the internal Markov decision process, the internal agent takes the category prediction action according to the observed current time step state, and predicts the probability that the sample belongs to the unknown class and the known class, which is expressed as: Among them, APN(·) is the action policy network, which consists of fully connected layers; The rewards are designed to: Among them, C is the number of known classes, y c For labels, is the category prediction of the internal agent in the t-th time step of the internal Markov decision process that the current sample belongs to the c-th category, and λ is the trade-off coefficient used to adjust the value of the reward.

3. A hyperspectral open set domain generalization method according to claim 2, characterized in that: In step 2, the optimal open set recognition strategy is learned through the feedback interaction process between the agent and the customized reinforcement learning environment, specifically: the interactive feedback process of the internal agent in the internal Markov decision process is repeated until a predefined number of interactions is reached, and the internal Markov decision process interaction trajectory of the internal agent and the customized reinforcement learning environment is obtained; based on the internal Markov decision process interaction trajectory, the policy gradient loss and the value function loss are calculated to optimize the open set recognition strategy of the internal Markov decision process; The policy gradient loss function is expressed as: in, Represents the internal state s I The distribution at time step t, γ is the discount factor, is a parameterized policy network that receives s I As input, it outputs the category prediction action with the maximum expected return in the current state according to the learned strategy. I , is the advantage function, which is used to evaluate the current policy network in a given state s I The output action is: Expresses the expectation of optimization; The value function loss It is expressed as: Where V(·) is the state value function, which represents the expected return of following the current strategy under the input state; R t is the reward obtained by the internal agent at the tth time step, and T is the predefined number of interactions in the internal Markov decision process.

4. The hyperspectral open set domain generalization method according to claim 1, characterized in that: In step 3, the cross-task shared knowledge induction task is modeled as an external Markov decision process, and a reinforcement learning environment is customized for the task, wherein the reinforcement learning environment includes the state, state transition function, action and reward of the external Markov decision process; specifically, The state of the external Markov decision process is: the state set of each time step in the internal Markov decision process is passed as the state to the agent in the external Markov decision process as the state of the agent in the external Markov decision process; The agent in the external Markov decision process is denoted as the external agent; The state transfer function of the external Markov decision process refers to: at each time step, the state of the external agent is converted into the state set of each time step in the next internal Markov decision process.

5. The hyperspectral open set domain generalization method according to claim 1, characterized in that: The action refers to: based on the memory strategy module of Mamba, in each time step, the current state of the external agent in the external Markov decision process is used as the input of the memory strategy module of Mamba, and the cross-task knowledge induction action of the current time step is output.

6. A hyperspectral open set domain generalization method according to claim 5, characterized in that: Consider the following continuous-time state-space model: h'(g)=Ah(g)+Bx(g) y(g)=Ch(g)+Dx(g) Among them, x(g) is the input sequence, h'(g) is the derivative of the hidden state, A is the state matrix, B is the input mapping matrix, C is the output mapping matrix, and D is the direct transfer matrix. Using zero-order hold, the continuous parameter is converted into a discrete parameter by introducing a time scale parameter △: in, and are the discrete parameters corresponding to A, B and C in the system, so the following state space model is obtained: Among them, h g is a memory vector that encodes contextual history information, x g is the input, y g is the output; The Mamba-based memory strategy module stores the current state of the external agent in the external Markov decision process at each time step. As input, output the cross-task knowledge induction action of the current time step Therefore, the above state space model can be rewritten as: h g is the memory vector, Cross-task knowledge induction action at the current time step.

7. A hyperspectral open set domain generalization method according to claim 6, characterized in that: A selection mechanism is used to obtain B, C and △, which are dynamically parameterized according to the input changes: B=f B (x g ) C=f C (x g ) △=f △ (x g ) Among them, f B (·),f C (·) and f △ (·) are all parameterized neural networks.

8. The hyperspectral open set domain generalization method according to claim 4, characterized in that: The reward of the external Markov decision process is a constant risk reward It is expressed as: in, Indicates that the external agent takes the knowledge induction action to obtain the memory state at the gth time step After that, the internal agent is based on the memory state Take the open set recognition reward corresponding to the category prediction action; Represents the internal agent based on the state Take the open set recognition reward corresponding to the category prediction action; β is an adjustable regularization coefficient.

9. A hyperspectral open set domain generalization method according to claim 8, characterized in that: The reward-driven external agent learns the optimal cross-task shared knowledge induction strategy from different open set recognition tasks, specifically: Repeat the process of interactive feedback of the agent in the external Markov decision process until the predefined number of interactions is reached, and obtain the interaction trajectory of the external Markov decision process between the agent and the customized environment; finally, calculate the policy gradient loss and value function loss based on the interaction trajectory to optimize the cross-task knowledge induction strategy of the external Markov decision process; Policy Gradient Loss Function for External Markov Decision Process Expressed as: in, Represents the external state s O The distribution at time step g; is a parameterized policy network that receives s O As input, it outputs the cross-task knowledge induction action a with the maximum expected return in the current state according to the learned strategy O ; is the advantage function, which is used to evaluate the current policy network in a given state s O The following output action.

10. A hyperspectral open set domain generalization method according to claim 9, characterized in that: The value function loss of the external Markov decision process is expressed as: V(·) is the state value function, which represents the expected return of following the current policy under the input state; R g is the reward obtained by the external agent at the gth time step, and G is the predefined number of interactions in the external Markov decision process.

Images