A hyperspectral image classification method, device, electronic device and storage medium

Abstract

The present invention relates to the technical field of shake-sensing image processing, and in particular to a hyperspectral image classification method, device, electronic device and storage medium, including: using a variational inference loss function to train a domain data extension model based on a diffusion model to obtain a trained domain data extension model; using the trained domain data extension model to expand source domain scene data to generate extended domain scene data; based on the extended domain scene data and the source domain scene data, using supervised contrast learning and contrast adversarial learning loss functions to train a discriminator network model to obtain a trained discriminator network model; using the trained discriminator network model to extract features and classify and predict target domain scene data to obtain the land cover category of the target domain scene data. Through a cross-scene hyperspectral image classification method based on a diffusion model, accurate and reliable cross-scene hyperspectral image classification is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of shake-sensing image processing, and in particular to a hyperspectral image classification method, device, electronic device and storage medium. Background Art

[0002] Hyperspectral images contain hundreds of continuous narrow spectral bands, with high spectral resolution and rich spectral spatial information, and are therefore widely used in the field of remote sensing image processing. Hyperspectral image classification is an important task in remote sensing image processing. It aims to assign a land cover category label to each pixel in the hyperspectral image to identify the material conditions of the land cover in the area, and plays a vital role in mineral exploration, agricultural planning, crop detection, urban construction planning, etc.

[0003] Most of the current hyperspectral image classification methods train and optimize the model by using a sufficient number of labeled samples with sufficiently accurate category labels, so as to obtain a hyperspectral image classification model with good classification accuracy to achieve the classification of a specific scene data. However, in the actual application process, due to the cost and complexity of hyperspectral remote sensing data acquisition and manual annotation, the amount of existing annotated hyperspectral remote sensing data is relatively limited. In addition, since hyperspectral remote sensing images are affected by factors such as sensor nonlinearity and different material compositions of surface cover types in the same area at different scales, different seasonal conditions and weather and climate conditions during the acquisition process, even if hyperspectral remote sensing data is collected for the same surface area, there will be certain spectral reflectance differences between the hyperspectral remote sensing images obtained when the sensors used or the seasonal climates taken are different. This makes the existing hyperspectral image classification methods have poor classification performance when facing such new scene data, and it is difficult to meet the needs of cross-scene classification.

[0004] In summary, how to design an accurate and reliable cross-scene hyperspectral image classification method is an urgent problem that needs to be solved. Summary of the invention

[0005] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0006] To this end, the first purpose of this application is to propose a hyperspectral image classification method to solve the problems of poor generalization and high classification error of cross-scene hyperspectral remote sensing image classification in existing technical means.

[0007] The second object of the present application is to provide a device.

[0008] The third objective of the present application is to provide an electronic device.

[0009] A fourth objective of the present application is to provide a computer-readable storage medium.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a hyperspectral image classification method, comprising:

[0011] The domain data extension model based on the diffusion model is trained using the loss function of variational inference to obtain a trained domain data extension model;

[0012] Expanding the source domain scene data using the trained domain data expansion model to generate extended domain scene data;

[0013] Based on the extended domain scene data and the source domain scene data, the discriminator network model is trained using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model;

[0014] The trained discriminator network model is used to perform feature extraction and classification prediction on the target domain scene data to obtain the land cover category of the target domain scene data.

[0015] Preferably, the loss function of variational inference is used to train the domain data expansion model based on the diffusion model to obtain the trained domain data expansion model, which includes:

[0016] Block-dividing the labeled pixels in the source domain scene data to obtain multiple hyperspectral image blocks of the source domain scenes and land cover category labels corresponding to the multiple hyperspectral image blocks of the source domain scenes;

[0017] Randomly masking or reordering the labels of each pixel point except the central pixel in the land cover category label to obtain a new land cover category label;

[0018] Extracting information from the new land cover category label, guiding the domain data extension model to extract different categories of feature learning from the hyperspectral image block of the source domain scene, and obtaining an extended domain hyperspectral image block corresponding to the new land cover category label;

[0019] The domain data extension model is trained using the loss function of the variational inference to obtain a trained domain data extension model.

[0020] Preferably, the calculation formula of the loss function of the variational inference is:

[0021]

[0022] in, is the loss function of variational inference, for The noise of the moment, is an untrained UNet network for rich map generation, is the UNet network parameter, Untrained fully connected network generated for endmembers, is the fully connected network parameter, is a statistical parameter, It means converting it into vector form. is a low-dimensional random vector used to generate the r-th abundance map, is a low-dimensional random vector used to generate the r-th endmember.

[0023] Preferably, the step of expanding the source domain scene data by using the trained domain data expansion model to generate the extended domain scene data includes:

[0024] Inputting the hyperspectral image blocks of the source domain scene and the land cover category labels corresponding to the hyperspectral image blocks of the source domain scene into the trained domain data extension model, randomly generating a set of extended domain data having geographic information differences with the source domain data and the land cover category labels corresponding to the extended domain data;

[0025] A plurality of extended domain data having geographic information differences with the source domain data and land cover category labels corresponding to the extended domain data are obtained, and each source domain scene hyperspectral image block obtained by division is sequentially input into the trained domain data extension module to obtain a set of extended domain data having differences therefrom, and to obtain extended domain scene data.

[0026] Preferably, the discriminator network model is trained based on the extended domain data and the source domain scene data using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model, including:

[0027] Build a discriminator network model;

[0028] Using the discriminator network model to perform feature extraction on the source domain scene data and the extended domain scene data to obtain source domain feature values ​​and extended domain feature values, and using the discriminator network model to perform classification processing on the source domain feature values ​​and the extended domain feature values ​​to obtain source domain feature classification results, source domain feature transformation results, extended domain feature classification results, and extended domain feature transformation results;

[0029] Based on the source domain feature classification result, the source domain feature transformation result, the extended domain feature classification result and the extended domain feature transformation result, supervised contrastive learning and contrastive adversarial learning loss functions are used to perform loss calculation to obtain a trained discriminator network model.

[0030] Preferably, the total loss function calculation formula for the discriminator network model training is:

[0031]

[0032] in, is the supervised contrastive learning loss function, To compare the adversarial learning loss function, is the weight of the supervised contrastive learning loss function, is the weight of the loss function for contrastive learning.

[0033] Preferably, the method further comprises: dynamically adjusting the statistics of the target domain scene data and the statistics of the source domain scene data using a dynamic statistics adjustment method.

[0034] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a hyperspectral image classification device, comprising:

[0035] The data domain extension module uses the loss function of variational inference to train the domain data extension model based on the diffusion model to obtain a trained domain data extension model;

[0036] An extended domain scene data module, which uses the trained domain data extension model to expand the source domain scene data to generate extended domain scene data;

[0037] A discriminator training module, based on the extended domain scene data and the source domain scene data, trains the discriminator network model using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model;

[0038] The prediction module uses the trained discriminator network model to perform feature extraction and classification prediction on the target domain scene data to obtain the land cover category of the target domain scene data.

[0039] To achieve the above-mentioned purpose, the third aspect of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0040] The memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory to implement any of the above methods.

[0042] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present application proposes a computer-readable storage medium, including computer-executable instructions stored in the computer-readable storage medium, and the computer-executable instructions are used to implement any of the methods described above when executed by a processor.

[0043] A hyperspectral image classification method provided by the present application utilizes a variational inference loss function to train a domain data extension model based on a diffusion model, and obtains extended domain scene data through the trained model. The obtained extended domain data not only has domain-invariant features consistent with the source domain, but also preserves inter-domain differences, which can help subsequent feature extraction modules and classification prediction modules to better learn domain-invariant features and inter-domain differences, and effectively solve the learning problem of inconsistent probability distribution of sample data of different domain scenes existing in existing cross-scene hyperspectral image classification methods, and provide accurate and reliable land cover material detection and regional division results for tasks such as mineral exploration, agricultural planning, crop detection, and urban construction planning.

[0044] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 A flowchart of a first specific embodiment of a hyperspectral image classification method provided by the present invention;

[0047] Figure 2 It is a structural diagram of the domain data expansion model based on the diffusion model;

[0048] Figure 3 Schematic diagram of the discriminator network model;

[0049] Figure 4 is a schematic diagram of a hyperspectral image classification system;

[0050] Figure 5 A structural block diagram of a hyperspectral image classification device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The core of the present invention is to provide a hyperspectral image classification method, device, electronic device and storage medium, which can achieve accurate and reliable cross-scene hyperspectral image classification through a cross-scene hyperspectral image classification method based on a diffusion model.

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] Please refer to Figure 1 , Figure 1 This is a flow chart of a first specific embodiment of a hyperspectral image classification method provided by the present invention; the specific operation steps are as follows:

[0054] Step S101: training a domain data extension model based on a diffusion model using a loss function of variational inference to obtain a trained domain data extension model;

[0055] Block-dividing the labeled pixels in the source domain scene data to obtain multiple hyperspectral image blocks of the source domain scenes and land cover category labels corresponding to the multiple hyperspectral image blocks of the source domain scenes;

[0056] Randomly masking or reordering the labels of each pixel point except the central pixel in the land cover category label to obtain a new land cover category label;

[0057] Extracting information from the new land cover category label, guiding the domain data extension model to extract different categories of feature learning from the hyperspectral image block of the source domain scene, and obtaining an extended domain hyperspectral image block corresponding to the new land cover category label;

[0058] The domain data extension model is trained using the loss function of the variational inference to obtain a trained domain data extension model.

[0059] The calculation formula of the loss function of the variational inference is:

[0060]

[0061] in, is the loss function of variational inference, for The noise of the moment, is an untrained UNet network for rich map generation, is the UNet network parameter, Untrained fully connected network generated for endmembers, is the fully connected network parameter, is a statistical parameter, It means converting it into vector form. is a low-dimensional random vector used to generate the r-th abundance map, is a low-dimensional random vector used to generate the r-th endmember.

[0062] Step S102: using the trained domain data extension model to expand the source domain scene data to generate extended domain scene data;

[0063] Inputting the hyperspectral image blocks of the source domain scene and the land cover category labels corresponding to the hyperspectral image blocks of the source domain scene into the trained domain data extension model, randomly generating a set of extended domain data having geographic information differences with the source domain data and the land cover category labels corresponding to the extended domain data;

[0064] A plurality of extended domain data having geographic information differences with the source domain data and land cover category labels corresponding to the extended domain data are obtained, and each source domain scene hyperspectral image block obtained by division is sequentially input into the trained domain data extension module to obtain a set of extended domain data having differences therefrom, and to obtain extended domain scene data.

[0065] Step S103: Based on the extended domain scene data and the source domain scene data, the discriminator network model is trained using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model;

[0066] Build a discriminator network model;

[0067] Using the discriminator network model to perform feature extraction on the source domain scene data and the extended domain scene data to obtain source domain feature values ​​and extended domain feature values, and using the discriminator network model to perform classification processing on the source domain feature values ​​and the extended domain feature values ​​to obtain source domain feature classification results, source domain feature transformation results, extended domain feature classification results, and extended domain feature transformation results;

[0068] Based on the source domain feature classification result, the source domain feature transformation result, the extended domain feature classification result and the extended domain feature transformation result, supervised contrastive learning and contrastive adversarial learning loss functions are used to perform loss calculation to obtain a trained discriminator network model.

[0069] The total loss function calculation formula for the discriminator network model training is:

[0070]

[0071] in, is the supervised contrastive learning loss function, To compare the adversarial learning loss function, is the weight of the supervised contrastive learning loss function, is the weight of the loss function for contrastive learning.

[0072] Step S104: using the trained discriminator network model to perform feature extraction and classification prediction on the target domain scene data to obtain the land cover category of the target domain scene data.

[0073] The method further includes: dynamically adjusting the statistics of the target domain scene data and the statistics of the source domain scene data using a dynamic statistics adjustment method.

[0074] This embodiment provides a hyperspectral image classification method, which trains a domain data extension module based on a diffusion model based on a loss function based on variational inference. Aiming at the problem that the existing cross-scene hyperspectral image classification method cannot effectively learn the target domain scene data information and extract domain invariant features, a domain data extension model based on a diffusion model is proposed, which effectively solves the problems and shortcomings of the existing cross-scene hyperspectral image classification method, such as inaccurate extraction of domain invariant features and insufficient learning of domain differences. In order to further learn and extract domain invariant features and domain difference information between different domain scene data, this method builds a discriminator network model with reference to the contrast adversarial learning method, adopts a dynamic statistics adjustment strategy, improves the accuracy of the estimated value, and optimizes the model performance.

[0075] Based on the above embodiment, this embodiment describes the hyperspectral image classification method as follows:

[0076] 1. Use the loss function based on variational inference to train the diffusion model-based domain data extension module;

[0077] like Figure 2 As shown, this embodiment proposes a domain data expansion module based on a diffusion model, which is mainly composed of a diffusion model submodule and an information reorganization submodule;

[0078] For the hyperspectral image data of the source domain scene, all the labeled pixels are first divided into blocks to obtain several hyperspectral image blocks of the source domain scene. , and the land cover category label corresponding to each image block ,in, R represents the field of real numbers, represents the field of natural numbers, is the number of spectral bands of the hyperspectral image block, represents the block size of the image block and is an odd number;

[0079] Then, the hyperspectral image blocks and its corresponding label They are input into the diffusion model and information reorganization module respectively. The information reorganization module The labels of all pixels except the central pixel are randomly masked or reordered to disrupt the land cover distribution of the source domain data and obtain new labels. ;

[0080] And by the newly obtained label Extract information to guide the diffusion model from the source domain scene hyperspectral image patches Extract features of different categories for learning and generate new labels Distributed extended domain hyperspectral image patches , and obtain the extended domain data which has geographic information differences with the source domain data.

[0081] In this process, in order to standardize the learning of source domain features and label information by the domain data extension module based on the diffusion model, a loss function based on variational inference is used. Constrain the training process of the module.

[0082] 2. Use the trained diffusion model-based domain data extension module to expand the source domain scene data and generate extended domain data containing domain invariant information and inter-domain difference information;

[0083] By inputting the hyperspectral image patches of the source domain scene into the module and its corresponding land cover category label , randomly generate a set of extended domain data that has geographic information differences with the source domain data And its corresponding land cover category label By repeating the above process several times, each source domain scene hyperspectral image block obtained by division is sequentially input into the domain data extension module based on the diffusion model to obtain a set of extended domain data that is different from it, thereby realizing the generation of extended domain data.

[0084] 3. Based on the source domain scene data and the extended domain data obtained in step 2, the discriminator network is trained using supervised contrastive learning and contrastive adversarial learning loss functions;

[0085] like Figure 3 As shown in the figure, referring to the contrastive adversarial learning method, a discriminator network model is built. For the source domain scene data and the extended domain scene data, they are first sent to the feature extraction module in pairs for feature extraction operation.

[0086] The feature extraction module is supervised and trained using sample data from two different domains to enhance the ability of the feature extraction module to distinguish differences between domains and extract domain-invariant features. The feature extraction module processes the paired source scene data and extended domain scene data to obtain source domain data features and extended domain data features, respectively. The extracted features are then sent to the classification prediction module for processing to obtain the classification results and feature transformation results of each domain data feature.

[0087] Based on the classification results and feature transformation results of the source domain data and the classification results and feature transformation results of the extended domain data, supervised contrastive learning and contrastive adversarial learning loss functions are used to calculate the loss and constrain the training process of the discriminator network. The forms of these two loss functions are:

[0088]

[0089]

[0090] in, represents the total number of samples, Indicates the total number of categories of samples. is a symbolic function, when the sample The true category is hour, Takes 1, otherwise takes 0. Representation sample Belongs to category The predicted probability of . Indicates taking the logarithm. The analogy is The total number of samples, Representation sample The feature transformation result is Indicates the category The positive and negative sample sets, is a hyperparameter, Represents an exponential function.

[0091] The total loss function of the discriminator network training is as follows:

[0092]

[0093] in, and Represents the loss function weight.

[0094] 4. Using the feature extraction module, classification prediction module, and dynamic statistics adjustment module in the trained discriminator network, perform feature extraction and classification prediction on the target domain scene data in turn to obtain the land cover category of the target domain scene data;

[0095] For the data of a given target domain scene, the whole hyperspectral image is first divided into blocks in the same way as step 1 to obtain several hyperspectral image blocks of the target domain scene. For any hyperspectral image patch of the target domain scene ,First, the dynamic statistics adjustment module is used to dynamically balance the ,statistics of the target domain scene data distribution and the ,statistics estimated for the source domain scene data in the trained ,model.

[0096] When faced with the problem of inaccurate statistical estimation in the test phase, previous methods usually discard the source domain statistics and directly use the statistics of the test batch data for normalization. However, the statistical estimation of the test batch data more reflects the statistical law of the current batch data, and its estimated value is not necessarily accurate, which will lead to performance degradation. For this reason, the present invention adopts a dynamic statistical adjustment strategy, which is in the form of the following:

[0097]

[0098]

[0099] in is the distance between the source domain statistics and the test data statistics, is the test data batch statistic and the The distance between the statistics of the images, is the source domain data statistic and the test batch data The distance between the statistics of the two images.

[0100] By dynamically adjusting the statistics of the target domain scene data and the source domain scene data with the help of the dynamic statistics adjustment module, the statistical differences of the distribution of different scene data can be effectively extracted, thereby helping the feature extraction module to better extract the domain-invariant features in the target domain scene data. Subsequently, the features of the target domain scene data are sent to the classification prediction module to obtain the land cover category label of the target domain scene data.

[0101] A hyperspectral image classification method provided by an embodiment of the present invention reorganizes and randomly masks its geographic information through an information reorganization module, so that the obtained extended domain data not only has domain-invariant features consistent with the source domain, but also preserves the differences between domains, which can help subsequent feature extraction modules and classification prediction modules to better learn domain-invariant features and differences between domains, and effectively solve the learning problem of inconsistent probability distributions of sample data of different domain scenes in the existing cross-scene hyperspectral image classification methods. By simultaneously using the statistical estimation of the source domain data in the trained model and the statistics of the current batch of target domain data in the test phase to perform dynamic weighting, the distribution of source domain data and target domain data is considered synchronously, so that the model can synchronously consider the statistical differences between the distribution of source domain data and target domain data during the reasoning process. In the face of some cases where the differences between domains are large, the sample probability distributions of the source domain and the target domain can be better weighed to achieve more stable and reliable cross-scene hyperspectral image classification.

[0102] Based on the above embodiment, this embodiment describes the hyperspectral image classification system as follows:

[0103] like Figure 4As shown, the hyperspectral image classification system includes a domain data expansion module based on a diffusion model, a feature extraction module, a classification prediction module and a dynamic statistics adjustment module, wherein:

[0104] The domain data extension module based on the diffusion model is used to expand and augment the hyperspectral image data of the source domain scene, generate extended domain data with domain differences from the source domain scene, and increase the diversity of training data.

[0105] Furthermore, the domain data extension module based on the diffusion model includes a diffusion model submodule and an information reorganization submodule. The diffusion model submodule is used to extract domain-invariant features from the source domain scene hyperspectral data to generate extended domain scene data; the information reorganization submodule is used to randomly reorganize and mask the land cover distribution of the source domain scene, and guide the diffusion model submodule to generate extended domain data consistent with the land cover distribution obtained after random reorganization and masking.

[0106] The feature extraction module uses a deep convolutional neural network to extract domain-invariant features and inter-domain difference information from the input hyperspectral image data, and generates a more robust and fine-grained domain-invariant feature vector.

[0107] The classification prediction module predicts the classification result by using the domain invariant feature vector extracted by the feature extraction module, calculates the land cover category label of the input feature vector and the result after feature transformation, and obtains the classification result of the input hyperspectral image block.

[0108] The dynamic statistics adjustment module, based on the dynamic statistics adjustment strategy, dynamically balances the statistics of the target domain scene data distribution and the statistical estimation of the source domain scene data in the trained model, extracts the statistical differences of data distribution in different domains, and helps the feature extraction module to better extract domain-invariant features in the target domain scene data.

[0109] Please refer to Figure 5 , Figure 5 A structural block diagram of a hyperspectral image classification device provided by an embodiment of the present invention; the specific device may include:

[0110] The data domain expansion module 100 uses a loss function of variational inference to train a domain data expansion model based on a diffusion model to obtain a trained domain data expansion model;

[0111] An extended domain scene data module 200 is used to expand the source domain scene data using the trained domain data expansion model to generate extended domain scene data;

[0112] The discriminator training module 300 trains the discriminator network model based on the extended domain scene data and the source domain scene data using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model;

[0113] The prediction module 400 uses the trained discriminator network model to perform feature extraction and classification prediction on the target domain scene data to obtain the land cover category of the target domain scene data.

[0114] A hyperspectral image classification device of the present embodiment is used to implement the aforementioned hyperspectral image classification method. Therefore, the specific implementation method of a hyperspectral image classification device can be seen in the embodiment part of a hyperspectral image classification method in the previous text. For example, the data domain extension module 100, the extended domain scene data module 200, the discriminator training module 300, and the prediction module 400 are respectively used to implement steps S101, S102, S103, and S104 in the aforementioned hyperspectral image classification method. Therefore, its specific implementation method can refer to the description of the corresponding various parts of the embodiment, which will not be repeated here.

[0115] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0116] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0117] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0118] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0119] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0120] The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0121] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0122] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0123] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0125] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0126] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0127] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0128] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A hyperspectral image classification method, characterized in that: include: The domain data extension model based on the diffusion model is trained using the loss function of variational inference to obtain a trained domain data extension model; Expanding the source domain scene data using the trained domain data expansion model to generate extended domain scene data; Based on the extended domain scene data and the source domain scene data, the discriminator network model is trained using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model; Using the trained discriminator network model to perform feature extraction and classification prediction on the target domain scene data, to obtain the land cover category of the target domain scene data; The loss function of variational inference is used to train the domain data expansion model based on the diffusion model, and the trained domain data expansion model is obtained, which includes: Block-dividing the labeled pixels in the source domain scene data to obtain multiple hyperspectral image blocks of the source domain scenes and land cover category labels corresponding to the multiple hyperspectral image blocks of the source domain scenes; Randomly masking or reordering the labels of each pixel point except the central pixel in the land cover category label to obtain a new land cover category label; Extracting information from the new land cover category label, guiding the domain data extension model to extract different categories of feature learning from the hyperspectral image block of the source domain scene, and obtaining an extended domain hyperspectral image block corresponding to the new land cover category label; Using the loss function of the variational inference to train the domain data extension model, to obtain a trained domain data extension model; The calculation formula of the loss function of the variational inference is: in, is the loss function of variational inference, for The noise of the moment, is an untrained UNet network for rich map generation, is the UNet network parameter, Untrained fully connected network generated for endmembers, is the fully connected network parameter, is a statistical parameter, It means converting it into vector form. is a low-dimensional random vector used to generate the r-th abundance map, is a low-dimensional random vector used to generate the r-th end member; The step of expanding the source domain scene data by using the trained domain data expansion model to generate the extended domain scene data includes: Inputting the hyperspectral image blocks of the source domain scene and the land cover category labels corresponding to the hyperspectral image blocks of the source domain scene into the trained domain data extension model, randomly generating a set of extended domain data having geographic information differences with the source domain data and the land cover category labels corresponding to the extended domain data; A plurality of extended domain data having geographic information differences with the source domain data and land cover category labels corresponding to the extended domain data are obtained, and each source domain scene hyperspectral image block obtained by division is sequentially input into the trained domain data extension module to obtain a set of extended domain data having differences therefrom, and to obtain extended domain scene data.

2. The hyperspectral image classification method according to claim 1, characterized in that: The discriminator network model is trained based on the extended domain data and the source domain scene data using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model, including: Build a discriminator network model; Using the discriminator network model to perform feature extraction on the source domain scene data and the extended domain scene data to obtain source domain feature values ​​and extended domain feature values, and using the discriminator network model to perform classification processing on the source domain feature values ​​and the extended domain feature values ​​to obtain source domain feature classification results, source domain feature transformation results, extended domain feature classification results, and extended domain feature transformation results; Based on the source domain feature classification result, the source domain feature transformation result, the extended domain feature classification result and the extended domain feature transformation result, supervised contrastive learning and contrastive adversarial learning loss functions are used to perform loss calculation to obtain a trained discriminator network model.

3. The hyperspectral image classification method according to claim 2, characterized in that: The total loss function calculation formula for the discriminator network model training is: in, is the supervised contrastive learning loss function, To compare the adversarial learning loss function, is the weight of the supervised contrastive learning loss function, is the weight of the loss function for contrastive learning.

4. The hyperspectral image classification method according to claim 1, characterized in that: The method further includes: dynamically adjusting the statistics of the target domain scene data and the statistics of the source domain scene data using a dynamic statistics adjustment method.

5. A hyperspectral image classification device, characterized in that: include: The data domain extension module uses the loss function of variational inference to train the domain data extension model based on the diffusion model to obtain a trained domain data extension model; An extended domain scene data module, which uses the trained domain data extension model to expand the source domain scene data to generate extended domain scene data; A discriminator training module, based on the extended domain scene data and the source domain scene data, trains the discriminator network model using supervised contrastive learning and contrastive adversarial learning loss functions to obtain a trained discriminator network model; A prediction module, using the trained discriminator network model to perform feature extraction and classification prediction on the target domain scene data to obtain the land cover category of the target domain scene data; The loss function of variational inference is used to train the domain data expansion model based on the diffusion model, and the trained domain data expansion model is obtained, which includes: Block-dividing the labeled pixels in the source domain scene data to obtain multiple hyperspectral image blocks of the source domain scenes and land cover category labels corresponding to the multiple hyperspectral image blocks of the source domain scenes; Randomly masking or reordering the labels of each pixel point except the central pixel in the land cover category label to obtain a new land cover category label; Extracting information from the new land cover category label, guiding the domain data extension model to extract different categories of feature learning from the hyperspectral image block of the source domain scene, and obtaining an extended domain hyperspectral image block corresponding to the new land cover category label; Using the loss function of the variational inference to train the domain data extension model, to obtain a trained domain data extension model; The calculation formula of the loss function of the variational inference is: in, is the loss function of variational inference, for The noise of the moment, is an untrained UNet network for rich map generation, is the UNet network parameter, Untrained fully connected network generated for endmembers, is the fully connected network parameter, is a statistical parameter, It means converting it into vector form. is a low-dimensional random vector used to generate the r-th abundance map, is a low-dimensional random vector used to generate the r-th end member; The step of expanding the source domain scene data by using the trained domain data expansion model to generate the extended domain scene data includes: Inputting the hyperspectral image blocks of the source domain scene and the land cover category labels corresponding to the hyperspectral image blocks of the source domain scene into the trained domain data extension model, randomly generating a set of extended domain data having geographic information differences with the source domain data and the land cover category labels corresponding to the extended domain data; A plurality of extended domain data having geographic information differences with the source domain data and land cover category labels corresponding to the extended domain data are obtained, and each source domain scene hyperspectral image block obtained by division is sequentially input into the trained domain data extension module to obtain a set of extended domain data having differences therefrom, and to obtain extended domain scene data.

6. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

Images