Snapshot hyperspectral image segmentation method and device, medium and product

Abstract

The invention discloses a snapshot hyperspectral image segmentation method and device, a medium and a product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a sample data set; training the initial SSNet model by adopting the sample data set until an output result of the initial SSNet model meets a set requirement, and taking the trained initial SSNet model as the SSNet model; acquiring a measurement image of the target scene; and adopting an SSNet model to obtain a snapshot hyperspectral segmentation image of the target scene based on the measurement image and the perception matrix. According to the invention, the hyperspectral image segmentation process can be simplified, and the hyperspectral image segmentation precision can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a snapshot hyperspectral image segmentation method, device, medium and product. Background Art

[0002] With the development of artificial intelligence, the application of machine reasoning is gradually replacing visual information acquisition in practice. Among them, hyperspectral image segmentation is an important reasoning task in fields such as remote sensing, autonomous driving, and precision agriculture. In the context of land cover, the main goal of hyperspectral image segmentation is to segment the pixels in a hyperspectral image into predefined categories, which helps to understand land cover, vegetation, and various surface features in detail. As a pre-operation of hyperspectral image segmentation, hyperspectral imaging technology provides guarantee for subsequent reasoning tasks. Traditional hyperspectral imaging and segmentation technologies rely on scanning hyperspectral cameras to acquire hyperspectral images, and then use machine learning or deep learning classifiers to predict the segmentation labels of the scene. However, traditional optical systems have disadvantages such as large volume, heavy weight, high cost, and slow acquisition speed, which severely limit the application scenarios of hyperspectral image segmentation. Theoretically, the imaging speed can be improved through the Compressed Sensing (CS) theory. The Coding Aperture Snapshot Spectral Imager (CASSI) realizes strip modulation by combining a fixed mask and a dispersive element, and then uses deep learning technology to quickly capture and reconstruct the HSI cube. This method reduces the volume of the optical system to a certain extent, but still requires complex optical elements for spectral modulation. To solve this problem, the spectral filter array method extends the traditional Bayer pattern. This method can acquire a spectral data cube without pre-optical devices, but it will waste more light throughput. The recently reported metasurfaces, disordered structures, and Fabry-Perot filters have finely tunable spectral filtering capabilities, so they can be used for spectral modulation within a specific spectral range. These components can achieve miniaturized on-chip integration, bringing new technical routes for integrated hyperspectral imaging technology. For example, the material coding method can modulate the incident light of nearly a hundred channels in the visible-near infrared range as a whole, and after reconstructing the hyperspectral image using a deep network, the segmentation task of the solution diffusion process is realized. However, segmenting after hyperspectral image reconstruction will increase the computational burden and may introduce errors during hyperspectral image reconstruction. Summary of the Invention

[0003] The purpose of the present application is to provide a snapshot hyperspectral image segmentation method, device, medium and product, which can simplify the process of hyperspectral image segmentation and improve the accuracy of hyperspectral image segmentation.

[0004] To achieve the above purpose, the present application provides the following solutions:

[0005] In a first aspect, the present application provides a snapshot hyperspectral image segmentation method, including:

[0006] Obtain a sample data set;

[0007] Train an initial SSNet model using the sample data set until the output result of the initial SSNet model meets the set requirements, and use the trained initial SSNet model as the SSNet model; the SSNet model includes: an input layer, a spectral feature extractor, a spatial-spectral feature extractor, and a classifier;

[0008] Obtain a measurement image of the target scene; the acquisition method of the measurement image is: use a snapshot hyperspectral camera to obtain the measurement image of the target scene;

[0009] Use the SSNet model to obtain a snapshot hyperspectral segmentation image of the target scene based on the measurement image and the sensing matrix.

[0010] Optionally, obtain the sensing matrix based on a Mie resonance metasurface; the Mie resonance metasurface includes microstructures; the microstructures are arranged in an array;

[0011] Integrate the Mie resonance metasurface on the camera to obtain a snapshot hyperspectral camera.

[0012] Optionally, the acquisition process of the sample data set includes:

[0013] Obtain an original hyperspectral image;

[0014] Perform compressive coding on the original hyperspectral image using the sensing matrix to obtain an original measurement image;

[0015] Obtain the sample data set based on the original measurement image and the sensing matrix.

[0016] Optionally, use the overall accuracy, average accuracy, and Cohen's Kappa coefficient to determine whether the output result of the initial SSNet model meets the set requirements.

[0017] Optionally, the coding function of the Mie resonance metasurface is expressed as:

[0018]

[0019] where C(δλ,i) represents the coding function, I(λ,i) represents the light intensity at wavelength λ and detector i, < > represents the average value of λ, and δλ represents the offset of wavelength λ.

[0020] Optionally, use the SSNet model to obtain predicted pixels based on the measurement image and the sensing matrix, including:

[0021] Using the input layer, based on the measurement image and the sensing matrix, obtain the image patches of the measurement image and the image patches of the sensing matrix;

[0022] Perform convolution processing on the image patches of the measurement image and the image patches of the sensing matrix, and perform feature fusion to obtain the fused image patches;

[0023] Perform convolution processing on the fused image patches to obtain the convolution-processed image patches;

[0024] Input the convolution-processed image patches into a spectral feature extractor and a spatial-spectral feature extractor respectively to obtain a first flattened image and a second flattened image;

[0025] Fuse the first flattened image and the second flattened image and input them into a classifier to obtain the predicted pixels.

[0026] Optionally, both the spectral feature extractor and the spatial-spectral feature extractor include a processing layer and a flattening layer; inputting the convolution-processed image patches into the spectral feature extractor and the spatial-spectral feature extractor respectively to obtain a first flattened image and a second flattened image includes:

[0027] Using the processing layer of the spectral feature extractor, based on the convolution-processed image, obtain a first processed image;

[0028] Using the flattening layer of the spectral feature extractor, based on the first processed image, obtain a first flattened image;

[0029] Using the processing layer of the spatial-spectral feature extractor, based on the convolution-processed image, obtain a second processed image;

[0030] Using the flattening layer of the spatial-spectral feature extractor, based on the second processed image, obtain a second flattened image.

[0031] Optionally, the processing layer includes a convolutional layer, a normalization layer, and an activation function.

[0032] In a second aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the snapshot hyperspectral image segmentation method described in any one of the above.

[0033] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the snapshot hyperspectral image segmentation method described in any one of the above are implemented.

[0034] Fourthly, the present application provides a computer program product, including a computer program, which when executed by a processor, implements the steps of the snapshot hyperspectral image segmentation method described in any one of the above.

[0035] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0036] The present application provides a snapshot hyperspectral image segmentation method, device, medium and product. The SSNet model is used to directly obtain the snapshot hyperspectral segmentation image of the target scene based on the measurement image and the sensing matrix of the target scene, without reconstructing the original hyperspectral scene image. The scene segmentation is directly implemented on the measurement image through the SSNet model, which can simplify the process of hyperspectral image segmentation, avoid the errors caused by reconstructing the hyperspectral scene image, and further improve the accuracy of snapshot hyperspectral image segmentation. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 It is a flowchart of a hyperspectral image segmentation method in an embodiment of the present application;

[0039] Figure 2 It is a schematic diagram of traditional scanning hyperspectral imaging and segmentation, snapshot hyperspectral imaging and segmentation, and the hyperspectral image segmentation method proposed by the present application provided in an embodiment of the present application;

[0040] Figure 3 It is a schematic diagram of the transmission spectral matrix of 25 metasurfaces provided in an embodiment of the present application;

[0041] Figure 4 It is a comparison chart of the C(δλ,i) values of the Pavia University dataset and the Mie resonance metasurface provided in an embodiment of the present application;

[0042] Figure 5 It is a schematic diagram of the correlation coefficient matrix of 25 metasurfaces provided in an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of four transmission modes of the Mie resonance metasurface provided in an embodiment of the present application;

[0044] Figure 7 It is a schematic diagram of the SSNet model provided in an embodiment of the present application;

[0045] Figure 8 Synthetic RGB image, measurement image, reference category map, and schematic diagram of segmentation qualitative results for segmenting the PaviaU scene using hyperspectral images provided by an embodiment of the present application;

[0046] Figure 9 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0047] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0048] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0049] In an exemplary embodiment, as Figure 1 shown, a hyperspectral image segmentation method is provided. This method is executed by a computer device, and specifically, it can be executed alone by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, taking the segmentation of hyperspectral images by this method as an example, it includes:

[0050] Step 100, obtain a sample data set.

[0051] Step 200, train an initial SSNet model using the sample data set until the output result of the initial SSNet model meets the set requirements, and use the trained initial SSNet model as the SSNet model. Among them, the SSNet model includes: an input layer, a spectral feature extractor, a spatio-spectral feature extractor, and a classifier.

[0052] Step 300, obtain a measurement image of the target scene. The acquisition method of the measurement image is: use a snapshot hyperspectral camera to obtain the measurement image of the target scene.

[0053] Step 400, use the SSNet model to obtain a snapshot hyperspectral segmentation image of the target scene based on the measurement image and the sensing matrix.

[0054] As an optional implementation manner, the acquisition process of obtaining the sample data set in step 100 can be: obtain an original hyperspectral image. Compress and encode the original hyperspectral image using the sensing matrix to obtain an original measurement image. Obtain the sample data set based on the original measurement image and the sensing matrix.

[0055] To obtain the measurement image of the hyperspectral scene, the above compression encoding process is implemented based on the CS theory. Among them, the measurement image y obtained after compression encoding can be expressed as: y = Φx + n. In the formula, y represents the measurement image, Φ represents the sensing matrix (i.e., the pattern matrix of the metasurface), x represents the hyperspectral scene image (i.e., the target scene, equivalent to the hyperspectral image reconstructed in the second row), and n represents the measurement noise.

[0056] As Figure 2 shown, Figure 2 the first row of Figure 2 represents that the traditional hyperspectral imaging and segmentation system performs the segmentation task after obtaining the hyperspectral image; Figure 2 the second row of Figure 2 represents that the snapshot hyperspectral imaging and segmentation system reconstructs the hyperspectral image by the deep network after obtaining the measurement image of the hyperspectral scene, and finally performs the segmentation task;

[0057] The measurement image of the scene can be obtained by compressing and encoding the hyperspectral scene image.

[0058] It should be noted that the above compression sensing model is to explain that the measurement image y can be obtained according to the sensing matrix Φ of the metasurface and the hyperspectral scene image x.

[0059] As an optional implementation manner, the sample data set in step 100 can adopt two commonly used data sets in the remote sensing scene, including the Pavia University dataset (Pavia University, PaviaU) and the Pavia Centre dataset (Pavia Centre, PaviaC).

[0060] In practical applications, the designed Mie resonance metasurface can be integrated on the chip surface of a CMOS camera to obtain a snapshot hyperspectral camera, and the measured image y can be obtained by using the snapshot hyperspectral camera. After setting the parameters of the metasurface (period, width, and height, and the period, width, and height of the metasurface are all within the sub-wavelength range), simulation can be carried out through FDTD software (fully known as FDTD Solutions), and the Mie resonance metasurface structure and sensing matrix Φ designed by simulation can be obtained. Among them, the FDTD algorithm is fully known as the finite-difference time-domain method, Finite-Difference Time-Domain.

[0061] As Figure 3 shows the transmission spectral matrix of 25 metasurfaces. Due to the Mie resonance of the TiO2 structure, many disordered valleys appear in these spectra. To quantitatively characterize the encoding performance of the Mie resonance metasurface for the target spectrum, the encoding function C(δλ,i) is set as follows: where I(λ,i) represents the light intensity at wavelength λ and detector i, < > represents the average value of λ, and δλ represents the offset of wavelength λ. As Figure 4 shown, when δλ = 0, the C(δλ,i) of the Pavia University dataset and the value of the Mie resonance metasurface normalized to 1 show similar full widths at half maximum, confirming the encoding performance of the Mie resonance metasurface. The correlation coefficient matrix, as Figure 5 shown, can be used to evaluate the correlation degree of different filters. The average value of all correlation coefficients is 0.35, indicating that there are significant differences between the different spectral responses involved. The four transmission modes (5×5) of the Mie resonance metasurface are as Figure 6 shown, and these random and different modulation modes can ensure the efficiency of spectral compression encoding.

[0062] As an alternative implementation, to improve the accuracy of hyperspectral image segmentation, as Figure 7 shown, step 400 includes:

[0063] 401. The input layer is based on the measurement image and the sensing matrix to obtain the image patches of the measurement image and the image patches of the sensing matrix. Perform convolution processing on the image patches of the measurement image and the image patches of the sensing matrix, and perform feature fusion to obtain the fused image patches. Perform convolution processing on the fused image patches to obtain the image patches after convolution processing. For example, in the input layer, the measurement image of the target scene and the sensing matrix of the Mie resonance metasurface are input simultaneously. After random training sampling, the original measurement image is cropped into multiple image patches of a set size (i.e., the size parameter of the input image of the SSNet model, such as 5×5 or 10×10), respectively obtaining the image patches of the measurement image and the image patches of the sensing matrix. Among them, the sensing matrix can extract the features of the sensing matrix as the input of the SSNet model, thereby increasing the feature extraction ability and robustness of the model.

[0064] It should be noted that the measurement image of the hyperspectral scene is obtained by using a snapshot hyperspectral camera, that is, the measurement image y. By inputting the designed metasurface parameters into the FDTD software above, the physical structure of the metasurface can be obtained. Then, based on this physical structure, the FDTD software simulates the transmission spectrum of the metasurface. After screening these transmission spectra through an optimization algorithm, 25 transmission spectra are finally obtained. After arranging them in order, a 5×5 sensing matrix is obtained. The 5×5 pattern is then repeated according to the actual image size to obtain the final sensing matrix (this is at the simulation level). In practical applications, both the designed Mie resonance metasurface and the camera chip have a fixed spatial resolution (i.e., the picture size). Therefore, the size of the measurement image obtained by taking pictures during practical applications is fixed, and the image needs to be cropped according to actual needs.

[0065] Perform 2D convolution processing (Conv 2D) on the image patches of the measurement image and the image patches of the sensing matrix, and perform feature fusion (Concatenation) to obtain the fused image patches. Perform 2D convolution processing on the fused image patches again to obtain the image patches after convolution processing.

[0066] 402. Input the image blocks after convolution processing into the spectral feature extractor and the spatio-spectral feature extractor respectively to obtain the first flattened image and the second flattened image. For example, input the image blocks after convolution processing into the spectral feature extractor and the spatio-spectral feature extractor respectively. Both of these feature extractors include a processing layer and a flattening layer. Specifically, both the spectral feature extractor and the spatio-spectral feature extractor include three processing layers and one flattening layer. The processing layer includes a convolutional layer, a normalization layer, and an activation function. In the processing layer, 3D convolution operation (Conv 3D), normalization processing (BN), and activation processing (the activation function is ReLU) are performed in sequence. The processing layer can also be represented as a Conv 3D - BN - ReLU layer. In the flattening layer, a flattening operation (Flatten) is performed. Compared with traditional 2D convolution, 3D convolution can extract features spanning spatial and spectral dimensions, and these features are very effective for distinguishing targets such as different classes of ground objects. To extract features at different levels, the kernel sizes of 3D convolution in the spectral feature extractor are (3, 3, 7) - (3, 3, 5) - (3, 3, 3), and the kernel sizes of 3D convolution in the spatio-spectral feature extractor are (7, 7, 3) - (5, 3, 3) - (3, 3, 3).

[0067] Use the processing layer of the spectral feature extractor to obtain the first processed image based on the image after convolution processing. Use the flattening layer of the spectral feature extractor to obtain the first flattened image based on the first processed image. Use the processing layer of the spatio-spectral feature extractor to obtain the second processed image based on the image after convolution processing. Use the flattening layer of the spatio-spectral feature extractor to obtain the second flattened image based on the second processed image.

[0068] 403. Input the first flattened image and the second flattened image after fusion into the classifier to obtain the snapshot hyperspectral segmentation image of the target scene. For example, fuse the first flattened image and the second flattened image (Fusion) to obtain the fused features. Input the fused features into the classifier to achieve pixel-level prediction, where the Dropout rate is set to 0.4. The classifier includes three processing layers, and in the three processing layers, Linear - ReLU - Dropout operations are performed in sequence. The output of the classifier is the classification label predicted for each pixel (i.e., Figure 7 the predicted pixels in Figure 7 The snapshot hyperspectral segmentation image of the target scene in

[0069] In addition, the cross-entropy loss function is adopted as the loss function of the SSNet model, which is a deep learning segmentation network model. The process of deep learning is to input an image, extract features through a convolutional network to obtain an output, then calculate the loss function for gradient backpropagation, update the network weights, and perform a new iteration, repeating this process. The loss function is LOSS CE It is calculated based on the predicted classification labels (i.e., predicted pixels) output by the SSNet model and the ground truth labels manually annotated, expressed as: Among them, and represent the reference label (i.e., the ground truth label) and the predicted classification label respectively, and M and L are the total number of mini-batch samples (i.e., image patches of the measured image) and the number of land cover classes respectively.

[0070] In an exemplary embodiment, for the training and validation of the above snapshot hyperspectral image segmentation method, snapshot hyperspectral image segmentation simulation is carried out for the remote sensing scenario. Two commonly used datasets in the remote sensing scenario are selected as sample datasets, namely the Pavia University dataset (PaviaU) and the Pavia Centre dataset (PaviaC). Both of these two datasets are continuously imaged using a reflective optical spectral imaging system (ROSIS-03), with a wavelength range of 0.43 - 0.86 μm and a total of 115 bands. Due to noise effects, some bands are deleted, so the data sizes of PaviaU and PaviaC are 610×340×103 and 1096×715×102 respectively. These two datasets also contain 9 different land cover classes. In addition to the segmentation accuracy of each single class, three commonly used metrics are also adopted to evaluate the final segmentation results, that is, it is determined whether the output results of the SSNet model meet the set requirements according to these three metrics, including the overall accuracy (OA), the average accuracy (AA), and Cohen's Kappa (k). They represent the overall performance of the segmentation, the average performance of each class, and the consistency of the segmentation results respectively.

[0071] In this embodiment, the SSNet model is implemented using the PyTorch framework and trained for 500 epochs on 1×RTX4090 GPU using the Adam optimizer. By default, the learning rate, training batch size, and image patch size are set to 0.01, 64, and 20 respectively.

[0072] The initial SSNet model was trained with 10% and 20% sampling in the PaviaU and PaviaC scenarios until OA, AA, and k in the 20% training sampling all exceeded 99%, obtaining the trained initial SSNet model, which was used as the SSNet model. The SSNet model was verified with 90% and 80% sampling in the PaviaU and PaviaC scenarios. And the unlabeled black background areas were tested respectively. Figure 8 The upper row in [figure] respectively shows the synthetic RGB image, measurement image, reference category map, and segmentation qualitative results of the PaviaU scenario. Figure 8 The lower row in [figure] labels the 9 land cover classes and colors of the reference category map and the segmentation result map. The quantitative classification results for the two scenarios are shown in Table 1. The segmentation OA, AA, and k for the two scenarios exceeded 99% (20% training sampling) respectively. It shows that the hyperspectral image segmentation method proposed in this application can achieve accurate land cover segmentation prediction.

[0073] Table 1 Quantitative classification result table

[0074]

[0075] In summary, this application proposes a snapshot hyperspectral image segmentation method. This method is based on Mie resonance metasurface and uses the derived SSNet model as a classifier to achieve accurate segmentation prediction in remote sensing scenarios. The designed Mie resonance metasurface can provide approximately 1096×715 spatial pixels and 103 spectral channels in the range of 430 - 860 nm. In addition, the derived SSNet model can achieve a prediction accuracy of over 99%.

[0076] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the snapshot hyperspectral image segmentation method. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a snapshot hyperspectral image segmentation method.

[0077] Those skilled in the art can understand that Figure 9 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0078] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0079] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0081] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0082] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0083] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0084] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for those of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A snapshot hyperspectral image segmentation method, characterized in that The hyperspectral image segmentation method includes: Obtaining a sample data set; Training an initial SSNet model using the sample data set until the output result of the initial SSNet model meets the set requirements, and taking the trained initial SSNet model as the SSNet model; the SSNet model includes: an input layer, a spectral feature extractor, a spatial-spectral feature extractor, and a classifier; Obtaining a measurement image of the target scene; the acquisition method of the measurement image is: using a snapshot hyperspectral camera to obtain the measurement image of the target scene; Using the SSNet model to obtain a snapshot hyperspectral segmentation image of the target scene based on the measurement image and the sensing matrix.

2. The snapshot hyperspectral image segmentation method according to claim 1, wherein Obtaining the sensing matrix based on the Mie resonance metasurface; the Mie resonance metasurface includes microstructures; the microstructures are arranged in an array; Integrating the Mie resonance metasurface on the camera to obtain a snapshot hyperspectral camera.

3. The snapshot hyperspectral image segmentation method according to claim 2, wherein The acquisition process of the sample data set includes: Obtaining an original hyperspectral image; Performing compressive coding on the original hyperspectral image using the sensing matrix to obtain an original measurement image; Obtaining the sample data set based on the original measurement image and the sensing matrix.

4. The snapshot hyperspectral image segmentation method according to claim 1, wherein Using the overall accuracy, average accuracy, and Cohen's Kappa coefficient to determine whether the output result of the initial SSNet model meets the set requirements.

5. The snapshot hyperspectral image segmentation method according to claim 2, characterized in that The coding function of the Mie resonance metasurface is expressed as: where C(δλ,i) represents the coding function, I(λ,i) represents the light intensity at wavelength λ and detector i, < > represents the average value of λ, and δλ represents the offset of wavelength λ.

6. The snapshot hyperspectral image segmentation method according to claim 1, wherein Using the SSNet model to obtain a snapshot hyperspectral segmentation image of the target scene based on the measurement image and the sensing matrix, including: Using the input layer to obtain an image block of the measurement image and an image block of the sensing matrix based on the measurement image and the sensing matrix; Performing convolution processing on the image block of the measurement image and the image block of the sensing matrix, and performing feature fusion to obtain a fused image block; Performing convolution processing on the fused image block to obtain a convolved image block; Inputting the convolved image block into the spectral feature extractor and the spatial-spectral feature extractor respectively to obtain a first flattened image and a second flattened image; Fusing the first flattened image and the second flattened image and inputting them into the classifier to obtain the snapshot hyperspectral segmentation image of the target scene.

7. The snapshot hyperspectral image segmentation method according to claim 6, wherein Both the spectral feature extractor and the spatial-spectral feature extractor include a processing layer and a flattening layer; inputting the convolved image block into the spectral feature extractor and the spatial-spectral feature extractor respectively to obtain a first flattened image and a second flattened image, including: Using the processing layer of the spectral feature extractor to obtain a first processed image based on the convolved image; Using the flattening layer of the spectral feature extractor to obtain a first flattened image based on the first processed image; Using the processing layer of the spatial-spectral feature extractor to obtain a second processed image based on the convolved image; The flattening layer of the spatial-spectral feature extractor obtains a second flattened image based on the second processed image.

8. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the snapshot hyperspectral image segmentation method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the snapshot hyperspectral image segmentation method according to any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the snapshot hyperspectral image segmentation method according to any one of claims 1-7.

Images