Micro-hyperspectral image classification model construction method, classification method and device
By combining microscopic hyperspectral imaging technology with deep learning to create image stitching and denoising algorithms, the difficulties of traditional hyperspectral imaging systems in identifying small targets have been solved, enabling efficient and accurate identification of microalgae and microplastics, and improving detection limits and identification efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2022-07-07
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies struggle to accurately, efficiently, and quickly identify microalgae and microplastics at the tens of micrometer scale, and traditional hyperspectral imaging systems face difficulties in identifying small-sized targets.
Microscopic hyperspectral imaging technology combined with deep learning image stitching and denoising algorithms is used to stitch and denoise microscopic hyperspectral images through an image homography transformation model, and to perform hyperspectral data dimensionality reduction to establish a classification model to identify microalgae and microplastics.
It improves the efficiency and accuracy of identifying microalgae and microplastics, raises the detection limit, solves the difficulties of traditional hyperspectral imagers in identifying small targets, and reduces the pressure of data storage and transmission.
Smart Images
Figure CN115376126B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of marine environmental monitoring and relates to a method for constructing a microscopic hyperspectral image classification model, a classification method, and a device. Background Technology
[0002] Microalgae are primary producers in marine ecosystems, performing photosynthesis to provide energy for numerous aquatic organisms and helping to reduce atmospheric carbon dioxide levels. However, pollution and toxicity from microplastics in the ocean can affect the stability of marine ecosystems. Furthermore, because microalgae are much smaller than microplastics, they easily adsorb and aggregate, making them susceptible to ingestion by aquatic organisms and posing a health risk through bioaccumulation in the food chain. Therefore, accurate, efficient, and rapid methods are urgently needed for the classification of microalgae and microplastics.
[0003] Currently, hyperspectral imaging systems and microscopic hyperspectral image processing methods have been reported, but most of them are for larger targets, such as aquatic organisms like oysters, mussels, and fish. Research on the identification of small-sized (tens of micrometers) microplastics and microalgae is still lacking. Summary of the Invention
[0004] Therefore, embodiments of this application provide a method for constructing a microscopic hyperspectral image classification model, a classification method, and an apparatus.
[0005] According to a first aspect of the embodiments of this application, a method for constructing a microscopic hyperspectral image classification model is provided, comprising:
[0006] Obtain microscopic hyperspectral images of samples containing a mixture of microalgae and microplastics;
[0007] Based on the image homography transformation model, the microscopic hyperspectral images in each spectral dimension are stitched together to obtain the hyperspectral images corresponding to each spectral dimension.
[0008] The stitching edges of the hyperspectral images are denoised.
[0009] The hyperspectral image after denoising is subjected to hyperspectral data dimensionality reduction processing;
[0010] A classification model is established based on the denoised hyperspectral image and the dimension-reduced hyperspectral image to identify the microscopic hyperspectral image.
[0011] Optionally, based on the image homography transformation model, the microscopic hyperspectral images in each spectral dimension are stitched together to obtain microscopic hyperspectral images corresponding to each spectral dimension, including:
[0012] Save the microscopic hyperspectral images and establish a raw spatial spectrum database, and label the mask tags of microplastics and microalgae;
[0013] Based on the location information of the microalgae and microplastic mixture sample, an affine transformation is performed on the image in the original spatial spectrum database;
[0014] Using the images in the original spatial spectrum database and the images after affine transformation as input, and the relationship of the affine transformation as output, an image homography transformation model based on a deep convolutional neural network is established.
[0015] Based on the mask label, an attention mechanism network branch is established for the image homography transformation model to reduce the morphological changes of microalgae and microplastic targets when the image undergoes homography transformation, thereby optimizing the image homography transformation model;
[0016] Based on the optimized image homography transformation model, microscopic hyperspectral images are stitched together according to each spectral dimension.
[0017] Optionally, the stitching edges of the hyperspectral image are denoised, including:
[0018] A1: Calculate the image signal-to-noise ratio for each spectral dimension of the hyperspectral images of microalgae and microplastics, and set two image signal-to-noise ratio thresholds S1 and S2 for the low-wavelength and high-wavelength parts, respectively, with corresponding wavelengths λ1 and λ2.
[0019] A2: Establish high- and low signal-to-noise ratio image pairs in the spectral dimension, where the wavelength range of the high signal-to-noise ratio image is (λ1, λ2) and the wavelength range of the low signal-to-noise ratio image is [λ0, λ1]∪[λ2, λ3].
[0020] A3: Randomly select high- and low signal-to-noise ratio image pairs to form a correspondence between high and low signal-to-noise ratio images;
[0021] A4: Using low signal-to-noise ratio (SNR) images as input and high SNR images as output, a spectral image denoising model based on a deep convolutional neural network is established according to the correspondence.
[0022] A5: Use the spectral image denoising model to test new hyperspectral images, calculate the average signal-to-noise ratio of all dimensions of the image, and if it is greater than 2*(S1+S2), the denoising model is considered effective; otherwise, repeat A1-A5 until an effective spectral image denoising model is obtained.
[0023] A6: Input the hyperspectral image into an effective spectral image denoising model for denoising processing.
[0024] Optionally, the hyperspectral image after denoising is subjected to hyperspectral data dimensionality reduction processing, including:
[0025] B1: The denoised hyperspectral image is clustered using a clustering algorithm to form n clusters, each cluster containing several wavelengths, where n is the target dimensionality reduction number;
[0026] B2: Select the cluster center fi for each cluster, where i = 1, 2, ..., n, to obtain the dimensionality reduction result [f1, f2, f3, ..., fn];
[0027] B3: Repeat operations B1 and B2 for n = 2, 3, ..., N-1 respectively to obtain N-2 sets of dimensionality reduction results, where N is the number of all wavelengths in the microscopic hyperspectral image;
[0028] B4: Based on the dimensionality reduction results, N-2 sets of dimensionality-reduced hyperspectral images corresponding to the wavelengths are obtained.
[0029] Optionally, a classification model is established based on the denoised hyperspectral image and the dimension-reduced hyperspectral image, including:
[0030] Based on the mask label, the denoised hyperspectral image is divided into a training set and a test set consisting of microplastics, microalgae, and background. The number of wavelengths contained in the denoised hyperspectral image is N.
[0031] Using the training set, a classification model CA is trained and built;
[0032] The test set was classified using the classification model CA to obtain the classification index F1. CA ;
[0033] Based on the mask labels, the dimensionality-reduced hyperspectral image is divided into a training set and a test set for microplastics, microalgae, and background.
[0034] For each set of dimensionality-reduced hyperspectral images, a classification model CB is trained and built on the training and test sets respectively. j , j is the number of groups, where j = 1, 2, ..., N-2;
[0035] The established classification model CB j The test set images of the dimensionality-reduced hyperspectral images were classified to obtain the classification index F1. CBj ;
[0036] Take the formula F1 CBj =w×F1 CA And the smallest j is j*, corresponding to the classification model CB j* As the final classification model, where w∈[1,+∞).
[0037] According to a second aspect of the embodiments of this application, an apparatus for constructing a microscopic hyperspectral image classification model is provided, comprising:
[0038] The first acquisition module is used to acquire microscopic hyperspectral images of the microalgae and microplastic mixture sample;
[0039] The stitching module is used to stitch together the microscopic hyperspectral images in each spectral dimension according to the image homography transformation model to obtain the hyperspectral images corresponding to each spectral dimension.
[0040] A noise reduction module is used to perform noise reduction processing on the stitching edges of the hyperspectral image;
[0041] The dimension reduction processing module is used to perform hyperspectral data dimension reduction processing on the denoised hyperspectral image;
[0042] The modeling module is used to build a classification model based on the denoised hyperspectral image and the dimension-reduced hyperspectral image to identify the microscopic hyperspectral image.
[0043] According to a third aspect of the embodiments of this application, a method for classifying microscopic hyperspectral images is provided, comprising:
[0044] Acquire the microscopic hyperspectral image of the object to be identified;
[0045] The microscopic hyperspectral image to be identified is input into the classification model described in the first aspect to obtain the identification result.
[0046] According to a fourth aspect of the embodiments of this application, a microscopic hyperspectral image classification device is provided, comprising:
[0047] The second acquisition module is used to acquire the microscopic hyperspectral image to be identified;
[0048] The recognition module is used to input the microscopic hyperspectral image to be recognized into the classification model described in the first aspect to obtain the recognition result.
[0049] According to a fifth aspect of the embodiments of this application, a microscopic hyperspectral image classification device is provided, comprising:
[0050] Light source, used to output a visible-near-infrared compensated light source with a stable spectral structure;
[0051] A microscopic hyperspectral imager is used for hyperspectral imaging of samples of microalgae and microplastic mixtures under a microscopic field of view.
[0052] The sample moving stage is movable during observation and is used to hold the sample, align different regions of the target object in the sample with the imaging range of the microscopic hyperspectral imager, and output position information.
[0053] A processor for executing the method described in the first aspect.
[0054] According to a sixth aspect of the embodiments of this application, an electronic device is provided, comprising:
[0055] One or more processors;
[0056] Memory, used to store one or more programs;
[0057] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first or third aspect.
[0058] The technical solutions provided by the embodiments of this application may include the following beneficial effects:
[0059] As can be seen from the above embodiments, this application, based on microscopic hyperspectral imaging technology, identifies microplastics and microalgae at the tens of micrometer scale, and improves the detection limit by combining microscopic image stitching technology. Addressing the issue of reduced signal-to-noise ratio at the edges of imaging wavelengths in traditional hyperspectral imagers, this method allows for more reliable data processing of images at these edge wavelengths. Furthermore, considering the high spectral dimensionality of traditional hyperspectral imaging methods and the difficulties in data storage and transmission, a spectral dimensionality reduction method for microalgae and microplastics has been established to improve recognition efficiency.
[0060] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0062] Figure 1 This is a flowchart illustrating a method for constructing a microscopic hyperspectral image classification model according to an exemplary embodiment.
[0063] Figure 2 This is a schematic diagram of a microscopic hyperspectral image mask label according to an exemplary embodiment;
[0064] Figure 3 This is a network architecture diagram of a homography transformation model according to an exemplary embodiment.
[0065] Figure 4 This is a network architecture diagram of a spectral image denoising model according to an exemplary embodiment.
[0066] Figure 5 This is a block diagram of a device for constructing a microscopic hyperspectral image classification model according to an exemplary embodiment.
[0067] Figure 6This is a flowchart illustrating a microscopic hyperspectral image classification method according to an exemplary embodiment.
[0068] Figure 7 This is a block diagram of a microscopic hyperspectral image classification device according to an exemplary embodiment. Detailed Implementation
[0069] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.
[0070] Example 1:
[0071] Figure 1 This is a flowchart illustrating a method for constructing a microscopic hyperspectral image classification model according to an exemplary embodiment, such as... Figure 1 As shown, the following steps may be included:
[0072] S11: Obtain microscopic hyperspectral images of the microalgae and microplastic mixture sample;
[0073] S12: Based on the image homography transformation model, the microscopic hyperspectral images in each spectral dimension are stitched together to obtain the hyperspectral images corresponding to each spectral dimension.
[0074] S13: Denoise the stitching edges of the hyperspectral image;
[0075] S14: Perform hyperspectral data dimensionality reduction processing on the denoised hyperspectral image;
[0076] S15: Based on the denoised hyperspectral image and the dimension-reduced hyperspectral image, establish a classification model to identify the microscopic hyperspectral image.
[0077] As can be seen from the above embodiments, this application improves the detection limit of hyperspectral imaging technology by combining microscopic hyperspectral imaging technology with deep learning-based image stitching technology, improves the signal-to-noise ratio of microscopic hyperspectral images by using denoising algorithms, and improves the hyperspectral image recognition efficiency by using hyperspectral data dimensionality reduction processing.
[0078] In the specific implementation of S11: obtain microscopic hyperspectral images of microalgae and microplastic mixture samples;
[0079] Specifically, a trace amount of seawater, microalgae, and microplastics are prepared on a glass slide. The target object is stained with a staining agent. A visible-near-infrared compensated light source with a stable spectral structure is used. The glass slide is mounted on a sample moving stage, which can be moved during observation. Different regions of the target object in the sample are aligned with the imaging range of the microscopic hyperspectral imager, and the position information is output. Then, the microscopic hyperspectral imager is used to perform hyperspectral imaging of the microalgae and microplastic mixture sample under a microscopic field of view, and microscopic hyperspectral images are acquired until the number of acquired images reaches a certain threshold.
[0080] In this embodiment, the hyperspectral imager used contains 103 wavelengths ranging from 400 to 1000 nm.
[0081] In the specific implementation of S12: based on the image homography transformation model, the microscopic hyperspectral images in each spectral dimension are stitched together to obtain the hyperspectral images corresponding to each spectral dimension; this step may include the following sub-steps:
[0082] S121: Save the microscopic hyperspectral image and establish the original spatial spectrum database, and label the mask tags of microplastics and microalgae; Figure 2 This is a schematic diagram of a microscopic hyperspectral image mask label according to an exemplary embodiment, wherein... Figure 2 (a) in the image is the original hyperspectral image. Figure 2 (b) in the image represents a microplastic mask. Figure 2 (c) in the figure represents a microalgae mask.
[0083] Specifically, all the microscopic hyperspectral image data acquired in S11 are saved in the computer, and three wavelengths are randomly selected from the 400 to 600 nm wavelengths of the hyperspectral image data of the same imaging area to synthesize a color image for confirming the location of the target object.
[0084] Establish a matrix of the same size as the hyperspectral microscopy image, and based on the color image, confirm and fill the corresponding values in the matrix as the corresponding hyperspectral microscopy image mask labels;
[0085] In this embodiment, value 0 represents a background mask label, value 1 represents a microplastic mask label, and value -1 represents a microalgae mask label.
[0086] S122: Perform an affine transformation on the image in the original spatial spectrum database based on the location information of the microalgae and microplastic mixture sample;
[0087] Specifically, the microscopic hyperspectral image in S121 and the position information of the sample output by the sample moving stage are obtained, a matrix of the same size as the stitched image is established, and the microscopic hyperspectral image is filled into the matrix through affine transformation according to the position information of the sample to obtain the affine transformed image.
[0088] S123: Using the images in the original spatial spectrum database and the images after affine transformation as inputs, and the relationship of the affine transformation as output, establish an image homography transformation model based on a deep convolutional neural network;
[0089] Specifically, such as Figure 3 This is the network architecture diagram for homography transformation, where convolution and pooling are common operations in image processing. During training, batch normalization (BN) is used to accelerate network training and improve its generalization ability; dropout is used to prevent overfitting. The cross-entropy loss function is proposed to calculate the dynamic region prediction loss during training; the L2 loss function with different scale summations is used to calculate the homography estimation loss. The exponential linear unit (ELU) is used as the activation function, and its expression is shown below, where x is the output after convolution and other operations.
[0090]
[0091] S124: Based on the mask label, establish an attention mechanism network branch for the image homography transformation model to reduce the morphological changes of microalgae and microplastic targets when the image undergoes homography transformation, thereby optimizing the image homography transformation model;
[0092] The microscopic hyperspectral image and mask label at the same imaging location were used as the input and output of the attention mechanism network branch, respectively, to train the network.
[0093] S125: Based on the optimized image homography transformation model, stitch together the microscopic hyperspectral image according to each spectral dimension.
[0094] Specifically, for each specific spectral dimension, the optimized image homography transformation model is used for transformation.
[0095] In a specific implementation of S13: denoising processing is performed on the stitching edges of the microscopic hyperspectral image; this step may include the following sub-steps:
[0096] S131: Calculate the image signal-to-noise ratio for each spectral dimension of the hyperspectral images of microalgae and microplastics, and set two image signal-to-noise ratio thresholds S1 and S2 for the low-wavelength and high-wavelength parts, respectively, with corresponding wavelengths λ1 and λ2.
[0097] Specifically, the microscopic hyperspectral imaging technique used in this embodiment includes N wavelengths uniformly distributed from λ0 to λ3, therefore, the following settings are made: For low wavelength, For high wavelengths; calculate the signal-to-noise ratio (SNR) of the corresponding microscopic hyperspectral image for each wavelength. Based on experience, the SNR thresholds can be set to S1 = 20 dB and S2 = 60 dB to obtain the corresponding wavelengths λ1 and λ2.
[0098] S132: Establish high- and low signal-to-noise ratio image pairs in the spectral dimension, where the wavelength range of the high signal-to-noise ratio image is (λ1, λ2) and the wavelength range of the low signal-to-noise ratio image is [λ0, λ1]∪[λ2, λ3].
[0099] Specifically, from the microscopic hyperspectral images, high signal-to-noise ratio images are selected in the wavelength range (λ1, λ2), and low signal-to-noise ratio images are selected in the wavelength range [λ0, λ1]∪[λ2, λ3], to form high-low signal-to-noise ratio image pairs.
[0100] S133: Randomly select high- and low signal-to-noise ratio single-channel image pairs to form a correspondence between high and low signal-to-noise ratio images;
[0101] Specifically, in the high-low signal-to-noise ratio image pair described in S132, several high signal-to-noise ratio images and several low signal-to-noise ratio images are randomly selected to form a correspondence between high signal-to-noise ratio and low signal-to-noise ratio images.
[0102] S134: Based on the correspondence between high signal-to-noise ratio (SNR) and low SNR images, a spectral image denoising model based on a deep convolutional neural network is established, using a single-channel low SNR image as input and a single-channel high SNR image as output.
[0103] Specifically, such as Figure 4 This is the network architecture diagram of a spectral image denoising model, where convolutional layers and pooling layers are commonly used operations in the field of image processing. Using the single-channel image of the low signal-to-noise ratio (SNR) image from the high SNR / low SNR image correspondence established in S133 as input and the single-channel image of the high SNR image as output, a spectral image denoising model based on a deep convolutional neural network is established.
[0104] S135: Test the new hyperspectral image using the spectral image denoising model, calculate the average signal-to-noise ratio of the image in all dimensions. If it is greater than 2*(S1+S2), the denoising model is considered effective; otherwise, repeat S131-S135.
[0105] Specifically, the distribution of the S11 microalgae and microplastic mixture sample is changed, and a new microscopic hyperspectral image is obtained using microscopic hyperspectral imaging technology. The spectral image denoising model is used to calculate the average signal-to-noise ratio of the image in all spectral dimensions of the new microscopic hyperspectral image. If it is greater than 2*(S1+S2), the denoising model is considered effective; otherwise, S131-S135 are repeated to achieve the purpose of denoising the stitching edges of the microscopic hyperspectral image and improve the image signal-to-noise ratio.
[0106] In the specific implementation of S14: performing hyperspectral data dimensionality reduction processing on the denoised hyperspectral image may include the following sub-steps:
[0107] S141: The denoised hyperspectral image is clustered using a clustering algorithm to form n clusters, each cluster containing several wavelengths, where n is the target dimensionality reduction number;
[0108] Specifically, n wavelengths are randomly selected from the N wavelengths contained in the denoised hyperspectral image as initial cluster centers. The single-channel image of the denoised hyperspectral image corresponding to each wavelength is used as the element corresponding to that wavelength. The distance between each element and the element corresponding to each cluster center is calculated, and each element is assigned to the nearest cluster center. Each cluster center and the wavelength corresponding to the element assigned to it represent a cluster. After all elements have been assigned, each cluster recalculates new cluster centers based on the elements within the cluster, and new clusters are formed based on these new cluster centers. This process is repeated until the cluster centers no longer change, forming the final n clusters.
[0109] S142: Select the cluster center fi of each cluster, where i = 1, 2, ..., n, to obtain the dimensionality reduction result [f1, f2, f3, ..., fn];
[0110] Specifically, from the n clusters formed by S141, the cluster center of each cluster is selected as fi, where i = 1, 2, ..., n, and the resulting n wavelength combinations [f1, f2, ..., fn] are used as the dimensionality reduction result.
[0111] S143: Repeat S141 and S142 for n = 2, 3, ..., N-1 respectively to obtain N-2 sets of dimensionality reduction results, where N is the number of all wavelengths in the microscopic hyperspectral image;
[0112] Specifically, S141 and S142 are repeated for n = 2, 3, ..., N-1 respectively, to obtain N-2 different wavelength combinations as N-2 sets of dimensionality reduction results, where N is the total number of wavelengths in the microscopic hyperspectral image.
[0113] S144: Based on the dimensionality reduction results, obtain N-2 sets of dimensionality-reduced hyperspectral images corresponding to the wavelengths.
[0114] Specifically, based on the N-2 sets of dimensionality reduction results obtained in S143, select the single-channel image of the dimensionality-reduced hyperspectral image corresponding to each wavelength in each set of dimensionality reduction results to obtain N-2 sets of dimensionality-reduced hyperspectral images.
[0115] The clustering algorithm generates N-2 sets of dimensionality-reduced hyperspectral images, maximizing the correlation within clusters and minimizing the correlation between clusters, thus creating conditions for the establishment of the recognition model.
[0116] In the specific implementation of S15: a classification model is established based on the denoised hyperspectral image and the dimension-reduced hyperspectral image to identify the microscopic hyperspectral image. This may include the following sub-steps:
[0117] S151: Based on the mask label, the denoised hyperspectral image is divided into a training set and a test set for microplastics, microalgae, and background, and the denoised hyperspectral image contains N wavelengths.
[0118] Specifically, based on the mask labels that include three categories: microalgae, microplastics, and background, the training set is composed of 70% of the pixels of the denoised hyperspectral image containing microplastics, microalgae, and the two categories of mask labels, and 70% of the pixels of the background mask. The remaining 30% of the pixels are used as the test set. The denoised hyperspectral image contains N wavelengths.
[0119] S152: Using the training set, train and build a classification model CA;
[0120] Specifically, the classification model CA is trained and established using the training set containing the three categories of microplastics, microalgae, and background.
[0121] S153: Classify the test set using the classification model CA to obtain the classification index F1. CA ;
[0122] The test set was classified using the classification model CA to obtain the classification index F1. CA The classification index F1 CA The calculation formula is as follows:
[0123]
[0124] Here, F1 is a single-class classification metric; TP is the number of predicted positive values that were actually positive, FP is the number of predicted positive values that were actually negative, and FN is the number of predicted negative values that were actually positive. The F1 score is obtained by averaging all F1 scores after calculating the F1 score for each class.CA .
[0125] S154: Based on the mask labels, the dimensionality-reduced hyperspectral image is divided into a training set and a test set for microplastics, microalgae, and background.
[0126] Specifically, based on the mask labels that include three categories: microalgae, microplastics, and background, 70% of the pixels containing the mask labels for microplastics and microalgae in the denoised hyperspectral image and 70% of the pixels in the background mask are used as the training set, and the remaining pixels are used as the test set.
[0127] S155: For each set of dimensionality-reduced hyperspectral images, train and build a classification model CB on the training and test sets respectively. j , j is the number of groups, where j = 1, 2, ..., N-2;
[0128] Specifically, using the training set of each set of dimensionality-reduced hyperspectral images containing three categories—microplastics, microalgae, and background—a classification model CB is trained and established. j j is the number of groups, where j = 1, 2, ..., N-2, and a total of N-2 classification models are established.
[0129] S156: Using the established classification model CB j The test set of dimensionality-reduced hyperspectral images was classified to obtain the classification index F1. CBj ;
[0130] Specifically, the established classification model CB is used. j The test set images of the dimensionality-reduced hyperspectral images were classified to obtain the classification index F1. CBj A total of N-2 classification indices F1 were obtained. CBj Each classification model CB j Corresponding classification index F1 CBj The calculation steps are the same as in S153, where j = 1, 2, ..., N-2.
[0131] S157: Take the formula F1. CBj =w×F1 CA And the smallest j is j*, corresponding to the classification model CB j* As the final classification model.
[0132] Take the formula F1 CBj =w×F1 CA And the smallest j is j*, corresponding to the classification model CB j* As the final classification model, where w∈[1,+∞), satisfying this formula ensures that the classification effect of the model does not degenerate, and taking the minimum value ensures that the classification model has sufficient generalization ability.
[0133] Corresponding to the aforementioned embodiments of the method for constructing a microscopic hyperspectral image classification model, this application also provides embodiments of an apparatus for constructing a microscopic hyperspectral image classification model.
[0134] Figure 5 This is a block diagram illustrating an apparatus for constructing a microscopic hyperspectral image classification model according to an exemplary embodiment. (Refer to...) Figure 5 The device includes a first acquisition module 21, a splicing module 22, a noise reduction module 23, a dimensionality reduction module 24, and a modeling module 25.
[0135] The first acquisition module 21 is used to acquire microscopic hyperspectral images of the microalgae and microplastic mixture sample;
[0136] The stitching module 22 is used to stitch together the microscopic hyperspectral images in each spectral dimension according to the affine transformation model to obtain the hyperspectral images corresponding to each spectral dimension.
[0137] The noise reduction module 23 is used to perform noise reduction processing on the stitching edges of the hyperspectral image;
[0138] Dimensionality reduction processing module 24 is used to perform hyperspectral data dimensionality reduction processing on the hyperspectral image after noise reduction processing;
[0139] Modeling module 25 is used to establish a classification model based on the denoised hyperspectral image and the dimension-reduced hyperspectral image to identify the microscopic hyperspectral image.
[0140] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0141] Example 2:
[0142] Figure 6 This is a flowchart illustrating a microscopic hyperspectral image classification method according to an exemplary embodiment, such as... Figure 6 As shown, the following steps may be included:
[0143] S21: Acquire the microscopic hyperspectral image to be identified;
[0144] S22: Input the microscopic hyperspectral image to be identified into the classification model described in Example 1 to obtain the identification result.
[0145] As can be seen from the above embodiments, this application improves the detection limit of hyperspectral imaging technology by combining microscopic hyperspectral imaging technology with microscopic image stitching technology; to address the difficulty of low spectral signal-to-noise ratio in hyperspectral images, a denoising algorithm is adopted to improve the signal-to-noise ratio of hyperspectral images, enabling images to be processed more reliably; to address the difficulty of high spectral dimensionality in traditional hyperspectral imaging methods, a spectral dimensionality reduction method for microalgae and microplastics is established, improving recognition efficiency.
[0146] Corresponding to the aforementioned embodiments of the microscopic hyperspectral image classification method, this application also provides embodiments of a microscopic hyperspectral image classification device.
[0147] Figure 7 This is a block diagram of a microscopic hyperspectral image classification device according to an exemplary embodiment. (Refer to...) Figure 7 The device includes a second acquisition module 21 and an identification module 22.
[0148] The second acquisition module 21 is used to acquire the microscopic hyperspectral image to be identified;
[0149] The identification module 22 is used to input the microscopic hyperspectral image to be identified into the classification model constructed in Example 1 to obtain the identification result.
[0150] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0151] Example 3:
[0152] This invention also provides a microscopic hyperspectral image classification device, comprising:
[0153] Light source, used to output a visible-near-infrared compensated light source with a stable spectral structure;
[0154] A microscopic hyperspectral imager is used for hyperspectral imaging of samples of microalgae and microplastic mixtures under a microscopic field of view.
[0155] The sample moving stage is movable during observation and is used to hold the sample, align different regions of the target object in the sample with the imaging range of the microscopic hyperspectral imager, and output position information.
[0156] A processor for executing the methods described in S11-S15.
[0157] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0158] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0159] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the above-described method for constructing a microscopic hyperspectral image classification model or a microscopic hyperspectral image classification method.
[0160] Accordingly, this application also provides a computer-readable storage medium storing computer instructions thereon, characterized in that, when the instructions are executed by a processor, they implement the method for constructing a microscopic hyperspectral image classification model or the microscopic hyperspectral image classification method as described above.
[0161] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.
[0162] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for constructing a microscopic hyperspectral image classification model, characterized in that, include: Obtain microscopic hyperspectral images of samples containing a mixture of microalgae and microplastics; Based on the image homography transformation model, the microscopic hyperspectral images in each spectral dimension are stitched together to obtain hyperspectral images corresponding to each spectral dimension. This step specifically includes: saving the microscopic hyperspectral images and establishing an original spatial spectrum database, and labeling the microplastics and microalgae with mask tags; performing affine transformations on the images in the original spatial spectrum database based on the positional information of the microalgae and microplastic mixture samples; establishing an image homography transformation model based on a deep convolutional neural network using the images in the original spatial spectrum database and the affine transformed images as input, and the relationship of the affine transformation as output; establishing an attention mechanism network branch for the image homography transformation model based on the mask tags to reduce the morphological changes of microalgae and microplastic targets during image homography transformation, thereby optimizing the image homography transformation model; and stitching the microscopic hyperspectral images based on each spectral dimension according to the optimized image homography transformation model. The hyperspectral images are denoised at their stitching edges. This step specifically includes: A1: Calculating the signal-to-noise ratio (SNR) of each spectral dimension for the hyperspectral images of microalgae and microplastics, and setting two SNR thresholds, S1 and S2, for the low and high wavelength portions, respectively, corresponding to wavelengths λ1 and λ2; A2: Establishing high- and low SNR image pairs in the spectral dimensions, where the wavelength range of the high SNR image is (λ1, λ2), and the wavelength range of the low SNR image is [λ0, λ1]∪[λ2, λ1]. [λ3]; A3: Randomly select high-to-low signal-to-noise ratio (SNR) image pairs to form a correspondence between high and low SNR images; A4: Using the low SNR image as input and the high SNR image as output, establish a spectral image denoising model based on a deep convolutional neural network according to the correspondence; A5: Test new hyperspectral images using the spectral image denoising model, calculate the average SNR of all dimensions of the image, and if it is greater than 2*(S1+S2), the denoising model is considered effective; otherwise, repeat A1-A5 until an effective spectral image denoising model is obtained; A6: Input the hyperspectral image into the effective spectral image denoising model for denoising processing. The hyperspectral image after denoising is subjected to hyperspectral data dimensionality reduction processing. This step specifically includes: B1: Clustering the denoised hyperspectral image using a clustering algorithm to form n clusters, each cluster containing several wavelengths, where n is the target dimensionality reduction number; B2: Selecting the cluster center fi of each cluster, where i=1, 2, ..., n, to obtain the dimensionality reduction results [f1, f2, f3, ..., fn]; B3: Repeating operations B1 and B2 for n=2, 3, ..., N-1 respectively, to obtain N-2 sets of dimensionality reduction results, where N is the total number of wavelengths in the microscopic hyperspectral image; B4: Obtaining N-2 sets of dimensionality-reduced hyperspectral images corresponding to the wavelengths based on the dimensionality reduction results. A classification model is established based on the denoised and dimensionality-reduced hyperspectral images to identify microscopic hyperspectral images. This step specifically includes: dividing the denoised hyperspectral image into training and testing sets for microplastics, microalgae, and background, based on the mask labels, wherein the denoised hyperspectral image contains N wavelengths; and training and establishing the classification model using the training set. Using the classification model The test set is classified to obtain the classification index F1. CA Based on the mask labels, the dimensionality-reduced hyperspectral images are divided into training and testing sets for microplastics, microalgae, and background; for each set of training and testing sets of dimensionality-reduced hyperspectral images, a classification model is trained and established respectively. , j is the number of groups, where j = 1, 2, ..., N-2; the established classification model is used. The test set images of the dimensionality-reduced hyperspectral images were classified to obtain the classification index F1. CBj Take the formula that satisfies And the smallest j is j*, corresponding to the classification model. As the final classification model, in .
2. A device for constructing a microscopic hyperspectral image classification model, characterized in that, The apparatus for performing the method of constructing a microscopic hyperspectral image classification model as described in claim 1 includes: The first acquisition module is used to acquire microscopic hyperspectral images of the microalgae and microplastic mixture sample; The stitching module is used to stitch together the microscopic hyperspectral images in each spectral dimension according to the image homography transformation model to obtain the hyperspectral images corresponding to each spectral dimension. A noise reduction module is used to perform noise reduction processing on the stitching edges of the hyperspectral image; The dimension reduction processing module is used to perform hyperspectral data dimension reduction processing on the denoised hyperspectral image; The modeling module is used to build a classification model based on the denoised hyperspectral image and the dimension-reduced hyperspectral image to identify the microscopic hyperspectral image.
3. A method for classifying microscopic hyperspectral images, characterized in that, include: Acquire the microscopic hyperspectral image of the object to be identified; The microscopic hyperspectral image to be identified is input into the classification model described in claim 1 to obtain the classification result.
4. A microscopic hyperspectral image classification device, characterized in that, The apparatus for performing the microscopic hyperspectral image classification method of claim 3 includes: The second acquisition module is used to acquire the microscopic hyperspectral image to be identified; The identification module is used to input the microscopic hyperspectral image to be identified into the classification model described in claim 1 to obtain the identification result.
5. A microscopic hyperspectral image classification device, characterized in that, include: Light source, used to output a visible-near-infrared compensated light source with a stable spectral structure; A microscopic hyperspectral imager is used for hyperspectral imaging of samples of microalgae and microplastic mixtures under a microscopic field of view. The sample moving stage is movable during observation and is used to hold the sample, align different regions of the target object in the sample with the imaging range of the microscopic hyperspectral imager, and output position information. A processor for performing the method of claim 1.
6. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in claim 1 or 3.