A water body microplastic identification method and system based on a hyperspectral sensor

Abstract

The application provides a water body microplastic identification method and system based on a hyperspectral sensor, relates to the technical field of microplastic identification, and is characterized in that the hyperspectral data of a water body to be analyzed are acquired based on a hyperspectral sensor, preprocessed, and used to acquire an end member library of water body components in the local area, match the pixels in the effective waveband spectral data with the end members in the end member library of water body components, form a two-dimensional end member matrix from the matched pixels, form a two-dimensional hyperspectral matrix from the effective waveband spectral data, construct a reconstruction cost function based on the abundance matrix and the two-dimensional end member matrix, optimize and solve the reconstruction cost function to determine the optimal abundance matrix, perform K-means clustering based on the row vectors of the abundance matrix, and complete the identification of the water body microplastics based on the pixels and the abundance matrix of the clustering centers.

Description

Technical Field

[0001] This invention relates to the field of microplastic identification technology, specifically to a method and system for identifying microplastics in water based on a hyperspectral sensor. Background Technology

[0002] With the acceleration of industrialization and the large-scale use of plastic products, microplastic pollution has become a global water environment problem, posing a potential threat to ecosystems and human health. Hyperspectral imaging technology, due to its ability to acquire continuous and detailed spectral information of ground objects, is widely used in remote sensing monitoring of microplastics in water bodies. Especially in open water areas such as natural reservoirs and lakes, hyperspectral sensors can achieve rapid and non-destructive detection of large water areas, providing an important technical means for the identification and quantification of microplastic pollution. However, the natural water environment is complex, and the surface layer of water often contains various interfering substances such as algae, suspended solids, and biofilms. Their spectral characteristics in the visible-near infrared band highly overlap with those of microplastics, leading to severe challenges for traditional methods in practical applications.

[0003] Existing hyperspectral-based microplastic identification methods largely rely on spectral thresholding or supervised classification algorithms. While these methods can identify plastics to some extent, their accuracy drops significantly in complex scenarios with algal cover or predominantly mixed pixels. Specifically, existing methods generally suffer from the following drawbacks: First, they lack fine-grained analysis of the spectral composition of mixed pixels, making it difficult to distinguish microplastic regions encased in algal films or coexisting with organic matter. Second, the clustering process is mostly based on original spectral features, failing to fully consider the spatial correlation between pixels and the similarity of material composition, easily leading to "same object, different spectra" or "same spectrum, different object" phenomena. Third, the number of clusters is usually manually set, and the initialization of cluster centers is random, resulting in unstable results and poor repeatability. Therefore, there is an urgent need to develop a microplastic identification method that can adapt to complex aquatic environments, integrate spectral and spatial information, and possess adaptive clustering capabilities.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for identifying microplastics in water based on a hyperspectral sensor, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for identifying microplastics in water based on a hyperspectral sensor, comprising the following steps:

[0008] Step 1: Acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data.

[0009] Step 2: Obtain the local water composition end-member library, match each pixel in the effective band spectral data with each end-member in the water composition end-member library, and construct a two-dimensional end-member matrix from the successfully matched pixels; expand the effective band spectral data into a two-dimensional hyperspectral matrix;

[0010] Step 3: Reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost.

[0011] Step 4: Extract the eigenvalues ​​of the abundance matrix. Based on the eigenvalues ​​of the abundance matrix, use the spectral gap method to determine the number of clusters k. Perform K-means clustering based on the row vectors of the abundance matrix.

[0012] Step 5: Identify microplastics in water bodies based on clustering results and abundance matrix.

[0013] Furthermore, the mean and standard deviation of reflectance of all pixels in each band are calculated. The absolute value of the mean reflectance of all pixels is divided by the standard deviation of all pixels in the same band to obtain the signal-to-noise ratio of the corresponding band. A preset signal-to-noise ratio threshold and standard deviation threshold are set. Bands that simultaneously satisfy the condition that the signal-to-noise ratio is less than the signal-to-noise ratio threshold and the standard deviation is less than the standard deviation threshold are called meaningless bands.

[0014] Furthermore, the hyperspectral data of the remaining bands are used to construct a hyperspectral matrix. This hyperspectral matrix is ​​then standardized, and the covariance matrix of the standardized hyperspectral matrix is ​​calculated. Eigenvalues ​​and corresponding eigenvectors of the covariance matrix are extracted. The standardized eigenvalues ​​are then sorted in descending order of their cumulative contribution rate. With a constraint of greater than 95%, the first I eigenvalues ​​that meet the constraint are taken as principal components. The component retention coefficients of the remaining bands are determined based on the components of each band on the principal components. The component retention coefficients of each band are sorted from largest to smallest. The first 10% of the bands are called effective bands, and the remaining bands are called low contribution bands. Low contribution bands are then removed.

[0015] The formula is:

[0016] ;

[0017] ;

[0018] in, For the front The cumulative contribution rate of each principal component Let be the component retention coefficient of the j-th residual band. Let be the eigenvalue corresponding to the i-th principal component, where i is the index of the principal component and j is the index of the residual band. The sum of all eigenvalues, This represents the value of the j-th residual band in the eigenvector corresponding to the i-th principal component.

[0019] Furthermore, the water composition end-member library contains complete spectral data of various pure substances in the local water body. The logic for matching each pixel in the effective band spectral data with each end-member in the water composition end-member library is as follows: calculate the spectral angular distance between the pixel and each effective band of each end-member in the water composition end-member library. If the minimum spectral angular distance is less than the spectral angular distance threshold of the corresponding end-member, then the match is successful.

[0020] Furthermore, the two-dimensional pixel coordinates are flattened into one dimension, and the flattened pixel coordinate information and pixel spectral information are used to construct a two-dimensional hyperspectral matrix, where the two-dimensional pixel coordinate space is the pixel space. Each pixel in the two-dimensional hyperspectral matrix is ​​reconstructed as a superposition of its endmembers using the abundance matrix and the two-dimensional endmember matrix. The reconstruction cost function is then constructed as follows:

[0021] ;

[0022] in, To reconstruct the cost function, It is a two-dimensional hyperspectral matrix, specifically a B-row, N-column matrix, where B represents the total number of pixels in the effective band spectral data, and N represents the total number of effective bands in the effective band spectral data. It is a two-dimensional endmember matrix, specifically an M-row N-column matrix. The abundance matrix is ​​specifically a B-row, M-column matrix, where M is the number of successfully matched endmembers. For the first pixel space The abundance vector of each cell in the abundance matrix The fourth neighbor in the 4th neighboring neighborhood The abundance vector of each cell in the abundance matrix, where... For the first A 4-neighbor neighborhood of 1 pixel Sparse parameters, For spatial smoothing parameters, , This is the index of a cell in the cell space; for The square of the Frobenius norm, for The L1 norm, for and The square of the Euclidean distance;

[0023] Solve for minimization The abundance matrix corresponding to the time.

[0024] Furthermore, the logic for determining the number of clusters k based on the spectral gap method is as follows: The eigenvalues ​​of the abundance matrix are arranged in ascending order, and the arrangement result is: ;in, Let be the t-th eigenvalue of the abundance matrix. The index of the eigenvalues ​​of the abundance matrix is ​​T, where T is the total number of eigenvalues ​​in the abundance matrix. Maximum k is the cluster number.

[0025] Furthermore, k row vectors are randomly selected from the row vector matrix of the abundance matrix as cluster centers. The Euclidean distance between each row vector of the abundance matrix and each cluster center is calculated. Each row vector of the abundance matrix is ​​assigned to the cluster corresponding to its nearest cluster center. The cluster center position is updated with the mean of the row vectors in each cluster. The iteration is performed until the preset number of iterations is reached, and the clustering result is output. The clustering result includes the pixel coordinate set of each category label.

[0026] Further, for pixels under the same category label, the abundance matrix component of each pixel that successfully matches an endmember in each abundance matrix is ​​calculated, called the endmember abundance component. The sum of the endmember abundance components of all pixels under the same category label is calculated, and the sum of all endmember abundance components of all pixels is also calculated. The sum of the endmember abundance components of all pixels for the same microplastic is divided by the sum of all endmember abundance components of all pixels to obtain the proportion of that microplastic in all pixels under the corresponding category label. This invention also provides a water microplastic identification system based on a hyperspectral sensor. The system is used to implement the aforementioned water microplastic identification method based on a hyperspectral sensor, specifically including:

[0027] The component extraction module is used to acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data.

[0028] The endmember matching module is used to obtain the local water composition endmember library, match each pixel in the effective band spectral data with each endmember in the water composition endmember library, and construct a two-dimensional endmember matrix from the successfully matched pixels; and expand the effective band spectral data into a two-dimensional hyperspectral matrix.

[0029] The reconstruction optimization module is used to reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost.

[0030] The similarity clustering module is used to determine the spectral clustering similarity matrix based on the abundance matrix and the spatial adjacency relationship between pixels; the similarity matrix is ​​normalized by Laplacian matrix to obtain the normalized similarity matrix; the number of clusters k is determined based on the spectral gap method; the normalized similarity matrix is ​​normalized; and K-means clustering is performed based on the normalized row vectors.

[0031] The final identification module is used to identify microplastics in water bodies based on the pixels of the cluster centers and the abundance matrix.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] This invention effectively distinguishes the spectral characteristics of microplastics from interfering substances such as algae by constructing a local water body component endmember library and combining it with spectral angular distance matching, significantly improving the accuracy of endmember identification in complex water environments. Based on this, by representing each pixel in the original hyperspectral image as a linear combination of several endmembers in the water body component endmember library, and constructing a reconstruction cost function with spatial smoothness and sparsity constraints for optimization, the spectral composition of mixed pixels is accurately analyzed. This reconstruction mechanism effectively solves the problem of spectral mixing between interfering substances such as algae and biofilms and microplastics, significantly improving the accuracy of microplastic identification in complex natural water bodies. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall method flow of the present invention.

[0035] Figure 2 This is a schematic diagram of the overall system structure of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0037] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0038] Example:

[0039] Please see Figure 1 The present invention provides a technical solution:

[0040] A method for identifying microplastics in water based on a hyperspectral sensor, comprising the following steps:

[0041] Step 1: Acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data.

[0042] Furthermore, the mean and standard deviation of reflectance of all pixels in each band are calculated. The absolute value of the mean reflectance of all pixels is divided by the standard deviation of all pixels in the same band to obtain the signal-to-noise ratio of the corresponding band. A preset signal-to-noise ratio threshold and standard deviation threshold are set. Bands that simultaneously satisfy the condition that the signal-to-noise ratio is less than the signal-to-noise ratio threshold and the standard deviation is less than the standard deviation threshold are called meaningless bands.

[0043] The signal-to-noise ratio (SNR) of each band is calculated using the uniform region method. The logic of the uniform region method for calculating the SNR is as follows: the average value of the signal is estimated as the signal strength, the standard deviation of the signal is the noise intensity, the average value is divided by the noise to obtain the SNR, and the absolute value of the mean reflectance of all pixels is divided by the standard deviation of all pixels in the same band to obtain the SNR of the corresponding band.

[0044] The method for setting the signal-to-noise ratio (SNR) threshold is as follows: Calculate the SNR of all bands, calculate the average SNR of all bands, and set 20% of this average value as the SNR threshold.

[0045] Low signal-to-noise ratio (SNR) bands, where the SNR is less than the threshold, are usually located at the edge of the sensor's response or in strong atmospheric absorption bands. Their signals are severely contaminated by noise, making the data unreliable. Retaining them would introduce a large number of errors and interfere with the accuracy of subsequent critical steps such as endmember matching, spectral unmixing, and cluster analysis.

[0046] The standard deviation threshold is set as follows: Calculate the standard deviation of all bands, calculate the average of the smallest 10% of the standard deviations, and compare it with the largest 5% of the standard deviations. Set the smaller one as the standard deviation threshold.

[0047] A small standard deviation for a wavelength band means that the reflectance values ​​of that band are very close and vary very little across all pixels. Such a band acts like a "constant," providing no effective information for distinguishing different substances. In hyperspectral identification, we rely on the differences in reflectance (i.e., spectral features) of different substances across different wavelength bands for differentiation. A band with no variation has zero information entropy, is ineffective for classification, and constitutes redundant information that needs to be filtered out.

[0048] Furthermore, the hyperspectral data of the remaining bands are used to construct a hyperspectral matrix. The hyperspectral matrix is ​​then subjected to min-max normalization, and the covariance matrix of the min-max normalized hyperspectral matrix is ​​calculated. Eigenvalues ​​and corresponding eigenvectors of the covariance matrix are extracted. The min-max normalized eigenvalues ​​are then sorted in descending order, and their cumulative contribution rate is used as the eigenvalues. With a constraint of greater than 95%, the first I eigenvalues ​​that meet the constraint are taken as principal components. The component retention coefficients of the remaining bands are determined based on the components of each band on the principal components. The component retention coefficients of each band are sorted from largest to smallest. The first 10% of the bands are called effective bands, and the remaining bands are called low contribution bands. Low contribution bands are then removed.

[0049] The formula is:

[0050] ;

[0051] ;

[0052] in, For the front The cumulative contribution rate of each principal component Let be the component retention coefficient of the j-th residual band. Let be the eigenvalue corresponding to the i-th principal component, where i is the index of the principal component and j is the index of the residual band. The sum of all eigenvalues, This represents the value of the j-th residual band in the eigenvector corresponding to the i-th principal component.

[0053] In PCA, each eigenvalue represents the amount of variance explained by its corresponding principal component (eigenvector). The principal component corresponding to the largest eigenvalue explains the largest variance in the data, which usually means that this principal component contains the most important information in the data. The cumulative contribution rate is determined by stacking the sorted eigenvalues ​​to determine how much information the stacked eigenvalues ​​retain, expressed as the cumulative contribution rate. With a constraint of greater than 95%, principal component selection can remove a large number of components with little information while retaining most of the information. For each principal component's corresponding eigenvector, the components reflecting the eigenvectors under each band... The larger the absolute value, the more sensitive the corresponding spectral band is to spectral differences. This represents the contribution of the corresponding principal component. Using the contribution of each principal component as a weight, the sensitivity of a certain band to the spectral differences under each principal component is weighted to obtain the comprehensive sensitivity of the corresponding band to the spectral differences among all principal components. The larger the value, the greater the overall sensitivity of the corresponding band to the spectral differences among all principal components. This is also beneficial for subsequent analysis of the degree of spectral differences, and its importance is also higher. Therefore, bands with larger retention coefficients are retained.

[0054] PCA achieves significant dimensionality reduction by condensing information from numerous original bands into a few principal components. Removing bands with low contribution rates to the principal components is equivalent to removing redundant information. This significantly reduces the amount of data required for subsequent steps (such as endmember matching, spectral unmixing, and clustering), thereby greatly improving the overall algorithm's speed and computational efficiency. Adjacent bands in spectral data often exhibit high correlation (i.e., multicollinearity). This collinearity can interfere with the stability of optimization algorithms and clustering. The principal components generated by PCA are orthogonal (uncorrelated). By selecting components with high contribution rates and bands sensitive to spectral differences, we retain the original bands corresponding to the principal components that constitute the main information, indirectly reducing the redundant correlation within the dataset. This makes numerical computation more stable and the solutions more reliable during subsequent optimization, such as constructing reconstruction cost functions, avoiding convergence issues or excessive errors caused by ill-conditioned data; the final identification relies on spectral clustering and K-means clustering. If the input data for clustering contains a large number of low-contribution bands with no discriminative power, these "noise" dimensions will dilute the truly effective spectral features, leading to the "curse of dimensionality" and making it difficult for clustering algorithms to accurately capture meaningful class structures. By using PCA to filter effective bands, we provide the clustering algorithm with cleaner, more compact data, which helps to form clearer inter-class separation and tighter intra-class aggregation, resulting in more accurate and stable microplastic pixel clustering results.

[0055] Step 2: Obtain the local water composition end-member library, match each pixel in the effective band spectral data with each end-member in the water composition end-member library, and construct a two-dimensional end-member matrix from the successfully matched pixels; expand the effective band spectral data into a two-dimensional hyperspectral matrix;

[0056] Each pixel in the pixel space is numbered in order from top to bottom and from left to right. The number of each pixel is used as the index of the row of the two-dimensional hyperspectral matrix. The effective bands are numbered in ascending order and used as the index of the column of the two-dimensional hyperspectral matrix to form the two-dimensional hyperspectral matrix.

[0057] Each pixel that successfully matches an endmember is arranged into a row vector according to its effective band in ascending order. The pixels that successfully match the endmember are then sorted according to their numbers in ascending order. The row vectors of the pixels that successfully match the endmember are then arranged into a matrix according to the sorting order. This matrix is ​​the two-dimensional endmember matrix.

[0058] Furthermore, the water component end-member library contains complete spectral data of various pure substances in the local water body. The logic for matching each pixel in the effective band spectral data with each end-member in the water component end-member library is as follows: calculate the spectral angular distance between the pixel and each effective band of each end-member in the water component end-member library. If the minimum spectral angular distance is less than the spectral angular distance threshold of the corresponding end-member, the match is successful. The spectral angular distance threshold for each substance can be obtained by consulting relevant literature for the corresponding substance, or by inviting experts in the field to demonstrate the properties of the corresponding substance. This is a conventional technical method in the field and will not be elaborated here. The various pure substances include, but are not limited to, water, various microplastics, and various algae.

[0059] Step 3: Reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost.

[0060] Furthermore, for ease of computation, the two-dimensional pixel coordinates are flattened into one dimension. The flattened pixel coordinate information and pixel spectral information are then used to construct a two-dimensional hyperspectral matrix, where the two-dimensional pixel coordinate space is the pixel space. Each pixel in the two-dimensional hyperspectral matrix is ​​reconstructed as a superposition of its endmembers using an abundance matrix and a two-dimensional endmember matrix.

[0061] The reconstruction cost function is constructed as follows:

[0062] ;

[0063] in, To reconstruct the cost function, It is a two-dimensional hyperspectral matrix, specifically a B-row, N-column matrix, where B represents the total number of pixels in the effective band spectral data, and N represents the total number of effective bands in the effective band spectral data. It is a two-dimensional endmember matrix, specifically an M-row N-column matrix. The abundance matrix is ​​specifically a B-row, M-column matrix, where M is the number of successfully matched endmembers. For the first pixel space The abundance vector of each cell in the abundance matrix The fourth neighbor in the 4th neighboring neighborhood The abundance vector of each cell in the abundance matrix, where... For the first A 4-neighbor neighborhood of 1 pixel The sparsity parameter is typically set between 0.001 and 0.1. This is a spatial smoothing parameter, typically taken as 0.01–1. , This is the index of a cell in the cell space; for The square of the Frobenius norm, for The L1 norm, for and The square of the Euclidean distance;

[0064] To reconstruct the original spectral matrix by representing each pixel as a linear combination of superimposed endmembers, It is the "sum of squared reconstruction errors" for the entire dataset; it reflects the accuracy of reconstructing the original spectral matrix using linear combination of endmembers. The smaller the error, the more correctly the unmixing model can interpret the observed spectrum. It is a sparsity constraint on the abundance matrix, used to encourage each cell to consist of only a few endmembers, minimizing the complexity of mixing. The abundance difference between a pixel and its neighboring pixels in the abundance space (abundance matrix) is used to characterize the overall smoothness of the abundance of neighboring pixels in the entire image, ensuring that the mixing ratio of adjacent pixels in the space does not have "abrupt changes" and avoiding random jumps in the unmixing results between adjacent pixels due to noise. The reconstruction cost function consists of the above three parts (reconstruction error part, sparsity constraint, and spatial smoothness constraint). Minimizing the reconstruction cost function is to ensure that the total reconstruction error of the unmixing is smaller, the mixing complexity is lower, and the reconstruction space is smoother.

[0065] The abundance matrix reflects the mixing ratio and spatial distribution of different substances (such as plastics, algae, and water) in each pixel. It describes the deconstruction process of "spectral mixing" in hyperspectral observations—that is, the reflected signal of each pixel is formed by the proportional superposition of several pure components. The abundance matrix not only reveals the spectral composition patterns of materials but also spatially presents their distribution patterns and interactions.

[0066] In algae-covered environments such as lakes or reservoirs, pixels are often formed by a mixture of various components, including microplastic particles, algal biofilms, and the water background. Traditional classification or clustering methods cannot distinguish this "mixed spectrum," resulting in excessive spectral differences during subsequent clustering and identification, leading to too many cluster labels and a decrease in microplastic recognition rates. By constructing an abundance matrix, complex mixed pixels can be decomposed into the independent spectral contributions of each substance, thereby achieving an accurate description of "covered," "partially mixed," or "weak-signal" plastic regions.

[0067] Solve for minimization The abundance matrix corresponding to the time can be obtained using existing techniques, including but not limited to multiplicative update method and alternating least squares method.

[0068] The solution process of the multiplicative update method is as follows: First, under the condition of a fixed endmember matrix, the optimization objective of the abundance matrix is ​​decomposed into a form that can be approximated stepwise. Then, based on the idea of ​​nonnegative matrix factorization, the gradient of the objective function with respect to the abundance matrix is ​​separated into positive and negative terms, so that the update of each term is performed in a proportional correction manner. Specifically, in each iteration, the algorithm calculates the positive and negative effects of the reconstruction term and the constraint term on the abundance matrix, respectively. By updating the ratio of the former as the gain term and the latter as the suppression term, the abundance value gradually approaches the optimal distribution while maintaining nonnegativity. As the iteration progresses, the reconstruction error of the pixel spectrum continuously decreases, and the abundance matrix gradually stabilizes at a local optimum, thereby obtaining the proportional distribution of each endmember in each pixel.

[0069] The alternating least squares method solves the overall optimization problem by dividing it into two analytically solvable subproblems and iteratively approximating the global optimum. The algorithm first fixes the endmember matrix and solves a least squares problem with non-negativity constraints for each pixel to obtain the optimal abundance vector under the current conditions. Then, with the abundance matrix fixed, it reverse-engineers the endmember spectra, allowing the new endmember combinations to better reconstruct the original data. These two steps are executed alternately, ensuring that the objective function monotonically decreases in each iteration until convergence.

[0070] Step 4: Extract the eigenvalues ​​of the abundance matrix. Based on the eigenvalues ​​of the abundance matrix, use the spectral gap method to determine the number of clusters k. Perform K-means clustering based on the row vectors of the abundance matrix.

[0071] Furthermore, the logic for determining the number of clusters k based on the spectral gap method is as follows: The eigenvalues ​​of the abundance matrix are arranged in ascending order, and the arrangement result is: ;in, Let be the t-th eigenvalue of the abundance matrix. The index of the eigenvalues ​​of the abundance matrix is ​​T, where T is the total number of eigenvalues ​​in the abundance matrix. Maximum k is the cluster number.

[0072] Furthermore, k row vectors are randomly selected from the row vector matrix of the abundance matrix as cluster centers. The Euclidean distance between each row vector of the abundance matrix and each cluster center is calculated. Each row vector of the abundance matrix is ​​assigned to the cluster corresponding to its nearest cluster center. The cluster center position is updated with the mean of the row vectors in each cluster. The iteration is performed until the preset number of iterations is reached, and the clustering result is output. The clustering result includes the pixel coordinate set of each category label.

[0073] Step 5: Identify microplastics in water bodies based on clustering results and abundance matrix.

[0074] For pixels under the same category label, calculate the abundance matrix component of each pixel that successfully matches an endmember in each abundance matrix. This component is called the endmember abundance component. Calculate the sum of the endmember abundance components of all pixels under the same category label, and then calculate the sum of all endmember abundance components of all pixels. Divide the sum of the endmember abundance components of all pixels for the same microplastic by the sum of all endmember abundance components of all pixels to obtain the proportion of that microplastic in all pixels under the corresponding category label.

[0075] Please see Figure 2 The present invention further provides a water microplastic identification system based on a hyperspectral sensor. This system is used to implement the aforementioned water microplastic identification method based on a hyperspectral sensor, specifically including:

[0076] The component extraction module is used to acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data.

[0077] The endmember matching module is used to obtain the local water composition endmember library, match each pixel in the effective band spectral data with each endmember in the water composition endmember library, and form a two-dimensional endmember matrix with the successfully matched pixels; and expand the effective band spectral data into a two-dimensional hyperspectral matrix.

[0078] The reconstruction optimization module is used to reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost.

[0079] The similarity clustering module is used to determine the spectral clustering similarity matrix based on the abundance matrix and the spatial adjacency relationship between pixels; the similarity matrix is ​​normalized by Laplacian matrix to obtain the normalized similarity matrix; the number of clusters k is determined based on the spectral gap method; the normalized similarity matrix is ​​normalized; and K-means clustering is performed based on the normalized row vectors.

[0080] The final identification module is used to identify microplastics in water bodies based on the pixels of the cluster centers and the abundance matrix.

[0081] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0082] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that cannot be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for identifying microplastics in water based on a hyperspectral sensor, characterized in that, The specific steps include: Step 1: Acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data. Step 2: Obtain the local water composition end-member library, match each pixel in the effective band spectral data with each end-member in the water composition end-member library, and construct a two-dimensional end-member matrix from the successfully matched pixels; expand the effective band spectral data into a two-dimensional hyperspectral matrix; Step 3: Reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost. The two-dimensional pixel coordinates are flattened into one dimension. The flattened pixel coordinate information and pixel spectral information are used to construct a two-dimensional hyperspectral matrix, where the two-dimensional pixel coordinate space is the pixel space. Each pixel in the two-dimensional hyperspectral matrix is ​​reconstructed as a superposition of its endmembers using an abundance matrix and a two-dimensional endmember matrix. The reconstruction cost function is then constructed as follows: in, To reconstruct the cost function, It is a two-dimensional hyperspectral matrix, specifically a B-row, N-column matrix, where B represents the total number of pixels in the effective band spectral data, and N represents the total number of effective bands in the effective band spectral data. It is a two-dimensional endmember matrix, specifically an M-row N-column matrix. The abundance matrix is ​​specifically a B-row, M-column matrix, where M is the number of successfully matched endmembers. For the first pixel space The abundance vector of each cell in the abundance matrix The fourth neighbor in the 4th neighboring neighborhood The abundance vector of each cell in the abundance matrix, where... For the first A 4-neighbor neighborhood of 1 pixel Sparse parameters, For spatial smoothing parameters, , This is the index of a cell in the cell space; for The square of the Frobenius norm, for The L1 norm, for and The square of the Euclidean distance; Solve for minimization The abundance matrix corresponding to the time; Step 4: Extract the eigenvalues ​​of the abundance matrix. Based on the eigenvalues ​​of the abundance matrix, use the spectral gap method to determine the number of clusters k. Perform K-means clustering based on the row vectors of the abundance matrix. Step 5: Identify microplastics in water bodies based on clustering results and abundance matrix.

2. The method for identifying microplastics in water based on a hyperspectral sensor according to claim 1, characterized in that: Calculate the mean and standard deviation of reflectance for all pixels in each band. Divide the absolute value of the mean reflectance of all pixels by the standard deviation of all pixels in the same band to obtain the signal-to-noise ratio (SNR) of the corresponding band. Set a preset SNR threshold and a standard deviation threshold. Bands that simultaneously satisfy the conditions of SNR less than the SNR threshold and standard deviation less than the standard deviation threshold are called meaningless bands.

3. The method for identifying microplastics in water based on a hyperspectral sensor according to claim 1, characterized in that: The remaining hyperspectral data are used to construct a hyperspectral matrix. This hyperspectral matrix is ​​then standardized, and the covariance matrix of the standardized hyperspectral matrix is ​​calculated. Eigenvalues ​​and corresponding eigenvectors of the covariance matrix are extracted. The standardized eigenvalues ​​are then sorted in descending order of their cumulative contribution rate. The constraint condition is greater than 95%. The first I eigenvalues ​​that meet the constraint condition for the first time are taken as principal components. The component retention coefficient of the remaining bands is determined based on the component of each band on the principal components. The component retention coefficient of each band is sorted from large to small. The first 10% of the bands are called effective bands, and the remaining bands are called low contribution bands. Low contribution bands are removed. The formula is: in, For the front The cumulative contribution rate of each principal component Let be the component retention coefficient of the j-th residual band. Let be the eigenvalue corresponding to the i-th principal component, where i is the index of the principal component and j is the index of the residual band. The sum of all eigenvalues, This represents the value of the j-th residual band in the eigenvector corresponding to the i-th principal component. The water composition end-member library contains complete spectral data of various pure substances in local water bodies. The logic for matching each pixel in the effective band spectral data with each end-member in the water composition end-member library is as follows: calculate the spectral angular distance between the pixel and each effective band of each end-member in the water composition end-member library. If the minimum spectral angular distance is less than the spectral angular distance threshold of the corresponding end-member, the match is successful.

4. The method for identifying microplastics in water based on a hyperspectral sensor according to claim 3, characterized in that: The logic for determining the cluster number k based on the spectral gap method is as follows: Arrange the eigenvalues ​​of the abundance matrix in ascending order, and the arrangement result is: ;in, Let be the t-th eigenvalue of the abundance matrix. The index of the eigenvalues ​​of the abundance matrix is ​​T, where T is the total number of eigenvalues ​​in the abundance matrix. Maximum k is the cluster number.

5. The method for identifying microplastics in water based on a hyperspectral sensor according to claim 4, characterized in that: In the row vector matrix of the abundance matrix, k row vectors are randomly selected as cluster centers. The Euclidean distance between each row vector of the abundance matrix and each cluster center is calculated. Each row vector of the abundance matrix is ​​assigned to the cluster corresponding to its nearest cluster center. The cluster center position is updated with the mean of the row vectors in each cluster. The process is iterated until the preset number of iterations is reached. The clustering result is output, which includes the pixel coordinate set of each category label.

6. The method for identifying microplastics in water based on a hyperspectral sensor according to claim 5, characterized in that: For pixels under the same category label, calculate the abundance matrix component of each pixel that successfully matches an endmember in each abundance matrix. This component is called the endmember abundance component. Calculate the sum of the endmember abundance components of all pixels under the same category label, and then calculate the sum of all endmember abundance components of all pixels. Divide the sum of the endmember abundance components of all pixels for the same microplastic by the sum of all endmember abundance components of all pixels to obtain the proportion of that microplastic in all pixels under the corresponding category label.

7. A water microplastic identification system based on a hyperspectral sensor, characterized in that: The system is used to implement the water microplastic identification method based on a hyperspectral sensor as described in any one of claims 1-6, specifically including: The component extraction module is used to acquire hyperspectral data of the water body to be analyzed based on a hyperspectral sensor, analyze the hyperspectral data to obtain the signal-to-noise ratio and standard deviation of each band, filter out meaningless bands based on the signal-to-noise ratio and standard deviation, perform PCA analysis on the remaining bands to remove low-contribution bands, and obtain effective band spectral data. The endmember matching module is used to obtain the local water composition endmember library, match each pixel in the effective band spectral data with each endmember in the water composition endmember library, and construct a two-dimensional endmember matrix from the successfully matched pixels; and expand the effective band spectral data into a two-dimensional hyperspectral matrix. The reconstruction optimization module is used to reconstruct the two-dimensional hyperspectral matrix based on the abundance matrix and the two-dimensional endmember matrix, construct the reconstruction cost function, and optimize the reconstruction cost function to obtain the abundance matrix with the minimum reconstruction cost. The two-dimensional pixel coordinates are flattened into one dimension. The flattened pixel coordinate information and pixel spectral information are used to construct a two-dimensional hyperspectral matrix, where the two-dimensional pixel coordinate space is the pixel space. Each pixel in the two-dimensional hyperspectral matrix is ​​reconstructed as a superposition of its endmembers using an abundance matrix and a two-dimensional endmember matrix. The reconstruction cost function is then constructed as follows: in, To reconstruct the cost function, It is a two-dimensional hyperspectral matrix, specifically a B-row, N-column matrix, where B represents the total number of pixels in the effective band spectral data, and N represents the total number of effective bands in the effective band spectral data. It is a two-dimensional endmember matrix, specifically an M-row N-column matrix. The abundance matrix is ​​specifically a B-row, M-column matrix, where M is the number of successfully matched endmembers. For the first pixel space The abundance vector of each cell in the abundance matrix The fourth neighbor in the 4th neighboring neighborhood The abundance vector of each cell in the abundance matrix, where... For the first A 4-neighbor neighborhood of 1 pixel Sparse parameters, For spatial smoothing parameters, , This is the index of a cell in the cell space; for The square of the Frobenius norm, for The L1 norm, for and The square of the Euclidean distance; Solve for minimization The abundance matrix corresponding to the time; The similarity clustering module is used to determine the spectral clustering similarity matrix based on the abundance matrix and the spatial adjacency relationship between pixels; the similarity matrix is ​​normalized by Laplacian matrix to obtain the normalized similarity matrix; the number of clusters k is determined based on the spectral gap method; the normalized similarity matrix is ​​normalized; and K-means clustering is performed based on the normalized row vectors. The final identification module is used to identify microplastics in water bodies based on the pixels of the cluster centers and the abundance matrix.

Images