A hyperspectral-based water quality monitoring method and system

Abstract

The application provides a hyperspectral-based water quality monitoring method and system, adopts a community discovery algorithm to divide all wave bands into a plurality of strongly correlated wave band communities; calculates a condition number of spectral data in each wave band community, determines first, second and third candidate features according to the condition number; combines all the first, second and third candidate features into a candidate feature pool, screens an optimal feature subset from the candidate feature pool according to a cost function considering model prediction error and feature quantity at the same time; and establishes a quantitative monitoring model of a water quality parameter by using the optimal feature subset and a measured value of the water quality parameter.

Description

Technical Field

[0001] This application belongs to the field of water quality monitoring, and in particular relates to a water quality monitoring method and system based on hyperspectral imaging. Background Technology

[0002] By analyzing subtle changes in the reflectance spectrum of water bodies, key water quality parameters such as chlorophyll a, suspended solids concentration, chemical oxygen demand (COD), and dissolved organic matter can be rapidly and extensively retrieved, providing an efficient technical means for the dynamic monitoring and management of the water environment. However, while hyperspectral data provides rich information, it also brings the curse of dimensionality. Hyperspectral data is characterized by a large number of bands, strong inter-band correlations, and high information redundancy. Directly using full-band spectral data for water quality monitoring not only significantly increases computational complexity and runtime, but more importantly, severe multicollinearity between bands can seriously affect the stability and prediction accuracy of the model, even leading to overfitting and reducing its generalization ability in practical applications. Existing research mainly uses two categories of methods to reduce the dimensionality of hyperspectral data: feature selection and feature extraction. Feature selection methods, such as correlation coefficient methods and stepwise regression methods, directly select the optimal subset from the original bands using evaluation indicators, but they easily overlook the synergistic effects between bands and may lose some useful information. Feature extraction methods, such as principal component analysis (PCA) and partial least squares (PLS), project the original high-dimensional data into a low-dimensional space through mathematical transformations to generate new comprehensive variables. However, they typically perform a uniform transformation on the entire spectral data, failing to consider the differences in multicollinearity across different spectral regions. Therefore, there is an urgent need to develop a hyperspectral water quality monitoring method capable of grading and diagnosing multicollinearity and implementing differentiated and targeted treatments. Summary of the Invention

[0003] To address the above problems, this invention proposes a hyperspectral-based water quality monitoring method, comprising the following steps: The hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters are obtained, and the hyperspectral data are preprocessed; the correlation coefficient matrix between each band after preprocessing is calculated, and the community detection algorithm is used to divide all bands into multiple strongly correlated band communities; Calculate the condition number of the spectral data within each band community. If the condition number is less than a first critical value, calculate the mutual information between all bands within the community and the measured water quality parameters, and sort them from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, iteratively select multiple bands from the sorted bands. In each iteration, select the bands with the maximum correlation to the water quality parameters and the minimum redundancy with the selected band subset, until a preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, perform discrete wavelet transform on the spectral data of the community and reconstruct the signal. Use the reconstructed signal as the second type of candidate features. If the condition number is greater than or equal to the second critical value, use the spectral data of the community as the independent variable and the measured water quality parameters as the dependent variable, and apply the orthogonal projection onto the latent structure method to extract the predicted components as the third type of candidate features. All candidate features of the first, second, and third categories are combined into a candidate feature pool. A feature selection algorithm is used to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features. The optimal feature subset and the measured values ​​of water quality parameters are then used to establish a quantitative monitoring model for water quality parameters.

[0004] Optionally, the preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

[0005] Optionally, the community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

[0006] Optionally, the step of performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, and using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

[0007] Optionally, the feature selection algorithm, based on a cost function that simultaneously considers model prediction error and the number of features, selects the optimal feature subset from the candidate feature pool, including: The sequential forward selection algorithm is employed, and the cost function is specifically as follows: , where RMSECV is the root mean square error using five-fold cross-validation, and N is the number of features in the current feature subset.

[0008] Optionally, the step of establishing a quantitative monitoring model for water quality parameters using the optimal feature subset and measured values ​​of water quality parameters includes: The model was constructed using support vector regression, and radial basis functions were selected as the kernel function.

[0009] The present invention also provides a hyperspectral-based water quality monitoring system, comprising the following modules: The segmentation module is used to acquire the hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters, and to preprocess the hyperspectral data; calculate the correlation coefficient matrix between each band after preprocessing, and use a community detection algorithm to divide all bands into multiple strongly correlated band communities; The candidate feature calculation module is used to calculate the condition number of spectral data within each band community. If the condition number is less than a first critical value, the mutual information between all bands within the community and the measured values ​​of water quality parameters is calculated and sorted from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, multiple bands are iteratively selected from the sorted bands. In each iteration, the bands with the maximum correlation to water quality parameters and the minimum redundancy with the selected band subset are selected until a preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, the spectral data of the community is subjected to discrete wavelet transform and the signal is reconstructed. The reconstructed signal is used as the second type of candidate features. If the condition number is greater than or equal to the second critical value, the spectral data of the community is used as the independent variable and the measured values ​​of water quality parameters are used as the dependent variable. The method of orthogonal projection onto the latent structure is applied to extract the predicted components as the third type of candidate features. The model building module is used to combine all first, second and third category candidate features into a candidate feature pool, and to use a feature selection algorithm to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features; and to use the optimal feature subset and the measured values ​​of water quality parameters to establish a quantitative monitoring model for water quality parameters.

[0010] Optionally, the preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

[0011] Optionally, the community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

[0012] Optionally, the step of performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, and using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

[0013] Optionally, the feature selection algorithm, based on a cost function that simultaneously considers model prediction error and the number of features, selects the optimal feature subset from the candidate feature pool, including: The sequential forward selection algorithm is employed, and the cost function is specifically as follows: , where RMSECV is the root mean square error using five-fold cross-validation, and N is the number of features in the current feature subset.

[0014] Optionally, the step of establishing a quantitative monitoring model for water quality parameters using the optimal feature subset and measured values ​​of water quality parameters includes: The model was constructed using support vector regression, and radial basis functions were selected as the kernel function.

[0015] Compared with existing technologies, this invention identifies strongly correlated band communities through a community detection algorithm, thus locating the collinearity problem. Based on this, the severity of collinearity within each community is quantitatively assessed by calculating the condition number. According to the assessment results, targeted dimensionality reduction strategies of varying intensities are adopted, such as retaining the optimal band, wavelet transform reconstruction, or orthogonal projection extraction. This approach maximizes the preservation of effective spectral information related to water quality parameters while eliminating varying degrees of redundant information and collinearity interference. The quantitative monitoring model established using the optimal feature subset selected through a cost function is not only more concise in structure but also exhibits higher predictive accuracy. Attached Figure Description

[0016] Figure 1 A schematic diagram for excluding bands; Figure 2 A schematic diagram of the smooth band; Figure 3 This is a schematic diagram illustrating the process of obtaining candidate features using condition numbers. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0018] In the first embodiment, the present invention proposes a water quality monitoring method based on hyperspectral imaging, comprising the following steps: S1. Obtain the hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters, and preprocess the hyperspectral data; calculate the correlation coefficient matrix between each band after preprocessing, and use a community detection algorithm to divide all bands into multiple strongly correlated band communities; In one embodiment, spectral reflectance data of water samples were collected under darkroom conditions using an ASD ground-based spectrometer, with a spectral range of 350 to 2500 nm. Simultaneously, the chemical oxygen demand (COD) of the water samples was measured using national standard methods such as the dichromate method. During preprocessing, the 350-399 nm and 2401-2500 nm bands, which exhibit significant noise at the beginning and end of the spectrum, were first removed. Figure 1 As shown, the spectral curves were then denoised using a smoothing method, and the spectral data were normalized using a standard normal transformation to eliminate the influence of physical factors such as sample surface scattering. Figure 2 As shown.

[0019] All N preprocessed spectral bands are used as N nodes in a graph. The Pearson correlation coefficient between any two bands is calculated on all water samples, and the absolute value of the correlation coefficient is used as the weight of the edge connecting the corresponding two nodes, thus constructing a weighted undirected network. Then, a community detection algorithm is applied to iteratively partition the network, and all band nodes are divided into different sets by maximizing the modularity function. Each set is a band community with high correlation between bands within the community and low correlation between sets.

[0020] In an optional embodiment, the preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

[0021] A Savitzky-Golay filter was applied, using a sliding window containing nine consecutive data points and fitted with a second-order polynomial to reduce instrument noise in the original spectral data. After denoising, a standard normal transformation was performed on each spectral sample. This transformation corrected for spectral line drift caused by sample surface physical properties and optical path variations through centering and scaling.

[0022] In an optional embodiment, the community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

[0023] Each band is abstracted as a node in the network graph, and the absolute value of the correlation coefficient between any two bands is calculated. This value is then used as the weight of the edge connecting the corresponding node, thus constructing a weighted undirected graph. The Louvain community detection algorithm is executed on the weighted undirected graph. By iteratively optimizing the community partitioning, the modularity of the network is maximized until the modularity value converges to the maximum and no longer increases, resulting in multiple band communities with high correlation within each band.

[0024] S2, calculate the condition number of the spectral data within each band community. If the condition number is less than the first critical value, calculate the mutual information between all bands within the community and the measured water quality parameters, and sort them from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, iteratively select multiple bands from the sorted bands. In each iteration, select the band with the maximum correlation to the water quality parameters and the minimum redundancy with the selected band subset, until the preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, perform discrete wavelet transform on the spectral data of the community and reconstruct the signal. Use the reconstructed signal as the second type of candidate features. If the condition number is greater than or equal to the second critical value, use the spectral data of the community as the independent variable and the measured water quality parameters as the dependent variable, and apply the orthogonal projection to the latent structure method to extract the predicted components as the third type of candidate features. The first critical value is less than the second critical value.

[0025] For a specific band community, the spectral data of all bands contained within it are extracted from all N water samples and constructed into an N x M matrix X, where N is the number of samples and M is the number of bands within the community. Singular Value Decomposition (SVD) is then performed on this data matrix X, yielding a series of singular values. The maximum value among all singular values ​​is selected. and minimum value By calculating the ratio of the two, i.e., the condition number This allows us to obtain the condition number of the spectral data within the community for that band, which can be used to quantitatively assess the intensity of multicollinearity among the bands within that community.

[0026] In one embodiment, the first critical value The value is 10, the second critical value. The condition number is set to 100. When the condition number is less than 10, multicollinearity is weak. For a given band community, the mutual information between the spectral data sequence of each band in the community and the measured water quality parameter sequences corresponding to all water samples is calculated. All bands in the community are then sorted in descending order according to the calculated mutual information values ​​to form an ordered candidate band list. An iterative screening process based on the maximum correlation and minimum redundancy criterion is initiated: the first band in the ordered list (i.e., the band with the largest mutual information with water quality parameters) is taken as the initial feature and placed into a temporary optimal feature subset S. Subsequently, in each iteration, all bands in the candidate list that have not yet been selected into subset S are traversed, and a comprehensive evaluation value is calculated for each of them. This evaluation value aims to measure the degree to which each band satisfies "maximum correlation with measured water quality parameters" and "minimum average correlation with existing bands in subset S". The band with the best comprehensive evaluation value in this iteration is selected and added to subset S. The iterative process continues until the number of bands in subset S reaches the preset upper limit, or the redundancy introduced by the newly added bands exceeds the preset threshold, at which point it terminates. The resulting optimal feature subset S, which contains one or more bands, is the first type of candidate feature representing the community.

[0027] Preferably, for any unselected band in the candidate feature pool Its comprehensive evaluation value The calculation formula is as follows:

[0028] in, The correlation term represents the candidate band. The larger the mutual information value between the band and the target water quality parameter y, the stronger the correlation between the band and the parameter to be measured. Redundant terms represent candidate bands. Compared with all existing bands in the currently selected optimal feature subset S The smaller the average mutual information, the less information the candidate band has that overlaps with the selected features, and the stronger its independence; |S| represents the number of bands contained in the current optimal feature subset S.

[0029] When the condition number is between 10 and 100, there is a moderate degree of multicollinearity. The spectral data of the community is decomposed into three levels using the db4 wavelet basis. The low-frequency approximation coefficients of the third level are extracted to reconstruct the signal and obtain new variables that can represent the main information of the community. When the condition number is greater than or equal to 100, severe multicollinearity exists. Using the spectral data of the community as X and the measured COD value as Y, an orthogonal projection onto a latent structure regression model is established, and the first predicted principal component is extracted as a new candidate feature. Figure 3 As shown.

[0030] In an alternative embodiment, the band with the highest mutual information with the measured water quality parameters within the community is extracted as a first-class candidate feature. Specifically, for a specific band community defined by a community detection algorithm, every spectral band contained within the community is traversed. For each band, its spectral data (e.g., reflectance or absorbance) on all water samples is used as one variable sequence, while the measured water quality parameters (e.g., chlorophyll a concentration) corresponding to all water samples are used as another variable sequence. The mutual information value between these two sequences is calculated. After calculating the mutual information values ​​between all bands and water quality parameters within the community, these values ​​are compared, and the band with the highest mutual information value is finally selected as the first-class candidate feature representing this community.

[0031] The specific steps for extracting predictive components using the orthogonal projection to latent structure method are as follows: Spectral data from all bands within a given band community exhibiting severe collinearity are used to construct the independent variable matrix X across all water samples. Simultaneously, the measured water quality parameters corresponding to these samples are used to construct the dependent variable vector Y. An OPLS regression model is then established with X as input and Y as output. The core of the OPLS regression model lies in decomposing the variation information of the independent variable matrix X into two parts: one part is predictive information linearly correlated with the dependent variable Y, and the other part is non-predictive information completely uncorrelated with Y (i.e., orthogonal). Through the calculation of the OPLS regression model, one or more predictive components that can best explain the variation in Y can be extracted from X. The predictive component itself is a composite variable integrating information from all bands within the community, and it serves as a third-class candidate feature representing the community.

[0032] In an optional embodiment, performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, then using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

[0033] Using the Daubechies4 wavelet as the basis function, a three-level discrete wavelet decomposition is performed on the spectral data of the target community, decomposing the signal into coefficients at different frequency levels. After decomposition, detail coefficients containing high-frequency noise are discarded, retaining only the low-frequency approximation coefficients of the third level, as they contain the main contour information of the signal. The signal is reconstructed using only these low-frequency approximation coefficients, generating a smooth spectral curve that represents the core information of the original community, serving as the second type of candidate feature.

[0034] In one embodiment, the first type of candidate feature is a specific single spectral band, the value of which is the spectral reflectance or absorbance of that band; the second type of candidate feature is a signal vector reconstructed by discrete wavelet transform, which is itself a set of numerical sequences representing the main outline and trend of a moderately collinear band community; the third type of candidate feature is a predicted component extracted from a highly collinear community via orthogonal projection to the latent structure method, which is a single scalar value that integrates all band information within the community and is highly correlated with water quality parameters.

[0035] S3, combine all candidate features of the first, second and third categories into a candidate feature pool, and use a feature selection algorithm to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features; use the optimal feature subset and the measured values ​​of water quality parameters to establish a quantitative monitoring model for water quality parameters.

[0036] The candidate features of categories I, II, and III generated from all band communities in the previous step are aggregated to form a candidate feature pool. A sequential forward selection algorithm is used for filtering. Starting from an empty set, each iteration selects a feature from the candidate feature pool and adds it to the current feature subset. The selection criterion is to minimize the cost function, which is defined as the sum of the root mean square error under 10-fold cross-validation and a penalty term. The penalty term is proportional to the number of selected features. This cost function is minimized until its value cannot be further reduced; the resulting feature subset is the optimal feature subset. The selected optimal feature subset is used as the input independent variable of the model, and the corresponding measured COD value is used as the output dependent variable. A COD quantitative monitoring model is constructed using the support vector regression algorithm. By training the model on the training dataset, the kernel function, penalty coefficient, and other hyperparameters of the support vector regression model are determined. The trained model can then be used to predict the COD concentration of unknown water samples.

[0037] In an optional embodiment, the feature selection algorithm, based on a cost function that simultaneously considers model prediction error and the number of features, selects the optimal feature subset from the candidate feature pool, including: The sequential forward selection algorithm is employed, and the cost function is specifically as follows: , where RMSECV is the root mean square error using five-fold cross-validation, and N is the number of features in the current feature subset.

[0038] The above process employs a sequential forward selection algorithm. In each iteration, all unselected features in the candidate feature pool are traversed, and one is temporarily added to the current feature subset. This expanded feature subset, along with corresponding measured water quality parameters, is used to train a prediction model such as support vector regression. Its performance is evaluated using k-fold cross-validation, and the root mean square error of the cross-validation is calculated. The total cost is calculated based on a preset cost function. In each iteration, the candidate feature that minimizes the total cost is selected and formally added to the feature subset. This process is repeated until adding any new feature no longer reduces the cost function; the resulting feature subset is the optimal feature subset. The goal of the cost function is to achieve a balance between model prediction accuracy and model complexity. By minimizing this cost function, the selected feature subset maintains a low prediction error while having a smaller number of features, avoiding model overfitting.

[0039] In an optional embodiment, the step of establishing a quantitative monitoring model for water quality parameters using the optimal feature subset and measured values ​​of water quality parameters includes: The model was constructed using support vector regression, and radial basis functions were selected as the kernel function.

[0040] The Support Vector Regression (SVR) algorithm is employed, using the selected optimal feature subset as the model's input variables and the corresponding measured water quality parameters as the output variables for model training. To effectively handle the complex nonlinear relationship between input features and output parameters, the model is configured with a radial basis function as the kernel function. This function maps the data to a high-dimensional space, thereby constructing a nonlinear regression model with high generalization ability and prediction accuracy.

[0041] In a second embodiment, the present invention provides a hyperspectral-based water quality monitoring system, comprising the following modules: The segmentation module is used to acquire the hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters, and to preprocess the hyperspectral data; calculate the correlation coefficient matrix between each band after preprocessing, and use a community detection algorithm to divide all bands into multiple strongly correlated band communities; The candidate feature calculation module is used to calculate the condition number of spectral data within each band community. If the condition number is less than a first critical value, the mutual information between all bands within the community and the measured values ​​of water quality parameters is calculated and sorted from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, multiple bands are iteratively selected from the sorted bands. In each iteration, the bands with the maximum correlation to water quality parameters and the minimum redundancy with the selected band subset are selected until a preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, the spectral data of the community is subjected to discrete wavelet transform and the signal is reconstructed. The reconstructed signal is used as the second type of candidate features. If the condition number is greater than or equal to the second critical value, the spectral data of the community is used as the independent variable and the measured values ​​of water quality parameters are used as the dependent variable. The method of orthogonal projection onto the latent structure is applied to extract the predicted components as the third type of candidate features. The model building module is used to combine all first, second and third category candidate features into a candidate feature pool, and to use a feature selection algorithm to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features; and to use the optimal feature subset and the measured values ​​of water quality parameters to establish a quantitative monitoring model for water quality parameters.

[0042] In an optional embodiment, the preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

[0043] In an optional embodiment, the community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

[0044] In an optional embodiment, performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, then using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

[0045] In an optional embodiment, the feature selection algorithm, based on a cost function that simultaneously considers model prediction error and the number of features, selects the optimal feature subset from the candidate feature pool, including: The sequential forward selection algorithm is employed, and the cost function is specifically as follows: , where RMSECV is the root mean square error using five-fold cross-validation, and N is the number of features in the current feature subset.

[0046] In an optional embodiment, the step of establishing a quantitative monitoring model for water quality parameters using the optimal feature subset and measured values ​​of water quality parameters includes: The model was constructed using support vector regression, and radial basis functions were selected as the kernel function.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0048] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0049] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A water quality monitoring method based on hyperspectral imaging, characterized in that, Includes the following steps: Acquire hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters, and preprocess the hyperspectral data; The correlation coefficient matrix between each band after preprocessing was calculated, and a community detection algorithm was used to divide all bands into multiple strongly correlated band communities. Calculate the condition number of the spectral data within each band community. If the condition number is less than the first critical value, calculate the mutual information between all bands within the community and the measured values ​​of water quality parameters, and sort them from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, iteratively select multiple bands from the sorted bands. In each iteration, select the bands with the maximum correlation to water quality parameters and the minimum redundancy with the selected band subset, until a preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, perform discrete wavelet transform on the spectral data of the community and reconstruct the signal. The reconstructed signal is used as the second type of candidate features. If the condition number is greater than or equal to the second critical value, then the predicted components are extracted as the third type of candidate features by using the spectral data of the community as the independent variable and the measured value of the water quality parameter as the dependent variable, and applying the method of orthogonal projection onto the potential structure. All candidate features of the first, second, and third categories are combined into a candidate feature pool. A feature selection algorithm is used to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features. The optimal feature subset and the measured values ​​of water quality parameters are then used to establish a quantitative monitoring model for water quality parameters.

2. The method according to claim 1, characterized in that, The preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

3. The method according to claim 1, characterized in that, The community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

4. The method according to claim 1, characterized in that, The step of performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, and using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

5. The method according to claim 1, characterized in that, The feature selection algorithm, based on a cost function that simultaneously considers model prediction error and the number of features, selects the optimal feature subset from the candidate feature pool, including: The sequential forward selection algorithm is employed, and the cost function is specifically as follows: , where RMSECV is the root mean square error using five-fold cross-validation, and N is the number of features in the current feature subset.

6. The method according to claim 1, characterized in that, The step of establishing a quantitative monitoring model for water quality parameters using the optimal feature subset and measured values ​​of water quality parameters includes: The model was constructed using support vector regression, and radial basis functions were selected as the kernel function.

7. A water quality monitoring system based on hyperspectral imaging, characterized in that, Includes the following modules: The segmentation module is used to acquire the hyperspectral data of the water sample to be tested and the corresponding measured values ​​of water quality parameters, and to preprocess the hyperspectral data; The correlation coefficient matrix between each band after preprocessing was calculated, and a community detection algorithm was used to divide all bands into multiple strongly correlated band communities. The candidate feature calculation module is used to calculate the condition number of spectral data within each band community. If the condition number is less than a first critical value, the mutual information between all bands within the community and the measured values ​​of water quality parameters is calculated and sorted from high to low mutual information values. Using the maximum correlation and minimum redundancy criterion, multiple bands are iteratively selected from the sorted bands. In each iteration, the bands with the maximum correlation to water quality parameters and the minimum redundancy with the selected band subset are selected until the preset selection quantity or redundancy threshold is met. The selected multiple bands are used as the first type of candidate features. If the condition number is greater than or equal to the first critical value and less than the second critical value, the spectral data of the community is subjected to discrete wavelet transform and the signal is reconstructed. The reconstructed signal is used as the second type of candidate features. If the condition number is greater than or equal to the second critical value, then the predicted components are extracted as the third type of candidate features by using the spectral data of the community as the independent variable and the measured value of the water quality parameter as the dependent variable, and applying the method of orthogonal projection onto the potential structure. The model building module is used to combine all first, second and third category candidate features into a candidate feature pool, and to use a feature selection algorithm to select the optimal feature subset from the candidate feature pool based on a cost function that simultaneously considers the model prediction error and the number of features; and to use the optimal feature subset and the measured values ​​of water quality parameters to establish a quantitative monitoring model for water quality parameters.

8. The system according to claim 7, characterized in that, The preprocessing of the hyperspectral data includes: The spectral data were denoised using a Savitzky-Golay filter with a smoothing window size of 9 data points and a polynomial order of 2. The smoothed spectral data were transformed using a standard normal variable.

9. The system according to claim 7, characterized in that, The community detection algorithm is used to divide all bands into multiple strongly correlated band communities, including: Treat all bands as nodes in a graph, and the absolute value of the correlation coefficient between bands as the edge weight. Apply the Louvain community discovery algorithm for iterative optimization until the modularity of the community division reaches its maximum value and no longer changes, thus completing the division of the band community.

10. The system according to claim 7, characterized in that, The step of performing discrete wavelet transform on the spectral data of the community and reconstructing the signal, and using the reconstructed signal as a second type of candidate feature, includes: Using db4 wavelet as the basis function, the spectral data of the community is decomposed into three levels. The low-frequency approximation coefficients after decomposition are retained, and the signal is reconstructed using the low-frequency approximation coefficients to obtain the second type of candidate features.

Images