A method for retrieving water quality parameters based on hyperspectral data analysis

Abstract

The present invention relates to the technical field of data analysis, and specifically discloses a method for inverting water quality parameters based on hyperspectral data analysis, comprising the following steps: S1: Divide the hyperspectral image into grids, and obtain hyperspectral curves based on the reflectance values of the pixel points within the grids; S2: Determine sub-grids according to the difference degree between the initial grids and the to-be-determined grids; determine the to-be-determined grids according to the existing sub-grids, and determine the sub-grids again, repeating the above steps until there are no new sub-grids; count the total number of sub-grids, and determine the target image according to the maximum total number of sub-grids; S3: Group the target image to ensure that the number of groups is greater than the number threshold; determine the representative images in the groups, and determine the training set and the validation set according to the representative images; train the water quality parameter inversion model based on the training set and the validation set, and invert the water quality parameters. The present invention can ensure the diversity of the training set and improve the accuracy of artificial intelligence water quality inversion.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and particularly relates to a method for inverting water quality parameters based on hyperspectral data analysis. Background Art

[0002] Hyperspectral refers to data obtained through remote sensing technology, usually referring to spectral information obtained by observing a target in multiple narrow bands. Different from traditional multispectral images (such as red, green, and blue), hyperspectral data can provide information on hundreds of bands, enabling a more detailed description of the spectral characteristics of an object.

[0003] With the rapid development of artificial intelligence technology, the application of artificial intelligence in the field of water quality parameter inversion is becoming increasingly important. Traditional water quality detection methods usually require complex experimental equipment and long-time sampling and analysis, while deep learning models can extract patterns from a large amount of water quality data and quickly and accurately invert various parameters in the water body, not only improving the efficiency and accuracy of water quality monitoring, but also providing strong support for real-time monitoring of water pollution and optimizing water resource management.

[0004] However, the application of artificial intelligence in the field of water quality parameter inversion also has certain drawbacks. The performance and accuracy of machine learning models highly depend on the diversity of training data. If the samples in the training set cannot fully cover various changes in the water body, the prediction ability of the model may be limited, which may lead to the inability of the artificial intelligence system to accurately reflect complex water quality changes in actual applications. Therefore, how to ensure the diversity of the training set is the key to improving the accuracy of artificial intelligence water quality inversion. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for inverting water quality parameters based on hyperspectral data analysis to solve the following technical problems:

[0006] Ensure the diversity of the training set and improve the accuracy of artificial intelligence water quality inversion.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A method for inverting water quality parameters based on hyperspectral data analysis includes the following steps:

[0009] S1: Set sampling points in the water body sampling area, obtain the hyperspectral images at the sampling points, divide the hyperspectral images into grids, the length and width of the grids are both preset values, and obtain hyperspectral curves based on the reflectance values of the pixel points in the grids at a preset target band, where the reflectance value represents the radiance value after radiometric correction;

[0010] S2: Mark grid i as the initial grid, mark the grids sharing the same edge with the initial grid as pending grids, and calculate the difference degree. Let f(x) represent the hyperspectral curve corresponding to the initial grid, x represent the target band, and g(x) represent the hyperspectral curve corresponding to the pending grid. Set the difference degree threshold Pys. When the difference degree P ≤ Pys, regard the pending grid as a sub-grid of the initial grid.

[0011] Mark the grids sharing the same edge with the sub-grid as the pending grids of the initial grid, repeat the above steps to determine the sub-grids of the initial grid again, and repeat the above steps until there are no new sub-grids for the initial grid.

[0012] Count the total number of sub-grids of the initial grid, determine the initial grid k corresponding to the maximum total number of sub-grids, and intercept the initial grid k and the corresponding sub-grids in the hyperspectral image as the target image.

[0013] S3: Group the target images. The difference degree between the initial grids of any two target images in the group is less than the difference degree threshold. When the total number of groups is less than or equal to the preset quantity threshold, determine new sampling points and repeat the above steps until the number of groups is greater than the quantity threshold for the first time.

[0014] Take the initial grid of the target image c in the group as the first grid, determine the difference degree between the first grid and the other initial grids in the group, and calculate the total difference degree Ptot. Determine the target image C corresponding to the minimum total difference degree and regard it as the representative image. Take all the representative images as the validation set and the other target images as the training set.

[0015] Train the pre-established water quality parameter inversion model based on the training set and the validation set, and invert the water quality parameters.

[0016] As a further solution of the present invention: In the step S3, the process of inverting the water quality parameters specifically includes:

[0017] Obtain the water quality parameters of the area corresponding to the target image. The water quality parameters include dissolved oxygen, suspended solid concentration, and algae concentration, and label the corresponding water quality parameters for the target images in the training set and the representative images in the validation set.

[0018] Establish a water quality parameter inversion model through a deep learning model, train the water quality parameter inversion model with the target images in the training set with labeled water quality parameters, and verify the water quality parameter inversion model with the representative images in the validation set with labeled water quality parameters. Invert the water quality parameters based on the verified water quality parameter inversion model.

[0019] As a further solution of the present invention: The process of obtaining the water quality parameters of the area corresponding to the target image specifically includes:

[0020] Taking the area corresponding to the target image as the collection area, setting a number of collection points at equal intervals within the collection area, sampling at the collection points to obtain the water quality parameters at the collection points, calculating the mean value of the s-th water quality parameter at the collection points, and obtaining the s-th water quality parameter of the collection area, where s = 1, 2, 3.

[0021] As a further solution of the present invention: During the process of calculating the mean value of the s-th water quality parameter at the collection points, when the difference between the s-th water quality parameter at a certain collection point and the corresponding mean value is greater than or equal to the preset value, removing the s-th water quality parameter at this collection point and recalculating the mean value.

[0022] As a further solution of the present invention: In step S2, during the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, taking the total number of sub-grids corresponding to the minimum total difference degree as the maximum total number of sub-grids.

[0023] As a further solution of the present invention: In step S3, during the process of determining the representative image, when the total difference degrees of two or more are the same and are the minimum total difference degree, taking the target image with the minimum total difference degree and the largest area as the representative image.

[0024] As a further solution of the present invention: In step S1, the specific process of obtaining the hyperspectral curve includes:

[0025] Based on the partial least squares regression method, determining the contribution degree of the waveband to the water quality parameter. When the contribution degree is greater than or equal to the preset contribution degree threshold, taking the corresponding waveband as the target waveband;

[0026] Calculating the average reflectance Aa represents the reflectance value of the a-th pixel point in the grid on the target waveband, n represents the total number of pixel points in the grid, generating coordinate points (b, APJb), APJb represents the average reflectance corresponding to the b-th target waveband, fitting the coordinate points to obtain the curve of the average reflectance changing with the target waveband, and taking it as the hyperspectral curve.

[0027] As a further solution of the present invention: In step S1, the hyperspectral image is obtained based on the sensor at the preset position.

[0028] Advantages of the present invention: In this solution, first, multiple sampling points are set within the water body sampling area, and hyperspectral images of each sampling point are obtained. Then, hyperspectral curves are generated based on the reflectance values of pixel points within the grid in the target band, thereby carefully capturing the spectral characteristics of different regions in the water body, ensuring the spatial resolution of hyperspectral data and the integrity of spectral information. Compared with traditional multispectral imaging, hyperspectral imaging can provide more continuous and detailed spectral data within a wider spectral range, enabling the system to identify minute and crucial spectral differences in the water body, and more accurately distinguish different types of suspended substances, dissolved substances, etc., thus laying a foundation for the accurate inversion of water quality parameters. Through grid division, large-scale hyperspectral images can be systematically processed, improving the efficiency and accuracy of data processing. In addition, the reflectance values after radiometric correction ensure the consistency and comparability of the data, providing a reliable basis for subsequent analysis and modeling. Then, by marking and calculating the difference degree between grids, grids with similar spectral characteristics are identified, and these similar grids are aggregated into sub-grids. Finally, the initial grid k with the largest number of sub-grids is determined, and it and its sub-grids are intercepted as the target image. By effectively clustering similar regions through the difference degree threshold, the redundancy of data is reduced, ensuring that each cluster represents a unique water quality characteristic region. Moreover, by identifying sub-grids, the ability to capture the heterogeneity within the water body is enhanced, ensuring the diversity and representativeness of the training data set. In addition, selecting the initial grid k with the largest total number of sub-grids as the target image ensures that the selected region has broad representativeness in the overall image, contributing to improving the training effect and prediction accuracy of the subsequent model. Next, by grouping to ensure that the target images within each group have similar initial grid characteristics, when the number of groups does not reach the preset threshold, new sampling points are dynamically determined to increase data diversity. Representative images are selected as the validation set, and the remaining images are used as the training set to train and validate the water quality parameter inversion model. The methods of grouping and dynamic sampling ensure that the training set covers different spectral characteristic regions in the water body, greatly enhancing the diversity and representativeness of the training data, thereby enhancing the generalization ability and prediction accuracy of the model. By using representative images as the validation set, the model performance can be effectively evaluated and optimized to avoid overfitting. At the same time, dynamically adjusting the sampling points enables the training data to adapt to various changes that may occur in the water body, further enhancing the reliability and practicality of the model in actual applications. Finally, by accurately annotating the training set and the validation set, a water quality parameter inversion mechanism is established using a deep learning model, and finally the accurate prediction and inversion of water quality parameters are achieved.The accurately labeled dataset provides a high-quality foundation for the training of deep learning models, ensuring that the models can learn effective features and patterns. The powerful expressive ability of deep learning models enables them to process complex hyperspectral data, which has rich and continuous spectral band information, allowing the models to extract tiny spectral feature changes that are closely related to the water quality state from hundreds or even more bands, thereby more accurately identifying key parameters such as dissolved oxygen concentration, suspended solid content, and algae distribution, and accurately retrieving the key water quality parameters. The present invention can ensure the diversity of the training set and improve the accuracy of artificial intelligence water quality inversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below with reference to the accompanying drawings.

[0030] Figure 1 It is a schematic flowchart of a method for retrieving water quality parameters based on hyperspectral data analysis according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] Please refer to Figure 1 As shown, the present invention is a method for retrieving water quality parameters based on hyperspectral data analysis, including the following steps:

[0033] S1: Set sampling points in the water body sampling area, obtain the hyperspectral images at the sampling points, divide the hyperspectral images into grids, the length and width of the grids are both preset values, and obtain the hyperspectral curves based on the reflectance values of the pixel points in the grids at the preset target bands, where the reflectance values represent the radiance values after radiometric calibration;

[0034] S2: Mark grid i as the initial grid, mark the grids that share the same side with the initial grid as the pending grids, and calculate the difference degree f(x) represents the hyperspectral curve corresponding to the initial grid, x represents the target band, g(x) represents the hyperspectral curve corresponding to the pending grid, set the difference degree threshold Pys, and when the difference degree P ≤ Pys, use the pending grid as the sub-grid of the initial grid;

[0035] Mark the grids that share the same side with the sub-grid as the pending grids of the initial grid, repeat the above steps to determine the sub-grids of the initial grid again, and repeat the above steps until there are no new sub-grids for the initial grid;

[0036] Count the total number of sub - grids of the initial grid, determine the initial grid k corresponding to the maximum total number of sub - grids, and intercept the initial grid k and the corresponding sub - grids in the hyperspectral image as the target image;

[0037] S3: Group the target images. The difference degree between the initial grids of any two target images in the group is less than the difference degree threshold. When the total number of groups is less than or equal to the preset quantity threshold, determine new sampling points and repeat the above steps until the number of groups is greater than the quantity threshold for the first time;

[0038] Take the initial grid of the target image c in the group as the first grid, determine the difference degree between the first grid and the remaining initial grids in the group, and calculate the total difference degree Ptot. Determine the target image C corresponding to the minimum total difference degree and use it as the representative image. Use all the representative images as the validation set and the remaining target images as the training set;

[0039] Based on the training set and the validation set, train the pre - established water quality parameter inversion model to invert the water quality parameters.

[0040] It should be noted that first, multiple sampling points are set within the water body sampling area, and hyperspectral images of each sampling point are obtained. Then, hyperspectral curves are generated based on the reflectance values of pixel points within the grid in the target wavelength band, so as to meticulously capture the spectral characteristics of different regions in the water body, ensuring the spatial resolution of hyperspectral data and the integrity of spectral information. Compared with traditional multispectral imaging, hyperspectral imaging can provide more continuous and detailed spectral data within a wider spectral range, enabling the system to identify tiny and crucial spectral differences in the water body, and more accurately distinguish different types of suspended substances, dissolved substances, etc., thus laying a foundation for the precise inversion of water quality parameters. Through grid division, large-scale hyperspectral images can be systematically processed, improving the efficiency and accuracy of data processing. In addition, the reflectance values after radiometric correction ensure the consistency and comparability of the data, providing a reliable basis for subsequent analysis and modeling. After that, by marking and calculating the difference degrees between grids, grids with similar spectral characteristics are identified, and these similar grids are aggregated into sub-grids. Finally, the initial grid k with the largest number of sub-grids is determined, and it and its sub-grids are intercepted as the target image. By effectively clustering similar regions through the difference degree threshold, the redundancy of data is reduced, ensuring that each cluster represents a unique water quality characteristic region. Moreover, by identifying sub-grids, the ability to capture the internal heterogeneity of the water body is enhanced, ensuring the diversity and representativeness of the training dataset. In addition, selecting the initial grid k with the largest total number of sub-grids as the target image ensures that the selected region has broad representativeness in the overall image, which helps to improve the training effect and prediction accuracy of the subsequent model. Then, by grouping to ensure that the target images within each group have similar initial grid characteristics, when the number of groups does not reach the preset threshold, new sampling points are dynamically determined to increase data diversity. Representative images are selected as the validation set, and the remaining images are used as the training set to train and validate the water quality parameter inversion model. The methods of grouping and dynamic sampling ensure that the training set covers different spectral characteristic regions in the water body, greatly enhancing the diversity and representativeness of the training data, thereby enhancing the generalization ability and prediction accuracy of the model. By using representative images as the validation set, the model performance can be effectively evaluated and optimized to avoid overfitting. At the same time, dynamically adjusting the sampling points enables the training data to adapt to various possible changes in the water body, further enhancing the reliability and practicality of the model in actual applications. Finally, by accurately annotating the training set and the validation set, a water quality parameter inversion mechanism is established using a deep learning model, and finally the precise prediction and inversion of water quality parameters are achieved.The accurately labeled dataset provides a high-quality foundation for the training of deep learning models, ensuring that the models can learn effective features and patterns. The powerful expressive ability of deep learning models enables them to process complex hyperspectral data, which has rich and continuous spectral band information, allowing the models to extract subtle spectral feature changes that are closely related to the water quality state from hundreds or even more bands, thereby more accurately identifying key parameters such as dissolved oxygen concentration, suspended solid content, and algae distribution, and accurately retrieving the key water quality parameters.

[0041] In another preferred embodiment of the present invention, in step S3, the process of retrieving water quality parameters specifically includes:

[0042] Obtain the water quality parameters of the area corresponding to the target image, where the water quality parameters include dissolved oxygen, suspended solid concentration, and algae concentration, and label the corresponding water quality parameters for the target images in the training set and the representative images in the validation set.

[0043] Establish a water quality parameter retrieval model through a deep learning model, train the water quality parameter retrieval model with the target images in the training set with labeled water quality parameters, and verify the water quality parameter retrieval model with the representative images in the validation set with labeled water quality parameters, and retrieve the water quality parameters based on the verified water quality parameter retrieval model.

[0044] It is worth noting that first, obtain the actually measured water quality parameters according to the water area corresponding to the target image, and these parameters include but are not limited to dissolved oxygen (DO), suspended solid concentration (TSS), and algae concentration (Chl-a or other algae indicators). By making one-to-one corresponding labels between these parameters and the corresponding target images, the association between remote sensing images (hyperspectral images) and water quality parameters can be established, providing basic data support for model learning; input the target images of the labeled training set into the deep learning model to enable the model to learn the mapping relationship between different spectral features and specific water quality parameters, and then use the representative images in the validation set with labeled water quality parameters to verify the performance of the preliminarily trained model. Finally, apply it to the target images without labeled water quality parameters or larger-scale water area images to achieve rapid and accurate retrieval of water quality parameters.

[0045] In another preferred embodiment of the present invention, the process of obtaining the water quality parameters of the area corresponding to the target image specifically includes:

[0046] Take the area corresponding to the target image as the collection area, set a number of collection points at equal distance intervals within the collection area, sample at the collection points to obtain the water quality parameters at the collection points, and calculate the mean value of the s-th water quality parameter at the collection points to obtain the s-th water quality parameter of the collection area, where s = 1, 2, 3.

[0047] In another preferred embodiment of the present invention, in the process of calculating the mean of the sth water quality parameter at the collection points, when the difference between the sth water quality parameter at a certain collection point and the corresponding mean is greater than or equal to a preset value, the sth water quality parameter at the collection point is removed and the mean is calculated again.

[0048] In another preferred embodiment of the present invention, in the step S2, in the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, the total number of sub-grids corresponding to the minimum total difference is taken as the maximum total number of sub-grids.

[0049] In another preferred embodiment of the present invention, in the step S3, in the process of determining the representative image, when two or more total differences are the same and have the minimum total difference, the target image with the minimum total difference and the largest area is used as the representative image.

[0050] In another preferred embodiment of the present invention, in step S1, obtaining a hyperspectral curve specifically includes:

[0051] The contribution of the band to the water quality parameters is determined based on the partial least squares regression method. When the contribution is greater than or equal to the preset contribution threshold, the corresponding band is used as the target band.

[0052] Calculate the average reflectivity Aa represents the reflectance value of the a-th pixel point in the grid in the target band, n represents the total number of pixels in the grid, and the coordinate point (b, APJb) is generated. APJb represents the average reflectance corresponding to the b-th target band. The coordinate point is fitted to obtain a curve of the average reflectance changing with the target band, which is used as the hyperspectral curve.

[0053] It is worth noting that in order to extract bands that are highly sensitive to water quality parameters from massive and continuously distributed spectral information, the partial least squares regression method is used to quantitatively evaluate the contribution of each spectral band to water quality parameters; compared with traditional multispectral data, the biggest feature of hyperspectral data is the richness and continuity of its spectral dimensions, that is, it has hundreds or even thousands of narrow-band spectra in the visible light, near-infrared and mid-infrared ranges, which provides more detailed spectral diagnostic information for accurately identifying water quality changes; in this process, the continuous spectral information provided by hyperspectral data enables these fitted average reflectance curves to show the trend of spectral changes with bands in detail, rather than just the rough information of a few discrete points. By taking the average value and fitting it on multiple target bands, it not only improves the stability of data processing, but also makes the maximum use of the continuity characteristics of hyperspectral data, making the obtained hyperspectral curves more representative and consistent.

[0054] In another preferred embodiment of the present invention, in the step S1, the hyperspectral image is acquired based on a sensor at a preset position.

[0055] The above has described in detail one embodiment of the present invention, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. Any equivalent changes and improvements made within the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. A water quality parameter inversion method based on hyperspectral data analysis, characterized in that: The following steps are involved: S1: Sampling points are set in the water body sampling area, a hyperspectral image at the sampling point is obtained, the hyperspectral image is divided into grids, the length and width of the grids are both preset values, and a hyperspectral curve is obtained based on the reflectance value of the pixel points in the grid at a preset target band, the reflectance value represents the radiation brightness value after radiation correction; S2: Mark grid i as the initial grid, mark the grids that have the same edge as the initial grid as pending grids, and calculate the difference f(x) represents the hyperspectral curve corresponding to the initial grid, x represents the target band, g(x) represents the hyperspectral curve corresponding to the pending grid, and a difference threshold Pys is set. When the difference P≤Pys, the pending grid is used as a subgrid of the initial grid; Mark the grid that has the same edge as the subgrid as the undetermined grid of the initial grid, repeat the above steps, determine the subgrid of the initial grid again, and repeat the above steps until there is no new subgrid of the initial grid; Counting the total number of subgrids of the initial grid, determining the initial grid k corresponding to the maximum total number of subgrids, and intercepting the initial grid k and the corresponding subgrids in the hyperspectral image as the target image; S3: grouping the target images, and the difference between the initial grids of any two target images in the group is less than the difference threshold. When the total number of groups is less than or equal to the preset number threshold, determining new sampling points, and repeating the above steps until the number of groups is greater than the number threshold for the first time; Taking the initial grid of the target image c in the group as the first grid, determining the difference between the first grid and the remaining initial grids in the group, and calculating the total difference Ptot, determining the target image C corresponding to the minimum total difference, taking it as the representative image, taking all the representative images as the verification set, and taking the remaining target images as the training set; Based on the training set and the validation set, a pre-established water quality parameter inversion model is trained to invert the water quality parameters; The specific steps of obtaining the hyperspectral curve include: The contribution of the band to the water quality parameters is determined based on the partial least squares regression method. When the contribution is greater than or equal to the preset contribution threshold, the corresponding band is used as the target band. Calculate the average reflectivity , A a represents the reflectance value of the ath pixel in the grid in the target band, n represents the total number of pixels in the grid, and the generated coordinate point (b, APJ b ), APJ b represents the average reflectivity corresponding to the b-th target band, and the coordinate points are fitted to obtain a curve showing the average reflectivity changing with the target band, which is used as a hyperspectral curve.

2. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In step S3, the process of inverting the water quality parameters specifically includes: Obtaining water quality parameters of the area corresponding to the target image, the water quality parameters including dissolved oxygen, suspended matter concentration and algae concentration, and annotating the target image in the training set and the representative image in the validation set with the corresponding water quality parameters; A water quality parameter inversion model is established through a deep learning model. The water quality parameter inversion model is trained by using target images in a training set that annotates water quality parameters. The water quality parameter inversion model is verified by using representative images in a verification set that annotates water quality parameters. Water quality parameters are inverted based on the verified water quality parameter inversion model.

3. The water quality parameter inversion method based on hyperspectral data analysis according to claim 2 is characterized in that: The process of obtaining the water quality parameters of the area corresponding to the target image specifically includes: The area corresponding to the target image is taken as the acquisition area, and a number of acquisition points are set at equal intervals in the acquisition area. Sampling is performed at the acquisition points to obtain the water quality parameters at the acquisition points. The mean of the s-th water quality parameter at the acquisition points is calculated to obtain the s-th water quality parameter of the acquisition area, s=1, 2, 3.

4. The water quality parameter inversion method based on hyperspectral data analysis according to claim 3 is characterized in that: In the process of calculating the mean of the sth water quality parameter at the collection points, when the difference between the sth water quality parameter at a certain collection point and the corresponding mean is greater than or equal to a preset value, the sth water quality parameter at the collection point is removed and the mean is calculated again.

5. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S2, in the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, the total number of sub-grids corresponding to the minimum total difference is taken as the maximum total number of sub-grids.

6. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S3, in the process of determining the representative image, when two or more total differences are the same and have the smallest total difference, the target image with the smallest total difference and the largest area is used as the representative image.

7. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S1, the hyperspectral image is acquired based on a sensor at a preset position.

Images