Water quality parameter inversion method based on hyperspectral data analysis

Abstract

The invention relates to the technical field of data analysis, and particularly discloses a water quality parameter inversion method based on hyperspectral data analysis, which comprises the following steps: S1, carrying out grid division on a hyperspectral image, and obtaining a hyperspectral curve based on reflectivity values of pixel points in a grid; s2, sub-grids are determined according to the difference degree of the initial grid and the to-be-determined grid; determining a to-be-determined grid according to the existing sub-grids, determining the sub-grids again, and repeating the steps until no new sub-grids exist; counting the total number of the sub-grids, and determining a target image according to the maximum total number of the sub-grids; s3, grouping the target images, and ensuring that the number of groups is greater than a number threshold; determining representative images in the groups, and determining a training set and a verification set according to the representative images; and training a water quality parameter inversion model based on the training set and the verification set, and performing inversion on the water quality parameters. The diversity of the training set can be ensured, and the accuracy of artificial intelligence water quality inversion is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a water quality parameter inversion method based on hyperspectral data analysis. Background Art

[0002] Hyperspectral refers to data obtained through remote sensing technology, usually referring to the spectral information of the target observed in multiple narrow bands. Unlike traditional multispectral images (such as red, green, and blue), hyperspectral data can provide information in hundreds or thousands of bands, thereby describing the spectral characteristics of the object in more detail.

[0003] With the rapid development of artificial intelligence technology, the application of artificial intelligence in the field of water quality parameter inversion is becoming increasingly important. Traditional water quality detection methods usually require complex experimental equipment and long-term sampling and analysis, while deep learning models can extract patterns from a large amount of water quality data and quickly and accurately invert various parameters in the water body. This not only improves the efficiency and accuracy of water quality monitoring, but also provides strong support for real-time monitoring of water pollution and optimization of water resources management.

[0004] However, the application of artificial intelligence in the field of water quality parameter inversion also has certain drawbacks. The performance and accuracy of machine learning models are highly dependent on the diversity of training data. If the samples in the training set cannot fully cover the various changes in the water body, the prediction ability of the model may be limited, which may lead to the inability of artificial intelligence systems to accurately reflect complex water quality changes in practical applications. Therefore, how to ensure the diversity of the training set is the key to improving the accuracy of artificial intelligence water quality inversion. Summary of the invention

[0005] The purpose of the present invention is to provide a water quality parameter inversion method based on hyperspectral data analysis to solve the following technical problems:

[0006] Ensure the diversity of training sets and improve the accuracy of artificial intelligence water quality inversion.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A water quality parameter inversion method based on hyperspectral data analysis includes the following steps:

[0009] S1: Sampling points are set in the water body sampling area, a hyperspectral image at the sampling point is obtained, the hyperspectral image is divided into grids, the length and width of the grids are both preset values, and a hyperspectral curve is obtained based on the reflectance value of the pixel points in the grid at a preset target band, the reflectance value represents the radiation brightness value after radiation correction;

[0010] S2: Mark grid i as the initial grid, mark the grids that have the same edge as the initial grid as pending grids, and calculate the difference f(x) represents the hyperspectral curve corresponding to the initial grid, x represents the target band, g(x) represents the hyperspectral curve corresponding to the undetermined grid, and a difference threshold Pys is set. When the difference P≤Pys, the undetermined grid is used as a subgrid of the initial grid;

[0011] Mark the grid that has the same edge as the subgrid as the undetermined grid of the initial grid, repeat the above steps, determine the subgrid of the initial grid again, and repeat the above steps until there is no new subgrid of the initial grid;

[0012] Counting the total number of subgrids of the initial grid, determining the initial grid k corresponding to the maximum total number of subgrids, and intercepting the initial grid k and the corresponding subgrids in the hyperspectral image as the target image;

[0013] S3: grouping the target images, and the difference between the initial grids of any two target images in the group is less than the difference threshold. When the total number of groups is less than or equal to the preset number threshold, determining new sampling points, and repeating the above steps until the number of groups is greater than the number threshold for the first time;

[0014] Taking the initial grid of the target image c in the group as the first grid, determining the difference between the first grid and the remaining initial grids in the group, and calculating the total difference Ptot, determining the target image C corresponding to the minimum total difference, taking it as the representative image, taking all the representative images as the verification set, and taking the remaining target images as the training set;

[0015] The pre-established water quality parameter inversion model is trained based on the training set and the validation set to invert the water quality parameters.

[0016] As a further solution of the present invention: in step S3, the process of inverting the water quality parameters specifically includes:

[0017] Obtaining water quality parameters of the area corresponding to the target image, the water quality parameters including dissolved oxygen, suspended matter concentration and algae concentration, and annotating the target image in the training set and the representative image in the validation set with the corresponding water quality parameters;

[0018] A water quality parameter inversion model is established through a deep learning model. The water quality parameter inversion model is trained by using target images in a training set that annotates water quality parameters. The water quality parameter inversion model is verified by using representative images in a verification set that annotates water quality parameters. Water quality parameters are inverted based on the verified water quality parameter inversion model.

[0019] As a further solution of the present invention: the process of obtaining the water quality parameters of the area corresponding to the target image specifically includes:

[0020] The area corresponding to the target image is taken as the acquisition area, and a number of acquisition points are set at equal intervals in the acquisition area. Sampling is performed at the acquisition points to obtain the water quality parameters at the acquisition points, and the mean of the s-th water quality parameter at the acquisition points is calculated to obtain the s-th water quality parameter of the acquisition area, s=1, 2, 3.

[0021] As a further solution of the present invention: in the process of calculating the mean of the sth water quality parameter at the collection points, when the difference between the sth water quality parameter at a certain collection point and the corresponding mean is greater than or equal to a preset value, the sth water quality parameter at the collection point is removed and the mean is calculated again.

[0022] As a further solution of the present invention: in the step S2, in the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, the total number of sub-grids corresponding to the minimum total difference is taken as the maximum total number of sub-grids.

[0023] As a further solution of the present invention: in the step S3, in the process of determining the representative image, when two or more total differences are the same and are the minimum total differences, the target image with the minimum total difference and the largest area is used as the representative image.

[0024] As a further solution of the present invention: in the step S1, obtaining the hyperspectral curve specifically includes:

[0025] The contribution of the band to the water quality parameters is determined based on the partial least squares regression method. When the contribution is greater than or equal to the preset contribution threshold, the corresponding band is used as the target band.

[0026] Calculate the average reflectivity Aa represents the reflectance value of the a-th pixel point in the grid in the target band, n represents the total number of pixels in the grid, and the coordinate point (b, APJb) is generated. APJb represents the average reflectance corresponding to the b-th target band. The coordinate point is fitted to obtain a curve of the average reflectance changing with the target band, which is used as the hyperspectral curve.

[0027] As a further solution of the present invention: in the step S1, the hyperspectral image is acquired based on a sensor at a preset position.

[0028] Beneficial effects of the present invention: In this solution, firstly, multiple sampling points are set in the water sampling area, and a hyperspectral image of each sampling point is obtained, and then a hyperspectral curve is generated through the reflectance value of the pixel points in the grid in the target band, so as to capture the spectral characteristics of different areas in the water body in detail, and ensure the spatial resolution of the hyperspectral data and the integrity of the spectral information. Compared with traditional multispectral imaging, hyperspectral imaging can provide more continuous and detailed spectral data in a wider spectral range, so that the system can identify small and critical spectral differences in the water body, and more accurately distinguish different types of suspended matter, dissolved matter, etc., thus laying a foundation for the accurate inversion of water quality parameters. The grid division can systematically process large-scale hyperspectral images and improve the efficiency and accuracy of data processing. In addition, the reflectance values ​​after radiation correction ensure the consistency and comparability of the data, providing a reliable basis for subsequent analysis and modeling. After that, by marking and calculating the difference between grids, grids with similar spectral characteristics are identified, and these similar grids are clustered into subgrids. Finally, the initial grid k with the most subgrids is determined, and it and its subgrids are intercepted as the target image. The difference threshold is used to effectively cluster similar areas, reducing the redundancy of the data and ensuring that each cluster represents a unique water quality characteristic area. And by identifying subgrids, the ability to capture the internal heterogeneity of the water body is enhanced, ensuring the diversity and representativeness of the training data set. In addition, the initial grid k with the largest number of subgrids is selected as the target image, ensuring that the selected area is widely representative in the overall image, which helps to improve the training effect and prediction accuracy of the subsequent model; then, grouping is used to ensure that the target images in each group have similar initial grid features. When the number of groups does not reach the preset threshold, new sampling points are dynamically determined to increase data diversity, and representative images are selected as validation sets, and the remaining images are used as training sets to train and verify the water quality parameter inversion model; grouping and dynamic The sampling method ensures that the training set covers different spectral feature areas in the water body, greatly improving the diversity and representativeness of the training data, thereby enhancing the generalization ability and prediction accuracy of the model. By using representative images as the validation set, the model performance can be effectively evaluated and optimized to avoid overfitting. At the same time, the sampling points are dynamically adjusted so that the training data can adapt to various changes that may occur in the water body, further improving the reliability and practicality of the model in practical applications. Finally, by accurately labeling the training set and validation set, a water quality parameter inversion mechanism is established using a deep learning model, ultimately achieving accurate prediction and inversion of water quality parameters.Accurately labeled data sets provide a high-quality foundation for the training of deep learning models, ensuring that the models can learn effective features and laws. The powerful expression ability of deep learning models enables them to process complex hyperspectral data. These data have rich and continuous spectral information, allowing the model to extract tiny spectral feature changes that are closely related to water quality from hundreds or even more bands, thereby more accurately identifying key parameters such as dissolved oxygen concentration, suspended matter content, and algae distribution, and accurately inverting key water quality parameters. The present invention can ensure the diversity of training sets and improve the accuracy of artificial intelligence water quality inversion. ; BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with the accompanying drawings.

[0030] Figure 1 It is a flow chart of a water quality parameter inversion method based on hyperspectral data analysis of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 As shown, the present invention is a water quality parameter inversion method based on hyperspectral data analysis, comprising the following steps:

[0033] S1: Sampling points are set in the water body sampling area, a hyperspectral image at the sampling point is obtained, the hyperspectral image is divided into grids, the length and width of the grids are both preset values, and a hyperspectral curve is obtained based on the reflectance value of the pixel points in the grid at a preset target band, the reflectance value represents the radiation brightness value after radiation correction;

[0034] S2: Mark grid i as the initial grid, mark the grids that have the same edge as the initial grid as pending grids, and calculate the difference f(x) represents the hyperspectral curve corresponding to the initial grid, x represents the target band, g(x) represents the hyperspectral curve corresponding to the undetermined grid, and a difference threshold Pys is set. When the difference P≤Pys, the undetermined grid is used as a subgrid of the initial grid;

[0035] Mark the grid that has the same edge as the subgrid as the undetermined grid of the initial grid, repeat the above steps, determine the subgrid of the initial grid again, and repeat the above steps until there is no new subgrid of the initial grid;

[0036] Counting the total number of subgrids of the initial grid, determining the initial grid k corresponding to the maximum total number of subgrids, and intercepting the initial grid k and the corresponding subgrids in the hyperspectral image as the target image;

[0037] S3: grouping the target images, and the difference between the initial grids of any two target images in the group is less than the difference threshold. When the total number of groups is less than or equal to the preset number threshold, determining new sampling points, and repeating the above steps until the number of groups is greater than the number threshold for the first time;

[0038] Taking the initial grid of the target image c in the group as the first grid, determining the difference between the first grid and the remaining initial grids in the group, and calculating the total difference Ptot, determining the target image C corresponding to the minimum total difference, taking it as the representative image, taking all the representative images as the verification set, and taking the remaining target images as the training set;

[0039] The pre-established water quality parameter inversion model is trained based on the training set and the validation set to invert the water quality parameters.

[0040] It should be noted that, firstly, multiple sampling points are set up in the water sampling area, and the hyperspectral image of each sampling point is obtained, and then the hyperspectral curve is generated by the reflectance value of the pixel points in the grid in the target band, so as to capture the spectral characteristics of different areas in the water body in detail, and ensure the spatial resolution of the hyperspectral data and the integrity of the spectral information. Compared with traditional multispectral imaging, hyperspectral imaging can provide more continuous and detailed spectral data in a wider spectral range, so that the system can identify small and critical spectral differences in the water body, and more accurately distinguish different types of suspended matter, dissolved matter, etc., thus laying the foundation for the accurate inversion of water quality parameters; Grid division can systematically process large-scale hyperspectral images and improve the efficiency and accuracy of data processing. In addition, the reflectance values ​​after radiation correction ensure the consistency and comparability of the data, providing a reliable basis for subsequent analysis and modeling. After that, by marking and calculating the difference between grids, grids with similar spectral characteristics are identified, and these similar grids are clustered into sub-grids. Finally, the initial grid k with the most sub-grids is determined, and it and its sub-grids are intercepted as the target image. The similar areas are effectively clustered through the difference threshold, which reduces the redundancy of the data and ensures that each cluster represents a unique water quality characteristic area. Identifying subgrids enhances the ability to capture internal heterogeneity of water bodies and ensures the diversity and representativeness of the training data set. In addition, the initial grid k with the largest number of subgrids is selected as the target image, which ensures that the selected area is widely representative in the overall image, which helps to improve the training effect and prediction accuracy of subsequent models. Then, grouping is used to ensure that the target images in each group have similar initial grid features. When the number of groups does not reach the preset threshold, new sampling points are dynamically determined to increase data diversity. Representative images are selected as validation sets, and the remaining images are used as training sets to train and verify the water quality parameter inversion model. Grouping and dynamic sampling The method ensures that the training set covers different spectral feature areas in the water body, greatly improving the diversity and representativeness of the training data, thereby enhancing the generalization ability and prediction accuracy of the model. By using representative images as the validation set, the model performance can be effectively evaluated and optimized to avoid overfitting. At the same time, the sampling points are dynamically adjusted so that the training data can adapt to various changes that may occur in the water body, further improving the reliability and practicality of the model in practical applications. Finally, by accurately labeling the training set and the validation set, the water quality parameter inversion mechanism is established using the deep learning model, and ultimately the accurate prediction and inversion of water quality parameters are achieved.Accurately labeled data sets provide a high-quality foundation for the training of deep learning models, ensuring that the models can learn effective features and laws. The powerful expression ability of deep learning models enables them to process complex hyperspectral data. These data have rich and continuous spectral information, allowing the model to extract tiny spectral feature changes that are closely related to water quality from hundreds or even more bands, thereby more accurately identifying key parameters such as dissolved oxygen concentration, suspended matter content, and algae distribution, and accurately inverting key water quality parameters. ;

[0041] In another preferred embodiment of the present invention, in step S3, the process of inverting the water quality parameters specifically includes:

[0042] Obtaining water quality parameters of the area corresponding to the target image, the water quality parameters including dissolved oxygen, suspended matter concentration and algae concentration, and annotating the target image in the training set and the representative image in the validation set with the corresponding water quality parameters;

[0043] A water quality parameter inversion model is established through a deep learning model. The water quality parameter inversion model is trained by using target images in a training set that annotates water quality parameters. The water quality parameter inversion model is verified by using representative images in a verification set that annotates water quality parameters. Water quality parameters are inverted based on the verified water quality parameter inversion model.

[0044] It is worth noting that the actual measured water quality parameters are first obtained according to the water area corresponding to the target image. These parameters include but are not limited to dissolved oxygen (DO), suspended solids concentration (TSS) and algae concentration (Chl-a or other algae indicators). By annotating these parameters with the corresponding target images one by one, the association between remote sensing images (hyperspectral images) and water quality parameters can be established, providing basic data support for model learning; the labeled training set target images are used to input the deep learning model, so that the model learns the mapping relationship between different spectral features and specific water quality parameters, and then the representative images in the verification set with labeled water quality parameters are used to verify the performance of the model that has been initially trained. Finally, it is applied to target images without labeled water quality parameters or water images of a larger range to achieve fast and accurate inversion of water quality parameters.

[0045] In another preferred embodiment of the present invention, the process of obtaining the water quality parameters of the area corresponding to the target image specifically includes:

[0046] The area corresponding to the target image is taken as the acquisition area, and a number of acquisition points are set at equal intervals in the acquisition area. Sampling is performed at the acquisition points to obtain the water quality parameters at the acquisition points, and the mean of the s-th water quality parameter at the acquisition points is calculated to obtain the s-th water quality parameter of the acquisition area, s=1, 2, 3.

[0047] In another preferred embodiment of the present invention, in the process of calculating the mean of the sth water quality parameter at the collection points, when the difference between the sth water quality parameter at a certain collection point and the corresponding mean is greater than or equal to a preset value, the sth water quality parameter at the collection point is removed and the mean is calculated again.

[0048] In another preferred embodiment of the present invention, in the step S2, in the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, the total number of sub-grids corresponding to the minimum total difference is taken as the maximum total number of sub-grids.

[0049] In another preferred embodiment of the present invention, in the step S3, in the process of determining the representative image, when two or more total differences are the same and have the minimum total difference, the target image with the minimum total difference and the largest area is used as the representative image.

[0050] In another preferred embodiment of the present invention, in step S1, obtaining a hyperspectral curve specifically includes:

[0051] The contribution of the band to the water quality parameters is determined based on the partial least squares regression method. When the contribution is greater than or equal to the preset contribution threshold, the corresponding band is used as the target band.

[0052] Calculate the average reflectivity Aa represents the reflectance value of the a-th pixel point in the grid in the target band, n represents the total number of pixels in the grid, and the coordinate point (b, APJb) is generated. APJb represents the average reflectance corresponding to the b-th target band. The coordinate point is fitted to obtain a curve of the average reflectance changing with the target band, which is used as the hyperspectral curve.

[0053] It is worth noting that in order to extract bands that are highly sensitive to water quality parameters from massive and continuously distributed spectral information, the partial least squares regression method is used to quantitatively evaluate the contribution of each spectral band to water quality parameters; compared with traditional multispectral data, the biggest feature of hyperspectral data is the richness and continuity of its spectral dimensions, that is, it has hundreds or even thousands of narrow-band spectra in the visible light, near-infrared and mid-infrared ranges, which provides more detailed spectral diagnostic information for accurately identifying water quality changes; in this process, the continuous spectral information provided by hyperspectral data enables these fitted average reflectance curves to show the trend of spectral changes with bands in detail, rather than just the rough information of a few discrete points. By taking the average value and fitting it on multiple target bands, it not only improves the stability of data processing, but also makes the maximum use of the continuity characteristics of hyperspectral data, making the obtained hyperspectral curves more representative and consistent.

[0054] In another preferred embodiment of the present invention, in the step S1, the hyperspectral image is acquired based on a sensor at a preset position.

[0055] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A water quality parameter inversion method based on hyperspectral data analysis, characterized in that: The following steps are involved: S1: Sampling points are set in the water body sampling area, a hyperspectral image at the sampling point is obtained, the hyperspectral image is divided into grids, the length and width of the grids are both preset values, and a hyperspectral curve is obtained based on the reflectance value of the pixel points in the grid at a preset target band, the reflectance value represents the radiation brightness value after radiation correction; S2: Mark grid i as the initial grid, mark the grids that have the same edge as the initial grid as pending grids, and calculate the difference f(x) represents the hyperspectral curve corresponding to the initial grid, x represents the target band, g(x) represents the hyperspectral curve corresponding to the undetermined grid, and a difference threshold Pys is set. When the difference P≤Pys, the undetermined grid is used as a subgrid of the initial grid; Mark the grid that has the same edge as the subgrid as the undetermined grid of the initial grid, repeat the above steps, determine the subgrid of the initial grid again, and repeat the above steps until there is no new subgrid of the initial grid; Counting the total number of subgrids of the initial grid, determining the initial grid k corresponding to the maximum total number of subgrids, and intercepting the initial grid k and the corresponding subgrids in the hyperspectral image as the target image; S3: grouping the target images, and the difference between the initial grids of any two target images in the group is less than the difference threshold. When the total number of groups is less than or equal to the preset number threshold, determining new sampling points, and repeating the above steps until the number of groups is greater than the number threshold for the first time; Taking the initial grid of the target image c in the group as the first grid, determining the difference between the first grid and the remaining initial grids in the group, and calculating the total difference Ptot, determining the target image C corresponding to the minimum total difference, taking it as the representative image, taking all the representative images as the verification set, and taking the remaining target images as the training set; The pre-established water quality parameter inversion model is trained based on the training set and the validation set to invert the water quality parameters.

2. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In step S3, the process of inverting the water quality parameters specifically includes: Obtaining water quality parameters of the area corresponding to the target image, the water quality parameters including dissolved oxygen, suspended matter concentration and algae concentration, and annotating the target image in the training set and the representative image in the validation set with the corresponding water quality parameters; A water quality parameter inversion model is established through a deep learning model. The water quality parameter inversion model is trained by using target images in a training set that annotates water quality parameters. The water quality parameter inversion model is verified by using representative images in a verification set that annotates water quality parameters. Water quality parameters are inverted based on the verified water quality parameter inversion model.

3. The water quality parameter inversion method based on hyperspectral data analysis according to claim 2 is characterized in that: The process of obtaining the water quality parameters of the area corresponding to the target image specifically includes: The area corresponding to the target image is taken as the acquisition area, and a number of acquisition points are set at equal intervals in the acquisition area. Sampling is performed at the acquisition points to obtain the water quality parameters at the acquisition points, and the mean of the s-th water quality parameter at the acquisition points is calculated to obtain the s-th water quality parameter of the acquisition area, s=1, 2, 3.

4. The water quality parameter inversion method based on hyperspectral data analysis according to claim 3 is characterized in that: In the process of calculating the mean of the sth water quality parameter at the collection points, when the difference between the sth water quality parameter at a certain collection point and the corresponding mean is greater than or equal to a preset value, the sth water quality parameter at the collection point is removed and the mean is calculated again.

5. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S2, in the process of determining the target image, when the total number of sub-grids corresponding to two or more initial grids is the same and is the maximum total number of sub-grids, the total number of sub-grids corresponding to the minimum total difference is taken as the maximum total number of sub-grids.

6. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S3, in the process of determining the representative image, when two or more total differences are the same and have the smallest total difference, the target image with the smallest total difference and the largest area is used as the representative image.

7. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S1, obtaining the hyperspectral curve specifically includes: The contribution of the band to the water quality parameters is determined based on the partial least squares regression method. When the contribution is greater than or equal to the preset contribution threshold, the corresponding band is used as the target band. Calculate the average reflectivity Aa represents the reflectance value of the a-th pixel point in the grid in the target band, n represents the total number of pixels in the grid, and the coordinate point (b, APJb) is generated. APJb represents the average reflectance corresponding to the b-th target band. The coordinate point is fitted to obtain a curve of the average reflectance changing with the target band, which is used as the hyperspectral curve.

8. The water quality parameter inversion method based on hyperspectral data analysis according to claim 1 is characterized in that: In the step S1, the hyperspectral image is acquired based on a sensor at a preset position.

Images