In-situ hyperspectral water quality parameter inversion method based on deep learning

Abstract

The invention discloses an in-situ hyperspectral water quality parameter inversion method based on deep learning. According to the method, a parallel network model is constructed by combining a convolutional neural network (CNN) and a Transformer, local features are extracted by using the CNN, global information is captured by using the Transformer, and comprehensive analysis of complex features and waveband information in in-situ hyperspectral reflectivity data is realized. In the model training process, the initial layer of the network is pre-trained by adopting a self-supervised learning strategy of random shielding and noise injection, and then the weight of the network is finely adjusted by utilizing labeled data, so that the understanding of the model on a data feature mode is enhanced. According to the method, the unlabeled data is fully utilized, and the training precision and generalization ability of the model are improved. Compared with a traditional explicit linear or nonlinear model constructed by depending on prior knowledge, the method has the advantages that the recessive correlation between spectrums can be captured, end-to-end water quality parameter inversion is realized, and the result is remarkably improved in the aspects of precision and generalization ability.

Description

Technical Field

[0001] The invention relates to an in-situ hyperspectral inversion method for water quality parameters based on deep learning, and belongs to the technical field of water quality monitoring. Background Art

[0002] Water quality monitoring is an extremely important part of the protection and management of rivers, lakes and reservoirs. The construction of a new accurate and real-time water quality monitoring technology system is a more favorable basic support and condition guarantee for water conservancy scientific and technological innovation and high-quality development. Water quality parameter inversion mainly uses remote sensing data or sensor data to automatically infer important water quality parameters such as turbidity, dissolved oxygen, chlorophyll, etc., thereby reducing the dependence of traditional monitoring methods on human resources. Water quality parameter inversion includes multiple key links, such as data preprocessing, feature extraction, model training and optimization. With the powerful nonlinear mapping capabilities of deep learning, the inversion model based on deep learning can achieve higher prediction accuracy under complex and changing environmental conditions, and is widely used in environmental protection, resource management, and water ecological monitoring.

[0003] At present, in the research field of water quality parameter inversion from hyperspectral data, there have been a large number of studies based on satellite remote sensing hyperspectral images (such as Landsat 8 and Sentinel-2). These methods are usually divided into semi-analytical methods, semi-empirical methods and machine learning methods. However, these methods often rely on prior knowledge to build explicit linear or nonlinear models for specific spectral bands, which makes it difficult to capture implicit complex correlations, and are limited to focusing on local spectral features, ignoring the global correlation of spectral bands. In addition, since the frequency of water quality monitoring and the frequency of field spectral data collection are difficult to match in time scale, the utilization rate of the collected hyperspectral data is low, and many spectral data lack the corresponding true values ​​of water quality parameters, and cannot be used as labeled data for deep learning model training, which leads to the problem of insufficient training data and waste of data resources. Summary of the invention

[0004] Purpose of the invention: In view of the problems and shortcomings in the prior art, the present invention provides an in-situ hyperspectral inversion method for water quality parameters based on deep learning, which combines convolutional neural network (CNN) with Transformer to construct a parallel network model, uses CNN to extract local features, and Transformer to capture global information. It also adopts a self-supervised learning strategy and uses unlabeled data to pre-train the initial layer of the network, so as to achieve a comprehensive analysis of hyperspectral reflectance data and accurately invert the water quality parameters of the measured water area.

[0005] Technical solution: A method for in-situ hyperspectral inversion of water quality parameters based on deep learning, the method comprising the following steps:

[0006] Step 1: The in-situ hyperspectral reflectance data collected on site constitute a data set. The water quality parameter values ​​of the target water area measured at the same time are taken as the true values. The data set is cleaned and the hyperspectral data without corresponding water quality parameters are removed. The data set is divided into a training set, a validation set, and a test set according to the set ratio;

[0007] Step 2: Build and train the CNN and Transformer parallel network models, including determining the model structure, selecting the model loss function, configuring the hyperparameters, and the model training and evaluation process, and finally obtain a model suitable for in-situ hyperspectral water quality parameter inversion;

[0008] Step 3: Combine the unlabeled data removed in step 1 and use a self-supervised learning strategy with random occlusion and noise injection to pre-train the initial layer of the network constructed in step 2;

[0009] Step 4: Replace the original network weight configuration in step 2 with the network weight configuration trained in step 3, and then fine-tune the network weight using the labeled data;

[0010] Step 5: After obtaining the in-situ hyperspectral water quality parameter inversion model, the in-situ hyperspectral reflectance data in the test set are used to invert the water quality parameters and evaluate the inversion accuracy and generalization ability of the model;

[0011] Step 6: Finally, the inversion model is applied to actual water quality monitoring.

[0012] In step 1, water quality parameters include turbidity, chlorophyll a, dissolved oxygen, conductivity, total nitrogen, and total phosphorus of the water body. Hyperspectral data includes the collected 400-900nm reflectance with a spectral resolution of 1nm. The collection of spectral data and water quality parameters is achieved through instruments and equipment such as spectrometers and automatic measuring stations.

[0013] The model structure in step 2 is composed of a convolution module and a Transformer module in parallel and then in series with a fully connected layer, wherein the convolution module is composed of a convolution layer and a normalization layer, and the Transformer module is composed of a multi-head attention, a normalization layer and a feedforward neural network, and is specifically implemented as follows:

[0014] (2.1) Use the convolutional module and the Transformer module to build two branches:

[0015] One branch uses CNN to extract local features, and the other branch uses Transformer to capture global information. Each branch contains multiple convolutional modules and Transformer modules. In order to fuse feature information of different dimensions, the features between branches are aligned and then added, and then sent to the next module of each branch. For deep training of the network, residual connections are added to the model. Finally, the outputs of the two branches are spliced ​​and input into the fully connected layer.

[0016] (2.2) To ensure effective learning of the entire network, the loss function is calculated on the fused output, and the mean squared error (MSE) is used as the loss function:

[0017]

[0018] in, is the inverted water quality parameter value, and y is the true value of the water quality parameter;

[0019] (2.3) The model is trained using the data set obtained in step 1, and Adam is used as the optimization algorithm for the training network to obtain the in-situ hyperspectral water quality parameter inversion model.

[0020] In the step (2.1), the convolution module is composed of a convolution layer and a normalization layer, which is specifically implemented as follows:

[0021] The module consists of two sizes of one-dimensional convolutional neural networks (1, 1) and (3, 1), activation function ReLU, batch normalization (Batch Normalization, BN) and residual connection. Given a batch data X = {x 1 , x 2 , …, x N}, where N is the batch size, x i is the feature vector of the input data, and the module output Y = {y 1 ,y 2 , …, y N} can be expressed as:

[0022] ReLU(x)=max(0,x)

[0023] Y = ReLU(X + BN(Conv(X))

[0024] Among them, Conv represents the convolution operation; BN represents batch normalization, and its formula is:

[0025]

[0026] Among them, μ B represents the batch mean, represents the batch variance, ε is a constant used to avoid division by zero errors, and γ and β are learnable parameters for scaling and translation, respectively.

[0027] In the step (2.1), the Transformer module consists of multi-head attention, layer normalization and feedforward neural network, which is specifically implemented as follows:

[0028] For a given input sequence X, each element is first transformed linearly to generate a query vector (Query) Q, a key vector (Key) K, and a value vector (Value) V, which are calculated as follows:

[0029] Q=X·W Q , K = X·W K , V = X·W V

[0030] Among them, W Q , W k , W v is a learnable weight matrix;

[0031] In the Transformer encoder, multi-head attention is used to capture the features of different subspaces, which is calculated as:

[0032] MutiHead(Q,K,V)=Concat(head 1 , head 2 , …, head N )W O

[0033] Among them, each head i is a separate self-attention calculation:

[0034]

[0035] in, is a scaling factor to prevent the gradient from disappearing due to excessive dot product; the outputs of all heads are concatenated and then transformed by a linear transformation W O Get the final result;

[0036] Then, the output of the multi-head self-attention MutiHead (Q, K, V) is fed into the feed-forward neural network FNN and output after residual connection and layer normalization:

[0037] FNN(X)=W 2 ·σ(W 1 ·x+b 1 )+b 2

[0038] Output = LayerNorm(X + FNN(X))

[0039] Among them, W i is the weight matrix, b i is the bias, σ is the activation function, ReLU is usually selected; LayerNorm standardizes the output of each layer of neurons in the input to reduce the gradient disappearance problem. The calculation method is:

[0040]

[0041] μ L and are the mean and variance of the output of the neurons in this layer, and ε is a small constant to prevent division by zero.

[0042] In step 3, the model initialization layer is pre-trained using a self-supervised learning strategy of random occlusion and noise injection, which is specifically implemented as follows:

[0043] The model initialization layer is set to the first convolution module of the convolution branch and the first Transformer encoder of the Transformer branch; first, given the input sequence L and C are the spectrum length and number of channels respectively. The initialization layer is self-supervised pre-trained by using the method of local random occlusion and noise injection and reconstruction. The mask ratio is defined as r. For each spectrum band of length L, an occlusion mask block of length k is randomly generated, where each mask block is r. 1 and r 2 The probability is random occlusion and random noise; the sequence and mask block are input into the initialization layer for reconstruction task to obtain the restored spectral band. The predicted spectral band is evaluated by the root mean square error loss function, which can be expressed as:

[0044]

[0045] Among them, N masked =r·L is the total length of the mask block, Refers to the i-th spectral band segment predicted after reconstruction.

[0046] The step 5 uses the normalized root mean square error and determination coefficient to evaluate the in-situ hyperspectral water quality parameter inversion model obtained in the step 4, and the formula is:

[0047]

[0048] Among them, y i is the actual water quality parameter value, is the mean of the actual values, is the predicted water quality parameter value, σ y is the standard deviation of the actual values.

[0049] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the in-situ hyperspectral inversion method for water quality parameters based on deep learning as described above is implemented.

[0050] A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the in-situ hyperspectral inversion method for water quality parameters based on deep learning.

[0051] Beneficial effects: Compared with the prior art, the in-situ hyperspectral inversion method for water quality parameters based on deep learning provided by the present invention has the following advantages:

[0052] (1) A deep learning method is used to invert water quality parameters from in-situ hyperspectral reflectance data based on convolutional neural networks (CNNs) and Transformers. The local feature extraction capability of CNNs and the global information capture advantage of Transformers are fully utilized to capture the complex correlations implicit in the data, thereby improving the accuracy and generalization ability of the model.

[0053] (2) CNN and Transformer are used to construct a parallel network model. Combined with the feature fusion mechanism, the two networks are used to extract features of different dimensions respectively, thereby enhancing the multi-dimensional feature expression ability of the model and taking into account the global correlation while paying attention to the local features of the spectrum.

[0054] (3) The self-supervised learning pre-training adopted has significantly improved the overall inversion effect. It makes full use of the unlabeled data that lacks the true values ​​of the corresponding water quality parameters, solves the problems of insufficient training data and waste of data resources, and improves the model's understanding of the characteristic patterns of hyperspectral data.

[0055] (4) The present invention solves the problem that traditional linear or nonlinear models rely on prior knowledge and have low generalization performance for inversion of different water areas, realizes end-to-end water quality parameter inversion, and has higher inversion accuracy and better generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of the structure of a convolutional neural network (CNN) and a Transformer parallel network and a self-supervised training model in an example implementation of the present invention;

[0058] Figure 3 This is a schematic diagram of the feature fusion mechanism in an example implementation of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0060] like Figure 1 As shown in the figure, the in-situ hyperspectral inversion method for water quality parameters based on deep learning includes the following steps:

[0061] Step 1: The in-situ hyperspectral reflectance data collected on site constitutes a data set, and the spectral band of 400-900nm with a resolution of 1nm that has the highest correlation with the inverted water quality parameters is selected to form a single spectral data. The water quality parameter values ​​of the target water area measured by the automatic measuring station or laboratory sampling at the same time are taken as the true value; the data set is cleaned, and the hyperspectral data without corresponding water quality parameters measured at the same time are removed; the remaining data is used as the data set and divided into training set, validation set and test set in a ratio of 7:2:1. Among them, water quality parameters include turbidity, chlorophyll a, dissolved oxygen, conductivity, total nitrogen and total phosphorus of the water body.

[0062] The processed spectral reflectance data is an array of shape (1, 500), such as [[0.0059853, 0.0050442, 0.0056415, …, 0.124342, 0.125087, …, 0.0010855]], which usually has numerical fluctuations within a certain band. Standardizing these data can make the reflectance data of different bands comparable, so as to better capture the patterns in the data. For a given spectral sequence x, the calculation method is:

[0063]

[0064] Where μ is the mean of the spectral data and σ is the standard deviation of the data;

[0065] Step 2: Use the CNN and Transformer parallel network model to make full use of the local feature extraction capability of CNN and the global information capture advantage of Transformer to capture the complex correlation implicit in the data, and use the feature fusion mechanism to extract features of different dimensions using two networks. The model structure consists of a convolution module and a Transformer module in parallel and then in series with a fully connected layer. The convolution module consists of a convolution layer and a normalization layer, and the Transformer module consists of a multi-head attention, a normalization layer, and a feedforward neural network.

[0066] First, two branches are constructed using the convolution module and the Transformer module. The convolution module consists of two one-dimensional convolutional neural networks of (1, 1) and (3, 1), activation function ReLU, batch normalization and residual connection. Given a batch data X = {x 1 , x 2 , …, x N}, where N is the batch size, x i is a spectral band in the input batch, and the module output Y = {y 1 ,y 2 , …, y N} can be expressed as:

[0067] ReLU(x)=max(0,x)

[0068] Y = ReLU(X + BN(Conv(X))

[0069] Among them, Conv represents the convolution operation; BN represents batch normalization, and its formula is:

[0070]

[0071] Among them, μ B represents the batch mean, represents the batch variance, ∈ is a constant used to avoid division by zero errors, and γ and β are learnable parameters for scaling and translation, respectively.

[0072] The Transformer module consists of multi-head attention, layer normalization and feed-forward neural network. For a given input sequence X, each element is first transformed linearly to generate a query vector (Query) Q, a key vector (Key) K, and a value vector (Val u e) V, which is calculated as:

[0073] Q=X·W Q , K = X·W K , V = X·W V

[0074] Among them, W Q , W k , W v is a learnable weight matrix;

[0075] In the Transformer encoder, multi-head attention is used to capture the features of different subspaces, which is calculated as:

[0076] MutiHead(Q,K,V)=Concat(head 1 , head 2 , …, head N )WO

[0077] When building the model, each encoder has 10 attention heads, that is, N = 10, and each head i is a separate self-attention calculation:

[0078]

[0079] in, is a scaling factor to prevent the gradient from disappearing due to excessive dot product; the outputs of all heads are concatenated and then transformed by a linear transformation W O Get the final result;

[0080] Then, the output of the multi-head self-attention MutiHead (Q, K, V) is fed into the feed-forward neural network FNN and output after residual connection and layer normalization:

[0081] FNN(X)=W 2 ·σ(W 1 ·x+b 1 )+b 2

[0082] Output = LayerNorm(X + FNN(X))

[0083] Among them, W i is the weight matrix, b i is the bias, σ is the activation function, ReLU is usually selected; LayerNorm standardizes the output of each layer of neurons in the input to reduce the gradient disappearance problem. The calculation method is:

[0084]

[0085] μ L and are the mean and variance of the output of the neurons in this layer, and ε is a small constant to prevent division by zero.

[0086] In order to fuse feature information of different dimensions, the branches are aligned and then the features are added. For the intermediate feature Y with a convolution module size of (batch size, number of channels, 500) C And the intermediate feature Y with convolution module size (batch size, 5, 100) T , respectively, after being dimensionally aligned by the (1, 1) convolution kernel and the linear layer, they are added together and then sent to the next module of each branch. In order to train the network in depth, residual connections are added to the model; finally, the outputs of the two branches are spliced ​​and input into the fully connected layer, and finally the water quality parameter Y is output. w ={y 1 ,y 2,y 3 , ..., y N}, where y i is the input spectral band x i The corresponding water quality parameters.

[0087] To ensure effective learning of the entire network, the loss function is calculated on the fused output, and the mean squared error (MSE) is used as the loss function:

[0088]

[0089] in, is the inverted water quality parameter value, and y is the true value of the water quality parameter;

[0090] Then, the data set obtained in step 1 is used to train the model, and Adam is used as the optimization algorithm for the training network to obtain the in-situ hyperspectral water quality parameter inversion model.

[0091] Step 3: Combine the unlabeled data removed in step 1, and use the self-supervised learning strategy of random occlusion and noise injection to pre-train the initial layer of the network constructed in step 2; the model initialization layer is the first convolution module of the convolution branch and the first Transformer encoder of the Transformer branch; First, given the input sequence 500 and 16 are the spectrum length and number of channels; the initialization layer is self-supervised pre-trained by using the method of local random occlusion and noise injection and reconstruction, and the mask ratio is defined as r = 0.6. For each spectrum band with a length of L = 500, an occlusion mask block with a length of k = 10 is randomly generated, where each mask block is r 1 = 0.6 and r 2 =0.4 is random occlusion and random noise; the sequence and mask block are input into the initialization layer for reconstruction task to obtain the restored spectral band. The predicted spectral band is evaluated by the root mean square error loss function, which can be expressed as:

[0092]

[0093] Among them, N masked =r·L=300 is the total length of the mask block, Refers to the i-th spectral band segment predicted after reconstruction.

[0094] Step 4: Replace the network weights of the initialization layer with the network weights obtained from self-supervised training, and then use the labeled data to fine-tune the network weights to obtain the in-situ hyperspectral water quality parameter inversion model.

[0095] Step 5: After obtaining the in-situ hyperspectral water quality parameter inversion model, the in-situ hyperspectral reflectance data in the test set are used to invert the water quality parameters to evaluate the model inversion accuracy and generalization ability; the normalized root mean square error and determination coefficient are used to evaluate the final inversion model, and the formula is:

[0096]

[0097] Among them, y i is the actual water quality parameter value, is the mean of the actual values, is the predicted water quality parameter value, σ y is the standard deviation of the actual values. During the evaluation process, if the model's R 2 The closer the score is to 1, the higher the NRMSE STD The smaller the score, the better the model performance is, indicating that the model performs well in both accuracy and explanatory power.

[0098] Step 6: Apply the trained and evaluated in-situ hyperspectral water quality parameter inversion model to the actual water quality monitoring task, accurately invert the water quality parameters from the water surface hyperspectral reflectance data transmitted in real time by the automatic measuring station, and realize real-time and accurate water quality monitoring of the target water area.

[0099] In the process of processing in-situ hyperspectral reflectance data, the present invention uses limited water quality parameter data for training, adopts the proposed hyperspectral data inversion method based on CNN and Transformer, first encodes the spectral bands through CNN and generates a multi-scale feature map, and then uses the global attention mechanism of the Transformer module to construct the long-distance dependency characteristics of water quality parameters. In order to improve the model's ability to analyze complex spectral information, the present invention uses a self-supervised learning method to pre-train the CNN initial layer and Transformer encoder based on random occlusion and random noise, thereby enhancing the model's ability to understand the characteristic patterns of hyperspectral data. The present invention gets rid of the dependence of traditional water quality parameter inversion models on prior knowledge, and can achieve efficient end-to-end water quality parameter inversion on a limited sample data set, with high accuracy and strong generalization ability.

[0100] Obviously, those skilled in the art should understand that the various steps of the in-situ hyperspectral inversion method for water quality parameters based on deep learning in the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A method for in-situ hyperspectral inversion of water quality parameters based on deep learning, characterized in that: A parallel network model is constructed by combining convolutional neural network (CNN) and Transformer. CNN is used to extract local features, while Transformer captures global information. A self-supervised learning strategy is adopted, and the initial layer of the network is pre-trained with unlabeled data to achieve a comprehensive analysis of hyperspectral reflectance data and accurately invert the water quality parameters of the measured waters.

2. The in-situ hyperspectral inversion method for water quality parameters based on deep learning according to claim 1, characterized in that: The steps include: Step 1: The in-situ hyperspectral reflectance data collected on site constitute a data set. The water quality parameter values ​​of the target water area measured at the same time are taken as the true values. The data set is cleaned and the hyperspectral data without corresponding water quality parameters are removed. The data set is divided into a training set, a validation set, and a test set according to the set ratio; Step 2: Build and train the CNN and Transformer parallel network models, including determining the model structure, selecting the model loss function, configuring the hyperparameters, and the model training and evaluation process, and finally obtain a model suitable for in-situ hyperspectral water quality parameter inversion; Step 3: Combine the unlabeled data removed in step 1 and use a self-supervised learning strategy with random occlusion and noise injection to pre-train the initial layer of the network constructed in step 2; Step 4: Replace the original network weight configuration in step 2 with the network weight configuration trained in step 3, and then fine-tune the network weight using the labeled data; Step 5: After obtaining the in-situ hyperspectral water quality parameter inversion model, the in-situ hyperspectral reflectance data in the test set are used to invert the water quality parameters and evaluate the inversion accuracy and generalization ability of the model; Step 6: Finally, the inversion model is applied to actual water quality monitoring.

3. The method for in-situ hyperspectral inversion of water quality parameters based on deep learning according to claim 2 is characterized in that: In step 1, water quality parameters include turbidity, chlorophyll a, dissolved oxygen, conductivity, total nitrogen, and total phosphorus of the water body. Hyperspectral data includes the collected 400-900nm reflectance with a spectral resolution of 1nm. The collection of spectral data and water quality parameters is achieved through instruments and equipment such as spectrometers and automatic measuring stations.

4. The in-situ hyperspectral inversion method for water quality parameters based on deep learning according to claim 2 is characterized in that: The model structure in step 2 is composed of a convolution module and a Transformer module in parallel and then in series with a fully connected layer, wherein the convolution module is composed of a convolution layer and a normalization layer, and the Transformer module is composed of a multi-head attention, a normalization layer and a feedforward neural network, and is specifically implemented as follows: (2.1) Use the convolutional module and the Transformer module to build two branches: One branch uses CNN to extract local features, and the other branch uses Transformer to capture global information. Each branch contains multiple convolutional modules and Transformer modules. In order to fuse feature information of different dimensions, the features between branches are aligned and then added, and then sent to the next module of each branch. For deep training of the network, residual connections are added to the model. Finally, the outputs of the two branches are spliced ​​and input into the fully connected layer. (2.2) To ensure effective learning of the entire network, the loss function is calculated on the fused output, and the mean squared error (MSE) is used as the loss function: in, is the inverted water quality parameter value, and y is the true value of the water quality parameter; (2.3) The model is trained using the data set obtained in step 1, and Adam is used as the optimization algorithm for the training network to obtain the in-situ hyperspectral water quality parameter inversion model.

5. The method for in-situ hyperspectral inversion of water quality parameters based on deep learning according to claim 4 is characterized in that: In the step (2.1), the convolution module is composed of a convolution layer and a normalization layer, which is specifically implemented as follows: The module consists of two sizes of one-dimensional convolutional neural networks (1, 1) and (3, 1), activation function ReLU, batch normalization (BatchNormalization, BN) and residual connection. Given a batch data X = {x1, x2, ..., x N }, where N is the batch size, x i is the feature vector of the input data, and the module output Y = {y1, y2, ..., y N } can be expressed as: ReLU(x)=max(0,x) Y = ReLU(X + BN(Conv(X)) Among them, Conv represents the convolution operation; BN represents batch normalization, and its formula is: Among them, μ B represents the batch mean, represents the batch variance, ε is a constant used to avoid division by zero errors, and γ and β are learnable parameters for scaling and translation, respectively.

6. The method for in-situ hyperspectral inversion of water quality parameters based on deep learning according to claim 4 is characterized in that: In the step (2.1), the Transformer module consists of multi-head attention, layer normalization and feedforward neural network, which is specifically implemented as follows: For a given input sequence X, each element is first transformed linearly to generate a query vector (Query) Q, a key vector (Key) K, and a value vector (Value) V, which are calculated as follows: Q=X·W Q ,K=X·W K ,V=X·W v Among them, W Q , W k , W v is a learnable weight matrix; In the Transformer encoder, multi-head attention is used to capture the features of different subspaces, which is calculated as: MutiHead(Q,K,V)=Concat(head1,head2,…,head N )W O Among them, each head i is a separate self-attention calculation: in, is a scaling factor to prevent the gradient from disappearing due to excessive dot product; the outputs of all heads are concatenated and then transformed by a linear transformation W O Get the final result; Then, the output of the multi-head self-attention MutiHead (Q, K, V) is fed into the feed-forward neural network FNN and output after residual connection and layer normalization: FNN(X)=W2·σ(W1·x+b1)+b2 Output = LayerNorm(X + FNN(X)) Among them, W i is the weight matrix, b i is the bias, σ is the activation function, ReLU is usually selected; LayerNorm standardizes the output of each layer of neurons in the input to reduce the gradient disappearance problem. The calculation method is: μ L and are the mean and variance of the output of the neurons in this layer, and ε is a small constant to prevent division by zero.

7. The in-situ hyperspectral inversion method for water quality parameters based on deep learning according to claim 2 is characterized in that: In step 3, the model initialization layer is pre-trained using a self-supervised learning strategy of random occlusion and noise injection, which is specifically implemented as follows: The model initialization layer is set to the first convolution module of the convolution branch and the first Transformer encoder of the Transformer branch; first, given the input sequence L and C are the spectrum length and the number of channels respectively; the initialization layer is self-supervised pre-trained by using the method of local random occlusion and noise injection and reconstruction, and the mask ratio is defined as r. For each spectrum band of length L, an occlusion mask block of length k is randomly generated, where each mask block is random occlusion and random noise with probabilities of r1 and r2; the sequence and mask block are input into the initialization layer for reconstruction task to obtain the restored spectrum band, and the root mean square error loss function is used to evaluate the predicted spectrum band. The loss function can be expressed as: Among them, N masked =r·L is the total length of the mask block, Refers to the i-th spectral band segment predicted after reconstruction.

8. The method for in-situ hyperspectral inversion of water quality parameters based on deep learning according to claim 2 is characterized in that: The step 5 uses the normalized root mean square error and determination coefficient to evaluate the in-situ hyperspectral water quality parameter inversion model obtained in the step 4, and the formula is: Among them, y i is the actual water quality parameter value, is the mean of the actual values, is the predicted water quality parameter value, σ y is the standard deviation of the actual values.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the in-situ hyperspectral inversion method for water quality parameters based on deep learning as described in any one of claims 1 to 9 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the in-situ hyperspectral inversion method for water quality parameters based on deep learning as described in any one of claims 1 to 9.

Images