Water quality detection method, device and equipment based on spectral technology and storage medium

Abstract

The present disclosure provides a water quality detection method and device based on spectral technology, equipment and storage medium, the method comprises the following steps: clustering first spectral data of a plurality of water samples to obtain a plurality of clustering data; each class of clustering data is labeled to obtain a plurality of labeled data, wherein the labeled data comprises clustering data with labeled information; training an integrated recognition network according to the labeled data to obtain a target integrated recognition network; input the second spectral data to be detected into the target integrated recognition network to obtain a plurality of probability values output by the target integrated recognition network, wherein the number of probability value categories is equal to the number of clustering data categories; according to the probability value, the water quality parameter corresponding to the second spectral data is obtained, which solves the technical problem of low detection accuracy and poor anti-interference of the current general water quality detection model in the prior art, and achieves the technical effect of improving the accuracy and anti-interference of water quality detection.

Description

Technical Field

[0001] This disclosure belongs to the field of environmental monitoring technology, and specifically relates to a water quality detection method, device, equipment and storage medium based on spectral technology. Background Technology

[0002] With the rapid development of spectral technology, water quality detection based on spectroscopy has overcome the negative effects of traditional chemical detection methods, offering advantages such as high efficiency, simplicity, and no secondary pollution. Currently, spectral technology is applied to online detection and safety early warning of drinking water, surface water, and industrial wastewater. Under laboratory conditions, it can avoid interference from other substances in the water on the measurement of organic pollutants, greatly increasing the accuracy of detecting specific water quality parameters.

[0003] However, in actual chemical applications, due to the significant differences in the composition of pollutants in different chemical processes and the complexity of the composition of the water being monitored, directly using a general water quality testing model to detect the water quality parameters of different water bodies will result in biases.

[0004] It is evident that existing technologies suffer from the technical problems of low accuracy and poor anti-interference capabilities of commonly used water quality testing models. Summary of the Invention

[0005] This disclosure proposes a water quality detection method, apparatus, equipment, and storage medium based on spectral technology to solve the technical problems of low detection accuracy and poor anti-interference in currently used water quality detection models.

[0006] In a first aspect, this disclosure provides a water quality detection method based on spectral technology, including:

[0007] Clustering of the first spectral data of various water samples yields multi-class clustered data;

[0008] Each cluster of data is labeled to obtain multi-class labeled data, which includes cluster data with label information;

[0009] The target integrated recognition network is obtained by training the integrated recognition network based on the labeled data;

[0010] The second spectral data to be detected is input into the target integrated recognition network to obtain the multi-class probability values ​​output by the target integrated recognition network. The number of classes in the probability values ​​is equal to the number of classes in the clustered data.

[0011] Based on the probability value, obtain the water quality parameters corresponding to the second spectral data.

[0012] Furthermore, clustering was performed on the first spectral data of various water samples, including:

[0013] A preset number of first centroids are randomly obtained based on the first spectral data;

[0014] Based on the distance of the water sample to the first centroid, the water sample is divided into a first category of samples of a predetermined type, wherein the predetermined number is equal to the predetermined number of types, and the predetermined number is less than or equal to the predetermined number of types of water sample.

[0015] Calculate the second centroid of each class of samples;

[0016] Based on the distance of the water sample to the second centroid, the water samples are divided into a second category of samples with a predetermined type.

[0017] Repeat the process of obtaining the first category sample and the second category sample in sequence until the first category sample and the second category sample are the same.

[0018] Furthermore, the ensemble recognition network is trained based on labeled data, including:

[0019] The labeled data is divided into a training dataset and a test dataset;

[0020] The training dataset is input into the ensemble recognition network to obtain the initial training results;

[0021] Based on the relationship between the initial training results and the test dataset, the types, number, and weights of classifiers in the ensemble recognition network are adjusted;

[0022] Repeat the steps of obtaining the initial training results and adjusting the types, number, and weights of classifiers in the ensemble recognition network until the preset performance indicators of the ensemble recognition network meet the preset range.

[0023] Furthermore, the multi-class probability values ​​output by the target ensemble recognition network are obtained, including:

[0024] Obtain the initial probability value corresponding to each water sample output by each classifier in the target integrated recognition network;

[0025] Based on the initial probability values ​​and the weights of each classifier, the probability values ​​of the second spectral data corresponding to each water sample are calculated.

[0026] Furthermore, based on the probability values, the water quality parameters corresponding to the second spectral data are obtained, including:

[0027] Sort the probability values ​​from largest to smallest;

[0028] Select a preset number of first probability values ​​according to the sorting, and obtain the water quality parameters of the water sample corresponding to the first probability value according to the preset rules.

[0029] Calculate the weighted average of the water samples corresponding to the first probability value, and use it as a water quality parameter, where the weight of the water quality parameter is the first probability value.

[0030] Furthermore, the water quality parameters of the water sample corresponding to the first probability value are obtained according to preset rules, including:

[0031] A regression model is trained based on any spectral data in the clustered data and the corresponding water quality parameters.

[0032] Repeat the steps of training the regression model until all spectral data in the clustered data are involved in the training process, resulting in multiple regression models. The number of regression models is equal to the number of clusters in the clustered data.

[0033] The water quality parameters of the water sample corresponding to the first probability value are obtained based on the regression model.

[0034] Secondly, this disclosure provides a water quality detection device based on spectral technology, comprising:

[0035] The clustering module is used to cluster the first spectral data of various water samples to obtain multi-class clustered data;

[0036] The labeling module is used to label each cluster of data to obtain multi-class labeled data, where the labeled data includes cluster data with labeling information;

[0037] The training module is used to train the integrated recognition network based on labeled data to obtain the target integrated recognition network;

[0038] The output module is used to input the second spectral data to be detected into the target integrated recognition network to obtain the multi-class probability values ​​output by the target integrated recognition network, wherein the number of classes of the probability values ​​is equal to the number of classes of the clustering data;

[0039] The acquisition module is used to obtain the water quality parameters corresponding to the second spectral data based on the probability value.

[0040] Furthermore, the clustering module includes a first acquisition unit, a first partitioning unit, a first calculation unit, a second partitioning unit, and a first repeating unit;

[0041] The first acquisition unit is used to randomly acquire a preset number of first centroids based on the first spectral data;

[0042] The first division unit is used to divide the water sample into a first category of water samples according to the distance of the water sample from the first centroid. The preset number is equal to the preset number of categories, and the preset number is less than or equal to the number of categories of water samples.

[0043] The first calculation unit is used to calculate the second centroid of each class of samples.

[0044] The second division unit is used to classify water samples into a second category of preset types according to the distance of the water sample from the second centroid.

[0045] The first repeating unit is used to repeatedly obtain the first category sample and the second category sample in sequence until the first category sample and the second category sample are the same.

[0046] Furthermore, the training module includes a third partitioning unit, an input unit, an adjustment unit, and a second repetition unit;

[0047] The third partitioning unit is used to divide the labeled data into training and testing datasets;

[0048] The input unit is used to input the training dataset into the integrated recognition network to obtain the initial training results;

[0049] The adjustment unit is used to adjust the types, number, and weights of classifiers in the ensemble recognition network based on the relationship between the initial training results and the test dataset;

[0050] The second repetition unit is used to sequentially repeat the steps of obtaining the initial training results and adjusting the types, number, and weights of classifiers in the integrated recognition network until the preset performance indicators of the integrated recognition network meet the preset range.

[0051] Furthermore, the output module includes a second acquisition unit and a second calculation unit;

[0052] The second acquisition unit is used to acquire the initial probability value corresponding to each water sample output by each classifier in the target integrated recognition network.

[0053] The second calculation unit is used to calculate the probability value of the second spectral data corresponding to each water sample based on the initial probability value and the weight of each classifier.

[0054] Furthermore, the acquisition module includes a sorting unit, a third acquisition unit, and a third calculation unit;

[0055] The sorting unit is used to sort the probability values ​​from largest to smallest.

[0056] The third acquisition unit is used to select a preset number of first probability values ​​according to the sorting, and to acquire the water quality parameters of the water sample corresponding to the first probability value according to the preset rules.

[0057] The third calculation unit is used to calculate the weighted average value of the water sample corresponding to the first probability value, which is used as a water quality parameter, wherein the weight of the water quality parameter is the first probability value.

[0058] Furthermore, the third acquisition unit includes a training group, a repeating component, and a first acquisition component;

[0059] The training component is used to train a regression model based on any spectral data in the clustered data and the corresponding water quality parameters.

[0060] The repeating component is used to repeat the steps of training the regression model until all spectral data in the clustered data participates in the training process, resulting in multiple regression models. The number of regression models is equal to the number of clusters in the clustered data.

[0061] The first acquisition component is used to obtain the water quality parameters of the water sample corresponding to the first probability value based on the regression model.

[0062] Thirdly, this disclosure provides an electronic device, including:

[0063] One or more processors;

[0064] Memory, used to store one or more programs;

[0065] When the one or more programs are executed by the one or more processors, the one or more processors implement the water quality detection method based on spectral technology as described in the first aspect.

[0066] Fourthly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the water quality detection method based on spectral technology as described in the first aspect.

[0067] This embodiment of the invention pre-clusters the spectral data of water samples, labels the clustering results, and inputs them into a training ensemble recognition network. The target ensemble recognition network is trained using supervised learning, resulting in higher accuracy and stronger anti-interference capabilities compared to traditional unsupervised models. Using this ensemble recognition network, the probability of the water body belonging to each water sample is obtained, along with the relationship between spectral data and water quality parameters to acquire the water quality parameters of the water body being tested. This solves the technical problems of low accuracy and poor anti-interference in currently used water quality detection models, achieving the technical effect of improving the accuracy and anti-interference of water quality detection.

[0068] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 A schematic flowchart of a water quality detection method based on spectral technology provided in this embodiment of the disclosure;

[0071] Figure 2 This is a schematic diagram of the structure of a water quality detection device based on spectral technology provided in an embodiment of the present disclosure;

[0072] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0074] The following disclosure provides numerous different embodiments or examples for implementing various structures of this disclosure. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this disclosure. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0075] To address the technical problems of low accuracy and poor interference resistance in currently used water quality testing models, this disclosure provides a water quality testing method based on spectral technology, such as... Figure 1 As shown: Figure 1 This is a schematic flowchart of a water quality detection method based on spectral technology provided in an embodiment of the present disclosure. The method includes:

[0076] S101: Cluster the first spectral data of various water samples to obtain multi-class cluster data;

[0077] The water samples can include various water treatment scenarios in actual chemical applications, including but not limited to rainwater from ditches, water from chemical flotation tanks, water from chemical sedimentation tanks, water from chemical regulating tanks, water from the outlets and effluents of sewage treatment plants, aquaculture water, surface water, and agricultural water.

[0078] Specifically, customized sensor terminal equipment containing a UV-Vis spectrometer can be used to perform spectral measurements on water samples to obtain the first spectral data.

[0079] S102: Label each cluster data to obtain multi-class labeled data, wherein the labeled data includes cluster data with label information;

[0080] It should be noted that labeling the spectral data of clustered water samples yields labeled clustered data, which can be used to train water quality detection models. This supervised learning approach allows for the acquisition of more accurate water quality detection models.

[0081] S103: Train the integrated recognition network based on the labeled data to obtain the target integrated recognition network;

[0082] The integrated recognition network can be a general water quality detection model or an integrated model composed of multiple classifiers, including but not limited to Gaussian process classifiers, support vector machine classifiers, and random forest classifiers.

[0083] Specifically, labeled clustered data can be input into an ensemble recognition network composed of multiple classifiers. By analyzing the performance indicators such as the classifier's recognition performance and confidence level for each type of clustered data, the weight, type, and number of classifiers for each classifier can be determined. An ensemble recognition network whose performance indicators meet the preset range can be used as the target ensemble recognition network.

[0084] S104: Input the second spectral data to be detected into the target integrated recognition network to obtain the multi-class probability values ​​output by the target integrated recognition network, wherein the number of classes of the probability values ​​is equal to the number of classes of the clustered data;

[0085] The second spectral data can be the spectral data corresponding to the water body to be tested, and the multi-class probability value can be the probability value of the water body to be tested corresponding to different water body samples.

[0086] Specifically, when the second spectral data is input into the target integrated recognition network, multiple classifiers in the target integrated recognition network will output the probability values ​​of the water body being tested corresponding to different water body samples.

[0087] S105: Based on the probability value, obtain the water quality parameters corresponding to the second spectral data.

[0088] Specifically, a correspondence between water quality parameters and spectral data can be established in advance, and the spectral data of the water body to be tested can be calculated based on the probability value. Then, the water quality parameters of the water body to be tested can be obtained based on the correspondence.

[0089] This embodiment of the invention pre-clusters the spectral data of water samples, labels the clustering results, and inputs them into a training ensemble recognition network. The target ensemble recognition network is trained using supervised learning, resulting in higher accuracy and stronger anti-interference capabilities compared to traditional unsupervised models. Using this ensemble recognition network, the probability of the water body belonging to each water sample is obtained, along with the relationship between spectral data and water quality parameters to acquire the water quality parameters of the water body being tested. This solves the technical problems of low accuracy and poor anti-interference in currently used water quality detection models, achieving the technical effect of improving the accuracy and anti-interference of water quality detection.

[0090] In one optional embodiment, clustering is performed on the first spectral data of multiple water samples, including:

[0091] A preset number of first centroids are randomly obtained based on the first spectral data;

[0092] Optionally, unsupervised clustering can be performed on multiple water samples first, and then a preset number of first centroids can be randomly selected.

[0093] Based on the distance of the water sample to the first centroid, the water sample is divided into a first category of samples of a predetermined type, wherein the predetermined number is equal to the predetermined number of types, and the predetermined number is less than or equal to the predetermined number of types of water sample.

[0094] Optionally, the distance from each water sample to the first centroid is calculated, and the water sample closest to each first centroid is classified into a class associated with that first centroid.

[0095] It should be noted that, assuming the clustering results are reasonable, the number of first centroids should be less than the number of water sample species to simplify the data categories and avoid problems such as low accuracy, poor generalization ability, and insufficient robustness of the regression model due to insufficient sample numbers.

[0096] Calculate the second centroid of each class of samples;

[0097] Based on the distance of the water sample to the second centroid, the water samples are divided into a second category of samples with a predetermined type.

[0098] Repeat the process of obtaining the first category sample and the second category sample in sequence until the first category sample and the second category sample are the same.

[0099] Specifically, if two consecutive clustering results are the same, it indicates that the clustering is successful; if they are different, the steps of obtaining the first and second category samples are repeated.

[0100] In one alternative embodiment, before clustering the water samples, the individual UV-Vis spectral data can be preprocessed and wavelength selected. Specifically, preprocessing methods such as wavelet smoothing and SG smoothing can be used to reduce interference signals during the spectral acquisition process. Cross-validation methods can be used to select the wavelength, but this disclosure does not limit this.

[0101] In one alternative embodiment, training the integrated recognition network based on labeled data includes:

[0102] The labeled data is divided into a training dataset and a test dataset;

[0103] It should be noted that the labeled data should be sampled from random sampling, and the amount of labeled data should be as large as possible to ensure that the characteristics and parameters of the spectral data corresponding to each type of water sample are learned.

[0104] The training dataset is input into the ensemble recognition network to obtain the initial training results;

[0105] Based on the relationship between the initial training results and the test dataset, the types, number, and weights of classifiers in the ensemble recognition network are adjusted;

[0106] The classifier can be set to be changeable. The classifiers included in the ensemble recognition network can have different performance and be unrelated to each other, including but not limited to random forest, support vector machine, logistic regression, etc.

[0107] For example, since different classifiers calculate class probabilities differently, taking logistic regression and support vector machines as examples, when the vector of spectral data of water samples is input into the logistic regression model, the logistic regression model calculates the score for each type of water sample. The calculation method is as follows: s m =ω m T x+b m m = 1, 2, ..., M, s m It is the score for each water sample, where ω m T It is the feature weight vector, b m It's a bias, where the score of vector x is converted into a probability by the Softmax function: y mFor the transformation result, j = 1,...,M, P(Y = m|x) represents the probability. The Softmax function guarantees that the sum of all probabilities is 1. Therefore, for each input vector x, a probability distribution P can be obtained, representing the probability that the spectral data belongs to each water sample. Alternatively, a Support Vector Machine (SVM) can be chosen as the classifier. SVM is a boundary distance-based classification algorithm, unlike probabilistic models, and does not directly output probabilities. The Platt scaling method can be used to train a logistic regression model to fit the output of the SVM to obtain the probability of each class. The output of the SVM can be viewed as a score indicating whether a sample belongs to a positive example, and then the logistic regression model can be used to fit the score to obtain the probability of each class. It should be noted that the calculation of class probabilities depends on the classifier used. Regardless of the algorithm used, the calculation process converts the output of a linear or nonlinear model into probability values.

[0108] Specifically, the trained ensemble recognition network is evaluated using a test dataset, and the classifiers in the ensemble recognition network are adjusted based on the evaluation results. The adjustment methods include, but are not limited to, adjusting the types, weights, number of classifiers, and hyperparameters of the classifiers.

[0109] Repeat the steps of obtaining the initial training results and adjusting the types, number, and weights of classifiers in the ensemble recognition network until the preset performance indicators of the ensemble recognition network meet the preset range.

[0110] The preset performance metrics include, but are not limited to, accuracy, precision, recall, and F1 score.

[0111] Specifically, grid search can be used to determine the optimal weights for each classifier. A set of possible weight values ​​can be pre-defined for the classifiers, and cross-validation can be used to test different combinations until the optimal weight combination is found. Alternatively, a preset performance index can be calculated based on the recognition results of the object ensemble recognition network, and the training process can be stopped when the preset performance index meets a preset range.

[0112] In one optional embodiment, obtaining the multi-class probability values ​​output by the target ensemble recognition network includes:

[0113] Obtain the initial probability value corresponding to each water sample output by each classifier in the target integrated recognition network;

[0114] Based on the initial probability values ​​and the weights of each classifier, the probability values ​​of the second spectral data corresponding to each water sample are calculated.

[0115] For example, when the spectral data of a water sample is input into a target ensemble recognition network, the network outputs probability values ​​for M types of water samples from N classifiers. The weighted average of these N classifiers' M probability values ​​yields the weighted probability value of the water sample to be detected. This calculation process can be expressed as: P m_weighted =w1*P 1m +w2*P 2m +w3*P 3m +...+w N-1 *P N-1m +w N *P Nm , where P m_weighted P represents the weighted probability value. Nm w represents the probability value output by the Nth classifier for the m-th water sample. N is the weight value of the Nth classifier.

[0116] In one optional embodiment, obtaining the water quality parameters corresponding to the second spectral data based on the probability value includes:

[0117] Sort the probability values ​​from largest to smallest;

[0118] Select a preset number of first probability values ​​according to the sorting, and obtain the water quality parameters of the water sample corresponding to the first probability value according to the preset rules.

[0119] Calculate the weighted average of the water samples corresponding to the first probability value, and use it as a water quality parameter, where the weight of the water quality parameter is the first probability value.

[0120] For example, the obtained M weighted probability values ​​can be sorted, for instance, as follows: P3_weighted>P4_weighted>P2_weighted>...>PM_weighted. This indicates that the probability value of the third type of water sample is greater than that of the fourth type, and the probability value of the fourth type is greater than that of the second type. The water body to be tested can be identified as the water sample corresponding to the highest probability, and the water quality parameters of the water sample corresponding to the highest probability can be obtained according to preset rules. Alternatively, the top three probability values ​​can be obtained, such as... P3_weighted、P4_weighted、P2_weighted The probability values ​​of the third, fourth, and second water samples are used as the basis for determining the water quality parameters of the samples corresponding to the top three probability values. These top three probability values ​​are then used as weights for the corresponding water samples. A weighted average of the water quality parameters of the samples corresponding to the top three probability values ​​is then calculated to obtain the water quality parameters of the water body to be tested. The specific calculation method is as follows: Wherein, COD3 represents the water quality parameter of the third type of water sample, COD4 represents the water quality parameter of the fourth type of water sample, COD2 represents the water quality parameter of the second type of water sample, and COD-final represents the water quality parameter of the water body to be tested. The calculation method of the water quality parameter of the water body to be tested can also be set according to the specific scenario, and the water sample can be combined and optimized by combining indicators such as the standard error of prediction (SEP). This disclosure does not impose any restrictions on this.

[0121] In one optional embodiment, the water quality parameters of the water sample corresponding to the first probability value are obtained according to a preset rule, including:

[0122] A regression model is trained based on any spectral data in the clustered data and the corresponding water quality parameters.

[0123] Repeat the steps of training the regression model until all spectral data in the clustered data are involved in the training process, resulting in multiple regression models. The number of regression models is equal to the number of clusters in the clustered data.

[0124] The water quality parameters of the water sample corresponding to the first probability value are obtained based on the regression model.

[0125] Specifically, by establishing a correlation between the spectral data of water samples and water quality parameters, multiple regression models can be established for the calculation of water quality parameters. This allows for the acquisition of water quality parameters based on the spectral data calculated from the probability values ​​output by the target integrated recognition network in the case of unknown samples.

[0126] For example, the regression model can be a partial least squares regression model. The goal of the least squares regression model is to identify the relationship between the independent and dependent variables as a small number of latent variables. These latent variables are extracted from the spectral data and water quality parameters through a linear combination method and used as inputs to the regression model for predicting the water quality parameters of the sample.

[0127] It should be noted that, compared with the prior art of training regression models based on the spectral data of all water samples, the embodiments of this disclosure first cluster the spectral data obtained from different scenarios and then establish a regression model. This can ensure the generalization ability of the regression model while retaining more potential variables, thereby making the regression model more accurate.

[0128] Based on the same concept, this disclosure provides a water quality detection device based on spectral technology, such as... Figure 2 As shown: Figure 2 This is a schematic diagram of a water quality detection device based on spectral technology provided in an embodiment of the present disclosure. The device includes:

[0129] Clustering module 201 is used to cluster the first spectral data of multiple water samples to obtain multi-class cluster data;

[0130] The labeling module 202 is used to label each cluster data to obtain multi-class labeled data, wherein the labeled data includes cluster data with labeling information;

[0131] Training module 203 is used to train the integrated recognition network based on labeled data to obtain the target integrated recognition network;

[0132] Output module 204 is used to input the second spectral data to be detected into the target integrated recognition network to obtain multi-class probability values ​​output by the target integrated recognition network, wherein the number of classes of the probability values ​​is equal to the number of classes of the clustered data;

[0133] The acquisition module 205 is used to acquire the water quality parameters corresponding to the second spectral data based on the probability value.

[0134] This embodiment of the invention pre-clusters the spectral data of water samples, labels the clustering results, and inputs them into a training ensemble recognition network. The target ensemble recognition network is trained using supervised learning, resulting in higher accuracy and stronger anti-interference capabilities compared to traditional unsupervised models. Using this ensemble recognition network, the probability of the water body belonging to each water sample is obtained, along with the relationship between spectral data and water quality parameters to acquire the water quality parameters of the water body being tested. This solves the technical problems of low accuracy and poor anti-interference in currently used water quality detection models, achieving the technical effect of improving the accuracy and anti-interference of water quality detection.

[0135] Furthermore, the clustering module 201 includes a first acquisition unit, a first partitioning unit, a first calculation unit, a second partitioning unit, and a first repeating unit;

[0136] The first acquisition unit is used to randomly acquire a preset number of first centroids based on the first spectral data;

[0137] The first division unit is used to divide the water sample into a first category of water samples according to the distance of the water sample from the first centroid. The preset number is equal to the preset number of categories, and the preset number is less than or equal to the number of categories of water samples.

[0138] The first calculation unit is used to calculate the second centroid of each class of samples.

[0139] The second division unit is used to classify water samples into a second category of preset types according to the distance of the water sample from the second centroid.

[0140] The first repeating unit is used to repeatedly obtain the first category sample and the second category sample in sequence until the first category sample and the second category sample are the same.

[0141] Furthermore, the training module 203 includes a third partitioning unit, an input unit, an adjustment unit, and a second repetition unit;

[0142] The third partitioning unit is used to divide the labeled data into training and testing datasets;

[0143] The input unit is used to input the training dataset into the integrated recognition network to obtain the initial training results;

[0144] The adjustment unit is used to adjust the types, number, and weights of classifiers in the ensemble recognition network based on the relationship between the initial training results and the test dataset;

[0145] The second repetition unit is used to sequentially repeat the steps of obtaining the initial training results and adjusting the types, number, and weights of classifiers in the integrated recognition network until the preset performance indicators of the integrated recognition network meet the preset range.

[0146] Furthermore, the output module 204 includes a second acquisition unit and a second calculation unit;

[0147] The second acquisition unit is used to acquire the initial probability value corresponding to each water sample output by each classifier in the target integrated recognition network.

[0148] The second calculation unit is used to calculate the probability value of the second spectral data corresponding to each water sample based on the initial probability value and the weight of each classifier.

[0149] Furthermore, the acquisition module 205 includes a sorting unit, a third acquisition unit, and a third calculation unit;

[0150] The sorting unit is used to sort the probability values ​​from largest to smallest.

[0151] The third acquisition unit is used to select a preset number of first probability values ​​according to the sorting, and to acquire the water quality parameters of the water sample corresponding to the first probability value according to the preset rules.

[0152] The third calculation unit is used to calculate the weighted average value of the water sample corresponding to the first probability value, which is used as a water quality parameter, wherein the weight of the water quality parameter is the first probability value.

[0153] Furthermore, the third acquisition unit includes a training component, a repetition component, and a first acquisition component;

[0154] The training component is used to train a regression model based on any spectral data in the clustered data and the corresponding water quality parameters.

[0155] The repeating component is used to repeat the steps of training the regression model until all spectral data in the clustered data participates in the training process, resulting in multiple regression models. The number of regression models is equal to the number of clusters in the clustered data.

[0156] The first acquisition component is used to obtain the water quality parameters of the water sample corresponding to the first probability value based on the regression model.

[0157] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The present disclosure also provides a device including a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other via the communication bus 304.

[0159] Memory 303 is used to store computer programs.

[0160] In one embodiment of this disclosure, when the processor 301 executes the program stored in the memory 303, it implements a water quality detection method based on spectral technology provided in any of the foregoing method embodiments, including:

[0161] Clustering of the first spectral data of various water samples yields multi-class clustered data;

[0162] Each cluster of data is labeled to obtain multi-class labeled data, which includes cluster data with label information;

[0163] The target integrated recognition network is obtained by training the integrated recognition network based on the labeled data;

[0164] The second spectral data to be detected is input into the target integrated recognition network to obtain the multi-class probability values ​​output by the target integrated recognition network. The number of categories of the probability values ​​is equal to the number of types of water samples.

[0165] Based on the probability value, obtain the water quality parameters corresponding to the second spectral data.

[0166] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a water quality detection method based on spectral technology as provided in any of the foregoing method embodiments, including:

[0167] Clustering of the first spectral data of various water samples yields multi-class clustered data;

[0168] Each cluster of data is labeled to obtain multi-class labeled data, which includes cluster data with label information;

[0169] The target integrated recognition network is obtained by training the integrated recognition network based on the labeled data;

[0170] The second spectral data to be detected is input into the target integrated recognition network to obtain the multi-class probability values ​​output by the target integrated recognition network. The number of categories of the probability values ​​is equal to the number of types of water samples.

[0171] Based on the probability value, obtain the water quality parameters corresponding to the second spectral data.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0173] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0174] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A water quality detection method based on spectral technology, characterized in that, include: Clustering of the first spectral data of various water samples yields multi-class clustered data; Each cluster of data is labeled to obtain multi-class labeled data, wherein the labeled data includes the cluster data with labeling information; The integrated recognition network is trained based on the labeled data to obtain the target integrated recognition network; The second spectral data to be detected is input into the target integrated recognition network to obtain multi-class probability values ​​output by the target integrated recognition network, wherein the number of categories of the probability values ​​is equal to the number of categories of the clustering data; Based on the probability value, obtain the water quality parameters corresponding to the second spectral data; The step of training the integrated recognition network based on the labeled data includes: The labeled data is divided into a training dataset and a test dataset; The training dataset is input into the integrated recognition network to obtain the initial training results; Based on the relationship between the initial training results and the test dataset, the types, number, and weights of classifiers in the integrated recognition network are adjusted; Repeat the steps of obtaining the initial training results and adjusting the types, number, and weights of classifiers in the integrated recognition network until the preset performance index of the integrated recognition network meets the preset range. The process of obtaining the multi-class probability values ​​output by the target integrated recognition network includes: Obtain the initial probability value corresponding to each type of water sample output by each classifier in the target integrated recognition network; Based on the initial probability value and the weight of each classifier, the probability value of the second spectral data corresponding to each of the water samples is calculated; The step of obtaining the water quality parameters corresponding to the second spectral data based on the probability value includes: The probability values ​​are sorted from largest to smallest; Select a preset number of first probability values ​​according to the sorting, and obtain the water quality parameters of the water sample corresponding to the first probability value according to the preset rules. Calculate the weighted average of the water samples corresponding to the first probability value, and use it as the water quality parameter, wherein the weight of the water quality parameter is the first probability value.

2. The method according to claim 1, characterized in that, The clustering of the first spectral data of multiple water samples includes: A preset number of first centroids are randomly obtained based on the first spectral data; Based on the distance of the water sample to the first centroid, the water sample is divided into a first category of preset types of samples, wherein the preset number is equal to the preset number of types, and the preset number is less than or equal to the number of types of the water sample. Calculate the second centroid of the samples in each class of classification; Based on the distance of the water sample to the second centroid, the water sample is divided into a second category of preset types. Repeat the process of obtaining the first-classified sample and the second-classified sample in sequence until the first-classified sample and the second-classified sample are the same.

3. The method according to claim 1, characterized in that, The step of obtaining the water quality parameters of the water sample corresponding to the first probability value according to a preset rule includes: A regression model is trained based on any spectral data in the clustered data and the corresponding water quality parameters. The steps of training the regression model are repeated until all spectral data in the clustered data participate in the training process, resulting in multiple regression models, wherein the number of regression models is equal to the number of categories in the clustered data; The water quality parameters of the water sample corresponding to the first probability value are obtained based on the regression model.

4. An apparatus for implementing the water quality detection method based on spectral technology according to any one of claims 1-3, characterized in that, include: The clustering module is used to cluster the first spectral data of various water samples to obtain multi-class clustered data; A labeling module is used to label each type of clustered data to obtain multi-class labeled data, wherein the labeled data includes the clustered data with labeling information; The training module is used to train the integrated recognition network based on the labeled data to obtain the target integrated recognition network; The output module is used to input the second spectral data to be detected into the target integrated recognition network to obtain multi-class probability values ​​output by the target integrated recognition network, wherein the number of categories of the probability values ​​is equal to the number of categories of the clustering data; The acquisition module is used to acquire the water quality parameters corresponding to the second spectral data based on the probability value.

5. The apparatus according to claim 4, characterized in that, The clustering module includes a first acquisition unit, a first partitioning unit, a first calculation unit, a second partitioning unit, and a first repeating unit; The first acquisition unit is used to randomly acquire a preset number of first centroids based on the first spectral data; The first division unit is used to divide the water sample into a first category of preset types of samples according to the distance of the water sample from the first centroid, wherein the preset number is equal to the preset number of types, and the preset number is less than or equal to the number of types of the water sample. The first calculation unit is used to calculate the second centroid of each class of the classified samples; The second division unit is used to divide the water sample into a second category of preset types of samples according to the distance of the water sample to the second centroid. The first repeating unit is used to repeatedly acquire the first classified sample and the second classified sample in sequence until the first classified sample and the second classified sample are the same.

6. The apparatus according to claim 4, characterized in that, The training module includes a third partitioning unit, an input unit, an adjustment unit, and a second repetition unit; The third partitioning unit is used to partition the labeled data into a training dataset and a test dataset; The input unit is used to input the training dataset into the integrated recognition network to obtain the initial training results; The adjustment unit is used to adjust the types, number, and weights of classifiers in the integrated recognition network according to the relationship between the initial training results and the test dataset; The second repeating unit is used to sequentially repeat the steps of obtaining the initial training result and adjusting the types, number, and weights of classifiers in the integrated recognition network until the preset performance index of the integrated recognition network meets the preset range.

7. The apparatus according to claim 4, characterized in that, The output module includes a second acquisition unit and a second calculation unit; The second acquisition unit is used to acquire the initial probability value output by each classifier in the target integrated recognition network, corresponding to each type of water sample; The second calculation unit is used to calculate the probability value of the second spectral data corresponding to each of the water samples based on the initial probability value and the weight of each of the classifiers.

8. The apparatus according to claim 7, characterized in that, The acquisition module includes a sorting unit, a third acquisition unit, and a third calculation unit; The sorting unit is used to sort the probability values ​​from largest to smallest; The third acquisition unit is used to select a preset number of first probability values ​​according to the sorting, and to acquire the water quality parameters of the water sample corresponding to the first probability value according to the preset rules. The third calculation unit is used to calculate the weighted average value of the water sample corresponding to the first probability value, as the water quality parameter, wherein the weight of the water quality parameter is the first probability value.

9. The apparatus according to claim 8, characterized in that, The third acquisition unit includes a training component, a repetition component, and a first acquisition component; The training component is used to train a regression model based on any spectral data in the clustered data and the water quality parameters corresponding to the spectral data. The repeating component is used to repeat the steps of training the regression model until all spectral data in the clustered data participates in the training process, resulting in multiple regression models, wherein the number of regression models is equal to the number of categories in the clustered data. The first acquisition component is used to acquire the water quality parameters of the water sample corresponding to the first probability value based on the regression model.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the water quality detection method based on spectral technology as described in any one of claims 1-3.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the water quality detection method based on spectral technology as described in any one of claims 1-3.

Images