Water body monitoring abnormal state recognition model training method

Abstract

The invention discloses a water body monitoring abnormal state recognition model training method. The method comprises the steps of obtaining original water quality monitoring data; carrying out noise reduction processing on the original water quality monitoring data by adopting wavelet transform to obtain reference data; training a pre-constructed neural network model by using the reference data set, calculating a difference value between a water quality prediction result output by the neural network model and a water quality result of an actually measured water sample, and taking meteorological data as a boundary condition of the neural network model; taking the data of which the difference value exceeds the identification threshold value as abnormal data, removing the abnormal data to form new reference data, and inputting the new reference data into the model again; repeating the steps until all the abnormal data are removed, and obtaining a trained model for water quality abnormity prediction; according to the method, water environment change caused by meteorological factor change is reduced, so that the accuracy of the model in identifying water abnormity is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality monitoring, and in particular to a water body monitoring abnormal state recognition model training method. Background Art

[0002] As the surface water environment continues to improve, the overall water quality of rivers improves, and people pay more and more attention to the water ecological environment, it is necessary to adopt more effective and refined monitoring methods for some exogenous organic pollution incidents that still exist. For the detection and identification of organic pollutants in water bodies, my country currently mainly uses non-specific indicators such as chemical oxygen demand (COD), total organic carbon (TOC), ultraviolet absorbance at 254nm (UV254) and conventional five parameters for water quality monitoring. However, since aromatic compounds and organic pollutants with complex structures are difficult to be completely oxidized, it is difficult to fully reflect the degree of pollution of pollutants in the water by relying solely on the above indicators, and it is difficult to distinguish the types of pollutants. For example, in 2021, the water quality monitoring data of a water source in a certain city showed no abnormalities, but the tap water had an obvious odor, and the traceability analysis showed that organic matter such as 2-ethylhexanol and methyl benzoate exceeded the standard.

[0003] At present, the specific detection and analysis of organic pollutants mainly include electrochemical analysis, chromatography, spectroscopy, etc. Among them, three-dimensional fluorescence spectroscopy has certain advantages in pollutant identification due to its small sample preparation requirements, simple experimental conditions, high sensitivity, fast detection speed, and good selectivity. However, due to the complexity of water bodies in rivers and pipe networks, coupled with problems such as background interference from natural dissolved organic matter and overlapping of fluorescent powders of water-soluble organic matter, it is not applicable enough in the identification of low-concentration pollutants in water bodies.

[0004] In addition, alarm identification based on the existing standard system does not effectively consider the influence of factors such as temperature, humidity, light and rainfall in the environment area, and adopts a one-size-fits-all approach, which is not conducive to the refined supervision of relevant regulatory departments. In the related art, a water quality monitoring method based on EMD and improved LSTM is proposed in the patent application document with publication number CN115563487A. The scheme obtains data such as social indicators, meteorological indicators, water quantity indicators and water quality indicators, and uses EMD (empirical mode decomposition) to process water quality indicator data, optimizes the parameters of LSTM neural network by S-SSA (improved adaptive least squares algorithm), divides time series data into multiple sub-LSTM networks and trains them; however, the scheme ignores that water quality data contains noise interference, which affects the accuracy and stability of model training. Moreover, the scheme divides time series data into multiple sub-LSTM networks for training. Although this method may be able to better capture the different dynamic characteristics of time series, it does not perform self-training and parameter correction on the model, and the adaptability and prediction accuracy of the model are poor. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to improve the accuracy of identifying abnormal water quality.

[0006] The present invention solves the above technical problems by the following technical means:

[0007] The present invention proposes a water body monitoring abnormal state recognition model training method, the method comprising:

[0008] S10, obtaining original water quality monitoring data, wherein the original water quality monitoring data includes water quality monitoring data, spectral data and mass spectral data;

[0009] S20, performing noise reduction processing on the original water quality monitoring data by using wavelet transform to obtain benchmark data;

[0010] S30, using the benchmark data set to train a pre-built neural network model, and calculating the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample, wherein the meteorological data is used as the boundary condition of the neural network model;

[0011] S40, removing the data whose difference exceeds the recognition threshold as abnormal data to form new benchmark data, and re-execute step S30;

[0012] S50: After all abnormal data are eliminated, normal training data is obtained to continue training the water quality abnormality prediction model.

[0013] Furthermore, the spectral data includes ultraviolet-visible absorption spectral data and three-dimensional fluorescence spectral data.

[0014] Furthermore, before the wavelet transform is used to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, the method further includes:

[0015] Performing a cleaning operation on the original water quality monitoring data to obtain pre-processed water quality monitoring data;

[0016] Accordingly, the wavelet transform is used to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, including:

[0017] Wavelet transform is used to reduce the noise of the pre-processed water quality monitoring data to obtain the benchmark data.

[0018] Furthermore, the neural network model adopts an LSTM neural network model.

[0019] Furthermore, after the original water quality monitoring data is subjected to noise reduction processing by wavelet transform to obtain the benchmark data, the method further comprises:

[0020] Integrate the benchmark data according to long-term and short-term time periods to form a time series data set;

[0021] Accordingly, the time series data set is input into a pre-built neural network model, and the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample is calculated.

[0022] Furthermore, the meteorological data includes rainfall, temperature, sunlight and dew point.

[0023] Furthermore, before removing the data whose difference exceeds the recognition threshold as abnormal data to form new benchmark data, the method further includes:

[0024] The identification threshold is determined according to standard water sample data under normal conditions.

[0025] In addition, the present invention also proposes a method for identifying abnormal states of water body monitoring, and uses the model trained by the water body monitoring abnormal state identification model training method as described above to perform water body anomaly identification.

[0026] In addition, the present invention also proposes a computer device, which includes a memory and a processor; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method as described above.

[0027] In addition, the present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0028] The advantages of the present invention are:

[0029] (1) The present invention introduces meteorological data as the boundary condition of the neural network model, incorporates the impact of meteorological factors on water quality into the water environment and water quality, and uses it as a normalized judgment. It uses the water quality prediction model to identify abnormal data points, and improves the water body data baseline value by eliminating abnormal data, reducing the changes in the water environment caused by changes in meteorological factors, thereby improving the accuracy of the model in identifying water body anomalies.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a method for training a water body monitoring abnormal state recognition model according to an embodiment of the present invention;

[0032] Figure 2 It is a principle block diagram of the training of the water body monitoring abnormal state recognition model in one embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of comparison between test data and benchmark data in one embodiment of the present invention;

[0034] Figure 4 It is a structural diagram of the LSTM neural network model in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 1 to Figure 2 As shown, an embodiment of the present invention proposes a water body monitoring abnormal state recognition model training method, the method comprising the following steps:

[0037] S10, obtaining original water quality monitoring data, wherein the original water quality monitoring data includes water quality monitoring data, spectral data and mass spectral data;

[0038] It should be noted that this embodiment can collect original water quality monitoring data by using common detection instruments to obtain original water sample data, wherein the water quality monitoring data is conventional water quality monitoring data, the spectral data includes ultraviolet-visible absorption spectral data and three-dimensional fluorescence spectral data, and the mass spectrum data is small mass spectrum data.

[0039] S20, performing noise reduction processing on the original water quality monitoring data by using wavelet transform to obtain benchmark data;

[0040] It should be noted that this embodiment uses wavelet transform to process data to reduce data noise and improve data quality.

[0041] S30, using the benchmark data set to train a pre-built neural network model, and calculating the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample, wherein the meteorological data is used as the boundary condition of the neural network model;

[0042] It should be noted that the meteorological data in this embodiment includes climate elements such as rainfall, temperature, dew point, and season. By using rainfall, temperature, light, dew point, etc. as the boundary conditions of the model, abnormal data in the water quality data can be eliminated. Using multiple meteorological factors such as rainfall, temperature, light, dew point, etc. as boundary conditions, a comprehensive prediction model is provided that can more accurately reflect complex water quality changes.

[0043] S40, removing the data whose difference exceeds the recognition threshold as abnormal data to form new benchmark data, and re-execute step S30;

[0044] It should be noted that the recognition threshold is an empirical value determined based on standard water sample data under normal conditions (not affected by rainfall, temperature, etc.) and is used for comparison with the model prediction results.

[0045] It should be noted that if the difference is less than the recognition threshold, the baseline water sample data is data with a low degree of interference from rainfall and temperature, and subsequent detection can be carried out; if the difference is greater than the recognition threshold, the abnormal data is removed, and then the new baseline water sample data after removing the abnormal data is again predicted by the model, and the corresponding water sample data is obtained, and this water sample data is compared with the recognition threshold again to obtain the result. This cycle is repeated to obtain water sample data within the recognition threshold range. In the subsequent detection process of the water sample data, the water sample data is less affected by external interference, thereby providing accuracy of detection.

[0046] S50: After all abnormal data are eliminated, normal training data is obtained to continue training the water quality abnormality prediction model.

[0047] It should be noted that this embodiment introduces meteorological data as the boundary condition of the neural network model, incorporates the impact of meteorological factors on water quality into the water environment and water quality, and uses it as a normalized judgment, uses the water quality prediction model to identify abnormal data points, and improves the water body data baseline value by eliminating abnormal data. For comparison between the original data and the baseline data, see Figure 3 ,The model performs self-training and parameter correction after detecting abnormal data. This dynamic adjustment mechanism helps to improve the adaptability and prediction accuracy of the model, thereby reducing the changes in the water environment caused by changes in meteorological factors, and improving the accuracy of the model in identifying water anomalies.

[0048] As a further preferred technical solution, before the step S20: using wavelet transform to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, the method further includes the following steps:

[0049] Performing a cleaning operation on the original water quality monitoring data to obtain pre-processed water quality monitoring data;

[0050] Accordingly, the wavelet transform is used to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, including:

[0051] Wavelet transform is used to reduce the noise of the pre-processed water quality monitoring data to obtain the benchmark data.

[0052] It should be noted that this embodiment effectively reduces the noise in the data through data cleaning and wavelet transformation, improves the data quality, and enhances the stability and accuracy of the model.

[0053] As a further preferred technical solution, Figure 4 As shown, the neural network model adopts the LSTM neural network model. LSTM (Long Short-Temm Memory) is a special recurrent neural network (RNN) that controls the flow of information by introducing a gate mechanism, so that it can process and predict important events with long time intervals.

[0054] Furthermore, in the step S20: after the original water quality monitoring data is subjected to noise reduction processing by wavelet transform to obtain the reference data, the method further comprises the following steps:

[0055] Integrate the benchmark data according to long-term and short-term time periods to form a time series data set;

[0056] Accordingly, the time series data set is input into a pre-built neural network model, and the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample is calculated.

[0057] It should be noted that, in this embodiment, the long term refers to months and weeks, and the short term refers to hours, minutes or even seconds. The benchmark data is organized into a collection of all test data for a month, a collection of all test data for a week, a collection of data for a day, etc. in time series to adapt to the characteristics of the LSTM neural network model.

[0058] In addition, another embodiment of the present invention further proposes a method for identifying abnormal states of water body monitoring, which uses a model trained by the water body monitoring abnormal state identification model training method described in the above embodiment to identify water body anomalies.

[0059] In addition, another embodiment of the present invention also proposes a computer device, which includes a memory and a processor; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the water body monitoring abnormal state recognition model training method as described above.

[0060] In addition, another embodiment of the present invention further proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the water body monitoring abnormal state recognition model training method as described above is implemented.

[0061] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0062] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0063] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0064] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0065] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A water body monitoring abnormal state recognition model training method, characterized in that: The method comprises: S10, obtaining original water quality monitoring data, wherein the original water quality monitoring data includes water quality monitoring data, spectral data and mass spectral data; S20, performing noise reduction processing on the original water quality monitoring data by using wavelet transform to obtain benchmark data; S30, using the benchmark data set to train a pre-built neural network model, and calculating the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample, wherein the meteorological data is used as the boundary condition of the neural network model; S40, removing the data whose difference exceeds the recognition threshold as abnormal data to form new benchmark data, and re-execute step S30; S50: After all abnormal data are eliminated, normal training data is obtained to continue training the water quality abnormality prediction model.

2. The water body monitoring abnormal state recognition model training method according to claim 1, characterized in that: The spectrum data includes ultraviolet-visible absorption spectrum data and three-dimensional fluorescence spectrum data.

3. The water body monitoring abnormal state recognition model training method according to claim 1, characterized in that: Before the wavelet transform is used to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, the method further includes: Performing a cleaning operation on the original water quality monitoring data to obtain pre-processed water quality monitoring data; Accordingly, the wavelet transform is used to perform noise reduction processing on the original water quality monitoring data to obtain the benchmark data, including: Wavelet transform is used to reduce the noise of the pre-processed water quality monitoring data to obtain the benchmark data.

4. The water body monitoring abnormal state recognition model training method according to claim 1, characterized in that: The neural network model adopts the LSTM neural network model.

5. The water body monitoring abnormal state recognition model training method according to claim 4, characterized in that: After the original water quality monitoring data is subjected to noise reduction processing by wavelet transform to obtain the benchmark data, the method further comprises: Integrate the benchmark data according to long-term and short-term time periods to form a time series data set; Accordingly, the time series data set is input into a pre-built neural network model, and the difference between the water quality prediction result output by the neural network model and the water quality result of the measured water sample is calculated.

6. The water body monitoring abnormal state recognition model training method according to claim 1, characterized in that: The meteorological data include rainfall, temperature, sunshine and dew point.

7. The water body monitoring abnormal state recognition model training method according to claim 1, characterized in that: Before removing the data whose difference exceeds the recognition threshold as abnormal data and forming new benchmark data, the method further includes: The identification threshold is determined according to standard water sample data under normal conditions.

8. A method for identifying abnormal conditions of water body monitoring, characterized in that: The model trained by the water body monitoring abnormal state recognition model training method as described in any one of claims 1 to 7 is used to recognize water body anomalies.

9. A computer device, characterized in that: The device includes a memory and a processor; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images