Method, system, equipment and medium for detecting solution impurities using water quality detector

Abstract

The present application relates to the field of water quality detection, and in particular to a method, system, device and medium for detecting solution impurities using a water quality detector. The method comprises: obtaining spectral data of an ion solution to be tested; constructing a correction observation window based on the local concentration gradient on the spectral data to extract local nonlinear features, the local nonlinear features including absorbance gradient variation features and fitting residual anomaly features; establishing a mapping relationship between the spectral data and the water quality detection results based on the local nonlinear features within the correction observation window, and generating a correction coefficient through a scattering correction algorithm; correcting the spectral data based on the mapping relationship and the correction coefficient; and identifying the impurity components in the ion solution based on the corrected spectral data. A stable mapping relationship between absorbance and impurity concentration can be maintained even in a strong ion interference environment, so that the measurement results can maintain reliable accuracy even in a high-concentration ion strength environment.

Description

Technical Field

[0001] The present application relates to the field of water quality testing, and in particular to a method, system, equipment and medium for detecting solution impurities using a water quality detector. Background Art

[0002] In the field of spectroscopic water quality testing, especially impurity analysis in high-concentration ion solutions, traditional methods have long faced a fundamental challenge: when the concentration of the ion solution being tested is too high, the strong interactions between the ions in the solution can trigger local nonlinear optical effects, causing the relationship between spectral absorbance and impurity concentration to deviate significantly from the linear relationship. This distortion phenomenon in high-concentration environments manifests itself on two levels: first, microscopic inhomogeneities caused by ion clustering cause the spectral curve to exhibit steep gradient jumps in specific wavelength bands; second, scattering effects and baseline drift cause superimposed interference in localized regions, resulting in residual anomalies that are difficult for traditional methods to capture.

[0003] Existing mainstream technologies, such as multivariate scatter correction (MSC) or global spectral regression models, essentially rely on the uniformity and linear response assumptions of the solution system. They perform overall correction or fitting of the full-band spectrum, but are unable to distinguish between nonlinear distortion in local areas and true impurity signals. This one-size-fits-all approach is bound to fail in high-concentration scenarios: global correction will over-smooth key characteristic peaks, while regression models based on the full spectrum will exhibit systematic deviations due to interference from local distortion points, ultimately leading to a significant decrease in impurity identification. More seriously, as the concentration increases, these distorted areas will show a dynamic diffusion trend. Traditional static algorithms can neither locate the source of distortion nor adaptively adjust the correction intensity, making the measurement results lose reference value in the critical high-concentration range. Summary of the Invention

[0004] In order to solve the problem that the existing technology cannot perform water quality testing on high-concentration ion solutions, the present application provides a method, system, equipment and medium for detecting impurities in a water quality detector. In a first aspect, the method provided by the present application includes the following steps:

[0005] Acquiring spectral data of the ion solution to be tested, wherein the spectral data includes wavelength and absorbance;

[0006] Constructing a correction observation window on the spectral data according to the local concentration gradient to extract local nonlinear features, wherein the local nonlinear features include absorbance gradient change features and fitting residual anomaly features;

[0007] According to the local nonlinear characteristics within the correction observation window, a mapping relationship between the spectral data and the water quality test results is established, and a correction coefficient is generated through a scattering correction algorithm;

[0008] Correcting the spectral data according to the mapping relationship and the correction coefficient;

[0009] Based on the corrected spectral data, impurity components in the ionic solution are identified.

[0010] Specifically, the concentration of the ion solution to be measured is ≥1 mol / L, and the method of constructing a correction observation window according to the local concentration gradient includes:

[0011] Calculating the absorbance gradient of adjacent data points in the spectral data;

[0012] When the absorbance gradient is greater than a preset threshold, the size of the correction observation window is reduced to focus on the high-concentration area.

[0013] Specifically, the method for establishing a mapping relationship between spectral data and water quality test results includes:

[0014] Within the correction observation window, using absorbance data as input variables and impurity concentration as output variables, a local linear regression model is fitted by the least squares method to establish a mapping relationship between absorbance and impurity concentration, wherein the local linear regression model is a model trained based on the absorbance of ion solution samples with known impurity concentrations;

[0015] Calculating the absorbance fitting residuals of each data point in the correction observation window and obtaining the standard deviation of the fitting residuals, wherein the fitting residuals are used to quantify the abnormal characteristics of the fitting residuals in the local nonlinear characteristics, and when the standard deviation is greater than a preset noise threshold, it is determined that the window has significant nonlinear distortion;

[0016] The standard deviation is used as a weight adjustment factor to update the weight parameter of the mapping relationship. The method for updating the weight parameter of the mapping relationship includes:

[0017] When the standard deviation is greater than the preset noise threshold, the weight of the current window mapping relationship is reduced;

[0018] When the standard deviation is less than or equal to the preset noise threshold, the weight of the current window mapping relationship is increased.

[0019] Specifically, the scatter correction algorithm is a multivariate scatter correction, and the method of generating a correction coefficient by using the scatter correction algorithm includes:

[0020] According to the absorbance gradient change characteristics extracted within the correction observation window, a gradient stable region is screened as a reference spectrum candidate set;

[0021] Calculating the similarity between each spectrum in the reference spectrum candidate set and the spectrum to be corrected at the residual abnormal feature points, and selecting the spectrum with the highest similarity as the reference spectrum;

[0022] Fitting the spectrum to be corrected by the least squares method With reference spectrum The linear relationship:

[0023] ;

[0024] in, and is the correction factor generated, To correct for scattering effects, Used to correct baseline shift;

[0025] When the residual abnormal feature points in the window account for more than 30%, Value to apply gradient compensation factor :

[0026] ;

[0027] in is the maximum absorbance gradient value in the window, is the average gradient value.

[0028] Specifically, the method for correcting the spectral data according to the mapping relationship and the correction coefficient includes:

[0029] Multiplying the correction coefficient by the weight adjustment factor of the mapping relationship to obtain a spectral correction factor γ, wherein the weight adjustment factor is a factor for adjusting the weight of the correction observation window when establishing the mapping relationship;

[0030] The absorbance in the spectral data is corrected using the spectral correction factor γ. Perform baseline correction to obtain the corrected absorbance , the correction formula is:

[0031] ;

[0032] in is the minimum absorbance within the correction observation window.

[0033] Specifically, the method for identifying impurity components in the ionic solution includes:

[0034] Matching the corrected spectral data with characteristic peaks of preset impurities to identify impurity components in the ionic solution, wherein the characteristic peaks of the preset impurities are stored in an impurity characteristic peak database, which is obtained by measuring spectral curves of known impurities with a standard spectrometer;

[0035] When identifying impurity components in the ionic solution, the characteristic peak matching tolerance threshold is dynamically adjusted according to the concentration of the ionic solution to be measured.

[0036] Specifically, the parameters of the characteristic peak include: peak center wavelength, half-peak width and peak height, and the method of matching the corrected spectral data with the characteristic peak of the preset impurity is based on matching the parameters of the characteristic peak.

[0037] In a second aspect, the system provided by this application includes:

[0038] A spectrum detection module, used to obtain spectrum data of the ion solution to be tested, wherein the spectrum data includes wavelength and absorbance;

[0039] a window measurement module, configured to construct a correction observation window on the spectral data according to the local concentration gradient, and to extract local nonlinear features, wherein the local nonlinear features include absorbance gradient variation features and fitting residual anomaly features;

[0040] A spectral correction module is used to establish a mapping relationship between spectral data and water quality test results based on the local nonlinear characteristics within the correction observation window, and to generate a correction coefficient through a scattering correction algorithm;

[0041] Correcting the spectral data according to the mapping relationship and the correction coefficient;

[0042] The water quality detection module identifies impurity components in the ionic solution based on the corrected spectral data.

[0043] This application has the following technical effects:

[0044] By accurately capturing the local nonlinear characteristics in the spectrum of high-concentration ion solutions, the inherent defects of traditional global correction methods in impurity identification are effectively overcome.

[0045] Based on a dynamically constructed correction observation window, targeted analysis of gradient jumps and residual anomalies is achieved, converting distorted signals, previously blurred by the global algorithm, into quantifiable parameters. This allows for a stable mapping between absorbance and impurity concentration even in environments with strong ion interference. This significantly improves the resolution of impurity components in high-concentration ranges, ensuring reliable measurement accuracy even in high-ion strength environments, resolving the challenge of concentration-related distortion in water quality testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding numbers represent the same or corresponding parts.

[0047] Figure 1 This is a flow chart of a method for detecting solution impurities using a water quality detector in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0049] In the field of spectroscopic water quality testing, impurity analysis in high-concentration ion solutions has long been constrained by the theoretical frameworks of traditional methods. Existing mainstream techniques, such as multivariate scattering correction (MSC) and global spectral regression models, rely on the idealized assumptions of homogeneous solutions and linear optical responses. This assumption reveals a critical flaw in high-concentration scenarios. When the concentration of the ion solution to be measured exceeds 1 mol / L, dense ion clusters in the solution trigger localized refractive index abrupt changes, leading to non-uniform scattering of the incident laser light at the microscopic scale. This scattering is not globally uniform, but rather forms a random gradient with abrupt changes, causing spur-like jitter in the absorbance curve at specific wavelengths. Furthermore, strong inter-ionic interactions distort the energy level transition paths of impurity molecules, resulting in shifts in characteristic peak positions and expansion of the half-peak width. Traditional MSC methods are unable to effectively detect these impurities because they force a global fit of the full spectrum data to the reference spectrum, making it impossible to distinguish between locally distorted regions and true impurity signals. Furthermore, linear regression models based on the full spectrum can introduce systematic bias due to interference from these localized distortion points. These flaws render measurement results in the high-concentration range ineffective for engineering applications.

[0050] In order to solve the above problems, the inventors of this application have innovatively established a local feature analysis mechanism based on dynamic windows, and proposed a method, system, device and medium for detecting solution impurities by a water quality detector, such as Figure 1 As shown, the method of the present application includes:

[0051] Acquiring spectral data of the ion solution to be tested, the spectral data including wavelength and absorbance;

[0052] A correction observation window is constructed on the spectral data according to the local concentration gradient to extract local nonlinear features, including absorbance gradient change characteristics and fitting residual anomaly characteristics;

[0053] Based on the local nonlinear characteristics within the correction observation window, a mapping relationship between spectral data and water quality test results is established, and a correction coefficient is generated through a scattering correction algorithm;

[0054] Correct the spectral data according to the mapping relationship and correction coefficient;

[0055] Identify impurity components in ionic solutions based on corrected spectral data.

[0056] In practice, the optical fiber spectrometer first acquires spectral data of the ion solution to be tested in the 200-800nm ​​band. This data includes a wavelength sequence and its corresponding absorbance matrix. To address the "burr spectrum" characteristic of high-concentration solutions, the system automatically initiates a correction observation window construction program, dynamically delineating the analysis area based on the local concentration gradient. The method for constructing a correction observation window based on the local concentration gradient includes the following:

[0057] Calculate the absorbance gradient of adjacent data points in spectral data;

[0058] When the absorbance gradient is greater than a preset threshold, the size of the correction observation window is reduced to focus on the high-concentration area.

[0059] For example, if the absorbance change rate between adjacent data points exceeds 0.05 Abs / nm, the system immediately shrinks the window width from the default 20nm to 5nm, locking in the core region of distortion. Within this window, the algorithm simultaneously extracts two key features: first, the absorbance gradient variation, capturing the sudden change signal caused by ion clustering by calculating the extreme value of the first-order derivative within the window; second, the residual anomaly fitting feature, which uses the least squares method to establish a temporary linear model between absorbance and wavelength within the window, and uses the residual standard deviation of the actual measured value and the model prediction value as a quantified indicator of the distortion intensity.

[0060] These local features then drive the intelligent correction system. When mapping spectral data to water quality results, the system uses local linear regression within the window rather than a global model. The methods include:

[0061] In the calibration observation window, the absorbance data is used as the input variable and the impurity concentration is used as the output variable. The least squares method is used to fit the local linear regression model to establish a mapping relationship between absorbance and impurity concentration. The local linear regression model is a model trained based on the absorbance of ion solution samples with known impurity concentrations.

[0062] Calculate the absorbance fitting residuals of each data point in the correction observation window and obtain the standard deviation of the fitting residuals. The fitting residuals are used to quantify the abnormal characteristics of the fitting residuals in the local nonlinear characteristics. When the standard deviation is greater than the preset noise threshold, it is determined that there is significant nonlinear distortion in the window.

[0063] The standard deviation is used as a weight adjustment factor to update the weight parameters of the mapping relationship. The method for updating the weight parameters of the mapping relationship includes:

[0064] When the standard deviation is greater than the preset noise threshold, the weight of the current window mapping relationship is reduced;

[0065] When the standard deviation is less than or equal to the preset noise threshold, the weight of the current window mapping relationship is increased.

[0066] Specifically, the system uses absorbance as input and impurity concentration as output, using spectra of impurity samples of known concentrations for training. During training, the system monitors the standard deviation of the fitting residuals in real time. When the standard deviation of the fitting residuals exceeds the preset noise threshold of 0.01, indicating the presence of strong nonlinear interference, the system automatically reduces the weight of the current window mapping relationship to 30% of the standard value to prevent distorted data from contaminating the overall model.

[0067] At the same time, the scatter correction algorithm implements a more sophisticated compensation strategy. The scatter correction algorithm is a multivariate scatter correction. The methods for generating correction coefficients through the scatter correction algorithm include:

[0068] According to the absorbance gradient change characteristics extracted within the correction observation window, the gradient stable region is selected as the reference spectrum candidate set;

[0069] Calculate the similarity between each spectrum in the reference spectrum candidate set and the spectrum to be corrected at the residual abnormal feature points, and select the spectrum with the highest similarity as the reference spectrum;

[0070] Fitting the spectrum to be corrected by the least squares method With reference spectrum The linear relationship:

[0071] ;

[0072] in, and is the correction factor generated, To correct for scattering effects, Used to correct baseline shift;

[0073] When the residual abnormal feature points in the window account for more than 30%, Value to apply gradient compensation factor :

[0074] ;

[0075] in is the maximum absorbance gradient value in the window, is the average gradient value.

[0076] When screening the reference spectrum, the system gives priority to regional data with similar gradient change characteristics, and then matches the residual anomaly distribution pattern through similarity calculation. After determining the reference spectrum, the coefficient k no longer simply corrects the overall scattering, but focuses on compensating for local distortion, especially when the residual anomaly in the window accounts for more than 30%, the gradient compensation factor is introduced. α This factor significantly enhances the correction strength in high gradient regions, and its physical essence is to enhance the photon path length compensation in the distorted region.

[0077] The final step in the correction process is spectral factor fusion. Based on the mapping relationship and correction coefficients, the methods for correcting spectral data include:

[0078] Multiplying the correction coefficient by the weight adjustment factor of the mapping relationship to obtain the spectral correction factor γ, wherein the weight adjustment factor is a factor that adjusts the weight of the correction observation window when establishing the mapping relationship;

[0079] The absorbance in the spectral data is corrected using the spectral correction factor γ Perform baseline correction to obtain the corrected absorbance , the correction formula is:

[0080] ;

[0081] in The minimum absorbance within the corrected observation window.

[0082] When the spectral correction factor γ is abnormally reduced due to high residuals during implementation, the system immediately performs baseline correction, abandoning the global baseline and using the minimum absorbance within the window as the A baseline to ensure accurate anchoring of the local correction, so that the distorted signal in the original spectrum is reconstructed into a clear characteristic spectrum. Finally, in the impurity identification stage, the method for identifying impurity components in ionic solutions includes:

[0083] Match the corrected spectral data with the characteristic peaks of preset impurities to identify the impurity components in the ion solution. The characteristic peaks of the preset impurities are stored in the impurity characteristic peak database. The impurity characteristic peak database is obtained by measuring the spectral curves of known impurities with a standard spectrometer. The parameters of the characteristic peaks include: peak center wavelength, half-peak width and peak height;

[0084] When identifying impurity components in ionic solutions, the characteristic peak matching tolerance threshold is dynamically adjusted based on the concentration of the ionic solution being tested. This dynamic tolerance strategy effectively avoids misjudgment of peak position drift caused by ion interference.

[0085] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line, or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., an SSD).

[0086] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0087] Obviously, the embodiments described above are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0088] It should be understood that when the terms "first," "second," etc. are used in the claims, specification, and drawings of this application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "comprise" and "comprising" used in the specification and claims of this application indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

Claims

1. A method for detecting solution impurities using a water quality detector, characterized in that: The following steps are involved: Acquiring spectral data of the ion solution to be tested, wherein the spectral data includes wavelength and absorbance; Constructing a correction observation window on the spectral data according to the local concentration gradient to extract local nonlinear features, wherein the local nonlinear features include absorbance gradient change features and fitting residual anomaly features; The concentration of the ion solution to be measured is ≥1 mol / L, and the method for constructing a correction observation window according to the local concentration gradient includes: Calculating the absorbance gradient of adjacent data points in the spectral data; When the absorbance gradient is greater than a preset threshold, the size of the correction observation window is reduced to focus on the high concentration area; According to the local nonlinear characteristics within the correction observation window, a mapping relationship between the spectral data and the water quality test results is established, and a correction coefficient is generated through a scattering correction algorithm; Methods for establishing a mapping relationship between spectral data and water quality test results include: Within the correction observation window, using absorbance data as input variables and impurity concentration as output variables, a local linear regression model is fitted by the least squares method to establish a mapping relationship between absorbance and impurity concentration, wherein the local linear regression model is a model trained based on the absorbance of ion solution samples with known impurity concentrations; Calculating the absorbance fitting residuals of each data point in the correction observation window and obtaining the standard deviation of the fitting residuals, wherein the fitting residuals are used to quantify the abnormal characteristics of the fitting residuals in the local nonlinear characteristics, and when the standard deviation is greater than a preset noise threshold, it is determined that the window has significant nonlinear distortion; The standard deviation is used as a weight adjustment factor to update the weight parameter of the mapping relationship. The method for updating the weight parameter of the mapping relationship includes: When the standard deviation is greater than the preset noise threshold, the weight of the current window mapping relationship is reduced; When the standard deviation is less than or equal to the preset noise threshold, the weight of the current window mapping relationship is increased; Correcting the spectral data according to the mapping relationship and the correction coefficient; Based on the corrected spectral data, impurity components in the ionic solution are identified.

2. The method according to claim 1, characterized in that The scatter correction algorithm is a multivariate scatter correction, and the method for generating a correction coefficient by using the scatter correction algorithm includes: According to the absorbance gradient change characteristics extracted within the correction observation window, a gradient stable region is screened as a reference spectrum candidate set; Calculating the similarity between each spectrum in the reference spectrum candidate set and the spectrum to be corrected at the residual abnormal feature points, and selecting the spectrum with the highest similarity as the reference spectrum; Fitting the spectrum to be corrected by the least squares method With reference spectrum The linear relationship: ； in, and is the correction factor generated, To correct for scattering effects, Used to correct baseline shift; When the residual abnormal feature points in the window account for more than 30%, Value to apply gradient compensation factor : ； in is the maximum absorbance gradient value in the window, is the average gradient value.

3. The method according to claim 1, characterized in that The method for correcting the spectral data according to the mapping relationship and the correction coefficient includes: Multiplying the correction coefficient by the weight adjustment factor of the mapping relationship to obtain a spectral correction factor γ, wherein the weight adjustment factor is a factor for adjusting the weight of the correction observation window when establishing the mapping relationship; The absorbance in the spectral data is corrected using the spectral correction factor γ. Perform baseline correction to obtain the corrected absorbance , the correction formula is: ； in is the minimum absorbance within the correction observation window.

4. The method according to claim 1, wherein The method for identifying impurity components in the ionic solution comprises: Matching the corrected spectral data with characteristic peaks of preset impurities to identify impurity components in the ionic solution, wherein the characteristic peaks of the preset impurities are stored in an impurity characteristic peak database, which is obtained by measuring spectral curves of known impurities with a standard spectrometer; When identifying impurity components in the ionic solution, the characteristic peak matching tolerance threshold is dynamically adjusted according to the concentration of the ionic solution to be measured.

5. The method according to claim 4, characterized in that The parameters of the characteristic peak include: peak center wavelength, half-peak width and peak height. The method of matching the corrected spectral data with the characteristic peak of the preset impurity is based on matching the parameters of the characteristic peak.

6. A system for detecting impurities in a solution using a water quality detector, operated using the method according to any one of claims 1 to 5, characterized in that: The system comprises: A spectrum detection module, used to obtain spectrum data of the ion solution to be tested, wherein the spectrum data includes wavelength and absorbance; a window measurement module, configured to construct a correction observation window on the spectral data according to the local concentration gradient, and to extract local nonlinear features, wherein the local nonlinear features include absorbance gradient variation features and fitting residual anomaly features; A spectral correction module is used to establish a mapping relationship between spectral data and water quality test results based on the local nonlinear characteristics within the correction observation window, and to generate a correction coefficient through a scattering correction algorithm; Correcting the spectral data according to the mapping relationship and the correction coefficient; The water quality detection module identifies impurity components in the ionic solution based on the corrected spectral data.

7. A computing device, characterized in that include: a memory for storing program instructions; A processor, configured to call the program instructions stored in the memory and execute the method according to any one of claims 1 to 5 according to the obtained program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images