Spectrum data preprocessing method and system of micro ultraviolet spectrometer

Abstract

This invention discloses a method and system for preprocessing spectral data of a miniature ultraviolet spectrometer. The method includes: acquiring raw spectral data of the target and determining calibration spectral data; sequentially performing dark current smoothing and dark current removal on the raw spectral data; acquiring the calibration spectral data and performing wavelength calibration on the raw spectral data after dark current smoothing and dark current removal based on the size of the wavelength calibration flag; and performing smoothing processing on the wavelength-calibrated raw spectral data. This invention sequentially performs dark current smoothing, dark current removal, wavelength calibration, and smoothing processing on the raw spectral data, enabling accurate detection of low-concentration gases under conditions of simultaneous measurement of multiple component gases.

Description

Technical Field

[0001] This invention relates to the field of spectral data processing technology, specifically to a method and system for preprocessing spectral data from a miniature ultraviolet spectrometer. Background Technology

[0002] Miniature ultraviolet spectrometers are used to detect raw spectral data of various toxic and harmful gases such as VOCs, NOx, and SO2. Preprocessing of the raw spectral data of these gases is necessary to achieve accurate detection of low concentrations of gases under conditions of simultaneous measurement of multiple components.

[0003] Existing spectral data preprocessing methods include centering and smoothing. Centering calculates the average value for each sample, subtracts these values ​​from the spectral data, and distributes the relevant data for all samples around the zero point, fully reflecting variation information and effectively removing the influence of information changes on the spectral data. Smoothing methods use moving average smoothing or Savitzky-Golay convolution smoothing to eliminate noise in the spectral data.

[0004] The aforementioned spectral data preprocessing methods cannot remove dark current data from the spectral data, nor do they perform wavelength calibration on the spectral data, thus failing to achieve accurate detection of low-concentration gases under conditions of simultaneous measurement of multiple component gases. Summary of the Invention

[0005] In view of this, the present invention proposes a spectral data preprocessing method and system for a miniature ultraviolet spectrometer, which is used for data preprocessing for the detection of various toxic and harmful gases such as VOCs, NOx, and SO2 in the field, and can achieve accurate detection of low-concentration gases under the condition of simultaneous measurement of multiple component gases.

[0006] The first aspect of the present invention provides a method for preprocessing spectral data of a miniature ultraviolet spectrometer. The method includes: acquiring raw spectral data of a target and determining calibration spectral data; sequentially performing dark current smoothing and dark current removal on the raw spectral data; acquiring calibration spectral data and performing wavelength calibration on the raw spectral data after dark current smoothing and dark current removal according to the size of the wavelength calibration flag; and performing smoothing processing on the raw spectral data after wavelength calibration.

[0007] Furthermore, the step of determining the calibration spectral data includes: determining whether the acquired raw spectral data is calibration spectral data; if the raw spectral data is calibration data, then the calibration spectral data needs to be re-stored; if the raw spectral data is not calibration data, then the stored calibration spectral data is used as the calibration spectral data.

[0008] Furthermore, the step of performing dark current smoothing on the original spectral data includes: extracting dark current data from the original spectral data; obtaining a dark current smoothing flag bit; and determining whether to perform smoothing processing on the dark current data in the original spectral data based on the magnitude of the dark current smoothing flag bit.

[0009] Furthermore, if the dark current smoothing flag is 1, the extracted dark current data is smoothed and the smoothed dark current value is calculated; if the dark current smoothing flag is 0, the extracted dark current data is not smoothed.

[0010] Furthermore, the step of removing dark current from the original spectral data includes: determining the size of the dark current smoothing flag; if the dark current smoothing flag is 1, then removing the calculated dark current smoothing value from the original spectral data; if the dark current smoothing flag is 0, then directly removing the unsmoothed dark current data from the original spectral data.

[0011] Furthermore, the step of performing wavelength calibration on the original spectral data includes: obtaining a wavelength calibration flag bit; determining the size of the wavelength calibration flag bit; if the wavelength calibration flag bit is 1, calculating a wavelength calibration amount based on the calibrated spectral data, and using the calculated wavelength calibration amount to perform wavelength calibration on the original spectral data; if the wavelength calibration flag bit is 0, using a default wavelength calibration amount to perform wavelength calibration on the original spectral data; and sequentially performing pixel calibration and sub-pixel calibration on the wavelength-calibrated original spectral data.

[0012] Furthermore, the SG filtering method or adaptive filter is used to smooth the original spectral data after wavelength calibration.

[0013] A second aspect of the present invention provides a spectral data preprocessing system for a miniature ultraviolet spectrometer. The system includes: a data acquisition module for acquiring raw spectral data of a target; a dark current smoothing module for determining calibration spectral data and performing dark current smoothing on the raw spectral data; a dark current removal module for removing dark current from the smoothed raw spectral data; a wavelength calibration module for acquiring calibration spectral data and performing wavelength calibration on the smoothed and removed raw spectral data based on the size of a wavelength calibration flag; and a data smoothing module for smoothing the wavelength-calibrated raw spectral data.

[0014] A third aspect of the present invention provides a spectral data preprocessing device for a miniature ultraviolet spectrometer, the device comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the spectral data preprocessing method for the miniature ultraviolet spectrometer as described above.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the spectral data preprocessing method for a miniature ultraviolet spectrometer as described above.

[0016] The aforementioned method and system for preprocessing spectral data of a miniature ultraviolet spectrometer first acquires raw spectral data. Dark current smoothing is then performed on the acquired raw spectral data containing information on VOCs, NOx, and SO2. After dark current smoothing, the dark current data contained in the raw spectral data is removed according to the dark current smoothing requirements. After dark current removal, calibration spectral data is obtained according to wavelength calibration requirements, and the raw spectral data is then used for wavelength calibration. Finally, the raw spectrum after dark current removal and wavelength calibration is smoothed, effectively removing noise from the spectral data and obtaining accurate spectral data. This enables precise detection of low-concentration gases under conditions of simultaneous measurement of multiple component gases. Attached Figure Description

[0017] For illustrative and not limiting purposes, the invention will now be described with reference to preferred embodiments thereof, particularly the accompanying drawings, in which:

[0018] Figure 1 This is a flowchart of a spectral data preprocessing method for a miniature ultraviolet spectrometer provided in an embodiment of the present invention;

[0019] Figure 2 This is a dark current smoothing flowchart provided in an embodiment of the present invention;

[0020] Figure 3 This is a flowchart of dark current removal provided in an embodiment of the present invention;

[0021] Figure 4 This is a flowchart of wavelength calibration provided in an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of the structure of a spectral data preprocessing system for a miniature ultraviolet spectrometer provided in another embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the structure of a spectral data preprocessing device for a miniature ultraviolet spectrometer provided in an embodiment of the present invention. Detailed Implementation

[0024] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0025] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. The described embodiments are merely some, not all, of the embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0027] Figure 1 This is a flowchart of a spectral data preprocessing method for a miniature ultraviolet spectrometer according to an embodiment of the present invention. This spectral data preprocessing method performs dark current smoothing on the acquired raw spectral data containing VOCs, NOx, and SO2 information. After dark current smoothing, the dark current data contained in the raw spectral data is removed according to the dark current smoothing requirements. After dark current removal, calibration spectral data is obtained according to wavelength calibration requirements, and the raw spectral data is wavelength calibrated using the calibration spectral data. Finally, the raw spectrum after dark current removal and wavelength calibration is smoothed, which can effectively remove noise from the spectral data and obtain accurate spectral data.

[0028] Please see Figure 1 The spectral data preprocessing method of this miniature ultraviolet spectrometer includes the following steps:

[0029] S100: Acquire the raw spectral data of the target and determine the calibration spectral data.

[0030] The target is toxic and harmful gases such as VOCs, NOx, and SO2 on site. Raw spectral data containing information on VOCs, NOx, and SO2 are collected by a miniature ultraviolet spectrometer.

[0031] Determine whether the acquired raw spectral data is for calibration. If it is, then the calibration data needs to be re-stored; otherwise, the calibration data does not need to be re-stored, and the calibration data already stored during the previous preprocessing can be used as the calibration data.

[0032] S200 performs dark current smoothing on the acquired raw spectral data.

[0033] Figure 2 This is a dark current smoothing flowchart provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 2 The specific implementation method of step S200 is as follows:

[0034] S201, extract dark current data from the original spectral data.

[0035] S202, based on the dark current smoothing flag, determines whether to smooth the dark current data in the original spectral data.

[0036] S203, if the dark current smoothing flag is 1, the extracted dark current data is smoothed and the smoothed dark current value is calculated; if the dark current smoothing flag is 0, the extracted dark current data is not smoothed.

[0037] In this embodiment, the original spectral data is smoothed for dark current starting from the dark current smoothing flag, so as to transform the dark current data into data that is easy to calculate.

[0038] Dark current smoothing methods include moving average filtering and Gaussian filtering. Mean filtering, a linear filter, is straightforward: it calculates the average value of pixels within a window region and sets this average as the pixel value at the anchor point. This algorithm is efficient and simple. However, its drawback is also apparent: calculating the mean blurs edge and feature information in the signal, resulting in the loss of many features.

[0039] Gaussian filtering, a type of linear filtering, is a commonly used filtering algorithm that smooths images using the distribution of a two-dimensional Gaussian function. The advantages of Gaussian filtering can be summarized by the characteristics of the Gaussian function. First, the two-dimensional Gaussian function is rotationally symmetric, ensuring uniform smoothness in all directions and preserving the edge characteristics of the original signal. Second, the Gaussian function is a single-valued function; the anchor point of the Gaussian convolution kernel is an extremum, monotonically decreasing in all directions. Anchor pixels are not excessively affected by pixels far from the anchor point, preserving the characteristics of feature points and edges. Third, in the frequency domain, the filtering process is not contaminated by high-frequency signals. Gaussian blur is essentially a form of mean blurring, but it uses a weighted average, with closer points having higher weights and farther points having lower weights. In simpler terms, Gaussian filtering is a process of weighted averaging of the entire signal; the value of each pixel is obtained by weighted averaging of its own value and the values ​​of its neighboring pixels.

[0040] S300 performs dark current removal on the original spectral data after dark current smoothing.

[0041] After the dark current smoothing is completed, the dark current data contained in the original spectral data is removed according to the dark current smoothing requirements.

[0042] Dark current removal filtering is a fundamental task in signal processing. The purpose of dark current removal is to selectively extract certain information from a signal, depending on the application environment. Filtering can remove noise from a signal, extract visual features of interest, allow signal resampling, and so on. Frequency domain analysis divides the signal into different parts from low to high frequencies. Low frequencies correspond to regions with small changes in signal intensity, while high frequencies are regions with very large changes in signal intensity.

[0043] To reduce the impact of noise on the original spectral data, highlight the effective characteristics of each detected gas in the original spectral data, and reduce errors, dark current removal is required after dark current smoothing. This embodiment performs dark current removal on the original spectral data based on the dark current smoothing flag.

[0044] Figure 3 This is a flowchart of dark current removal provided in an embodiment of the present invention. Figure 3 As shown, the specific implementation method of step S300 is as follows:

[0045] S301, obtain the dark current smoothing flag.

[0046] S302 determines whether to remove dark current data from the original spectral data based on the dark current smoothing flag.

[0047] S303, if the dark current smoothing flag is 1, the calculated dark current smoothing value will be removed from the original spectral data; if the dark current smoothing flag is 0, the unsmoothed dark current data will be directly removed from the original spectral data.

[0048] S400 performs wavelength calibration on the raw spectral data after dark current removal.

[0049] After the dark current removal is completed, obtain the calibration spectral data according to the wavelength calibration requirements, and start wavelength calibration on the original spectral data after dark current removal.

[0050] Figure 4 This is a flowchart of wavelength calibration provided in an embodiment of the present invention. Figure 4 As shown, the specific implementation method of step S400 is as follows:

[0051] S401, obtain the wavelength calibration flag.

[0052] S402, determine the size of the wavelength calibration flag.

[0053] S403, if the wavelength calibration flag is 1, then wavelength calibration is required for the original spectral data after dark current removal. The wavelength calibration amount is calculated based on the stored calibration spectral data, and the calculated wavelength calibration amount is used to perform wavelength calibration on the original spectral data after dark current removal. Then, pixel calibration and sub-pixel calibration are performed on the original data. If the wavelength calibration flag is 0, then the default wavelength calibration amount is used to perform wavelength calibration on the original spectral data after dark current removal, and then pixel calibration and sub-pixel calibration are performed on the original spectral data.

[0054] By calibrating the spectral data, the wavelength of the raw spectral data of the spectrometer is calibrated, which effectively improves the accuracy of the data.

[0055] The S500 smooths the raw spectral data after wavelength calibration.

[0056] Finally, the original spectral data, after dark current removal and wavelength calibration, is smoothed, thus completing the spectral data preprocessing.

[0057] The smoothing methods used in this embodiment include SG filtering (Savitzky Golay Filter), adaptive filtering, and other methods.

[0058] The core idea of ​​the Savitzky Golay Filter is to perform weighted filtering on the data within a window, but its weights are obtained by least-squares fitting of a given high-order polynomial. Its advantage lies in its ability to more effectively preserve signal variation information while smoothing the filtering process. The Savitzky Golay Filter better preserves the observational information of the data and is more suitable for situations where data variation is critical.

[0059] Adaptive filters do not require prior knowledge of the statistical characteristics of the input signal and noise when processing spectral signals. The filter itself learns or estimates the signal's statistical characteristics during operation and adjusts its parameters accordingly to achieve the optimal filtering effect. If the statistical characteristics of the spectral signal change, it can track these changes and readjust its parameters to restore optimal filtering performance. Adaptive filtering is an effective method for smoothing spectral data.

[0060] By smoothing the raw spectral data after processes such as dark current removal and wavelength calibration, noise in the spectral data can be effectively eliminated.

[0061] The above-mentioned spectral data preprocessing method first performs dark current smoothing on the acquired raw spectral data containing VOCs, NOx, and SO2 information based on the dark current smoothing flag. After dark current smoothing, the dark current data contained in the raw spectral data is removed according to the dark current smoothing requirements. After dark current removal, calibration spectral data is obtained according to the wavelength calibration requirements, and the wavelength of the raw spectral data is calibrated using the calibration spectral data. Finally, the raw spectrum after dark current removal and wavelength calibration is smoothed, which can effectively remove noise from the spectral data and obtain the prepared spectral data.

[0062] Figure 5 This is a schematic diagram of the structure of a spectral data preprocessing system for a miniature ultraviolet spectrometer provided in another embodiment of the present invention.

[0063] In this embodiment, the spectral data preprocessing system 20 of the miniature ultraviolet spectrometer can be applied to a computer device. The spectral data preprocessing system 20 may include multiple functional modules composed of program code segments. The program code of each program segment in the spectral data preprocessing system 20 can be stored in the memory of the computer device and executed by at least one processor of the computer device to achieve (see details). Figure 1 (Description) Spectral data preprocessing function of miniature ultraviolet spectrometer.

[0064] In this embodiment, the spectral data preprocessing system 20 of the miniature ultraviolet spectrometer can be divided into multiple functional modules according to its functions. These functional modules may include: a data acquisition module 201, a dark current smoothing module 202, a dark current removal module 203, a wavelength calibration module 204, and a data smoothing module 205. The term "module" in this invention refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0065] The data acquisition module 201 is used to acquire the raw spectral data of the target.

[0066] The dark current smoothing module 202 is used to determine the calibration spectral data and perform dark current smoothing on the original spectral data.

[0067] The dark current removal module 203 is used to remove dark current from the original spectral data after dark current smoothing.

[0068] The wavelength calibration module 204 is used to acquire calibration spectral data and perform wavelength calibration on the original spectral data after dark current smoothing and dark current removal based on the size of the wavelength calibration flag.

[0069] The data smoothing module 205 is used to smooth the original spectral data after wavelength calibration.

[0070] The aforementioned spectral data preprocessing system first performs dark current smoothing on the acquired raw spectral data containing VOCs, NOx, and SO2 information based on the dark current smoothing flag. After dark current smoothing, the system removes dark current data from the raw spectral data according to the dark current smoothing requirements. After dark current removal, the system obtains calibration spectral data according to wavelength calibration requirements and uses the calibration spectral data to perform wavelength calibration on the raw spectral data. Finally, the system smooths the raw spectrum after dark current removal and wavelength calibration, effectively removing noise from the spectral data and obtaining the prepared spectral data.

[0071] For the method embodiments described above, see [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram of the spectral data preprocessing device for the miniature ultraviolet spectrometer provided by the present invention. The device 30 may include:

[0072] Memory 301 is used to store computer programs;

[0073] Processor 302, when executing the computer program, performs the following steps:

[0074] Collect the raw spectral data of the target and determine the calibration spectral data; sequentially perform dark current smoothing and dark current removal on the raw spectral data; obtain the calibration spectral data, and perform wavelength calibration on the raw spectral data after dark current smoothing and dark current removal according to the size of the wavelength calibration flag; perform smoothing processing on the raw spectral data after wavelength calibration.

[0075] For a description of the device provided by the present invention, please refer to the above method embodiments; the present invention will not be described in detail here.

[0076] Corresponding to the above method embodiments, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps:

[0077] Collect the raw spectral data of the target and determine the calibration spectral data; sequentially perform dark current smoothing and dark current removal on the raw spectral data; obtain the calibration spectral data, and perform wavelength calibration on the raw spectral data after dark current smoothing and dark current removal according to the size of the wavelength calibration flag; perform smoothing processing on the raw spectral data after wavelength calibration.

[0078] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] For a description of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments; the present invention will not be described in detail here.

[0080] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0081] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for preprocessing spectral data of a miniature ultraviolet spectrometer, characterized in that, include: Acquire the raw spectral data of the target and determine the calibration spectral data; The original spectral data were sequentially subjected to dark current smoothing and dark current removal; Acquire calibration spectral data, and perform wavelength calibration on the original spectral data after dark current smoothing and dark current removal based on the size of the wavelength calibration flag. Smooth the raw spectral data after wavelength calibration; The steps for determining the calibration spectral data include: Determine whether the acquired raw spectral data is calibration spectral data; If the original spectral data is used for calibration, then the calibration spectral data needs to be stored again. If the original spectral data is not the data used for calibration, then the stored spectral data used for calibration will be used as the calibration spectral data. The step of performing dark current smoothing on the original spectral data includes: Extract dark current data from the original spectral data; Obtain the dark current smoothing flag bit, and determine whether to smooth the dark current data in the original spectral data based on the size of the dark current smoothing flag bit; If the dark current smoothing flag is 1, the extracted dark current data is smoothed and the smoothed dark current value is calculated; if the dark current smoothing flag is 0, the extracted dark current data is not smoothed. The step of performing dark current removal on the original spectral data includes: Determine the size of the dark current smoothing flag bit; If the dark current smoothing flag is 1, the calculated dark current smoothing value will be removed from the original spectral data. If the dark current smoothing flag is 0, the unsmoothed dark current data in the original spectral data will be removed directly.

2. The spectral data preprocessing method for a miniature ultraviolet spectrometer according to claim 1, characterized in that, The steps for wavelength calibration of the raw spectral data include: Obtain the wavelength calibration flag; Determine the size of the wavelength calibration flag; If the wavelength calibration flag is 1, the wavelength calibration amount is calculated based on the calibration spectral data, and the calculated wavelength calibration amount is used to perform wavelength calibration on the original spectral data; if the wavelength calibration flag is 0, the default wavelength calibration amount is used to perform wavelength calibration on the original spectral data. The original spectral data after wavelength calibration is then subjected to pixel calibration and sub-pixel calibration in sequence.

3. The spectral data preprocessing method for a miniature ultraviolet spectrometer according to claim 1, characterized in that, The original spectral data after wavelength calibration is smoothed using the SG filtering method or an adaptive filter.

4. The system for spectral data preprocessing of a miniature ultraviolet spectrometer according to any one of claims 1-3, characterized in that, include: The data acquisition module is used to acquire the raw spectral data of the target; A dark current smoothing module is used to determine the calibration spectral data and perform dark current smoothing on the raw spectral data; The dark current removal module is used to remove dark current from the original spectral data after dark current smoothing. The wavelength calibration module is used to acquire calibration spectral data and perform wavelength calibration on the original spectral data after dark current smoothing and dark current removal based on the size of the wavelength calibration flag. The data smoothing module is used to smooth the raw spectral data after wavelength calibration.

5. A spectral data preprocessing device for a miniature ultraviolet spectrometer, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the spectral data preprocessing method for the miniature ultraviolet spectrometer as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the spectral data preprocessing method for the miniature ultraviolet spectrometer as described in any one of claims 1 to 3.

Images