Method, device and readable storage medium for automatic removal of 3D fluorescence spectrum scattering noise

Abstract

The present invention proposes a method, device, and readable storage medium for automatically removing scattering noise from 3D fluorescence spectra, belonging to the technical fields of spectral data analysis and optical signal processing. The present invention aims to address the problems of reduced signal intensity, loss of useful information, high cost, poor versatility, and complex operation associated with conventional methods for removing scattering noise in the prior art. The proposed method includes the steps of data acquisition, preprocessing, identification and removal of scattering noise, and data output, and automatically removes scattering noise through an image recognition algorithm and a specific mathematical model. The present invention is suitable for 3D fluorescence spectral data processing, can improve data accuracy, and is widely used in spectral data analysis in fields such as environmental monitoring, biomedicine, and materials science.

Description

Technical Field

[0001] The present invention relates to the technical field of spectral data analysis and optical signal processing, and in particular to a method and device for automatically removing scattering noise from 3D fluorescence spectra, and a readable storage medium thereof. The method integrates an unsupervised machine learning algorithm (DBSCAN) with statistical anomaly detection (Z-score) and is suitable for spectral data preprocessing in the fields of environmental monitoring, biomedicine, chemical analysis, etc. Background Art

[0002] In the field of fluorescence spectroscopy, 3D fluorescence spectroscopy is widely used due to its ability to provide rich information about the structure and properties of substances. It plays an important role in numerous fields, including environmental monitoring, biomedicine, and materials science. However, in practical applications, 3D fluorescence spectral data is often severely affected by scattering noise, which poses a significant challenge to the accurate interpretation and subsequent analysis of spectral data.

[0003] Traditional methods have many shortcomings when dealing with scattered noise. For example, while some hardware-based filtering techniques can suppress noise to a certain extent, they often reduce the overall strength of the signal, resulting in the loss of useful information. Furthermore, these hardware devices are often costly, increasing the economic burden of spectral analysis and limiting their application in resource-limited scenarios. Furthermore, while some software algorithms may perform well under specific conditions, they make numerous assumptions about the characteristics of spectral data, often requiring extensive manual parameter adjustments in practical applications. This not only increases operational complexity but also places high demands on the operator's expertise and experience. Furthermore, these algorithms lack versatility, making it difficult to achieve ideal denoising effects when faced with different types of spectral data and application scenarios.

[0004] With the continuous development of spectral analysis technology and the growing demand for its application, the market is increasingly in need of a technical solution that can efficiently, accurately and automatically remove scattering noise from 3D fluorescence spectra.

[0005] Therefore, developing a method for removing scattered noise that does not require human intervention, is efficient, accurate, and adaptable is a key technical challenge in this field. Summary of the Invention

[0006] The embodiments of the present invention provide a method, device, and readable storage medium for automatically removing 3D fluorescence spectrum scattering noise. These methods address the current state of the art in 3D fluorescence spectrum scattering noise processing, where traditional methods rely on manual operation, are inefficient, and produce unreliable results, while deep learning methods face problems such as high data labeling costs and insufficient generalization capabilities.

[0007] The core technology of this invention is to build a two-layer noise recognition mechanism by combining the unsupervised machine learning algorithm DBSCAN with the statistical anomaly detection method Z-score, so as to realize the fully automatic recognition, elimination and spectral surface reconstruction of scattered noise in 3D fluorescence spectra, without the need for manual data labeling, and significantly improve the automation level and accuracy of noise processing.

[0008] In a first aspect, the present invention provides a method for automatically removing scattering noise from a 3D fluorescence spectrum, the method comprising the following steps:

[0009] Preprocess the original 3D fluorescence spectrum data to construct a three-dimensional data structure of excitation wavelength, emission wavelength and fluorescence intensity;

[0010] Based on the spatial density distribution of three-dimensional data structure, the DBSCAN clustering algorithm is used to identify potential scattering noise points;

[0011] Based on the statistical distribution characteristics, the Z-score method is used to conduct a secondary verification of the clustering results to determine the final noise point set;

[0012] Interpolate and reconstruct the spectral data after noise removal to restore the continuity of the spectral surface;

[0013] The denoising effect is evaluated through quantitative indicators.

[0014] Furthermore, the preprocessing step includes converting the raw spectral data into a three-dimensional matrix and eliminating instrument response differences through Z-score normalization, as follows:

[0015]

[0016] in, is the global mean of fluorescence intensity, representing the trend of the data set; is the standard deviation of fluorescence intensity, representing the degree of data dispersion; Indicates the excitation wavelength and emission wavelength Fluorescence intensity value at the intersection point; is the standard score.

[0017] Furthermore, the DBSCAN clustering algorithm includes:

[0018] The standardized three-dimensional data points are reconstructed into a spatial point set, and the sparse noise point area is identified by setting the neighborhood radius and the minimum number of samples.

[0019] Furthermore, the initial value of the neighborhood radius is 0.1, the initial value of the minimum number of samples is 5, and they can be adaptively adjusted according to data characteristics.

[0020] Furthermore, the Z-score secondary check includes:

[0021] Calculate the absolute value of the Z-score of each data point, and determine the points with an absolute value greater than a preset threshold as statistical outliers. The preset threshold is 3.

[0022] Furthermore, the interpolation reconstruction adopts bidirectional cubic spline interpolation, and segmented interpolation along the excitation wavelength axis and the emission wavelength axis to ensure a smooth transition of the spectral surface.

[0023] Furthermore, the quantitative indicator is the signal-to-noise ratio, and the denoising effect is evaluated by calculating the ratio of signal energy to noise energy.

[0024] In a second aspect, the present invention provides a device for automatically removing 3D fluorescence spectrum scattering noise, comprising:

[0025] Data preprocessing module: used to load raw spectral data and perform standardization processing to construct a three-dimensional data structure;

[0026] Noise detection module: Integrates the DBSCAN clustering algorithm and the Z-score verification algorithm to identify and mark scattered noise points;

[0027] Interpolation filling module: Based on the cubic spline interpolation algorithm, the surface of the spectral data after noise removal is reconstructed;

[0028] Verification output module: evaluate the denoising effect through the signal-to-noise ratio indicator and output the optimized spectral data.

[0029] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned method for automatically removing 3D fluorescence spectrum scattering noise.

[0030] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the above-mentioned method for automatically removing 3D fluorescence spectral scattering noise.

[0031] The main contributions and innovations of the present invention are as follows:

[0032] 1. High-precision removal of scattered noise: By adopting advanced image recognition algorithms and specific mathematical models, it can more accurately identify and remove scattered noise from 3D fluorescence spectral data. Compared with traditional methods, it causes less damage to spectral data, effectively retains useful information, and improves data accuracy and reliability.

[0033] 2. No need for complex parameter adjustment: This overcomes the defect of traditional software algorithms that require a lot of manual parameter adjustment, realizes the automatic removal of scattering noise, reduces the dependence on the operator's professional knowledge and experience, makes spectral data analysis simpler and easier, and can meet the needs of a wider range of users.

[0034] 3. Good versatility: The method of the present invention is applicable to various types of 3D fluorescence spectral data and different application scenarios, such as environmental monitoring, biomedicine, materials science and other fields. It has strong versatility and wide applicability, and has more advantages than existing technologies.

[0035] 4. Reduce hardware costs: Compared with traditional hardware-based filtering technology, the present invention does not require the purchase of additional expensive hardware equipment, which can effectively reduce the economic cost of spectral analysis, making spectral analysis technology more economical and popularizable.

[0036] 5. Improve analysis efficiency: The method of automatically removing scattered noise can quickly process large amounts of 3D fluorescence spectral data, reducing the time for manual operation and data processing, improving the efficiency of spectral data analysis, and helping to accelerate the research and application process in related fields.

[0037] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0039] Figure 1 is a flow chart of a method for automatically removing scattering noise from a 3D fluorescence spectrum according to an embodiment of the present invention;

[0040] Figure 2 is an unprocessed three-dimensional fluorescence image according to an embodiment of the present invention, wherein the highlighted area is a scattered area;

[0041] Figure 3 is a comparison diagram of a conventional method according to an embodiment of the present invention and the method of the present invention;

[0042] Figure 4 3 is a comparison diagram of a traditional method according to an embodiment of the present invention and the method of the present invention.

[0043] Figure 5 is a comparison chart of the noise removal performance of the method according to an embodiment of the present invention and a traditional algorithm, wherein the first picture is an unprocessed original picture;

[0044] Figure 6FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0046] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0047] In the existing technology of 3D fluorescence spectral scattering noise processing, traditional methods rely on manual operation, are inefficient and produce unreliable results, while deep learning methods face the problems of high data labeling costs and insufficient generalization capabilities.

[0048] Based on this, the present invention solves the problems existing in the prior art by combining the unsupervised machine learning algorithm DBSCAN with the statistical anomaly detection method Z-score.

[0049] Example 1

[0050] The present invention aims to propose a method for automatically removing scattering noise from 3D fluorescence spectra. Figure 1 , the method comprises the following steps:

[0051] Step 1: Acquisition and digital preprocessing of 3D fluorescence spectral data. To address the high-dimensional spectral characteristics of water pollution detection, 3D fluorescence spectral data is first collected and converted into a digital matrix format, a three-dimensional data structure consisting of excitation wavelength (Ex), emission wavelength (Em), and fluorescence intensity. This data structure ensures uniformity and automation in subsequent algorithm processing.

[0052] like Figure 2 The figure shows an unprocessed three-dimensional fluorescence image (the highlighted area is the scattered area). The unprocessed scattered fluorescence image can be seen, and the contour line structure cannot be seen. It is used to compare the effects before and after removing the scattering.

[0053] Therefore, in this embodiment, step 1 specifically includes:

[0054] Step 101: Read the original 3D fluorescence spectrum into a structured three-dimensional matrix in the form of:

[0055]

[0056] Where i and j represent the discrete sampling point numbers of the excitation wavelength and emission wavelength; Indicates the excitation wavelength and emission wavelength The fluorescence intensity value at the intersection reflects the luminescence characteristics of the substance under this Ex / Em combination.

[0057] Step 102: The instrument response differences in the raw data are preprocessed by Z-score normalization:

[0058]

[0059] in is the global mean of fluorescence intensity, representing the trend of the data set; is the standard deviation of fluorescence intensity, representing the degree of data dispersion; Indicates the excitation wavelength and emission wavelength Fluorescence intensity value at the intersection point; is the standard score; this step ensures that the spectral data of different batches and samples are comparable and have a unified dimension.

[0060] Eliminating instrument response variations through Z-score normalization is the foundation of this invention's automated, high-precision, and cross-scenario denoising. Its technical value lies not only in resolving data scaling issues but also in providing a unified mathematical framework for the subsequent integration of unsupervised learning and statistical methods.

[0061] Step 2: DBSCAN clustering anomaly detection based on spatial reconstruction. Taking into account the spatial distribution characteristics of fluorescence intensity, a three-dimensional point set consisting of (Ex, Em, Intensity) is constructed. The density clustering algorithm DBSCAN is then introduced to identify scattered noise points with abnormal density. Compared to traditional denoising strategies based solely on intensity thresholds, this method combines spatial position and intensity information to achieve structured detection of complex background scattering patterns, demonstrating the targeted improvement and domain-specific adaptation of this technology to the context of aquatic environmental spectral data.

[0062] In this embodiment, step 2 specifically includes:

[0063] Step 201: Perform spatial reconstruction on the normalized fluorescence data points, and represent each data point as:

[0064]

[0065] Constructing a three-dimensional point set ,in .

[0066] Step 202: Use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to perform clustering and identify sparse density areas, i.e., potential scattered noise points.

[0067] Step 203: Select the neighborhood radius: the empirical initial value is 0.1, and it is adjusted according to the density distribution of the data feature space.

[0068] Step 204: Minimum number of samples : Generally set to 5 or select the best value through grid search.

[0069] Step 205: For each point, calculate the number of its neighboring points within the ε radius.

[0070] Step 206: If a point has a certain number of neighbors, it is defined as a “core point”; otherwise, it is defined as a “boundary point” or a “noise point”.

[0071] Step 207: The noise point set is recorded as .

[0072] This step can effectively identify isolated scattered points (such as noise and Raman scattering peaks) and other abnormal detection benchmarks.

[0073] Step 3: Z-score-based statistical screening and misjudgment correction mechanism. To address the potential misjudgment of boundary noise points by the DBSCAN algorithm, the Z-score statistical outlier removal method is introduced as a secondary verification mechanism to screen for potential residual outliers. By calculating the Z-score value and setting an appropriate threshold, the robustness and stability of the detection are enhanced at the level of statistical significance. This "dual-mechanism combined" strategy is an algorithmic innovation that addresses the problem of weak-intensity perturbations in the water background spectrum being easily misidentified.

[0074] In this embodiment, step 3 specifically includes:

[0075] Step 301: Based on the Z-score algorithm, perform secondary verification on the noise judgment result:

[0076] Step 302: For each point , calculate the absolute value of its Z-score, if it satisfies:

[0077]

[0078] The outliers in the statistical sense are considered to be outliers, and the outlier set is constructed .

[0079] Step 303: The final noise point set is the intersection of the two results:

[0080]

[0081] This mechanism can effectively eliminate misjudgments caused by data sparsity or numerical errors, ensuring the accuracy and robustness of noise recognition.

[0082] Step 4: Spectral surface reconstruction using bidirectional cubic spline interpolation. In the sparse regions after scatter removal, bidirectional cubic spline interpolation (2D cubic spline) is used for surface reconstruction. Interpolation is performed along both the excitation and emission wavelength axes during reconstruction to ensure a natural transition between physical continuity and mathematical smoothness. This method is superior to linear interpolation or single-axis interpolation and is particularly suitable for complex water sample backgrounds where high-precision restoration of local signal continuity is required.

[0083] In this embodiment, step 4 specifically includes:

[0084] Step 401: Supplement the normalized spatial region using 2D cubic spline interpolation:

[0085] Step 402: interpolate on each wavelength slice (along the EM axis);

[0086] Step 403: interpolate on each band slice (along the EX axis);

[0087] Step 404: The reconstruction process ensures that each spectrum segment is connected to the next spectrum segment continuously, ensuring a smooth transition of smoothness and curvature.

[0088] The reconstructed spectral surface retains the original data morphology and effectively restores the signal stability of the local area, making it suitable for data analysis and feature extraction in subsequent intervals.

[0089] Step 5: SNR-based quantitative evaluation mechanism. To comprehensively evaluate the denoising effect, the signal-to-noise ratio (SNR) is introduced as a quantitative metric to assess model performance from the perspective of information retention. This evaluation strategy strengthens the method's engineering verifiability and domain adaptability, providing an objective evaluation basis for subsequent modeling and feature extraction, and demonstrating the method's systematic advantages in practical water quality monitoring applications.

[0090] In this embodiment, step 5 specifically includes:

[0091] Step 501: Evaluate the denoising quality using quantitative indicators. For example, the signal-to-noise ratio (SNR) is used to evaluate the denoising quality:

[0092]

[0093] Among them, the signal-to-noise ratio (SNR) is a core indicator in the field of signal processing used to measure the ratio of the effective components to the noise in the signal; A is the signal energy; B is the noise energy.

[0094] like Figure 3 and 4 As shown in the figure, traditional methods typically use a fixed region cutting method, which can easily lead to the accidental excision of some fluorescence peaks, affecting data integrity and analysis results. The present invention has greater universality and can flexibly cut out the scattering area for different samples, while ensuring a smoother and more natural connection between the excised area and the normal area, avoiding mutations, thereby improving data quality and the accuracy of subsequent analysis.

[0095] like Figure 5 To verify the technical effects of the present invention, we will use algorithm comparison to verify its effectiveness. Traditional manual denoising methods rely on subjective judgment (such as "the degree of noise removal is visible to the naked eye") and lack quantitative standards. However, the present invention uses the SNR formula to convert the denoising effect into a specific numerical value, for example:

[0096] The SNR is low before denoising (e.g. Figure 5 1.1013dB in comparison), indicating that noise dominates;

[0097] After denoising, the SNR is significantly improved (e.g. Figure 5 7.999dB in the figure), which proves that the effective signal ratio has increased and the noise has been effectively suppressed.

[0098] It can be seen that the method provided by the present invention significantly improves the overall performance of noise removal in this field. According to the technical indicators, it can be seen that the SNR is significantly improved compared with the traditional algorithm. Under the action of the new algorithm, it is possible to remove the noise signal while ensuring that the data signal is not lost to the greatest extent.

[0099] In the present invention, SNR is used to:

[0100] Parameter tuning: Optimize the neighborhood radius (ε), minimum number of samples (MinPts), or Z-score threshold of DBSCAN by maximizing the SNR value;

[0101] Scenario adaptation: For different types of spectral data (such as water bodies and biological samples), the algorithm's versatility is verified using SNR as an indicator.

[0102] In summary, the SNR formula, as the core quantification of denoising effectiveness in this invention, not only provides an objective means of technical verification but also establishes a complete technical chain of "algorithm optimization - effect evaluation - scenario adaptation." This invention innovatively integrates physical optical properties with machine learning decision models, achieving multi-dimensional cross-validation of abnormal areas through a three-level collaborative filtering architecture (physical denoising - spatial clustering - statistical verification).

[0103] The following are supplementary explanations of professional terms that are not explained in detail in this invention, which are explained in combination with the commonly known meanings in the field:

[0104] 1. DBSCAN (Density Clustering Algorithm)

[0105] Full name: Density-Based Spatial Clustering of Applications with Noise

[0106] Definition: An unsupervised machine learning algorithm that identifies clusters and noise points based on the local density of data points. The core idea is that if the density of data points in a region exceeds a set threshold (defined by the neighborhood radius ε and the minimum number of samples, MinPts), it is considered a valid signal cluster containing "core points." Conversely, points with sparse density are considered noise points.

[0107] Application of the present invention: Identify scattering noise points with abnormal density (such as isolated Raman scattering peaks) in three-dimensional spectral space through DBSCAN.

[0108] 2. Z-score (standard score)

[0109] Meaning: An indicator used in statistics to measure the degree of deviation of a single data point from the mean of a data set. The calculation formula is:

[0110] Where X is the data point value, is the mean, is the standard deviation. The larger the absolute value of the Z-score, the more the data point deviates from the mean.

[0111] The present invention is applied to set a threshold (e.g., | Z |> 3) to identify statistical outliers, which serves as a secondary check on the DBSCAN clustering results to eliminate misjudged noise points.

[0112] 3. Rayleigh Scattering

[0113] Definition: Elastic scattering occurs when light interacts with molecules. The wavelength of the scattered light is the same as the incident light. In fluorescence spectroscopy, Rayleigh scattering appears as a high-intensity, narrow peak along the excitation wavelength and is a major source of noise.

[0114] Scenario of the present invention: As typical scattering noise in 3D fluorescence spectra, it is identified and eliminated through the collaborative mechanism of DBSCAN and Z-score.

[0115] 4. Raman Scattering

[0116] Meaning: Light is inelastically scattered from molecules, causing the scattered light wavelength to shift due to differences in molecular vibrational energy levels. In fluorescence spectroscopy, Raman scattering appears as a broad peak with a fixed offset from the excitation wavelength, interfering with effective signal extraction.

[0117] Scenario of the present invention: "Density sparse outliers" that are the focus of DBSCAN algorithm identification are eliminated through dual detection of spatial density and statistical distribution.

[0118] 5. Cubic Spline Interpolation

[0119] Meaning: A piecewise interpolation method that fits data points through a cubic polynomial function to ensure the continuity of the first-order and second-order derivatives between adjacent segments, thereby achieving a smooth transition.

[0120] Application of the present invention: Bidirectional cubic spline interpolation (along the excitation wavelength axis and the emission wavelength axis) is used to fill the gaps in the spectral data after denoising, restore the continuity of the surface, and avoid mutations or distortions that may be introduced by linear interpolation.

[0121] 6. Signal-to-Noise Ratio (SNR)

[0122] Meaning: A quantitative indicator that measures the ratio of effective components to noise in a signal. The calculation formula is:

[0123] The unit is decibel (dB). A larger value indicates higher signal quality. A is the signal energy; B is the noise energy.

[0124] Application of the present invention: As the core indicator for evaluating denoising effect, by calculating the ratio of signal energy to noise energy, the algorithm's ability to suppress scattered noise and signal fidelity can be objectively verified.

[0125] 7. Core Point / Border Point / Noise Point DBSCAN Algorithm Terminology:

[0126] Core point: a point that contains at least MinPts data points within the neighborhood radius ε and belongs to the core area of ​​the high-density signal cluster;

[0127] Boundary point: A point that is located in the neighborhood of a core point but has less than MinPts data points in its own neighborhood and belongs to the edge of the signal cluster;

[0128] Noise points: points that are neither core points nor boundary points belong to the low-density noise area.

[0129] Application of the present invention: By determining the type of data point (core point / noise point), effective signals and scattered noise can be accurately distinguished.

[0130] 8. 3D Fluorescence Spectroscopy: A spectroscopic technique that detects and analyzes the fluorescence emissions of a substance in three dimensions to obtain information about its structure, composition, and properties. It typically includes three dimensions: excitation wavelength, emission wavelength, and fluorescence intensity. It can more comprehensively reflect the optical properties of a substance and is widely used in environmental monitoring, biomedicine, materials science, and other fields.

[0131] 9. Scattering noise: During spectral measurements, non-fluorescent signals generated by the interaction of light with matter can interfere with the accurate measurement of fluorescence spectra. This interfering signal is called scattering noise. Common scattering noise includes Rayleigh scattering and Mie scattering. These can lead to increased background signals and spectral line broadening in spectral data, affecting the accuracy and reliability of spectral analysis.

[0132] Example 2

[0133] Based on the same concept, the present invention also proposes a device for automatically removing 3D fluorescence spectrum scattering noise, comprising:

[0134] Data preprocessing module: used to load raw spectral data and perform standardization processing to construct a three-dimensional data structure;

[0135] Among them, this module first preprocesses the input spectral data to eliminate the deviation caused by inconsistent data scales or other external factors, and provides standardized input for subsequent descattering processing.

[0136] Noise detection module: Integrates the DBSCAN clustering algorithm and the Z-score verification algorithm to identify and mark scattered noise points;

[0137] This module uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to perform cluster analysis on preprocessed data. By analyzing the local density distribution of the data in three-dimensional spectral space, it effectively identifies and removes low-density noise point areas, achieving preliminary identification and automatic removal of scattered noise. This algorithm has the advantages of not requiring a specified number of clusters and being sensitive to irregular distributions, making it suitable for identifying scattering anomalies in complex backgrounds. To further eliminate potential residual outliers based on preliminary denoising, a statistical detection method based on the Z-score is introduced. This module quantitatively identifies and removes outliers in the spectral matrix by measuring the degree to which the data deviates from the overall mean, compensating for potential omissions in fuzzy boundary regions caused by density clustering methods and improving the accuracy and completeness of overall noise removal.

[0138] Interpolation filling module: Based on the cubic spline interpolation algorithm, the surface of the spectral data after noise removal is reconstructed;

[0139] This module uses cubic spline interpolation to reconstruct and smooth missing regions caused by denoising, restoring the continuity and structural consistency of the spectral surface. This module ensures that denoising does not introduce new mutations or distortions, while improving the visualization of spectral data and the stability of subsequent analysis.

[0140] Verification output module: evaluate the denoising effect through the signal-to-noise ratio indicator and output the optimized spectral data.

[0141] Example 3

[0142] This embodiment also provides an electronic device, referring to Figure 6 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0143] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0144] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0145] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0146] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the methods for automatically removing scattering noise from a 3D fluorescence spectrum in the above embodiments.

[0147] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0148] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0149] The input / output device 408 is used to input or output information.

[0150] Example 4

[0151] This embodiment further provides a readable storage medium, in which a computer program is stored. The computer program includes program code for controlling a process to execute a process, and the process includes the method for automatically removing scattering noise from a 3D fluorescence spectrum according to the first embodiment.

[0152] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0153] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0154] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0155] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A method for automatically removing scattering noise from 3D fluorescence spectra, characterized in that: The following steps are involved: Preprocess the original 3D fluorescence spectrum data to construct a three-dimensional data structure of excitation wavelength, emission wavelength and fluorescence intensity; Based on the spatial density distribution of three-dimensional data structure, the DBSCAN clustering algorithm is used to identify potential scattering noise points; Based on the statistical distribution characteristics, the Z-score method is used to conduct a secondary verification of the clustering results to determine the final noise point set; Interpolation and reconstruction are performed on the spectral data after noise removal to restore the continuity of the spectral surface; wherein the interpolation and reconstruction adopts bidirectional cubic spline interpolation, and segmented interpolation along the excitation wavelength axis and the emission wavelength axis to ensure a smooth transition of the spectral surface; The denoising effect is evaluated by a quantitative index, wherein the quantitative index is the signal-to-noise ratio, and the denoising effect is evaluated by calculating the ratio of signal energy to noise energy.

2. The method for automatically removing 3D fluorescence spectrum scattering noise according to claim 1, wherein: The preprocessing step includes converting the original spectral data into a three-dimensional matrix and eliminating instrument response differences through Z-score standardization. The formula is: in, is the global mean of fluorescence intensity, representing the trend of the data set; is the standard deviation of fluorescence intensity, representing the degree of data dispersion; Indicates the excitation wavelength and emission wavelength Fluorescence intensity value at the intersection point; is the standard score.

3. The method for automatically removing scattering noise from a 3D fluorescence spectrum according to claim 1, wherein: The DBSCAN clustering algorithm includes: The standardized three-dimensional data points are reconstructed into a spatial point set, and the sparse noise point area is identified by setting the neighborhood radius and the minimum number of samples.

4. The method for automatically removing scattering noise from a 3D fluorescence spectrum according to claim 3, wherein: The initial value of the neighborhood radius is 0.1, the initial value of the minimum number of samples is 5, and they can be adaptively adjusted according to data characteristics.

5. The method for automatically removing 3D fluorescence spectrum scattering noise according to claim 1, wherein: The Z-score secondary check includes: The absolute value of the Z-score of each data point is calculated, and points with an absolute value greater than a preset threshold are determined as statistical outliers. The preset threshold is 3.

6. A device for automatically removing scattering noise from a 3D fluorescence spectrum, characterized in that: include: Data preprocessing module: used to load raw spectral data and perform standardization processing to construct a three-dimensional data structure; Noise detection module: Integrates the DBSCAN clustering algorithm and the Z-score verification algorithm to identify and mark scattered noise points; Interpolation filling module: Based on the cubic spline interpolation algorithm, the spectral data after noise removal is reconstructed. The interpolation reconstruction adopts bidirectional cubic spline interpolation, and interpolates along the excitation wavelength axis and the emission wavelength axis in sections to ensure a smooth transition of the spectral surface. Verification output module: The denoising effect is evaluated by a quantitative indicator, and the optimized spectral data is output. The quantitative indicator is the signal-to-noise ratio, and the denoising effect is evaluated by calculating the ratio of signal energy to noise energy.

7. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for automatically removing 3D fluorescence spectrum scattering noise according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the method for automatically removing 3D fluorescence spectrum scattering noise according to any one of claims 1 to 5.

Images