Spectral data processing method and spectral data processing apparatus

Abstract

The application provides a spectrum data processing method and a spectrum data processing device, and relates to the technical field of data processing. The method comprises the following steps: obtaining a plurality of original spectrum data corresponding to a target object; performing a preprocessing operation on each piece of original spectrum data to obtain a plurality of pieces of preprocessed spectrum data; determining a noise screening threshold according to spectrum intensity information corresponding to each wavelength point in each piece of preprocessed spectrum data; performing a first screening on each piece of preprocessed spectrum data according to the noise screening threshold to obtain a first spectrum data set after screening; determining standard spectrum data corresponding to the target object according to the first spectrum data set after screening; and performing a second screening according to the standard spectrum data to obtain a plurality of target spectrum data corresponding to the target object. The method effectively overcomes the limitations of traditional single-dimensional screening. Noise screening is performed based on a dynamically determined noise screening threshold, and an internal standard spectrum is generated based on clustering and morphological consistency analysis, so that the screening accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a spectrum data processing method and a spectrum data processing device. BACKGROUND

[0002] High-quality spectrum data screening is a key pre-step to ensure the accuracy and reliability of Raman, infrared and other spectrum analysis results. Spectrum analysis technology can obtain molecular fingerprint information of a substance by detecting the interaction between the substance and light, and is widely used in the fields of biomedicine, material science and environmental detection. However, in the actual acquisition process, factors such as instrument noise, environmental interference, sample pollution, and operation fluctuations (such as defocusing or unstable laser power) can cause a large number of abnormal and low-quality spectra, and directly using these spectra will affect the accuracy of subsequent qualitative identification and quantitative analysis.

[0003] Existing spectrum screening methods mainly include manual experience visual inspection or fixed threshold screening methods.

[0004] However, the manual screening method relies on the subjective judgment of the operator, is inefficient and has poor consistency; the method using fixed threshold has poor self-adaptability and weak generalization ability. SUMMARY

[0005] The present application aims to overcome the deficiencies in the prior art, and provides a spectrum data processing method and a spectrum data processing device to facilitate objective and high-precision spectrum data screening.

[0006] To achieve the above-mentioned purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0007] In a first aspect, the embodiments of the present application provide a spectrum data processing method, comprising:

[0008] Obtaining a plurality of original spectrum data corresponding to a target object, each original spectrum data comprising spectrum intensity information corresponding to different wavelength points;

[0009] Performing a preprocessing operation on each original spectrum data to obtain a plurality of preprocessed spectrum data;

[0010] Determining a noise screening threshold according to the spectrum intensity information corresponding to each wavelength point in each preprocessed spectrum data;

[0011] Performing a first screening on each preprocessed spectrum data according to the noise screening threshold to obtain a first screened spectrum data set;

[0012] Determining standard spectrum data corresponding to the target object according to the first screened spectrum data set;

[0013] According to the standard spectrum data, second screening is performed to obtain a plurality of target spectrum data corresponding to the target object.

[0014] Optionally, the determining the noise screening threshold according to the spectrum intensity information of each wavelength point in each piece of preprocessed spectrum data comprises:

[0015] According to the spectrum intensity information of adjacent wavelength points in each piece of preprocessed spectrum data, peak intensity fluctuation information of each piece of preprocessed spectrum data is determined.

[0016] According to the peak intensity fluctuation information of each piece of preprocessed spectrum data, the noise screening threshold is determined.

[0017] Optionally, the determining the peak intensity fluctuation information of each piece of preprocessed spectrum data according to the spectrum intensity information of adjacent wavelength points comprises:

[0018] According to the spectrum intensity information of adjacent wavelength points, a spectrum intensity difference value of the adjacent wavelength points is determined.

[0019] According to the spectrum intensity difference value of the adjacent wavelength points, a spectrum intensity difference mean value is determined.

[0020] According to the spectrum intensity difference value of the adjacent wavelength points, the spectrum intensity difference mean value, and the number of wavelength points, the peak intensity fluctuation information of the preprocessed spectrum data is determined.

[0021] Optionally, the determining the noise screening threshold according to the peak intensity fluctuation information of each piece of preprocessed spectrum data comprises:

[0022] According to the peak intensity fluctuation information of each piece of preprocessed spectrum data, a peak intensity fluctuation mean value and a peak intensity fluctuation standard deviation are determined.

[0023] According to the peak intensity fluctuation mean value and the peak intensity fluctuation standard deviation, the noise screening threshold is determined.

[0024] Optionally, the first screening of each piece of preprocessed spectrum data according to the noise screening threshold to obtain a first spectrum data set after screening comprises:

[0025] Each piece of preprocessed spectrum data is traversed, and for a current preprocessed spectrum data traversed, if the peak intensity fluctuation information of the current preprocessed spectrum data is less than or equal to the noise screening threshold, the current preprocessed spectrum data is added to the first spectrum data set.

[0026] Optionally, the determining the standard spectrum data corresponding to the target object according to the first spectrum data set after screening comprises:

[0027] acquire reference spectral data corresponding to the target object;

[0028] perform cluster analysis on each piece of spectral data in the first filtered spectral data set according to the reference spectral data, to obtain a second filtered spectral data set;

[0029] generate standard spectral data corresponding to the target object according to spectral intensity information corresponding to each wavelength point of each piece of spectral data in the second filtered spectral data set.

[0030] Optionally, the second filtering according to the standard spectral data to obtain a plurality of target spectral data corresponding to the target object comprises:

[0031] perform second filtering on each piece of spectral data in the second spectral data set according to the standard spectral data, to obtain a plurality of target spectral data corresponding to the target object.

[0032] Optionally, the second filtering according to the standard spectral data on each piece of spectral data in the second spectral data set to obtain a plurality of target spectral data corresponding to the target object comprises:

[0033] determine a morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data according to the standard spectral data;

[0034] perform second filtering on each piece of spectral data in the second spectral data set according to the morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data, to obtain a plurality of target spectral data corresponding to the target object.

[0035] Optionally, the second filtering on each piece of spectral data in the second spectral data set according to the morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data comprises:

[0036] traverse each piece of spectral data in the second spectral data set, and if a morphological similarity index of a current piece of spectral data and the standard spectral data is greater than or equal to a preset correlation threshold, add the current piece of spectral data to a third spectral data set;

[0037] use each piece of spectral data in the third spectral data set as the plurality of target spectral data corresponding to the target object.

[0038] Optionally, the preprocessing operation comprises one or more of the following: noise processing, spectral baseline fitting and correction, filtering processing, and normalization processing.

[0039] In a second aspect, the embodiments of the present application further provide a spectrum data processing apparatus, comprising: an acquisition module, a determination module, and a screening module;

[0040] The acquisition module is configured to acquire a plurality of original spectrum data corresponding to a target object, each piece of original spectrum data comprising spectrum intensity information corresponding to different wavelength points;

[0041] The determination module is configured to perform a preprocessing operation on each piece of original spectrum data to obtain a plurality of pieces of preprocessed spectrum data;

[0042] The determination module is configured to determine a noise screening threshold according to the spectrum intensity information corresponding to each wavelength point in each piece of preprocessed spectrum data;

[0043] The screening module is configured to perform a first screening on each piece of preprocessed spectrum data according to the noise screening threshold to obtain a first spectrum data set after screening;

[0044] The screening module is configured to determine standard spectrum data corresponding to the target object according to the first spectrum data set after screening;

[0045] The screening module is configured to perform a second screening according to the standard spectrum data to obtain a plurality of pieces of target spectrum data corresponding to the target object.

[0046] Optionally, the determination module is specifically configured to determine peak intensity fluctuation information of each piece of preprocessed spectrum data according to the spectrum intensity information of adjacent wavelength points in each piece of preprocessed spectrum data;

[0047] The noise screening threshold is determined according to the peak intensity fluctuation information of each piece of preprocessed spectrum data.

[0048] Optionally, the determination module is specifically configured to determine a spectrum intensity difference value of adjacent wavelength points according to the spectrum intensity information of the adjacent wavelength points;

[0049] A spectrum intensity difference mean value is determined according to the spectrum intensity difference value of the adjacent wavelength points;

[0050] The peak intensity fluctuation information of the preprocessed spectrum data is determined according to the spectrum intensity difference value of the adjacent wavelength points, the spectrum intensity difference mean value, and the number of wavelength points.

[0051] Optionally, the determination module is specifically configured to determine a peak intensity fluctuation mean value and a peak intensity fluctuation standard deviation according to the peak intensity fluctuation information of each piece of preprocessed spectrum data;

[0052] The noise screening threshold is determined according to the peak intensity fluctuation mean value and the peak intensity fluctuation standard deviation.

[0053] Optionally, the screening module is specifically configured to traverse each piece of preprocessed spectral data, and for current preprocessed spectral data traversed, if peak intensity fluctuation information of the current preprocessed spectral data is less than or equal to the noise screening threshold, the current preprocessed spectral data is added to the first spectral data set.

[0054] Optionally, the screening module is specifically configured to obtain reference spectral data corresponding to the target object.

[0055] According to the reference spectral data, cluster analysis is performed on each piece of spectral data in the screened first spectral data set, to obtain a second spectral data set screened.

[0056] According to spectral intensity information corresponding to each wavelength point of each piece of spectral data in the screened second spectral data set, standard spectral data corresponding to the target object is generated.

[0057] Optionally, the screening module is specifically configured to perform second screening on each piece of spectral data in the second spectral data set according to the standard spectral data, to obtain a plurality of target spectral data corresponding to the target object.

[0058] Optionally, the screening module is specifically configured to determine a morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data according to the standard spectral data.

[0059] According to the morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data, second screening is performed on each piece of spectral data in the second spectral data set, to obtain a plurality of target spectral data corresponding to the target object.

[0060] Optionally, the screening module is specifically configured to traverse each piece of spectral data in the second spectral data set, and for current spectral data traversed, if a morphological similarity index of the current spectral data and the standard spectral data is greater than or equal to a preset correlation threshold, the current spectral data is added to a third spectral data set.

[0061] Each piece of spectral data in the third spectral data set is taken as a plurality of target spectral data corresponding to the target object.

[0062] Optionally, the preprocessing operation includes one or more of the following: noise processing, spectral baseline fitting and correction, filtering processing, and normalization processing.

[0063] In a third aspect, an electronic device is provided, and the electronic device includes a processor, a storage medium, and a bus. The storage medium stores machine readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus. The processor executes the machine readable instructions to implement the spectrum data processing method provided in the first aspect.

[0064] In a fourth aspect, a computer readable storage medium is provided, and the storage medium stores a computer program. When the computer program is run by a processor, the computer program performs the spectrum data processing method provided in the first aspect.

[0065] The present application has the following beneficial effects:

[0066] The present application provides a spectrum data processing method and a spectrum data processing device, including: obtaining a plurality of original spectrum data corresponding to a target object; performing a preprocessing operation on each piece of original spectrum data to obtain a plurality of pieces of preprocessed spectrum data; determining a noise screening threshold according to the spectrum intensity information of each wavelength point in each piece of preprocessed spectrum data; performing a first screening on each piece of preprocessed spectrum data according to the noise screening threshold to obtain a first screened spectrum data set; determining standard spectrum data corresponding to the target object according to the first screened spectrum data set; and performing a second screening according to the standard spectrum data to obtain a plurality of target spectrum data corresponding to the target object. The present application screens the spectrum data by combining the double screening standards of noise and morphological consistency, which can effectively overcome the limitations of traditional single-dimensional screening. For noise screening, the noise screening threshold is dynamically determined according to the statistical characteristics of each batch of data, so that the screening standard can adapt to different batches and different signal-to-noise ratio levels of the data set, effectively avoiding the problems of excessive strictness or looseness caused by manually setting a fixed threshold, and ensuring the scientificity and universality of the screening condition. For the screening of morphological consistency, the internal standard spectrum is generated based on clustering and morphological consistency analysis is performed, which eliminates the dependence on external standards or prior knowledge, ensures the objectivity and universality of the morphological consistency screening process, enables the system to automatically identify and retain the most representative and highest quality spectrum in the current data set in an unsupervised mode, significantly reduces the dependence on professional experience, and provides a reliable guarantee for large sample and high throughput spectrum analysis. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0068] Figure 1 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure One ;

[0069] Figure 2 A schematic diagram of low-quality spectral data under different influencing factors provided in the embodiments of this application;

[0070] Figure 3 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Two ;

[0071] Figure 4 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Three ;

[0072] Figure 5 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Four ;

[0073] Figure 6 This application provides a schematic diagram illustrating noise screening in an embodiment.

[0074] Figure 7 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Five ;

[0075] Figure 8 This is a schematic diagram of a clustering analysis result provided in an embodiment of this application;

[0076] Figure 9 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Six ;

[0077] Figure 10 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Seven ;

[0078] Figure 11 This is a schematic diagram illustrating the morphological consistency screening results provided in an embodiment of this application.

[0079] Figure 12 This is a schematic diagram illustrating the overall screening results provided in an embodiment of this application.

[0080] Figure 13 This is a schematic diagram of the dual-standard dynamic threshold filtering results provided in the embodiments of this application;

[0081] Figure 14 This is a schematic diagram of the overall process of a spectral data processing method provided in an embodiment of this application;

[0082] Figure 15A schematic diagram of a spectrum data processing device provided by an embodiment of the present application is shown in FIG. 1.

[0083] Figure 16 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will combine the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purpose of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts used in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0085] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0086] It should be noted that the term “comprising” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0087] The screening of high-quality spectra is a key pre-step to ensure the accuracy and reliability of Raman, infrared, and other spectral analysis results. Spectral analysis technology can obtain molecular fingerprint information of substances by detecting the interaction between substances and light, and is widely used in biomedical, material science, environmental detection, and other fields. However, in the actual acquisition process, factors such as instrument noise, environmental interference, sample contamination, and operation fluctuations (such as defocusing or unstable laser power) will introduce a large number of abnormal and low-quality spectra, directly affecting the accuracy of subsequent qualitative identification and quantitative analysis.

[0088] The existing spectrum screening methods mainly include manual experience visual inspection, single noise index threshold filtering or simple statistical outlier rejection strategy. The manual screening method relies on the subjective judgment of the operator, is low in efficiency and poor in consistency, and is difficult to cope with the high-throughput spectrum data scene. Some automatic methods use fixed thresholds (such as setting a uniform signal-to-noise ratio or an upper limit of intensity fluctuation) to independently evaluate a single spectrum, which lacks consideration of the overall distribution characteristics of the data and the shape consistency between spectra, and is prone to miss screening of high-noise spectra or mistakenly delete effective spectra. In addition, some methods try to identify outliers through dimension reduction methods such as principal component analysis, but when the data heterogeneity is high, it is easy to confuse real biological variation with collection abnormalities, and the reliability is limited.

[0089] The existing technology has the following limitations: first, it fails to combine noise level and shape similarity for comprehensive discrimination; second, it lacks an adaptive mechanism for dynamically generating thresholds based on the data itself, and has weak generalization ability; third, it relies too much on manual participation or parameter experience setting, which is not conducive to standardization and automation. Therefore, the current spectrum screening process still faces the problem of balancing efficiency, consistency and accuracy.

[0090] Therefore, the present application provides a spectrum data processing method, which combines the dual screening standards of noise and shape consistency for spectrum data screening, which can effectively overcome the limitations of traditional single-dimensional screening. For noise screening, the noise screening threshold is dynamically determined according to the statistical characteristics of each batch of data, so that the screening standard can adapt to different batches and different signal-to-noise ratio levels of the data set, effectively avoiding the problem of over-tight or over-loose caused by manually setting a fixed threshold, and ensuring the scientificity and universality of the screening condition. For shape consistency screening, internal standard spectra are generated based on clustering and shape consistency analysis is performed, eliminating the dependence on external standards or prior knowledge, so that the system can automatically identify and retain the most representative and highest quality spectra in the current data set in an unsupervised mode, significantly reducing the dependence on professional experience, and providing a reliable guarantee for large sample and high-throughput spectrum analysis.

[0091] Figure 1 Flowchart of the spectrum data processing method provided by the embodiments of the present application Figure One The execution subject of the present application can be a computer device, as shown in Figure 1 The method comprises the following steps.

[0092] S101, acquiring a plurality of original spectrum data corresponding to a target object.

[0093] Each piece of original spectrum data includes spectrum intensity information corresponding to different wavelength points.

[0094] The target object can refer to an object to be measured. The target object is irradiated by a set light source, and a plurality of pieces of original spectral data corresponding to the target object can be acquired. Different regions or different positions on the target object irradiated by the same light source can correspond to a piece of spectral data.

[0095] Different spectral technologies have different application ranges. According to scene requirements, the irradiation light source used can be different. The present application can be applied to a one-dimensional spectral scene such as Raman spectrum, fluorescence spectrum, near-infrared spectrum, ultraviolet spectrum, visible spectrum, and the like.

[0096] Generally, spectral data can be composed of two-dimensional information. The abscissa of the spectrum is usually used to locate the specific waveband of light, that is, the wavelength or frequency. The ordinate of the spectrum is usually used to reflect the energy or signal strength of light at the wavelength, that is, the spectral intensity information. Therefore, the original spectral data includes spectral intensity information corresponding to different wavelength points.

[0097] S102, each piece of original spectral data is preprocessed to obtain each piece of preprocessed spectral data.

[0098] Figure 2 The low-quality spectral data under the influence of different factors provided by the embodiments of the present application is shown in the following figure, Figure 2 (a) in the figure is low-quality spectral data caused by a temporary failure of a detector or loss of a signal; Figure 2 (b) in the figure is low-quality spectral data caused by carbonization of a sample irradiated by a laser; Figure 2 (c) in the figure is spectral distortion caused by excessive cosmic rays; Figure 2 (d) in the figure is signal blurring caused by inaccurate focusing on the focal plane. If the original spectral data without preprocessing and screening contains the above-mentioned abnormalities, the accuracy of subsequent analysis will be seriously affected.

[0099] In some embodiments, a series of preprocessing operations can be performed on each piece of original spectral data acquired, to eliminate various disturbances and system errors in the original spectrum, and to provide a high-quality, standardized spectral data basis for subsequent qualitative or quantitative analysis.

[0100] Optionally, the preprocessing operation includes but is not limited to noise processing, spectral baseline fitting and correction, filtering processing, and normalization processing.

[0101] The preprocessing procedure can be as follows: removing cosmic rays in the original spectral data to eliminate random high-intensity noise interference; deducting background signals to reduce interference from the sample substrate; performing noise reduction processing to improve the signal-to-noise ratio while retaining effective spectral features; fitting and correcting the spectral baseline to eliminate baseline drift caused by fluorescence background and the like; implementing smoothing filtering to reduce the influence of high-frequency noise; and finally normalizing the spectral intensity to eliminate intensity differences caused by fluctuations in experimental conditions, so that the data is in a unified dimension, facilitating subsequent analysis and comparison.

[0102] Of course, the manner of preprocessing is not limited to the above examples, for example, wavelet transform or median filtering can also be used for noise reduction, and standard normal variable transformation (SNV) or vector normalization can be used instead of maximum normalization.

[0103] S103, according to the spectral intensity information corresponding to each wavelength point in each piece of preprocessed spectral data, determine a noise screening threshold.

[0104] For each piece of preprocessed spectral data obtained after preprocessing, the noise screening threshold of the target object under the current batch of spectral data can be dynamically determined according to the spectral intensity information corresponding to each wavelength point in each piece of preprocessed spectral data.

[0105] Since the spectral data corresponding to different target objects are different, the spectral data characteristics of different target objects are different, and through the method, the noise screening threshold can be dynamically and adaptively determined, instead of relying on a fixed noise screening threshold, and the rationality and reliability of spectral data screening can be determined.

[0106] S104, according to the noise screening threshold, first screen each piece of preprocessed spectral data to obtain a first screened spectral data set.

[0107] After screening each piece of preprocessed spectral data by the noise screening threshold, spectral data with obvious noise pollution and transient interference, i.e., high-noise spectral data, can be screened out, and the first spectral data set is obtained. Each piece of spectral data in the first spectral data set is low-noise spectral data.

[0108] S105, according to the first screened spectral data set, determine the standard spectral data corresponding to the target object.

[0109] Based on the first screening based on noise, further correlation screening based on spectral morphology consistency can be performed. First, based on each piece of spectral data in the first screened spectral data set, clustering analysis can be performed to determine the standard spectral data corresponding to the target object.

[0110] The standard spectrum data here is not a pre-existing external reference standard (such as an ideal spectrum in a standard substance library), but an "average template" that best represents high-quality and consistent morphology spectrum automatically extracted from the current batch of spectrum data of the target object.

[0111] The standard spectrum data generated by the present scheme is used because the applicability of the reference spectrum depending on manual selection or known standard samples is poor, and the present scheme automatically generates standard spectrum data based on the characteristics of the current data itself, which has better adaptability.

[0112] S106, according to the standard spectrum data, performing a second screening to obtain a plurality of target spectrum data corresponding to the target object.

[0113] According to the generated standard spectrum data, the spectrum data after noise screening can be screened again for morphology consistency to exclude spectrum data with low morphology correlation with the standard spectrum data, and finally obtain a plurality of target spectrum data corresponding to the target object.

[0114] Therefore, the plurality of target spectrum data corresponding to the target object is the spectrum data that meets the low noise and morphology consistency at the same time and is screened from the original spectrum data.

[0115] Through the above screening method, low-quality spectrum data caused by various interferences can be eliminated to the maximum extent while retaining effective spectrum features. The plurality of target spectrum data obtained can be used for qualitative or quantitative analysis of the target object.

[0116] In some embodiments, based on the characteristic absorption peak (corresponding to a specific molecule or functional group) of the target spectrum data, the chemical or biological components contained in the object can be determined, including qualitative identification and quantitative calculation. Through the peak position, peak shape, peak intensity and other information of the spectrum, the microstructure or macroscopic morphology related characteristics of the object can be analyzed. Through the change of the spectrum with the external conditions or the state of the object, the physical state, quality level, health condition and other attributes of the object can be deduced.

[0117] In summary, the spectrum data processing method provided in the embodiment comprises: obtaining a plurality of original spectrum data corresponding to a target object; performing a preprocessing operation on each piece of original spectrum data to obtain a plurality of pieces of preprocessed spectrum data; determining a noise screening threshold according to spectrum intensity information corresponding to each wavelength point in each piece of preprocessed spectrum data; performing a first screening on each piece of preprocessed spectrum data according to the noise screening threshold to obtain a first spectrum data set after screening; determining standard spectrum data corresponding to the target object according to the first spectrum data set after screening; and performing a second screening according to the standard spectrum data to obtain a plurality of target spectrum data corresponding to the target object. The spectrum data is screened by combining the double screening standards of noise and morphological consistency, which can effectively overcome the limitations of traditional single-dimensional screening. For noise screening, the noise screening threshold is dynamically determined according to the statistical characteristics of each batch of data, so that the screening standard can adapt to different batches and different signal-to-noise ratio levels of the data set, effectively avoiding the problems of excessive strictness or looseness caused by manually setting a fixed threshold, and ensuring the scientificity and universality of the screening condition. For the screening of morphological consistency, the internal standard spectrum is generated based on clustering, and the morphological consistency analysis is performed, which eliminates the dependence on external standards or prior knowledge, so that the system can automatically identify and retain the most representative and highest-quality spectrum in the current data set in an unsupervised mode, significantly reducing the dependence on professional experience, and providing reliable guarantee for large-sample and high-throughput spectrum analysis.

[0118] Figure 3 Flowchart of the spectrum data processing method provided in the embodiment of the application Figure Two Optionally, in step S103, the noise screening threshold is determined according to the spectrum intensity information corresponding to each wavelength point in each piece of preprocessed spectrum data, comprising:

[0119] S201, peak intensity fluctuation information of each piece of preprocessed spectrum data is determined according to spectrum intensity information of adjacent wavelength points in each piece of preprocessed spectrum data.

[0120] In some embodiments, for each piece of preprocessed spectrum data, the standard deviation of the spectrum intensity difference of adjacent wavelength points can be calculated, and the standard deviation is defined as the peak intensity fluctuation information F of this piece of preprocessed spectrum data. The peak intensity fluctuation information F is used to quantify the smoothness and noise level of the spectrum signal.

[0121] In this way, the peak intensity fluctuation information F of each piece of preprocessed spectrum data can be obtained.

[0122] S202, the noise screening threshold is determined according to the peak intensity fluctuation information of each piece of preprocessed spectrum data.

[0123] Based on the peak intensity fluctuation information F of the current batch of pre-processed spectral data of the target object, a noise screening threshold can be further calculated. That is, the noise screening threshold is calculated based on the same batch of spectral data. When the target object changes or the illumination light source changes, the original spectral data corresponding to the target object will change, thereby resulting in the noise screening threshold finally obtained.

[0124] Therefore, according to the peak intensity fluctuation value calculated by each batch of spectral data itself, the noise screening threshold is dynamically determined. The screening standard can be self-adapted to different signal-to-noise ratio levels and data characteristics, which is a key means to realize automatic and high-precision screening.

[0125] Figure 4 Flowchart of the spectral data processing method provided by the embodiment of the present application Figure Three In step S201, the peak intensity fluctuation information of each piece of pre-processed spectral data is determined according to the spectral intensity information of adjacent wavelength points in each piece of pre-processed spectral data, including:

[0126] S301, the spectral intensity difference of adjacent wavelength points is determined according to the spectral intensity information of adjacent wavelength points.

[0127] In some embodiments, the spectral intensity information of the pre-processed spectral data can be calculated by the following formula:

[0128]

[0129] Wherein, The spectral intensity difference of adjacent wavelength points, that is, the spectral intensity difference between the i th and i+1 th wavelength points.

[0130] Wherein, for each pair of adjacent wavelength points, there is one . That is, assuming that a piece of pre-processed spectral data contains 10 wavelength points, the first wavelength point and the second wavelength point correspond to one , the second wavelength point and the third wavelength point correspond to one , and so on until the ninth wavelength point and the tenth wavelength point are calculated .

[0131] S302, the spectral intensity difference average is determined according to the spectral intensity difference of adjacent wavelength points.

[0132] The above indicates the spectral intensity difference average, wherein, , that is, the average value of the spectral intensity difference of each pair of adjacent wavelength points calculated above is obtained .

[0133] S303. Determine the peak intensity fluctuation information of the preprocessed spectral data based on the spectral intensity difference between adjacent wavelength points, the average spectral intensity difference, and the number of wavelength points.

[0134] In the formula, N represents the number of wavelength points. When there are ten wavelength points, nine can be calculated. Calculate nine The standard deviation is used to obtain the peak intensity fluctuation information F of the preprocessed spectral data.

[0135] Each preprocessed spectral data point undergoes the above steps to obtain its corresponding peak intensity fluctuation information F. Generally, a higher F value indicates that the spectrum is more severely affected by noise or anomalous interference.

[0136] Figure 5 A flowchart illustrating the spectral data processing method provided in the embodiments of this application. Figure Four In step S202, based on the peak intensity fluctuation information of each preprocessed spectral data, a noise screening threshold is determined, including:

[0137] S401. Based on the peak intensity fluctuation information of each preprocessed spectral data, determine the mean peak intensity fluctuation and the standard deviation of peak intensity fluctuation.

[0138] Alternatively, the mean and standard deviation of peak intensity fluctuation can be calculated using the following formula:

[0139]

[0140]

[0141] in, This represents the average peak intensity fluctuation. This represents the standard deviation of peak intensity fluctuation. This represents the peak intensity fluctuation information of the spectral data, where, .

[0142] S402. Determine the noise screening threshold based on the mean peak intensity fluctuation and the standard deviation of peak intensity fluctuation.

[0143] In some embodiments, the sum of the mean peak intensity fluctuation and the standard deviation of peak intensity fluctuation can be used as the noise screening threshold, that is, the noise screening threshold is set to... .

[0144] The noise screening threshold calculation method of this approach allows the obtained noise screening threshold to dynamically and adaptively change with the characteristics and data quality of the same batch of spectral data, thereby improving the screening accuracy during noise screening.

[0145] Optionally, in step S104, the pre-processed spectral data is subjected to first screening according to a noise screening threshold, to obtain a first screened spectral data set, including: traversing the pre-processed spectral data, and for the current pre-processed spectral data traversed, if the peak intensity fluctuation information of the current pre-processed spectral data is less than or equal to the noise screening threshold, the current pre-processed spectral data is added to the first spectral data set.

[0146] In some embodiments, based on the noise screening threshold determined above, the pre-processed spectral data can be subjected to first screening. Wherein, each pre-processed spectral data can be traversed in turn, and for the pre-processed spectral data currently traversed, the peak intensity fluctuation information F of the pre-processed spectral data is compared with the determined noise screening threshold, and if the peak intensity fluctuation information F of the pre-processed spectral data is less than or equal to the noise screening threshold, the pre-processed spectral data is added to the first spectral data set as low-noise spectral data. Thus, the peak intensity fluctuation information F exceeding the noise screening threshold is screened out.

[0147] For the calculation of the peak intensity fluctuation information F, in addition to the calculation by the standard deviation of the spectral intensity difference of adjacent wavelength points, statistical quantities such as mean absolute deviation or coefficient of variation can also be used for calculation; at the same time, the setting of the noise screening threshold can also be obtained by the box plot method or robust statistical quantity calculation.

[0148] Figure 6 A noise screening display diagram is provided for the embodiments of the present application. As shown in Figure 6 The horizontal coordinate represents the peak intensity fluctuation information F, and the vertical coordinate represents the number of spectra; the dashed line in the figure represents the noise screening threshold determined based on the present scheme, the histogram statistics result of the spectral data with the peak intensity fluctuation information F greater than the noise screening threshold is located to the right of the noise screening threshold, and the histogram statistics result of the spectral data with the peak intensity fluctuation information F less than or equal to the noise screening threshold is located to the left of the noise screening threshold. Through noise screening, each spectral data located to the right of the noise screening threshold can be screened out.

[0149] Figure 7 The flowchart of the spectral data processing method provided for the embodiments of the present application is shown in Figure Five Optionally, in step S105, the standard spectral data corresponding to the target object is determined according to the first screened spectral data set, including:

[0150] S501, obtaining the reference spectral data corresponding to the target object.

[0151] In some embodiments, high-precision instruments can be used to measure the reference spectral data corresponding to the target object. This reference spectral data is a standard instance in the real world. However, even if the same object is measured using high-precision instruments, spectral fluctuations can occur, and therefore a single measurement result cannot represent the common characteristics of the spectral data of the target object.

[0152] S502, according to the reference spectral data, performing cluster analysis on each piece of spectral data in the screened first spectral data set to obtain a screened second spectral data set.

[0153] In this embodiment, cluster analysis can be performed on each piece of spectral data in the first spectral data set to extract the spectral data with the largest quantity and the most consistent form with the reference spectral data, thereby obtaining the screened second spectral data set.

[0154] The cluster analysis can be based on the Euclidean distance, and each piece of spectral data in the first spectral data set is divided into several categories by calculating the Euclidean distance between each piece of spectral data in the first spectral data set and the reference spectral data, until convergence.

[0155] A Gaussian mixture model can also be used to model the spectral distribution probability, estimate the parameters by using an Expectation-Maximization Algorithm (EM) algorithm, and select the optimal cluster by using a Bayesian Information Criterion (BIC) criterion.

[0156] S503, generating standard spectral data corresponding to the target object according to the spectral intensity information corresponding to each wavelength point of each piece of spectral data in the screened second spectral data set.

[0157] The standard spectral data corresponding to the target object can be obtained by averaging each piece of spectral data in the second spectral data set. That is, the spectral intensity information at the corresponding wavelength point of each piece of spectral data in the second spectral data set is averaged to obtain the standard spectral intensity information corresponding to each wavelength point, and the standard spectral data corresponding to the target object is obtained from the standard spectral intensity information corresponding to each wavelength point.

[0158] Of course, in addition to the median or principal component analysis of each piece of spectral data in the second spectral data set, the standard spectral data corresponding to the target object can also be constructed.

[0159] In this embodiment, without human intervention or prior knowledge, the internal standard spectrum for form comparison can be automatically generated by performing cluster analysis on the noise-screened spectral data and calculating the average spectrum, and the method ensures the objectivity and universality of the form consistency screening process.

[0160] Figure 8 A clustering analysis result schematic diagram provided by an embodiment of the present application can form a core cluster located in a middle position, and each spectrum data in the core cluster constitutes a second spectrum data set.

[0161] Optionally, in step S503, the second filtering is performed according to the standard spectrum data, and a plurality of target spectrum data corresponding to the target object is obtained, including: performing the second filtering on each spectrum data in the second spectrum data set according to the standard spectrum data, and obtaining the plurality of target spectrum data corresponding to the target object.

[0162] Correlation analysis can be performed on each spectrum data in the second spectrum data set and the standard spectrum data, and the second filtering is performed on each spectrum data in the second spectrum data set based on the correlation analysis result.

[0163] Figure 9 A flowchart of a spectrum data processing method provided by an embodiment of the present application Figure Six Optionally, in the above step, the second filtering is performed on each spectrum data in the second spectrum data set according to the standard spectrum data, and the plurality of target spectrum data corresponding to the target object is obtained, including:

[0164] S601, determining a shape similarity index of each spectrum data in the second spectrum data set and the standard spectrum data according to the standard spectrum data.

[0165] In an implementation manner, the Pearson correlation coefficient C of each spectrum data in the second spectrum data set and the standard spectrum data can be calculated, the Pearson correlation coefficient C is taken as the shape similarity index, the second filtering is performed based on the Pearson correlation coefficient C, and the plurality of target spectrum data corresponding to the target object is obtained.

[0166] S602, performing the second filtering on each spectrum data in the second spectrum data set according to the shape similarity index of each spectrum data in the second spectrum data set and the standard spectrum data, and obtaining the plurality of target spectrum data corresponding to the target object.

[0167] Generally, the higher the Pearson correlation coefficient C, the more consistent the shape of the spectrum data with the standard spectrum data. Optionally, a correlation threshold value can be set, and the second filtering is performed on each spectrum data in the second spectrum data set by using the correlation threshold value and the Pearson correlation coefficient C of each spectrum data in the second spectrum data set and the standard spectrum data, so as to obtain the target spectrum data meeting the shape consistency.

[0168] In another implementation manner, the cosine similarity or dynamic time warping or other measurement manners can also be used to measure the shape correlation between each spectrum data and the standard spectrum data.

[0169] For example, the cosine similarity between each piece of spectral data and the standard spectral data is calculated, and the cosine similarity is taken as the shape similarity index between the spectral data and the standard spectral data.

[0170] Figure 10 Flowchart of the spectral data processing method provided by the embodiment of the present application Figure Seven Optionally, in step S601, the second filtering is performed on each piece of spectral data in the second spectral data set according to the shape similarity index of each piece of spectral data in the second spectral data set and the standard spectral data, including:

[0171] S701, each piece of spectral data in the second spectral data set is traversed, and for the current spectral data traversed, if the shape similarity index of the current spectral data and the standard spectral data is greater than or equal to a preset correlation threshold, the current spectral data is added to the third spectral data set.

[0172] Taking the Pearson correlation coefficient C as the shape similarity index for illustration, each piece of spectral data in the second spectral data set can be traversed in turn, and the Pearson correlation coefficient C of the current spectral data traversed is compared with the preset correlation threshold, if the Pearson correlation coefficient C of the current spectral data is greater than or equal to the preset correlation threshold, it is considered that the shape consistency of the current spectral data and the standard spectral data is high, and the spectral data can be reserved and added to the third spectral data set.

[0173] Through the above filtering, the spectral data with the Pearson correlation coefficient C less than the preset correlation threshold can be filtered out, and only the spectral data with high similarity to the standard spectral data is reserved.

[0174] For example, the preset correlation threshold can be set to 0.8, and the Pearson correlation coefficient C is usually between 0 and 1.

[0175] S702, each piece of spectral data in the third spectral data set is taken as a plurality of target spectral data corresponding to the target object.

[0176] Through the above-mentioned double filtering, that is, noise filtering and shape consistency filtering, each piece of spectral data in the third spectral data set obtained is the spectral data that meets the noise and shape filtering conditions at the same time, that is, a plurality of target spectral data with low noise and high shape consistency is obtained. Thus, under the premise of retaining effective spectral features, low-quality data caused by various interferences is maximally filtered out.

[0177] Figure 11A morphology consistency screening result display schematic diagram provided by an embodiment of the present application is shown in FIG. 6. In the diagram, the abscissa represents peak intensity fluctuation information F, the ordinate represents a morphology similarity index, a dashed line represents a set correlation threshold, and spectral data below the dashed line is spectral data with a morphology similarity index less than the correlation threshold, i.e., spectral data that needs to be screened out of the second spectral data set. Spectral data above the dashed line is target spectral data that is screened out.

[0178] It should be noted that the screening order of the double-standard screening process provided in the present application can be adaptively adjusted. The above is to take noise screening as the first screening and to combine correlation screening to complete the second screening. In actual application, correlation screening can be performed first, and then noise screening can be performed.

[0179] Of course, other screening indexes, such as a signal-to-noise ratio screening index or a peak position consistency screening index, can also be included on the basis of double-standard screening.

[0180] The present application can be implemented in software, embedded in a hardware system, or in a cloud platform. The present application is suitable for Raman spectrum, fluorescence, near-infrared, ultraviolet-visible spectrum, and other one-dimensional spectrum signal quality screening scenarios.

[0181] Figure 12 A whole screening result display schematic diagram provided by an embodiment of the present application is shown in FIG. 7. Figure 12 (a) in FIG. 7 shows a plurality of original spectral data. Figure 12 (b) in FIG. 7 shows a schematic diagram of screening a plurality of preprocessed spectral data by the double screening mechanism of the present application. Figure 12 (c) in FIG. 7 shows target spectral data obtained by final screening.

[0182] Figure 13 A double-standard dynamic threshold screening result schematic diagram provided by an embodiment of the present application is shown in FIG. 8. Figure 13 As shown in FIG. 8, a first dashed line perpendicular to the x-axis represents a noise screening threshold, a second dashed line perpendicular to the y-axis represents a correlation threshold, spectral data to the right of the first dashed line is spectral data that does not meet the condition and is screened out by noise screening, spectral data below the second dashed line is spectral data that does not meet the condition and is screened out by morphology consistency screening, and target spectral data obtained by final screening is shown in the upper left corner of the scatter plot.

[0183] Figure 14A schematic diagram of the overall process of a spectrum data processing method provided by an embodiment of the present application is shown. First, according to the requirements, the spectrum of the target object is collected to obtain a plurality of original spectrum data, and then a series of preprocessing is performed on the plurality of original spectrum data to obtain a plurality of preprocessed spectrum data; the peak intensity fluctuation information is calculated, and the normal distribution (mean and standard deviation) of the same group of data is combined to perform first screening; the correlation threshold is set, the spectrum data after the first screening is correlated with the standard spectrum data to perform correlation analysis, and the second screening is realized, and finally the high-quality target spectrum data is output.

[0184] In summary, the spectrum data processing method provided by the embodiment includes: obtaining a plurality of original spectrum data corresponding to a target object; performing a preprocessing operation on each original spectrum data to obtain a plurality of preprocessed spectrum data; determining a noise screening threshold according to the spectrum intensity information of each wavelength point in each preprocessed spectrum data; performing first screening on each preprocessed spectrum data according to the noise screening threshold to obtain a first screened spectrum data set; determining standard spectrum data corresponding to the target object according to the first screened spectrum data set; and performing second screening according to the standard spectrum data to obtain a plurality of target spectrum data corresponding to the target object. The present scheme can effectively overcome the limitations of traditional single-dimensional screening by combining the double screening standards of noise and morphology consistency for screening spectrum data. For noise screening, the noise screening threshold is dynamically determined according to the statistical characteristics of each batch of data, so that the screening standard can adapt to different batches and different signal-to-noise ratio levels of the data set, effectively avoiding the problem of over-tight or over-loose caused by manually setting a fixed threshold, and ensuring the scientificity and universality of the screening condition. For the screening of morphology consistency, the internal standard spectrum is generated based on clustering and morphology consistency analysis is performed, which eliminates the dependence on external standards or prior knowledge, ensures the objectivity and universality of the morphology consistency screening process, enables the system to automatically identify and retain the most representative and highest quality spectrum in the current data set in an unsupervised mode, significantly reduces the dependence on professional experience, and provides a reliable guarantee for large sample and high throughput spectrum analysis.

[0185] The following describes the device, equipment, storage medium and the like for performing the spectrum data processing method provided by the present application. For specific implementation processes and technical effects, please refer to the above description, which will not be repeated here.

[0186] Figure 15 A schematic diagram of a spectrum data processing device provided by an embodiment of the present application is shown. The functions implemented by the spectrum data processing device correspond to the steps of the method described above. As shown in the figure, the device includes an acquisition module 100, a determination module 200 and a screening module 300. Figure 15

[0187] ​The acquisition module 100 is configured to acquire a plurality of original spectrum data corresponding to the target object, each of the original spectrum data including spectrum intensity information corresponding to different wavelength points.

[0188] The determination module 200 is configured to perform a preprocessing operation on each of the original spectrum data to obtain a plurality of preprocessed spectrum data.

[0189] The determination module 200 is configured to determine a noise screening threshold according to the spectrum intensity information corresponding to each of the wavelength points in each of the preprocessed spectrum data.

[0190] The screening module 300 is configured to perform a first screening on each of the preprocessed spectrum data according to the noise screening threshold to obtain a first screened spectrum data set.

[0191] The screening module 300 is configured to determine standard spectrum data corresponding to the target object according to the first screened spectrum data set.

[0192] The screening module 300 is configured to perform a second screening according to the standard spectrum data to obtain a plurality of target spectrum data corresponding to the target object.

[0193] Optionally, the determination module 200 is specifically configured to determine peak intensity fluctuation information of each of the preprocessed spectrum data according to the spectrum intensity information of adjacent wavelength points in each of the preprocessed spectrum data.

[0194] The noise screening threshold is determined according to the peak intensity fluctuation information of each of the preprocessed spectrum data.

[0195] Optionally, the determination module 200 is specifically configured to determine a spectrum intensity difference value of adjacent wavelength points according to the spectrum intensity information of the adjacent wavelength points.

[0196] The spectrum intensity difference mean value is determined according to the spectrum intensity difference value of the adjacent wavelength points.

[0197] The peak intensity fluctuation information of the preprocessed spectrum data is determined according to the spectrum intensity difference value of the adjacent wavelength points, the spectrum intensity difference mean value, and the number of wavelength points.

[0198] Optionally, the determination module 200 is specifically configured to determine a peak intensity fluctuation mean value and a peak intensity fluctuation standard deviation according to the peak intensity fluctuation information of each of the preprocessed spectrum data.

[0199] The noise screening threshold is determined according to the peak intensity fluctuation mean value and the peak intensity fluctuation standard deviation.

[0200] Optionally, the screening module 300 is specifically configured to traverse each piece of preprocessed spectral data, and for the current preprocessed spectral data traversed, if the peak intensity fluctuation information of the current preprocessed spectral data is less than or equal to the noise screening threshold, the current preprocessed spectral data is added to the first spectral data set.

[0201] Optionally, the screening module 300 is specifically configured to obtain the reference spectral data corresponding to the target object.

[0202] According to the reference spectral data, the clustering analysis is performed on each piece of spectral data in the screened first spectral data set, and a second spectral data set screened is obtained.

[0203] According to the spectral intensity information corresponding to each wavelength point of each piece of spectral data in the screened second spectral data set, the standard spectral data corresponding to the target object is generated.

[0204] Optionally, the screening module 300 is specifically configured to perform second screening on each piece of spectral data in the second spectral data set according to the standard spectral data, and obtain a plurality of target spectral data corresponding to the target object.

[0205] Optionally, the screening module 300 is specifically configured to determine the morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data according to the standard spectral data.

[0206] According to the morphological similarity index of each piece of spectral data in the second spectral data set and the standard spectral data, the second screening is performed on each piece of spectral data in the second spectral data set, and a plurality of target spectral data corresponding to the target object is obtained.

[0207] Optionally, the screening module 300 is specifically configured to traverse each piece of spectral data in the second spectral data set, and for the current spectral data traversed, if the morphological similarity index of the current spectral data and the standard spectral data is greater than or equal to a preset correlation threshold, the current spectral data is added to a third spectral data set.

[0208] Each piece of spectral data in the third spectral data set is taken as a plurality of target spectral data corresponding to the target object.

[0209] Optionally, the preprocessing operation includes one or more of the following: noise processing, spectral baseline fitting and correction, filtering processing, and normalization processing.

[0210] The above device is used to execute the method provided by the foregoing embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0211] The above modules can be one or more integrated circuits configured to implement the above methods, for example, one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling code, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke program code. For another example, the modules can be integrated together to implement a system-on-a-chip (SOC).

[0212] The above modules can be connected or communicated with each other via wired connections or wireless connections. The wired connections can include metal cables, optical cables, hybrid cables, etc., or any combination thereof. The wireless connections can include connections in the form of LAN, WAN, Bluetooth, ZigBee, or NFC, etc., or any combination thereof. Two or more modules can be combined into a single module, and any one module can be divided into two or more units. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working processes of the system and device described above can refer to the corresponding processes in the method embodiments, which will not be described herein again.

[0213] Figure 16 A structural schematic diagram of an electronic device provided in an embodiment of the present application is shown in FIG. 1. The terminal can be a computing device with data processing function.

[0214] The device includes a processor 801 and a storage medium 802.

[0215] The storage medium 802 is configured to store programs, and the processor 801 invokes the programs stored in the storage medium 802 to execute the above method embodiments. The specific implementation manners and technical effects are similar, and will not be described here again.

[0216] The storage medium 802 stores program codes, and when the program codes are executed by the processor 801, the processor 801 executes various steps in the spectrum data processing method according to various exemplary embodiments of the present application described in the above “Exemplary Method” section of the specification.

[0217] The processor 801 can be a general processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, and can implement or execute the methods, steps and logical block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly embodied as completed by a hardware processor, or completed by a combination of hardware and software modules in the processor.

[0218] The storage medium 802 is a non-volatile computer readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The storage medium can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type storage medium, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic storage medium, magnetic disk, optical disk, etc. The storage medium is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The storage medium 802 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0219] Optionally, the present application also provides a program product, such as a computer readable storage medium, comprising a program for executing the above-mentioned method embodiments when executed by a processor.

[0220] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0221] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0222] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0223] The integrated unit realized in the form of software functional unit can be stored in a computer readable storage medium. The software functional unit stored in a storage medium includes a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) execute part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (English: Read-Only Memory, ROM for short), a random access memory (English: Random Access Memory, RAM for short), a magnetic disk or an optical disk, and various program code storage media.

Claims

1. A method of processing spectral data, characterized by, The method comprises the following steps: acquiring a plurality of original spectral data corresponding to a target object, each piece of original spectral data comprising spectral intensity information corresponding to different wavelength points; performing a preprocessing operation on each piece of original spectral data to obtain a plurality of pieces of preprocessed spectral data; determining a noise screening threshold according to the spectral intensity information corresponding to each wavelength point in each piece of preprocessed spectral data; performing a first screening on each piece of preprocessed spectral data according to the noise screening threshold to obtain a first set of screened spectral data; determining standard spectral data corresponding to the target object according to the first set of screened spectral data; performing a second screening according to the standard spectral data to obtain a plurality of pieces of target spectral data corresponding to the target object; the step of determining the noise screening threshold according to the spectral intensity information corresponding to each wavelength point in each piece of preprocessed spectral data comprises the following steps: determining peak intensity fluctuation information of each piece of preprocessed spectral data according to the spectral intensity information of adjacent wavelength points in each piece of preprocessed spectral data; determining a peak intensity fluctuation mean value and a peak intensity fluctuation standard deviation according to the peak intensity fluctuation information of each piece of preprocessed spectral data; determining the noise screening threshold according to the peak intensity fluctuation mean value and the peak intensity fluctuation standard deviation; the step of determining the peak intensity fluctuation information of each piece of preprocessed spectral data according to the spectral intensity information of adjacent wavelength points in each piece of preprocessed spectral data comprises the following steps: determining a spectral intensity difference value of adjacent wavelength points according to the spectral intensity information of adjacent wavelength points; determining a spectral intensity difference mean value according to the spectral intensity difference value of adjacent wavelength points; determining the peak intensity fluctuation information of preprocessed spectral data according to the spectral intensity difference value of adjacent wavelength points, the spectral intensity difference mean value, and the number of wavelength points; the step of determining the standard spectral data corresponding to the target object according to the first set of screened spectral data comprises the following steps: acquiring reference spectral data corresponding to the target object; performing cluster analysis on each piece of spectral data in the first set of screened spectral data according to the reference spectral data to obtain a second set of screened spectral data; generating the standard spectral data corresponding to the target object according to the spectral intensity information corresponding to each wavelength point of each piece of spectral data in the second set of screened spectral data.

2. The method of claim 1, wherein, the step of performing the first screening on each piece of preprocessed spectral data according to the noise screening threshold to obtain the first set of screened spectral data comprises the following steps: traversing each piece of preprocessed spectral data, and if the peak intensity fluctuation information of a current piece of preprocessed spectral data is less than or equal to the noise screening threshold, adding the current piece of preprocessed spectral data to the first set of spectral data.

3. The method of claim 1, wherein, the step of performing the second screening according to the standard spectral data to obtain a plurality of pieces of target spectral data corresponding to the target object comprises the following steps: performing the second screening on each piece of spectral data in the second set of spectral data according to the standard spectral data to obtain a plurality of pieces of target spectral data corresponding to the target object.

4. The method of claim 3, wherein, The second screening of each piece of spectral data in the second spectral data set according to the morphological similarity indexes of each piece of spectral data in the second spectral data set and the standard spectral data comprises: traversing each piece of spectral data in the second spectral data set, and if a morphological similarity index of a current piece of spectral data traversed is greater than or equal to a preset correlation threshold, adding the current piece of spectral data to a third spectral data set; each piece of spectral data in the third spectral data set is taken as the plurality of target spectral data corresponding to the target object.

5. The method of claim 4, wherein, The method comprises: acquiring a plurality of pieces of original spectral data corresponding to a target object, each piece of original spectral data comprising spectral intensity information corresponding to different wavelength points; performing a preprocessing operation on each piece of original spectral data to obtain a plurality of pieces of preprocessed spectral data; 6. A spectral data processing apparatus, characterized by comprising: determining a noise screening threshold according to the spectral intensity information corresponding to each wavelength point in each piece of preprocessed spectral data; performing first screening on each piece of preprocessed spectral data according to the noise screening threshold to obtain a first spectral data set after screening; determining standard spectral data corresponding to the target object according to the first spectral data set after screening; performing second screening according to the standard spectral data to obtain a plurality of pieces of target spectral data corresponding to the target object; the determination module is specifically configured to determine peak intensity fluctuation information of each piece of preprocessed spectral data according to spectral intensity information of adjacent wavelength points in each piece of preprocessed spectral data; determine a peak intensity fluctuation mean value and a peak intensity fluctuation standard deviation according to the peak intensity fluctuation information of each piece of preprocessed spectral data; determine the noise screening threshold according to the peak intensity fluctuation mean value and the peak intensity fluctuation standard deviation; the determination module is specifically configured to determine a spectral intensity difference value of adjacent wavelength points according to the spectral intensity information of the adjacent wavelength points; determine a spectral intensity difference mean value according to the spectral intensity difference value of the adjacent wavelength points; determine the peak intensity fluctuation information of the preprocessed spectral data according to the spectral intensity difference value of the adjacent wavelength points, the spectral intensity difference mean value, and the number of wavelength points. ​ ​ ​ ​

Images