Raman spectrum data preprocessing method and system for pathogenic microorganisms

Abstract

The invention relates to a Raman spectrum data preprocessing method and a Raman spectrum data preprocessing system for pathogenic microorganisms, which are applied to the field of spectrum data processing, and the method comprises the following steps: obtaining Raman spectrum data of the pathogenic microorganisms, the Raman spectrum data comprising spectrum intensity data of Raman spectrum; performing normalization processing on the spectral intensity data to obtain normalized spectral intensity data; marking the normalized spectral intensity data in a preset rectangular coordinate system, and obtaining position information of the normalized spectral intensity data in the rectangular coordinate system; performing polar coordinate representation on the normalized spectral intensity data according to the position information, and obtaining a spectral data angle according to an included angle between the spectral intensity data after polar coordinate representation and a polar axis; and performing matrix representation on the spectral data angle to obtain and output matrix spectral data. The method has the technical effects that the feature extraction difficulty is reduced, and the possibility of losing effective information is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of spectral data processing, and in particular to a method and system for preprocessing Raman spectral data of pathogenic microorganisms. Background Art

[0002] Infectious diseases are a large class of diseases caused by pathogenic microorganisms, which are harmful to human health and can be life-threatening when seriously infected. Traditional methods for detecting pathogenic microorganisms in clinical practice include morphological detection, immunological detection, mass spectrometry, etc., but these traditional detection methods all have certain disadvantages, such as long detection time and inability to effectively detect rare pathogens. In recent years, Raman spectroscopy has been increasingly used in microbial detection. It can quickly and non-destructively detect characteristic chemical components in microbial cells. Since each molecule has its characteristic Raman spectrum, Raman spectroscopy has a prominent advantage in detecting pathogenic microorganisms. After obtaining spectral data using Raman spectroscopy, artificial intelligence methods are currently commonly used to process and analyze the data.

[0003] Commonly used artificial intelligence methods include K-nearest neighbor algorithm, decision tree algorithm, support vector machine algorithm and convolutional neural network algorithm which has emerged in recent years. The core of artificial intelligence methods is to effectively extract features from data. However, the artificial intelligence algorithms currently used directly process and analyze the original data. The original Raman spectrum data is a one-dimensional data. When extracting feature values ​​from one-dimensional data, there may be a situation where incoherent data features cannot be extracted, making feature extraction difficult and may lead to the loss of effective information. The loss of effective information will cause misjudgment and misdiagnosis, which will lead to inspection accidents. Summary of the invention

[0004] In order to help solve the problem that feature extraction is difficult and may lead to loss of effective information, the present application provides a Raman spectroscopy data preprocessing method and system for pathogenic microorganisms.

[0005] In a first aspect, the present application provides a method for preprocessing Raman spectroscopy data of pathogenic microorganisms, which adopts the following technical solution: the method comprises:

[0006] Acquiring Raman spectrum data of pathogenic microorganisms, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum;

[0007] Normalizing the spectral intensity data to obtain normalized spectral intensity data;

[0008] Marking the normalized spectral intensity data in a preset rectangular coordinate system to obtain position information of the normalized spectral intensity data in the rectangular coordinate system;

[0009] Performing polar coordinate representation on the normalized spectral intensity data according to the position information, and obtaining the spectral data angle according to the angle between the spectral intensity data after polar coordinate representation and the polar axis;

[0010] The spectral data angles are represented by a matrix to obtain and output matrix spectral data.

[0011] Through the above technical solution, the original one-dimensional Raman spectral data is converted into matrix data. The matrix data is two-dimensional data. After preprocessing, the data dimension is increased, making the spectral data more visualized, solving the problem that characteristic values ​​can only be extracted from adjacent elements in subsequent artificial intelligence algorithms. If it is desired to extract the characteristics of an element, it can be extracted from non-adjacent elements, so that if there is a characteristic association between adjacent and distant elements, it can also be extracted. On the basis of the original, more effective information is mined and can be displayed intuitively, thereby reducing the difficulty of feature extraction and the possibility of losing effective information, thereby reducing the misjudgment rate and the possibility of inspection accidents, so that the needs of precision medicine can be met.

[0012] In a specific embodiment, after obtaining the Raman spectrum data of the pathogenic microorganism, the method further comprises:

[0013] Setting the spectral intensity data outside the preset range as abnormal data;

[0014] Eliminating the abnormal data in the spectral intensity data to obtain reasonable spectral intensity data;

[0015] The step of normalizing the spectral intensity data to obtain normalized spectral intensity data comprises:

[0016] The reasonable spectral intensity data is normalized to obtain normalized spectral intensity data.

[0017] Through the above technical solution, the spectral intensity data can be limited to a reasonable range; due to some accidental or other factors, the collected data may contain abnormally large or abnormally small data, so the data needs to be cleaned and then output; the data outside the range is marked as abnormal data and the abnormal data is removed, so that the spectral intensity data can be limited to a reasonable range for subsequent processing, thereby reducing the impact of abnormal data on subsequent processing and analysis.

[0018] In a specific possible implementation manner, after removing the abnormal data in the spectral intensity data to obtain reasonable spectral intensity data, the method further includes:

[0019] Compare the abnormal data with the preset standard value:

[0020] If the value corresponding to the abnormal data is greater than the preset standard value, subtract the preset reasonable value from the value corresponding to the abnormal data to obtain spectral intensity reduction data;

[0021] If the value corresponding to the abnormal data is less than the preset standard value, the value corresponding to the abnormal data is added with a preset reasonable value to obtain spectral intensity expansion data;

[0022] Inserting the spectral intensity reduction data and the spectral intensity expansion data into the spectral intensity reasonable data according to the position information of the spectral intensity reduction data and the spectral intensity expansion data;

[0023] The inserted spectral intensity reasonable data is set as the spectral intensity complete data;

[0024] The normalizing the reasonable spectral intensity data to obtain normalized spectral intensity data comprises:

[0025] The complete spectrum intensity data is normalized to obtain normalized spectrum intensity data.

[0026] Through the above technical solution, the abnormal data can be adjusted to a reasonable range and inserted into the reasonable spectral intensity data to ensure the integrity of the data; the abnormal data may have an impact on the subsequent processing and analysis process, but if the abnormal data is directly eliminated, some valid information implied in the abnormal data may also be eliminated. Therefore, the abnormal data can be adjusted by adding or subtracting and then inserted into the reasonable spectral intensity data, which can retain some valid information, ensure the integrity of the data, and reduce the impact on the subsequent data processing and analysis process.

[0027] In a specific implementation scheme, after obtaining the Raman spectrum data of the pathogenic microorganism, the method further includes:

[0028] Transforming the spectral intensity data into frequency domain representation and obtaining spectral intensity data represented by a plurality of sine functions;

[0029] The step of normalizing the spectral intensity data to obtain normalized spectral intensity data comprises:

[0030] Normalization processing is performed on the spectral intensity data represented by the plurality of sinusoidal functions to obtain normalized spectral intensity data.

[0031] Through the above technical solution, data can be converted from the time domain to the frequency domain. By converting the spectral intensity data to the frequency domain through time-frequency transformation, it is expressed as the superposition of several sine functions, and the complete spectral intensity data can be expressed by formula. After the data is expressed by formula in the frequency domain, it is easier to obtain the frequency, amplitude and other information of the data, which can make the subsequent processing process simpler.

[0032] In a specific implementation manner, after transforming the spectral intensity data into frequency domain representation and obtaining spectral intensity data represented by a plurality of sine functions, the method further comprises:

[0033] Sort the plurality of sine functions according to amplitude values, and obtain an amplitude function set within a preset range;

[0034] Sort the plurality of sinusoidal functions according to the magnitude of the frequency values, and obtain a frequency function set within a preset range;

[0035] Compare the amplitude function set with the frequency function set, and set the function that overlaps the amplitude function set with the frequency function set as the noise function set;

[0036] Identifying and eliminating a set of noise functions contained in the plurality of sinusoidal functions to obtain denoised spectral intensity data;

[0037] The normalizing process of the spectral intensity data represented by the sine functions to obtain normalized spectral intensity data comprises:

[0038] The denoised spectral intensity data is normalized to obtain normalized spectral intensity data.

[0039] Through the above technical scheme, the regular and difficult-to-find noise can be eliminated; there will be noise interference in the process of collecting raw data, most of which are caused by the collection equipment itself or the external environment. Therefore, most of the noise will be regular and difficult to be found and processed when removing abnormal data. Therefore, according to the two limiting conditions of amplitude and frequency, the noise signals that are difficult to be found in the spectral intensity data are eliminated, thereby improving the quality of the data, and reducing the impact of noise on the data when extracting characteristic values ​​later.

[0040] In a specific embodiment, the position information includes the spectral intensity data index (i) and the i-th normalized spectral intensity data (y′ i );

[0041] The performing polar coordinate representation on the normalized spectral intensity data according to the position information comprises:

[0042]

[0043] Where i is the spectral intensity data label; x′ i is the i-th normalized spectral intensity data; is the angle between the i-th spectral intensity data and the polar axis; N is the length of the spectral intensity data; r i The polar diameter length of the i-th spectral intensity data from the pole.

[0044] In a specific embodiment, the spectral data angle includes the angle between the i-th spectral intensity data and the polar axis.

[0045] The step of performing matrix representation on the spectral data angle to obtain and output matrix spectral data comprises:

[0046]

[0047]

[0048] Wherein, i, j are the spectral intensity data labels; is the angle between the i-th spectral intensity data and the polar axis; M i，j is the element in the matrix, that is, the spectral data of the i-th and j-th after the cosine function operation; M is the final output matrix spectral data.

[0049] In a second aspect, the present application provides a Raman spectroscopy data preprocessing device for pathogenic microorganisms, which adopts the following technical solution: the device comprises:

[0050] A data acquisition module, used to acquire Raman spectrum data of pathogenic microorganisms, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum;

[0051] A data normalization module, used for performing normalization processing on the spectral intensity data to obtain normalized spectral intensity data;

[0052] A position marking module, used for marking the normalized spectral intensity data in a preset rectangular coordinate system, and obtaining position information of the normalized spectral intensity data in the rectangular coordinate system;

[0053] A coordinate conversion module, used for performing polar coordinate representation according to the position information, and obtaining the spectral data angle according to the angle between the spectral intensity data after polar coordinate representation and the polar axis;

[0054] The data dimension conversion module is used to perform matrix representation on the spectral data angle to obtain and output matrix spectral data.

[0055] In a third aspect, the present application provides a computer device, which adopts the following technical solution: it includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and executes a Raman spectroscopy data preprocessing method for any of the pathogenic microorganisms mentioned above.

[0056] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: storing a computer program that can be loaded by a processor and execute the Raman spectroscopy data preprocessing method of any of the above-mentioned pathogenic microorganisms.

[0057] In summary, this application has the following beneficial technical effects:

[0058] The original one-dimensional Raman spectral data is converted into matrix data, which is two-dimensional data. After preprocessing, the data dimension is increased, making the spectral data more visual, solving the problem that characteristic values ​​can only be extracted from adjacent elements in subsequent artificial intelligence algorithms. If you want to extract the characteristics of an element, you can extract it from non-adjacent elements, so that if there is a characteristic association between adjacent and distant elements, it can also be extracted. On the basis of the original, more effective information is mined and can be displayed intuitively, thereby reducing the difficulty of feature extraction and the possibility of losing effective information, thereby reducing the misjudgment rate and the possibility of inspection accidents, so that the needs of precision medicine can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 1 is a flow chart of a method for preprocessing Raman spectroscopy data of pathogenic microorganisms in an embodiment of the present application;

[0060] Figure 2 is a Raman spectrum data diagram of the original pathogenic microorganism obtained in the examples of the present application;

[0061] Figure 3 is a polar coordinate data diagram obtained by expressing the Raman spectrum data of pathogenic microorganisms in the embodiment of the present application in polar coordinates;

[0062] Figure 4 This is a result diagram of two-dimensional data obtained by using the Raman spectrum data preprocessing method of pathogenic microorganisms in the embodiment of the present application;

[0063] Figure 5 is a schematic diagram of a Raman spectroscopy data preprocessing device for pathogenic microorganisms in an embodiment of the present application;

[0064] Figure 6 It is a schematic diagram used to embody a computer device in an embodiment of the present application.

[0065] Figure numerals: 501, data acquisition module; 502, data normalization module; 503, position marking module; 504, coordinate conversion module; 505, data dimension conversion module. DETAILED DESCRIPTION

[0066] The following is combined with Figure 1-6 This application is described in further detail.

[0067] The embodiment of the present application discloses a method for preprocessing Raman spectroscopy data of pathogenic microorganisms, which is used for a processing system that converts original one-dimensional Raman spectroscopy data into two-dimensional data and then performs subsequent data processing and analysis. Generally speaking, after a user collects Raman spectroscopy data of pathogenic microorganisms through a Raman spectroscopy detection device, the data is input into a processing system. The original collected Raman spectroscopy data of pathogenic microorganisms is one-dimensional data, that is, the input of the processing system is one-dimensional spectral data, and the output of the system after processing is matrix spectral data, which is two-dimensional data. The reason for converting one-dimensional data into two-dimensional data is that artificial intelligence methods are generally used to process and analyze Raman spectroscopy data. The core of processing and analysis is to effectively extract eigenvalues, and one-dimensional data has obvious disadvantages when extracting eigenvalues ​​using artificial intelligence methods. There may be a situation where incoherent data features cannot be extracted, which makes feature extraction difficult and may cause the loss of effective information.

[0068] Reference Figure 1 , the Raman spectroscopy data preprocessing method of pathogenic microorganisms includes the following steps:

[0069] S10, obtaining Raman spectrum data of the pathogenic microorganism, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum.

[0070] Specifically, the user collects the Raman spectrum data of pathogenic microorganisms through the Raman spectrum detection equipment and inputs it into the processing system. The original collected Raman spectrum data of pathogenic microorganisms is a one-dimensional data, which usually includes the spectral intensity data of the Raman spectrum; for example, referring to Figure 2 , which is the Raman spectrum data of the original pathogenic microorganisms obtained by the user through the Raman spectrum detection equipment. It is a one-dimensional data. The Raman intensity shown on the vertical axis in the data graph is the spectrum intensity data.

[0071] S20, performing normalization processing on the spectral intensity data to obtain normalized spectral intensity data.

[0072] Specifically, the spectral intensity data is scaled to a preset range, such as [-1, 1], by normalization processing. The normalization processing can be:

[0073]

[0074] Where i is the spectral intensity data number; y is the spectral intensity data of the acquired Raman spectrum; y i is the ith spectral intensity data value; y′ i is the ith normalized spectral intensity data.

[0075] S30, marking the normalized spectral intensity data in a preset rectangular coordinate system, and obtaining position information of the normalized spectral intensity data in the rectangular coordinate system.

[0076] Specifically, after normalization, the position of the spectral intensity data is marked in the rectangular coordinate system, and the position information of each data is obtained.

[0077] S40, expressing the normalized spectral intensity data in polar coordinates according to the position information, and obtaining the spectral data angle according to the angle between the spectral intensity data expressed in polar coordinates and the polar axis.

[0078] Specifically, the position information of each data in the rectangular coordinate system includes the data label information and the spectral intensity data value. According to the data label information and the data value, the data can be represented in polar coordinates. After the polar coordinate representation, the angle between the data and the polar axis can be obtained; for example, referring to Figure 3 The data is expressed in polar coordinates to obtain a polar coordinate data graph, from which the angle between the data and the polar axis can be obtained, that is, the spectral data angle.

[0079] In one embodiment, the position information of each data in the rectangular coordinate system is converted into polar coordinates. The specific representation method may be:

[0080]

[0081] Where i is the spectral intensity data label; y′ i is the i-th normalized spectral intensity data; is the angle between the i-th spectral intensity data and the polar axis; N is the length of the spectral intensity data; r i The polar diameter length of the i-th spectral intensity data from the pole.

[0082] After each normalized spectral intensity data is expressed in polar coordinates, the angle between the spectral intensity data and the polar axis can be obtained to obtain the spectral data angle, that is, The value of .

[0083] S50, performing matrix representation on the spectrum data angle to obtain and output matrix spectrum data.

[0084] Specifically, the angle between the obtained data and the polar axis, that is, Afterwards, the angle can be expressed in a matrix, and finally the matrix spectrum data is obtained, which is the two-dimensional spectrum data obtained after the conversion; for example, referring to Figure 4 ,The two-dimensional spectral data results obtained after matrix representation can be ,visualized using a two-dimensional graph, making the data results more ,intuitive.

[0085] In one embodiment, according to the obtained spectral angle data After matrix representation, data dimension conversion can be completed to obtain matrix spectrum data, that is, two-dimensional spectrum data. The specific calculation method of each element in the matrix can be:

[0086]

[0087] Where i, j are the spectral intensity data labels; is the angle between the i-th spectral intensity data and the polar axis; M i，j is the spectral data of the i-th and j-th after the cosine function operation.

[0088] Calculate each element in the matrix and generate the following N×N matrix M:

[0089]

[0090] In the present application, the object to be processed is one-dimensional Raman spectral data of pathogenic microorganisms. The original one-dimensional Raman spectral data is converted into matrix data. The matrix data is two-dimensional data. After preprocessing, the data dimension is increased, making the spectral data more visualized, solving the problem that only characteristic values ​​can be extracted from adjacent elements in subsequent artificial intelligence algorithms. If it is desired to extract the characteristics of an element, it can be extracted from non-adjacent elements, so that if there is a characteristic association between adjacent and distant elements, it can also be extracted. On the basis of the original, more effective information is mined and can be displayed intuitively, thereby reducing the difficulty of feature extraction and reducing the possibility of loss of effective information, thereby reducing the misjudgment rate and the possibility of inspection accidents, so that the needs of precision medicine can be met.

[0091] In one embodiment, considering that due to some accidental or other factors, the collected data may contain abnormally large or abnormally small data, which are called abnormal data. The abnormal data has an impact on the data processing and analysis process and the final processing result. Therefore, after obtaining the Raman spectrum data of pathogenic microorganisms, the following steps can be performed:

[0092] After the step of obtaining the Raman spectrum data of the pathogenic microorganism, the spectrum intensity data outside the preset range is first set as abnormal data. For example, a constant t is set to determine whether each input spectrum intensity data is within the preset range, for example, whether it is within the range of [-t, t]. If the data is outside the range of [-t, t], it is set as abnormal data; secondly, the abnormal data is removed from the spectrum intensity data to obtain reasonable spectrum intensity data; the reasonable spectrum intensity data and abnormal data can be expressed as:

[0093]

[0094] Where y is the spectral intensity data of the acquired Raman spectrum; t is a preset constant, and the preset range is [-t, t]; when the spectral intensity data is within the preset range, it is the reasonable spectral intensity data, that is, y a is the reasonable data of spectral intensity; the data outside the preset range is abnormal data, that is, y b is abnormal data.

[0095] Afterwards, the spectral intensity reasonable data was used for normalization.

[0096] In the present application, due to some accidental or other factors, the collected data may contain abnormally large or small data, so the data needs to be cleaned before output; the data outside the preset range is set as abnormal data and the abnormal data is removed, so that the spectral intensity data can be subsequently processed within a reasonable range, reducing the interference of abnormal data on subsequent processing and analysis.

[0097] In one embodiment, considering that if the abnormal data is directly removed, part of the valid information implicit in the abnormal data will be removed together, thereby affecting the accuracy of the subsequent processing results, the following steps can be performed after the abnormal data in the spectral intensity data is removed to obtain reasonable spectral intensity data:

[0098] First, the abnormal data is compared with the preset standard value. If the value corresponding to the abnormal data is greater than the preset standard value, the preset reasonable value is subtracted from the value corresponding to the abnormal data; if the value corresponding to the abnormal data is less than the preset standard value, the preset reasonable value is added to the value corresponding to the abnormal data; for example, the preset standard value is set to 0, and the preset reasonable value is set to t, then the value corresponding to the abnormal data is compared with 0, if the value corresponding to the abnormal data is greater than 0, then the value corresponding to the abnormal data is subtracted from t to obtain the spectrum intensity reduction data, and if the value corresponding to the abnormal data is less than 0, then the value corresponding to the abnormal data is added to t to obtain the spectrum intensity expansion data; according to the above method, the processing of abnormal data can be expressed as:

[0099]

[0100] Among them, y b is abnormal data; t is the preset reasonable value; 0 is the preset standard value; when y b > 0, the output is y after deducting the preset reasonable value t from the abnormal data c ,y c That is the spectrum intensity reduction data; when y b When <0, the abnormal data is added with the preset reasonable value t and the output is y d ,y d That is the spectral intensity expansion data.

[0101] Afterwards, according to the position information of the spectral intensity reduction data and the spectral intensity expansion data, the processed abnormal data is inserted into the original position in the data to obtain the spectral intensity complete data; and then the spectral intensity complete data is used for normalization processing.

[0102] In the present application scheme, for the processing of abnormal data, if the abnormal data is directly eliminated, some of the valid information hidden therein may be removed together, thereby affecting the results of subsequent processing. Therefore, the abnormal data is adjusted and inserted into the reasonable spectral intensity data, which can retain some valid information, ensure the integrity of the data, improve the data quality, and reduce the impact on the subsequent data processing and analysis process.

[0103] It should be noted that, considering the impact of abnormal data itself and the abnormal data processing method on the analysis and processing process and results, the method of processing the abnormal data in the spectral intensity data to obtain the complete spectral intensity data can be expressed as:

[0104]

[0105] In the above representation, t is a preset reasonable value, y is the spectral intensity data of the acquired Raman spectrum, and y′ is the complete spectral intensity data; first, the preset range is limited, for example, the range is limited to [-t, t]; secondly, the spectral intensity data outside [-t, t] is set as abnormal data and removed, that is, the spectral intensity data outside the range is set as abnormal data y b And remove it to obtain reasonable data of spectral intensity, that is, to obtain y a ,y a In the range of [-t,t]; then y b Compared with the preset constant, for example, the preset constant is set to 0, that is, y b Compared with 0, if y b =0, since the range [-t, t] contains the value 0, when y b = 0, y b It can be regarded as data within the range and attributed to the reasonable data of spectral intensity; if y b >, then y b -t and then inserted into the spectral intensity reasonable data according to the position information in the original data, that is, inserted into y a In; when y b When <0, then y b +t and then output, and insert it into the reasonable spectral intensity data according to the position information in the original data, that is, insert it into y a Finally, the complete spectrum intensity data is obtained, that is, y′. Then, the complete spectrum intensity data is normalized to obtain the normalized spectrum intensity data.

[0106] In one embodiment, considering that the data cannot be represented intuitively and the amplitude, frequency and other information of the data cannot be directly observed, the following steps can be performed after obtaining the Raman spectrum data of the pathogenic microorganism:

[0107] After obtaining the spectral intensity data, the spectral intensity data is first transformed into the frequency domain to obtain the spectral intensity data represented by several sine functions. The methods of transforming the data into the frequency domain include Fourier transform, wavelet transform, etc., which are not limited here. The embodiment of the present application takes Fourier transform as an example for explanation; after the data is transformed into the frequency domain by Fourier transform, it can be represented by the superposition of several sine functions, and the representation method can be:

[0108]

[0109] Where n is the number of sinusoidal functions required to represent the spectral intensity data. Due to the need to meet the requirements of fast preprocessing, the range of n is determined to be [0,100]; A i is the amplitude of the i-th sine function; ω i is the frequency of the i-th sine function; is the phase of the i-th sine function; R is a constant, and each Raman spectrum data corresponds to a different R.

[0110] Then the spectral intensity data represented by several sinusoidal functions are normalized.

[0111] In the present application scheme, the spectral intensity data is converted to the frequency domain through time-frequency transformation and represented as the superposition of several sinusoidal functions. The spectral intensity data can be expressed by a formula. After the data is expressed by a formula in the frequency domain, it is easier to obtain information such as the frequency and amplitude of the data, thereby simplifying the subsequent processing process.

[0112] In one embodiment, considering that regular noise interferes with data processing and analysis, the following steps may be performed after the spectral intensity data is transformed into frequency domain representation and the spectral intensity data represented by a plurality of sine functions is obtained:

[0113] After the spectral intensity data is expressed by several sine functions, the number of sine functions can be obtained as n. First, the sine functions are sorted according to their amplitudes, that is, A iThe values ​​are sorted by size, and the sorting method can be from large to small or from small to large, which is not limited here; the embodiment of the present application takes the sorting method from large to small as an example for explanation; after the sorting is completed, the last m1 sine functions in the amplitude value sorting are determined, where m1=0.2×n, where m1 and n represent the number of sine functions, so both are integers. If the calculated result of m1 is a non-integer, the non-integer is rounded down to obtain m1, that is, m1 sine functions with smaller amplitude values ​​are obtained and set as the amplitude function set. Then, the sine functions are sorted according to their frequency, that is, ω i The values ​​are sorted by size, and the sorting method can be from large to small or from small to large, which is not limited here; the embodiment of the present application takes the sorting method from large to small as an example for explanation; after the sorting is completed, the last m2 sine functions in the frequency value sorting are determined, where m2=0.1×n, where m2 and n represent the number of sine functions, so both are integers. If the calculated result of m2 is a non-integer, the non-integer is rounded down to obtain m2, that is, m2 sine functions with smaller amplitude values ​​are obtained and recorded as the frequency function set. After obtaining the amplitude function set and the frequency function set, the two function sets are compared, and the functions that overlap in the two function sets are obtained and set as the noise function set. Finally, among the several sine functions representing the spectral intensity data, the noise function set is identified and eliminated to obtain the denoised spectral intensity data that has been finally screened; then the denoised spectral intensity data is normalized.

[0114] In the present application, since there will be noise interference in the original data during the collection process, most of this noise interference is caused by the collection equipment itself or the external environment. Therefore, most of the noise will be regular and difficult to be discovered and processed when removing abnormal data. Therefore, according to the two limiting conditions of amplitude and frequency, the noise signals that are difficult to be discovered in the spectral intensity data are eliminated according to the mutual constraints, so that the data can be cleaned more accurately, the data quality can be improved, and the impact of noise on the data can be reduced when the characteristic values ​​are extracted later.

[0115] It should be noted that the operations of screening abnormal data and processing the abnormal data and removing environmental noise after converting to the frequency domain can be processed in the same embodiment, that is, after obtaining the original data, the abnormal data is first screened for abnormal data, and after the abnormal data is processed, the data is normalized, and then the data is converted to the frequency domain representation, and the regular noise in the data set is removed; after two data cleaning processes, the data quality can be improved, and the noise or interference in the data set can be removed as much as possible, reducing the impact in the subsequent feature extraction process.

[0116] Considering that the originally collected data may contain abnormally large or small data, as well as some regular noise or interference, the data is cleaned twice to achieve accurate data cleaning and improve the data quality. The impact of noise on the data can be reduced when the eigenvalues ​​are subsequently extracted. The process of converting the data to frequency domain representation can make the data representation more intuitive.

[0117] Figure 1 FIG. 1 is a flow chart of a method for preprocessing Raman spectroscopy data of pathogenic microorganisms in one embodiment. Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps may be executed in other orders; and Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0118] Based on the above method, the embodiment of the present application also discloses a Raman spectroscopy data preprocessing device for pathogenic microorganisms.

[0119] Reference Figure 5 , the device includes the following modules:

[0120] A data acquisition module 501 is used to acquire Raman spectrum data of pathogenic microorganisms, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum;

[0121] A data normalization module 502, used for normalizing the spectral intensity data to obtain normalized spectral intensity data;

[0122] A position marking module 503 is used to mark the normalized spectral intensity data in a preset rectangular coordinate system to obtain position information of the normalized spectral intensity data in the rectangular coordinate system;

[0123] A coordinate conversion module 504 is used to perform polar coordinate representation according to the position information, and obtain the spectral data angle according to the angle between the spectral intensity data after polar coordinate representation and the polar axis;

[0124] The data dimension conversion module 505 is used to perform matrix representation on the spectral data angles to obtain and output matrix spectral data.

[0125] In one embodiment, the data acquisition module 501 is also used to set the spectral intensity data outside the preset range as abnormal data; eliminate the abnormal data in the spectral intensity data to obtain reasonable spectral intensity data; normalize the spectral intensity data to obtain normalized spectral intensity data, including: normalize the reasonable spectral intensity data to obtain normalized spectral intensity data.

[0126] In one embodiment, the data acquisition module 501 is also used to compare the abnormal data with the preset standard value: if the numerical value corresponding to the abnormal data is greater than the preset standard value, the numerical value corresponding to the abnormal data is subtracted from the preset reasonable value to obtain spectral intensity reduction data; if the numerical value corresponding to the abnormal data is less than the preset standard value, the numerical value corresponding to the abnormal data is added to the preset reasonable value to obtain spectral intensity expansion data; according to the position information of the spectral intensity reduction data and the spectral intensity expansion data, the spectral intensity reduction data and the spectral intensity expansion data are inserted into the spectral intensity reasonable data; the inserted spectral intensity reasonable data is set as the spectral intensity complete data; normalizing the spectral intensity reasonable data to obtain normalized spectral intensity data includes: normalizing the spectral intensity complete data to obtain normalized spectral intensity data.

[0127] In one embodiment, the data acquisition module 501 is also used to transform the spectral intensity data into a frequency domain representation and obtain spectral intensity data represented by several sine functions; normalizing the spectral intensity data to obtain normalized spectral intensity data includes: normalizing the spectral intensity data represented by several sine functions to obtain normalized spectral intensity data.

[0128] In one embodiment, the data acquisition module 501 is also used to sort a number of sinusoidal functions according to the size of the amplitude value, and obtain an amplitude function set within a preset range; sort a number of sinusoidal functions according to the size of the frequency value, and obtain a frequency function set within a preset range; compare the amplitude function set with the frequency function set, and set the function that overlaps the amplitude function set with the frequency function set as a noise function set; identify and eliminate the noise function set contained in the number of sinusoidal functions to obtain denoised spectral intensity data; normalize the spectral intensity data represented by the number of sinusoidal functions to obtain normalized spectral intensity data, including: normalize the denoised spectral intensity data to obtain normalized spectral intensity data.

[0129] In one embodiment, the position information includes the spectral intensity data index (i) and the i-th normalized spectral intensity data (y′ i );

[0130] The coordinate conversion module 504 is specifically used for:

[0131]

[0132] Where i is the spectral intensity data label; y′ i is the i-th normalized spectral intensity data; is the angle between the i-th spectral intensity data and the polar axis; N is the length of the spectral intensity data; r i The polar diameter length of the i-th spectral intensity data from the pole.

[0133] In one embodiment, the spectral data angle includes the angle between the i-th spectral intensity data and the polar axis.

[0134] The data dimension conversion module 505 is specifically used for:

[0135]

[0136]

[0137] Wherein, i, j are the spectral intensity data labels; is the angle between the i-th spectral intensity data and the polar axis; M i，j is the element in the matrix, that is, the spectral data of the i-th and j-th after the cosine function operation; M is the final output matrix spectral data.

[0138] The Raman spectroscopy data preprocessing device for pathogenic microorganisms provided in the embodiments of the present application can be applied to the Raman spectroscopy data preprocessing method for pathogenic microorganisms provided in the above embodiments. For relevant details, refer to the above method embodiments. The implementation principles and technical effects are similar and will not be repeated here.

[0139] It should be noted that: the Raman spectrum data preprocessing device for pathogenic microorganisms provided in the embodiments of the present application only uses the division of the above-mentioned functional modules / functional units as an example when performing Raman spectrum data preprocessing of pathogenic microorganisms. In practical applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the Raman spectrum data preprocessing device for pathogenic microorganisms is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the Raman spectrum data preprocessing method for pathogenic microorganisms provided in the above method embodiment and the implementation method of the Raman spectrum data preprocessing device for pathogenic microorganisms provided in this embodiment belong to the same concept. The specific implementation process of the Raman spectrum data preprocessing device for pathogenic microorganisms provided in this embodiment is detailed in the above method embodiment, and will not be repeated here.

[0140] The embodiment of the present application also discloses a computer device.

[0141] Specifically, if Figure 6As shown, the computer device can be a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or otherwise. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned chips.

[0142] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0143] The embodiment of the present application also discloses a computer-readable storage medium.

[0144] Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art will understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the process of the implementation of the above-mentioned methods. Among them, the storage medium may be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.

[0145] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A method for preprocessing Raman spectroscopy data of pathogenic microorganisms, characterized in that: The method comprises: Acquiring Raman spectrum data of pathogenic microorganisms, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum; Normalizing the spectral intensity data to obtain normalized spectral intensity data; Marking the normalized spectral intensity data in a preset rectangular coordinate system to obtain position information of the normalized spectral intensity data in the rectangular coordinate system; Performing polar coordinate representation on the normalized spectral intensity data according to the position information, and obtaining the spectral data angle according to the angle between the spectral intensity data after polar coordinate representation and the polar axis; The spectral data angles are represented by a matrix to obtain and output matrix spectral data.

2. The method according to claim 1, characterized in that: After obtaining the Raman spectrum data of the pathogenic microorganism, the method further includes: Setting the spectral intensity data outside the preset range as abnormal data; Eliminating the abnormal data in the spectral intensity data to obtain reasonable spectral intensity data; The step of normalizing the spectral intensity data to obtain normalized spectral intensity data comprises: The reasonable spectral intensity data is normalized to obtain normalized spectral intensity data.

3. The method according to claim 2, characterized in that: After removing the abnormal data in the spectral intensity data to obtain reasonable spectral intensity data, the method further includes: Compare the abnormal data with the preset standard value: If the value corresponding to the abnormal data is greater than the preset standard value, subtract the preset reasonable value from the value corresponding to the abnormal data to obtain spectral intensity reduction data; If the value corresponding to the abnormal data is less than the preset standard value, the value corresponding to the abnormal data is added to the preset reasonable value to obtain the spectrum intensity expansion data; Inserting the spectral intensity reduction data and the spectral intensity expansion data into the spectral intensity reasonable data according to the position information of the spectral intensity reduction data and the spectral intensity expansion data; The inserted spectral intensity reasonable data is set as the spectral intensity complete data; The normalizing the reasonable spectral intensity data to obtain normalized spectral intensity data includes: The complete spectrum intensity data is normalized to obtain normalized spectrum intensity data.

4. The method according to claim 1, characterized in that: After obtaining the Raman spectrum data of pathogenic microorganisms, it also includes: Transforming the spectral intensity data into frequency domain representation and obtaining spectral intensity data represented by a plurality of sinusoidal functions; The step of normalizing the spectral intensity data to obtain normalized spectral intensity data comprises: Normalization processing is performed on the spectral intensity data represented by the plurality of sinusoidal functions to obtain normalized spectral intensity data.

5. The method according to claim 4, characterized in that: After transforming the spectral intensity data into frequency domain representation and obtaining spectral intensity data represented by a plurality of sine functions, the method further includes: Sort the plurality of sine functions according to amplitude values, and obtain an amplitude function set within a preset range; Sort the plurality of sinusoidal functions according to the magnitude of the frequency values, and obtain a frequency function set within a preset range; Compare the amplitude function set with the frequency function set, and set the function where the amplitude function set and the frequency function set overlap as the noise function set; Identifying and eliminating a set of noise functions contained in the plurality of sinusoidal functions to obtain denoised spectral intensity data; The normalizing process of the spectral intensity data represented by the sine functions to obtain normalized spectral intensity data comprises: The denoised spectral intensity data is normalized to obtain normalized spectral intensity data.

6. The method according to claim 1, characterized in that: The position information includes the spectral intensity data index (i) and the i-th normalized spectral intensity data (y′ i ); The performing polar coordinate representation on the normalized spectral intensity data according to the position information comprises: Where i is the spectral intensity data label; y′ i is the i-th normalized spectral intensity data; is the angle between the i-th spectral intensity data and the polar axis; N is the length of the spectral intensity data; r i The polar diameter length of the i-th spectral intensity data from the pole.

7. The method according to claim 1, characterized in that: The spectral data angle includes the angle between the i-th spectral intensity data and the polar axis The obtaining and outputting matrix spectrum data by performing matrix representation on the spectrum data angles comprises: Where i, j are the spectral intensity data labels; is the angle between the i-th spectral intensity data and the polar axis; M i，j is the element in the matrix, that is, the spectral data of the i-th and j-th after the cosine function operation; M is the final output matrix spectral data.

8. A Raman spectroscopy data preprocessing device for pathogenic microorganisms, characterized in that: The device comprises: A data acquisition module (501) is used to acquire Raman spectrum data of pathogenic microorganisms, wherein the Raman spectrum data includes spectrum intensity data of the Raman spectrum; A data normalization module (502), used for performing normalization processing on the spectral intensity data to obtain normalized spectral intensity data; A position marking module (503) is used to mark the normalized spectral intensity data in a preset rectangular coordinate system, and obtain the position information of the normalized spectral intensity data in the rectangular coordinate system; A coordinate conversion module (504) is used to perform polar coordinate representation according to the position information, and obtain the spectral data angle according to the angle between the spectral intensity data after polar coordinate representation and the polar axis; The data dimension conversion module (505) is used to perform matrix representation on the spectral data angle to obtain and output matrix spectral data.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

Images