A solid waste classification method based on spectrometer

Abstract

The present invention discloses a spectrometer-based solid waste classification method, which relates to the field of solid waste analysis technology and includes the following steps: S1. Preliminary classification: Obtaining a sample of unknown solid waste, observing the characteristics of the sample, and obtaining characteristic information; S2. Selecting a spectrometer: Selecting a spectrometer based on the characteristic information; Analyzing the sample using the spectrometer to obtain a spectrogram; The spectrogram includes a spectral curve; S3. Spectrogram preprocessing: Normalizing the data of the spectral curve; S4. Similarity analysis: For the spectral curves of multiple samples, arbitrarily selecting two samples for comparison to obtain a similarity value; if the similarity value is within a preset threshold range, the two samples being compared belong to the same type of solid waste. This method has low operating costs, requires low professional expertise from staff, and can achieve rapid and efficient analysis and classification of solids.

Description

Technical Field

[0001] The present invention relates to the technical field of solid waste analysis, and in particular to a solid waste classification method based on a spectrometer. Background Art

[0002] Environmental incidents in hazardous waste identification are of great concern due to their suddenness, complexity, and urgency. These incidents primarily include the dumping of unknown solid waste and historical landfills of unknown solid waste. According to the "Technical Specifications for Hazardous Waste Identification" (HJ 298-2019), hazardous waste identification requires the number of samples to be determined based on the mass of the solid waste to be identified. The amount of solid waste accumulated in dumping incidents is typically tens of tons, while the amount of solid waste landfilled in historical landfill incidents is generally over a thousand tons.

[0003] Due to the large amount of these substances accumulated / landfilled, environmental management officers, hazardous waste identification technicians, and other environmental professionals are required to collect dozens to hundreds of samples. They then need to select representative samples for initial screening, and use the results of these initial screening samples to determine subsequent testing parameters. This initial screening process typically relies heavily on the technicians' experience.

[0004] Therefore, collecting dozens to hundreds of samples will lead to a surge in project monitoring costs, labor costs, and time costs, which is not adapted to the urgency of environmental events. Improper selection of initial screening samples will also lead to distorted assessment of hazardous properties of hazardous wastes, and there is a possibility that hazardous wastes will be misjudged as general solid wastes.

[0005] The current technical solutions tend to conduct qualitative analysis and component analysis on unknown substances, that is, to determine the type of substance and the components it contains.

[0006] The prior art provides a method and process for extracting fingerprint features from solid waste, including: obtaining solid waste samples from a predetermined industry, obtaining predetermined fingerprint factors, using the fingerprint factors as observations, calculating statistical parameters based on the rank sum of each fingerprint factor in the predetermined industry, and then determining whether the specified fingerprint factors meet the screening criteria based on the statistical parameters, thereby screening out the corresponding fingerprint factors; constructing a fingerprint test set, introducing each screened fingerprint factor into the fingerprint test set as a reference variable, or removing it from the fingerprint test set; obtaining a comparison index, and determining the optimal fingerprint factor combination based on the test parameters and the comparison index. However, this approach has limitations in emergency situations such as dumping incidents. It cannot quickly identify or classify substances, and it interferes with subsequent work, such as sample collection in hazardous waste identification. Furthermore, the substances present in environmental incidents are complex and may be a mixture of one or more substances. Consequently, there is a high probability that the complex composition of the substances will interfere with each other, making it impossible to match the substances with an established fingerprint library. Furthermore, due to the variable composition of substances in environmental incidents, established fingerprint libraries are generally not universal.

[0007] The existing technology also provides a solid waste fingerprint feature extraction method based on statistical entropy. Specifically, a numerical indicator of solid waste is selected, and the numerical indicator is evenly divided into n levels and n events. After normalization, the ratio of the statistical entropy to the maximum entropy is calculated using the statistical entropy method as a relative ratio. The calculated relative ratio is then compared with the judgment relative ratio. If the calculated relative ratio is less than or equal to the judgment relative ratio, the numerical indicator is judged to be a characteristic fingerprint of solid waste. However, this type of solution requires high technical requirements in the numerical indicator selection step, and requires a deep understanding of the substance to be analyzed. Summary of the Invention

[0008] To address the challenges of existing technologies, the present invention provides a spectrometer-based solid waste classification method. This method allows for rapid on-site identification and analysis of materials found at solid waste dumps or environmental incidents, determining whether multiple substances are of the same type. It also features low operating costs and minimal staff expertise.

[0009] The technical solution of the present invention is achieved as follows:

[0010] A solid waste classification method based on a spectrometer comprises the following steps:

[0011] S1. Preliminary classification: Obtain samples of unknown solid waste, observe the characteristics of the samples, and obtain characteristic information;

[0012] S2. Selecting a spectrometer: selecting a spectrometer based on the characteristic information; the spectrometer may be a portable, movable device or an immovable device in a laboratory; analyzing the sample using the spectrometer to obtain a spectrum; the spectrum includes a spectrum curve;

[0013] S3, spectrum preprocessing: normalizing the data of the spectrum curve;

[0014] S4. Similarity analysis: For the spectral curves of the multiple samples, arbitrarily select two samples for comparison to obtain a similarity value; if the similarity value is within a preset threshold range, the two compared samples belong to the same type of solid waste.

[0015] The classification method presented here provides more scientifically accurate on-site classification results for substances of unknown origin. It is particularly suitable for portable spectroscopic analysis instruments, offers low cost and high timeliness, and requires minimal technical expertise. This method can provide theoretical support for environmental industry practitioners, such as environmental management law enforcement officers and hazardous waste identification technicians, to reduce sample size and select representative samples during hazardous waste identification. This method is applicable to both outdoor sample collection and in-laboratory processing.

[0016] As a further optimization of the above scheme, the preliminary classification is to preliminarily classify the samples according to their appearance, and the categories include liquid materials, amorphous or organic materials, high water content materials, and dark materials.

[0017] As a further optimization of the above solution, the spectrometer includes an X-ray fluorescence spectrometer, an X-ray diffractometer, a Fourier transform infrared spectrometer, and a Raman spectrometer.

[0018] Each type of spectrometer is used to analyze specific materials. X-ray fluorescence spectrometers are not suitable for non-solid samples, such as gases or liquids; X-ray diffractometers are not suitable for amorphous or non-crystalline materials; Fourier transform infrared spectroscopy is not suitable for water-sensitive samples; and Raman spectrometers are not suitable for samples that strongly absorb and scatter laser light, such as certain black or dark materials.

[0019] As a further optimization of the above solution, the normalization process is:

[0020]

[0021] Wherein, y represents the vertical coordinate value of the data point of the spectrum curve; max and y min Respectively represent the maximum vertical coordinate value and the minimum vertical coordinate value; y i and y i ′Respectively represent the ordinate values ​​of the data point of the ith item of the spectral curve before and after normalization processing.

[0022] Normalization aims to eliminate signal intensity variations caused by differences in intrinsic components within the sample (such as the content and activity of vibrational groups). After normalization, the ordinate of the spectral curve is mapped to a range of 0 to 1, converting it to dimensionless data. Dimensionless data, which lacks specific units, is often used to eliminate the influence of physical dimensions, allowing comparisons between different units of measurement.

[0023] As a further optimization of the above solution, the spectrum preprocessing further includes smoothing the data of the spectrum curve;

[0024] The smoothing process is performed using a Savitizky-Golay filter:

[0025]

[0026] Among them, (x i ,y i ) represents the data point of the spectral curve, i is the index of the current data point; X i Represents x i The corresponding smoothing window matrix; a, p and θ i are the constant term, coefficient and coefficient vector in the fitting polynomial respectively; m is the size of the smoothing window; Represents finding the polynomial coefficient θ i , so that X i θ i and Y i The Euclidean distance is the smallest; H i is the hat matrix; Y i is a vector consisting of the vertical coordinates of the data points within the smoothing window; Y i The result after filtering.

[0027] Smoothing is designed to remove noise and interference from the curve, making it more suitable for subsequent analysis and comparison. This solution uses a Savitizky-Golay filter to process the horizontal and vertical coordinate points in the spectrum. This smoothing is achieved by fitting a polynomial within a local window of data, ensuring that the polynomial optimally fits the data points within the window.

[0028] Alternatively, the smoothing methods also include moving average, exponential smoothing, polynomial fitting, Bezier curve fitting, local weighted scatter point smoothing method (Loess), Kalman filter, wavelet transform and other methods.

[0029] As a further optimization of the above solution, the spectrum preprocessing further includes removing invalid peaks of the spectrum curve; and modifying the vertical coordinate values ​​of the data points corresponding to the selected invalid peaks to preset intensity values.

[0030] Invalid peak correction aims to remove invalid peaks from a spectrum. For example, a portable Raman spectrometer may produce wide invalid peaks when detecting interference from strongly fluorescent substances, or may be affected by interference from cosmic rays during the detection process. Invalid peaks can be automatically identified using machine learning, deep learning, and other methods, using equipment and algorithms, or manually determined by technicians based on their expertise.

[0031] As a further optimization of the above scheme, when the change trend of the spectral curve is monotonically changing, the similarity value is calculated using the Spearman correlation coefficient, which is specifically calculated as follows:

[0032]

[0033] Among them, r s That is the similarity value; A i and B i The ordinate values ​​corresponding to the two spectral curves respectively; i is the index of the ordinate value in one spectral curve; n represents the number of data points in one spectral curve; rank(·) represents the rank.

[0034] The Spearman correlation coefficient measures the monotonic relationship between two variables. Rank(·) is used to sort each value in a dataset and assign a rank. For example, if we sort each value in variables A and B separately, we can arrange the values ​​in the dataset from smallest to largest (or largest to smallest) and rank them sequentially: the smallest value is assigned rank 1, the next smallest value is assigned rank 2, and so on.

[0035] As a further optimization of the above solution, when the variation trend of the spectral curve is irregular, the calculation of the similarity value includes adopting a correlation coefficient algorithm based on Euclidean distance conversion, and the specific calculation is:

[0036]

[0037] Wherein, r is the similarity value; (x i ,y i ) and (x i ′ ,y i ′ ) represent the coordinates of the i-th data point of the two spectral curves respectively; n represents the number of data points in one spectral curve; σ x and σ y Indicates the standard deviation of all data points on the horizontal and vertical axes respectively; u x and u y Represents the average value of all data points on the horizontal and vertical axes respectively; d is the Euclidean distance.

[0038] As a further optimization of the above scheme, the method further includes the following steps: S5, cluster analysis: for a number of unclassified samples, cluster analysis algorithms are used to classify them.

[0039] In similarity analysis, samples with similarity values ​​within the preset threshold range can be clustered, but there may still be samples whose similarity values ​​cannot be within the preset threshold range. In this case, the clustering algorithm is used to perform high-precision analysis and classification on the remaining unclassified samples.

[0040] As a further optimization of the above scheme, the clustering algorithm is calculated as follows:

[0041]

[0042] Wherein, x represents the spectral curve; μ i (0) represents the center of the i-th cluster before the iteration begins; C i (t) represents the set of all data assigned to the i-th cluster in the t-th iteration; μ i (t+1) represents the center of the i-th cluster at the r+1th iteration; k represents the number of classifications;

[0043] When μ i (t+1) =μ i (t) When the number of iterations reaches the preset value, the iteration is terminated.

[0044] The K-Means clustering algorithm is an iterative distance-based clustering analysis algorithm. It divides the spectral curve into k preset clusters (categories) and calculates the mean of all points at the center of each cluster. By updating the cluster center and recalculating the mean of all points within the cluster, the resulting k value at the end of the iteration is the final cluster, and each cluster corresponds to a category.

[0045] Compared with the prior art, the present invention achieves the following beneficial effects:

[0046] This invention provides a spectrometer-based solid waste classification method that can improve the scientificity and accuracy of on-site classification of materials of unknown origin. Its main advantages include low cost, high efficiency, and minimal operator expertise. This method can provide a theoretical basis for hazardous waste identification work, helping environmental management departments, law enforcement personnel, and hazardous waste identification technicians reduce sample size and select representative samples, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 1 is a flow chart of a solid waste classification method based on a spectrometer provided in an embodiment of the present invention;

[0048] Figure 2 Schematic diagrams of the effects before (left) and after (right) normalization processing provided by an embodiment of the present invention;

[0049] Figure 3 Schematic diagrams of the effects before (left) and after (right) smoothing provided by an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of the effect before (left) and after (right) the invalid peak removal process provided by an embodiment of the present invention;

[0051] Figure 5 1 is a schematic diagram showing the effect of similarity value calculation based on the Spearman correlation coefficient provided by an embodiment of the present invention;

[0052] Figure 6 2 is a schematic diagram showing the effect of similarity value calculation based on the correlation coefficient algorithm of Euclidean distance conversion provided by an embodiment of the present invention;

[0053] Figure 7 A schematic diagram illustrating the effect of cluster analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0055] This embodiment provides a solid waste classification method based on a spectrometer, such as Figure 1 As shown, the following steps are included:

[0056] S1. Preliminary Classification: Obtain a sample of unknown solid waste, observe the sample's characteristics, and obtain characteristic information. Specifically, in this embodiment, preliminary classification involves categorizing the sample based on its appearance, including liquid materials, amorphous or organic materials, highly water-rich materials, dark materials, or other materials.

[0057] S2. Select a spectrometer: Select a spectrometer based on the characteristic information; the spectrometer is a portable device. In this embodiment, the spectrometer includes a portable X-ray fluorescence spectrometer, a portable X-ray diffractometer, a mobile Fourier transform infrared spectrometer, a portable Raman spectrometer, or other spectrometers.

[0058] Each type of spectrometer is used to analyze specific materials. Portable X-ray fluorescence spectrometers are not suitable for non-solid samples, such as gases or liquids; portable X-ray diffractometers are not suitable for amorphous or amorphous materials; mobile Fourier transform infrared spectrometers are not suitable for water-sensitive samples; and portable Raman spectrometers are not suitable for samples that strongly absorb and scatter laser light, such as certain black or dark materials.

[0059] Analyzing the sample using a spectrometer to obtain a spectrum graph; the spectrum graph includes a spectrum curve;

[0060] S3. Spectral Preprocessing: Normalize, smooth, and remove invalid peaks from the spectral curve data. Save the spectrum obtained after analysis of the substance to be analyzed by the portable spectrometer, export the coordinate points in the spectrum, and preprocess the spectrum using mathematical methods to facilitate subsequent curve similarity comparison.

[0061] like Figure 2 As shown, in this embodiment, the normalization process is:

[0062]

[0063] Where y represents the vertical coordinate value of the data point of the spectrum curve; max and y minRespectively represent the maximum vertical coordinate value and the minimum vertical coordinate value; y i and y i ′ Respectively represent the ordinate values ​​of the i-th data point of the spectral curve before and after normalization.

[0064] Normalization aims to eliminate signal intensity variations caused by differences in intrinsic components within the sample (such as the content and activity of vibrational groups). After normalization, the ordinate of the spectral curve is mapped to a range of 0 to 1, converting it to dimensionless data. Dimensionless data, which lacks specific units, is often used to eliminate the influence of physical dimensions, allowing comparisons between different units of measurement.

[0065] like Figure 3 As shown, in this embodiment, the spectrum graph preprocessing also includes smoothing the data of the spectrum curve;

[0066] The smoothing process is done using the Savitizky-Golay filter:

[0067]

[0068] Among them, (x i ,y i ) represents the data point of the spectrum curve, i is the index of the current data point; X i Represents x i The corresponding smoothing window matrix; a, p and θ i are the constant term, coefficient and coefficient vector in the fitting polynomial respectively; m is the size of the smoothing window; Represents finding the polynomial coefficient θ i , so that X i θ i and Y i The Euclidean distance is the smallest; H i is the hat matrix; Y i is a vector consisting of the vertical coordinates of the data points within the smoothing window; Y i The result after filtering.

[0069] Smoothing is designed to remove noise and interference from the curve, making it more suitable for subsequent analysis and comparison. This solution uses a Savitizky-Golay filter to process the horizontal and vertical coordinate points in the spectrum. This smoothing is achieved by fitting a polynomial within a local window of data, ensuring that the polynomial optimally fits the data points within the window.

[0070] Alternatively, the smoothing methods also include moving average, exponential smoothing, polynomial fitting, Bezier curve fitting, local weighted scatter point smoothing method (Loess), Kalman filter, wavelet transform and other methods.

[0071] like Figure 4 As shown, in this embodiment, the spectrum preprocessing further includes removing invalid peaks of the spectrum curve; and the vertical coordinate values ​​of the data points corresponding to the selected invalid peaks are modified to preset intensity values.

[0072] Invalid peak correction aims to remove invalid peaks from a spectrum. For example, a portable Raman spectrometer may produce wide invalid peaks when detecting interference from strongly fluorescent substances, or may be affected by interference from cosmic rays during the detection process. Invalid peaks can be automatically identified using machine learning, deep learning, and other methods, using equipment and algorithms, or manually determined by technicians based on their expertise.

[0073] S4. Similarity Analysis: For the spectral curves of multiple samples, randomly select two samples for comparison and obtain a similarity value. If the similarity value is within a preset threshold range, the two samples are considered to be of the same type of solid waste. If the similarity between any two or more spectral curves is greater than 0.90, the curves are considered to be of the same type. Furthermore, if the similarity is less than 0.50, the compared curves are considered to be of different types. The specific preset threshold range can be set as needed; in this embodiment, the threshold range is [0.9, 1].

[0074] In this embodiment, when the changing trend of the spectral curve is monotonically changing, such as monotonically increasing or monotonically decreasing, the similarity value is calculated using the Spearman correlation coefficient; if the changing trend of the spectral curve is irregularly changing, such as fluctuating, the similarity value is calculated using a correlation coefficient algorithm based on Euclidean distance conversion.

[0075] Specifically, such as Figure 5 As shown, in this embodiment, the Spearman correlation coefficient is calculated as:

[0076]

[0077] Among them, r s Is the similarity value; A i and B iThey correspond to the ordinate values ​​of two spectral curves respectively; i is the index of the ordinate value in a spectral curve; n represents the number of data points in a spectral curve; rank(·) represents the rank.

[0078] The Spearman correlation coefficient measures the monotonic relationship between two variables. Rank(·) is used to sort each value in a dataset and assign a rank. For example, if we sort each value in variables A and B separately, we can arrange the values ​​in the dataset from smallest to largest (or largest to smallest) and rank them sequentially: the smallest value is assigned rank 1, the next smallest value is assigned rank 2, and so on.

[0079] like Figure 6 As shown in the figure, the specific calculation of the correlation coefficient algorithm based on Euclidean distance conversion is:

[0080]

[0081] Among them, r is the similarity value; (x i ,y i ) and (x i ′ ,y i ′ ) represent the coordinates of the i-th data point of the two spectral curves respectively; n represents the number of data points in a spectral curve; σ x and σ y Indicates the standard deviation of all data points on the horizontal and vertical axes respectively; u x and u y Represents the average value of all data points on the horizontal and vertical axes respectively; d is the Euclidean distance.

[0082] S5, cluster analysis: for several samples that have not been classified, cluster analysis algorithm is used to classify them. Figure 7 As shown, the clustering algorithm is calculated as follows:

[0083]

[0084] Where x represents the spectral curve; μ i (0) represents the center of the i-th cluster before the iteration begins; C i (t) represents the set of all data assigned to the i-th cluster in the t-th iteration; μ i (t+1) represents the center of the i-th cluster at the r+1th iteration; k represents the number of classifications;

[0085] When μ i (t+1) =μ i(t) When the number of iterations reaches the preset value, the iteration is terminated.

[0086] In similarity analysis, samples with similarity values ​​within the preset threshold range can be clustered, but there may still be samples whose similarity values ​​cannot be within the preset threshold range. In this case, the clustering algorithm is used to perform high-precision analysis and classification on the remaining unclassified samples.

[0087] The K-Means clustering algorithm is an iterative distance-based clustering analysis algorithm. It divides the spectral curve into k preset clusters (categories) and calculates the mean of all points at the center of each cluster. By updating the cluster center and recalculating the mean of all points within the cluster, the resulting k value at the end of the iteration is the final cluster, and each cluster corresponds to a category.

[0088] Based on the disclosure and teachings of the above description, those skilled in the art may also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and modifications and variations of the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although certain specific terms are used in this description, these terms are only for convenience of description and do not constitute any limitation to the present invention.

Claims

1. A solid waste classification method based on a spectrometer, characterized in that: The steps include: S1. Preliminary classification: Obtain a sample of unknown solid waste, observe the characteristics of the sample, and obtain characteristic information; perform preliminary classification of the sample based on its appearance; S2. Select a spectrometer: Select a spectrometer based on the characteristic information; the spectrometer is portable and movable; the spectrometer includes an X-ray fluorescence spectrometer, an X-ray diffractometer, a Fourier transform infrared spectrometer, and a Raman spectrometer, corresponding to the types of the sample, respectively, liquid materials, amorphous or organic materials, high water content materials, and dark materials; Analyzing the sample using the spectrometer to obtain a spectrum graph; the spectrum graph includes a spectrum curve; S3, spectrum preprocessing: normalizing the data of the spectrum curve; the normalization process is: Wherein, y represents the vertical coordinate value of the data point of the spectrum curve; max and y min Respectively represent the maximum vertical coordinate value and the minimum vertical coordinate value; y i and y i ′ Respectively represent the ordinate values ​​of the data point of the ith item of the spectral curve before and after normalization; S4. Similarity analysis: for the spectral curves of the multiple samples, arbitrarily select two samples for comparison to obtain a similarity value; Wherein, when the variation trend of the spectral curve is monotonically changing, the similarity value is calculated by using the Spearman correlation coefficient; when the variation trend of the spectral curve is irregularly changing, the similarity value is calculated by using a correlation coefficient algorithm based on Euclidean distance conversion; If the similarity value is within a preset threshold range, the two compared samples belong to the same type of solid waste.

2. The solid waste classification method based on a spectrometer according to claim 1, characterized in that: The spectrum graph preprocessing further includes smoothing the data of the spectrum curve.

3. The solid waste classification method based on spectrometer according to claim 1, characterized in that: The spectrum preprocessing further includes removing invalid peaks of the spectrum curve; and modifying the vertical coordinate values ​​of the data points corresponding to the selected invalid peaks to preset intensity values.

4. The method for solid waste classification based on a spectrometer according to claim 1, characterized in that: When the variation trend of the spectral curve is monotonically changing, the similarity value is calculated using the Spearman correlation coefficient, which is specifically calculated as follows: Among them, r s That is the similarity value; A i and B i The ordinate values ​​corresponding to the two spectral curves respectively; i is the index of the ordinate value in one spectral curve; n represents the number of data points in one spectral curve; rank(·) represents the rank.

5. The solid waste classification method based on spectrometer according to claim 1, characterized in that: When the variation trend of the spectral curve is irregular, the calculation of the similarity value includes adopting a correlation coefficient algorithm based on Euclidean distance conversion, specifically calculated as follows: Wherein, r is the similarity value; (x i ,y i ) and (x i ′ ,y i ′ ) represent the coordinates of the i-th data point of the two spectral curves respectively; n represents the number of data points in one spectral curve; σ x and σ y Indicates the standard deviation of all data points on the horizontal and vertical axes respectively; u x and u y Represents the average value of all data points on the horizontal and vertical axes respectively; d is the Euclidean distance.

6. The method for solid waste classification based on a spectrometer according to claim 1, characterized in that: The method further includes the following steps: S5, cluster analysis: for a number of unclassified samples, cluster analysis algorithms are used to classify them.

7. The method for solid waste classification based on spectrometer according to claim 6, characterized in that: The cluster analysis algorithm is calculated as follows: Wherein, x represents the spectral curve; μ i (0) represents the center of the i-th cluster before the iteration begins; C i (t) represents the set of all data assigned to the i-th cluster in the t-th iteration; μ i (t+1) represents the center of the i-th cluster at the r+1th iteration; k represents the number of classifications; When μ i (t+1) =μ i (t) When the preset number of iterations is reached, the iteration is terminated.

Images