Deep learning driven carbon scrap automatic detection and quantitative analysis method

Abstract

The invention relates to a deep learning-driven carbon chip automatic detection and quantitative analysis method, which comprises the following steps: (1) digital image acquisition: making a carbon chip sample subjected to experimental treatment into a microscope glass slide, and generating a digital image by using a high-resolution glass slide scanner; (2) establishing an annotation data performing format conversion and annotation on the obtained digital image, and clearly marking outlines of carbon scraps, lycopodium clavatum and other impurities in the image so as to establish the annotation database; (3) deep learning model training: dividing the annotation database into a training set, a verification set and a test set; and (4) performing automatic identification and quantitative analysis on the carbon chips: performing automatic identification on the digital image of the sample to be identified by adopting a deep learning model passing precision verification, counting the number of the carbon chips and the lycopodium spores, calculating other statistical indexes of the carbon chips, and further performing quantitative reconstruction analysis on a fire event. The method has the characteristics of high precision, high speed, high universality, simplicity and convenience in operation and suitability for statistics of a large number of samples.

Description

Technical Field

[0001] The present invention relates to the field of quantitative reconstruction of fire events in the study of ecological environment changes, and in particular to a deep learning-driven method for automatic detection and quantitative analysis of charcoal. Background Art

[0002] In recent years, climate warming has led to an increase in extreme wildfires, which have seriously affected human health and the Earth system. In order to provide reference information in a natural state, researchers have increasingly focused on the reconstruction of wildfire history. Charcoal, as an inorganic carbon compound produced by the burning of vegetation, is called a "fossil of fire" and can record information such as the frequency of past wildfires, the type of burned vegetation, and climatic conditions. Its analysis not only helps to accurately reconstruct the history of ancient fires, but also provides important support for the quantitative study of modern fires. Therefore, the accurate identification of charcoal is of key significance in the study of fire events.

[0003] At present, the identification of charcoal mainly relies on manual visual counting under a microscope. The specific process includes: first weighing the collected soil samples, adding 10% HCl and 40% HF to remove carbonates and silicates, and adding Lycopodium spores to assist in calculating the concentration. Then prepare microscope slides, and finally visually count the charcoal and Lycopodium spores under a microscope. The total number of charcoal in the sample is calculated by the ratio of the number of charcoal to Lycopodium spores in multiple fields of view. According to estimates, an experienced experimenter needs 4-8 hours to complete the identification of a sample. Since this process is time-consuming and requires a high level of professionalism from the identification personnel, manual counting remains a tedious task.

[0004] Although researchers have developed automatic charcoal identification algorithms, most of these algorithms are based on statistical features designed with prior knowledge. The specific method is to segment all objects (including charcoal and its approximations) in the scanned film by grayscale threshold in open source software such as ImageJ, and then count the morphological features of each object, such as color, texture, mean and median. Objects that meet the conditions are classified as charcoal, and those that do not meet the conditions are non-charcoal. However, these methods have two urgent problems: 1) Since artificially designed features are used for identification, the deposition environment and scanning equipment differences of charcoal samples may affect the accuracy, and the grayscale threshold and statistical feature threshold need to be manually adjusted to distinguish charcoal from background and non-charcoal. 2) The morphology of Lycopodium spores is more complex than that of charcoal, and it is difficult to design effective morphological features for accurate identification. Therefore, the identification of charcoal and Lycopodium spores cannot be carried out simultaneously, and the number of Lycopodium spores still needs to be manually counted. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a deep learning-driven automatic detection and quantitative analysis method for carbon chips with high accuracy, high speed, strong versatility, simple operation, and suitability for statistics of a large number of samples.

[0006] To solve the above problems, the present invention provides a deep learning driven carbon dust automatic detection and quantitative analysis method, comprising the following steps: ⑴Digital image acquisition: The experimentally treated charcoal samples were prepared into microscope slides and digital images were generated using a high-resolution slide scanner; ⑵Establish annotation database: The acquired digital images are converted and annotated to clearly mark the outlines of charcoal dust, lycopodium and other impurities in the images, thereby establishing an annotation database; ⑶ Deep learning model training: The labeled database is divided into training set, validation set and test set in the ratio of 70%, 15% and 15%, respectively. The YOLOv8 deep learning target detection model is introduced. The training set and validation set are used to train and preliminarily evaluate the deep learning model. The test set is used to test the accuracy of the deep learning algorithm to evaluate the performance of the model. ⑷Automatic identification and quantitative analysis of carbon chips: A deep learning model with verified accuracy is used to automatically identify the digital images of the samples to be identified, count the number of charcoal and Lycopodium spores, calculate statistical indicators including the concentration, aspect ratio, and size ratio of the charcoal, and further perform quantitative reconstruction and analysis of the fire event.

[0007] In step (2), the annotation database is established in the following manner: ① Collect scanned images of carbon dust samples of various deposition types in TIFF format; ②Convert the scanned image in TIFF format to JPG or PNG format and crop it into multiple sub-images according to the window size of 1000×1000 pixels; ③ Label all charcoal, Lycopodium spores and impurities using Labelme software, clearly marking their categories and outlines; ④ Summarize all sub-graphs and their corresponding annotation information to establish the annotation database required for deep learning; ⑤ Perform random sampling checks on the annotation database to verify the consistency between the image and the label; if it does not meet the accuracy requirements, the non-matching sub-images need to be re-annotated.

[0008] The charcoal samples of various sedimentary types in step ① refer to different types of samples including topsoil, air pollen, loess, strata, lakes and oceans.

[0009] In the step ④, the number of charcoal scraps marked in the marking database is not less than 1,500, and the number of Lycopodium spores is not less than 200.

[0010] In step (4), the automatic identification and quantitative analysis of carbon chips are carried out according to the following method: ⅰAfter obtaining the model that has passed the accuracy test, the digital image of the sample to be identified is input into the model, and the model outputs the category, confidence level and contour coordinates of all objects in the image; ii) Screen out the charcoal and Lycopodium spores with a confidence level greater than 0.5, and obtain their quantity and shape parameters; ⅲ Summarize the quantity and shape information of charcoal particles and Lycopodium spores in the sub-map, and reconstruct the fire event quantitatively based on the principle of charcoal science.

[0011] Compared with the prior art, the present invention has the following advantages: 1. Significantly improved the automation and applicability of carbon dust identification.

[0012] (1) In actual operation, in addition to identifying charcoal, it is also necessary to identify Lycopodium spores used to calculate charcoal indicators. However, the prior art only realizes the identification of charcoal, and Lycopodium spores still need to be manually identified. The present invention realizes the simultaneous identification of charcoal and Lycopodium spores, eliminates the dependence on manual labor, and thus significantly improves the automation level of the charcoal identification system.

[0013] (2) The differences in the deposition environment and scanning equipment of the carbon samples may affect the accuracy of the existing methods, resulting in the need for manual adjustment of parameters in actual operation. The present invention collects samples of various deposition types and expands the data set using data enhancement technology, which can realize automatic identification of samples of different deposition environments and has higher applicability.

[0014] 2. Further improved accuracy and consistency.

[0015] ⑴ The prior art uses statistical features based on prior knowledge to identify charcoal, while the present invention uses convolutional neural networks in deep learning to automatically extract the features of charcoal and Lycopodium spores, which can capture deeper details and thus improve the accuracy of recognition.

[0016] (2) In actual operation, manual visual statistics rely on the experience of the experimenter, and different operators may obtain significantly different results for the same sample. In addition, the operator may reduce the level of meticulousness when working continuously, thus affecting the accuracy of the results. The present invention is not affected by artificial subjective factors and significantly improves the consistency of the results.

[0017] 3. Significantly improved the speed of carbon dust identification.

[0018] ⑴ The current mainstream method for charcoal identification relies on professionals to conduct visual counting under a microscope. The identification time for each sample is about 4 to 8 hours, resulting in high economic and time costs. The existing method takes about 20 minutes to identify charcoal, but additional time is required for manual visual counting of Lycopodium. The present invention shortens the overall identification time to 25 seconds by simultaneously identifying charcoal and Lycopodium, significantly improving the efficiency of charcoal identification.

[0019] (2) When the number of samples is large, the existing recognition method consumes a lot of time. The present invention can use GPU to process the scanned images of the samples in parallel, further reducing the total working time. Therefore, it can provide strong data support for the quantitative reconstruction of large space fire events. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0021] Figure 1 Schematic diagram of the steps of acquiring digital images and establishing a labeling database of the present invention. Among them: A shows the spatial distribution of samples collected by the present invention; B is the process of laboratory extraction and making glass slides; C and D represent the input of glass slides into a high-resolution scanner device to scan digital images (TIFF format), and the digital images are converted into formats (PNG or JPG format) and cut into sub-images of 1000×1000 pixels; E is a screenshot of the interface for labeling charcoal and lycopodium in the sub-image in LabelMe5.3.1 labeling software; F represents the aggregation of labeling information to obtain a labeling database.

[0022] Figure 2 The process of sample recognition using the deep learning YOLOv8 target detection method is demonstrated. After the sample digital image (size 1000×1000) is input into the model, the backbone network extracts features from the image, and then the neck network fuses the extracted features. The detection network then performs recognition and predicts the category, position and outline of each object in the image. Finally, the number of charcoal and lycopodium, the major and minor axis lengths of the charcoal, and other statistical indicators are obtained through post-processing.

[0023] Figure 3 The visual recognition result of the charcoal sample of the present invention is shown in Figure 1. A is the scanned image of the charcoal sample; B is the result of the automatic recognition of the model of the present invention, where MC indicates that the object is charcoal, LyP indicates that the object is Lycopodium spore, the rotating rectangle is the circumscribed rectangle of the charcoal, the green line is the outline of the charcoal, and the number indicates the possibility that the model considers the object to be charcoal or Lycopodium spore.

[0024] Figure 4The effect of the number of annotations on the recognition accuracy in the present invention is shown in Figure 1. A shows the effect of the number of charcoal annotations on the recognition accuracy of deep learning; and B shows the effect of the number of Lycopodium spore annotations on the recognition accuracy of deep learning.

[0025] Figure 5 It is a block diagram of the technical route of the present invention.

[0026] Figure 6 The comparison chart of the results of Example 1 of the present invention and manual visual statistics on 64 samples. Among them: A is the comparison of the deep learning model results and the number of charcoal scraps obtained by manual statistics; B is the comparison of the deep learning model recognition results and the number of Lycopodium spores obtained by manual statistics; C is the comparison of the deep learning model recognition results and the charcoal scrap length-to-circle ratio index results obtained by manual statistics; D is the comparison of the deep learning model recognition results and the charcoal scrap size ratio index results obtained by manual statistics. DETAILED DESCRIPTION

[0027] like Figure 1~2 As shown, a deep learning-driven carbon dust automatic detection and quantitative analysis method includes the following steps: ⑴Digital image acquisition: The experimentally treated charcoal samples were prepared on microscope slides and digital images were generated using a high-resolution slide scanner.

[0028] ⑵Establish annotation database: The acquired digital images are converted and annotated to clearly mark the outlines of charcoal, lycopodium and other impurities in the images, thereby establishing an annotation database. The specific establishment process is as follows: ① Collect scanned images of charcoal samples of various sedimentary types in TIFF format. Charcoal samples of various sedimentary types refer to different types of samples including topsoil, air pollen, loess, strata, lakes and oceans.

[0029] ②Convert the scanned image in TIFF format to JPG or PNG format, and crop it into multiple sub-images according to the window size of 1000×1000 pixels.

[0030] ③ Use Labelme software to label all charcoal, Lycopodium spores and impurities, and clearly mark their categories and outlines.

[0031] ④ Summarize all sub-graphs and their corresponding annotation information to establish the annotation database required for deep learning. To ensure the accuracy and generalization ability of the model, the number of annotated charcoal fragments in the annotation database should be no less than 1,500 and the number of Lycopodium spores should be no less than 200.

[0032] ⑤ Perform random sampling checks on the annotation database to verify the consistency between the image and the label; if it does not meet the accuracy requirements, the non-matching sub-images need to be re-annotated.

[0033] ⑶ Deep learning model training: The labeled database is divided into training set, validation set and test set in the ratio of 70%, 15% and 15%. The YOLOv8 deep learning target detection model is introduced. The training set and validation set are used to train and preliminarily evaluate the deep learning model. The test set is used to test the accuracy of the deep learning algorithm to evaluate the performance of the model.

[0034] ⑷Automatic identification and quantitative analysis of carbon chips: The digital images of the samples to be identified are automatically identified using a deep learning model with verified accuracy, the number of charcoal and Lycopodium spores is counted, statistical indicators including the concentration, aspect ratio, and size ratio of charcoal are calculated, and further quantitative reconstruction and analysis of the fire event are performed. The specific methods are as follows: ⅰAfter obtaining the model that has passed the accuracy test, the digital image of the sample to be identified is input into the model, and the model outputs the category, confidence level and contour coordinates of all objects in the image; ii) Screen out the charcoal and Lycopodium spores with a confidence level greater than 0.5, and obtain their quantity and shape parameters; ⅲ Summarize the quantity and shape information of charcoal particles and Lycopodium spores in the sub-map, and reconstruct the fire event quantitatively based on the principle of charcoal science.

[0035]

Working Principle

[0036] The present invention introduces the YOLOv8 target detection algorithm to automatically identify the charcoal scanning film. Compared with the existing method, the simultaneous identification of charcoal and lycopodium is achieved without the need for hyperparameter setting and additional manual intervention.

[0037] like Figure 2As shown in the figure, the framework of YOLOv8 mainly consists of three parts: backbone, neck and detection head. The backbone part adopts a network structure similar to CSPDarknet to extract image features. It uses a convolution kernel with a size of 3x3, a step size of 1, and a padding of 1 to extract features from the input image. The neck part is mainly used for feature fusion. It adopts the PANet (Path Aggregation Network) structure. Through path enhancement and top-down feature fusion, it integrates feature information from different levels, thereby improving the model's detection ability for targets of different scales.

[0038] When the scanned digital image of the sample is input into the model, the features of the object in the image are extracted by the backbone network, and then the features of the object in the image are fused by the neck network. The detection network outputs the object information detected by the model, including category labels, bounding box coordinates, confidence scores, and segmentation masks. The category label indicates the category to which the object belongs, which is one of charcoal, lycopodium, or impurities; the bounding box coordinates are used to describe the location of the object in the image; the confidence score is used to evaluate the model's confidence in the detection results; the segmentation mask is used to annotate the target area pixel by pixel. The segmentation mask is a binary image with the same size as the original input image, in which the value of each pixel indicates whether the pixel belongs to the target category. Therefore, for objects whose identification category is charcoal, combined with their segmentation masks, the length and width of their circumscribed rectangles can be obtained as the major and minor axis lengths of the charcoal, and the area is the product of the number of pixels in the segmentation mask and the area of ​​the pixels. For objects whose identification category is lycopodium, their total number is counted.

[0039] Example 1 1. Digital image acquisition: ⑴ Obtaining charcoal sediment samples. To ensure the applicability of the model, the present invention collects various types of sediment samples including air pollen, topsoil, loess, lakes, oceans, etc. In fact, the required sediment samples can be collected according to research needs. The spatial distribution of the samples collected by the present invention is as follows: Figure 1 As shown in A.

[0040] ⑵ According to the palynological extraction method (such as hydrofluoric acid method, heavy liquid extraction method, etc.), the sample is made into a microscope slide for automatic counting of charcoal chips ( Figure 1 B).

[0041] ⑶ Use a high-resolution slide scanner to scan the slide into a digital image ( Figure 1 C).

[0042] 2. Establishment of annotation database A labeled database is established for model training and learning. In this example, all charcoal fragments, lycopodium spores, and impurities in 5987 sub-graphs are labeled.

[0043] ⑴Convert the TIFF scanned image into JPG or PNG format, and cut out multiple sub-images according to the 1000×1000 size window ( Figure 1 D).

[0044] ⑵ Label all charcoal, Lycopodium spores and impurities, and clearly mark their categories and outlines. This example was completed in the deep learning labeling software Labelme 5.3.1 ( Figure 2 E). This example labels the categories and contour coordinates of charcoal, Lycopodium spores, and impurities.

[0045] ⑶ Collect all sub-images and corresponding annotation information to establish the database required for deep learning. In this example, the database contains 5697 sub-images, including 21094 charcoal particles, 6241 Lycopodium spores, and 19964 impurities.

[0046] Experiments show that when a dataset of about 1,500 charcoal particles and 200 Lycopodium spores is used for training, the model's recognition accuracy for charcoal particles and Lycopodium spores has reached 91% and 95%. Therefore, the present invention recommends that the minimum number of annotations be determined as 1,500 charcoal particles and 200 Lycopodium spores. When the number of annotations is increased, the model recognition accuracy can be further improved, such as Figure 4 As shown in the figure. When the number of charcoal fragments and lycopodium spores annotated in this example is increased to 21,094, the recognition accuracy of the model for charcoal fragments reaches 94%, and the recognition accuracy for lycopodium spores reaches 99%. The above accuracies were tested on a larger dataset, including 3,093 charcoal fragments and 967 lycopodium spores.

[0047] ⑷ Perform random sampling checks on the database to verify the consistency between the image and the label. If it does not meet the accuracy requirements, the mismatched sub-images need to be re-labeled.

[0048] 3. Deep learning model training: The established database is divided into training set, validation set and test set in the ratio of 70%, 15% and 15%. The YOLOv8 deep learning target detection model is introduced, and the training set and validation set are used for model training and preliminary evaluation. The test set is used to test the accuracy of the deep learning algorithm. The running environment of this example is python3.9, and the YOLOv8 code is version 8.2.45 in the ultralytics official library.

[0049] Experiments show that the YOLOv8 deep learning target detection framework provides five benchmark models, namely YOLOv8n, YOLOv8m, YOLOv8s, YOLOv8l and YOLOv8x, with increasing model complexity. Among them, YOLOv8n has an accuracy of 93.8% for charcoal and 99.0% for Lycopodium spores, and can complete the recognition of a sample in about 25 seconds. Combined with the specific situation of this example, YOLOv8n was selected as the optimal model based on the trade-off between speed and accuracy.

[0050] 4. Automatic identification and quantitative analysis of carbon chips: The deep learning model that has passed accuracy verification is post-processed to count the number of charcoal and lycopodium, calculate other statistical indicators of the charcoal, and further perform quantitative reconstruction analysis of the fire event.

[0051] ⑴ After obtaining a model that has passed the accuracy test, the digital image of the sample to be identified is input into the model. The model will label each object with its category (charcoal or Lycopodium spores), confidence (i.e., the reliability score of the identification), and the contour coordinates describing the object boundary. The visualization results are as follows: Figure 3 shown.

[0052] ⑵ Filter out objects with a confidence level greater than 0.5 from the model output to ensure that only high-confidence recognition results are processed. For Lycopodium spores, only their number needs to be considered. For charcoal, further analysis includes quantitative statistics and shape parameter calculation. The specific implementation method is to fit a circumscribed rectangle to its contour coordinates. The length and width of the circumscribed rectangle are used as the major and minor axes of the charcoal, respectively. The circumscribed rectangle is implemented by the cv2.minAreaRect(points) method in the opencv library in python, where points represents the contour coordinates of the charcoal. The area of ​​the charcoal is obtained by calculating the number of pixels within the contour.

[0053] The charcoal dust statistical index obtained by the present invention was compared with the manual visual statistical results on 64 samples in a section. Figure 6 The correlation between the results of the present invention and the manual statistical results is above 0.97, indicating that the present invention can effectively and automatically detect and quantitatively analyze the carbon dust in the sample. Among them, the number of carbon dust and the number of Lycopodium obtained by the present invention are consistent with the number obtained by manual statistics ( Figure 6 A and B), the calculated aspect ratio and size ratio statistical indicators are consistent with the manual results in terms of trend ( Figure 6 C and D), that is, the results of the AD graph show that the model recognition results are highly consistent with the manual statistical results in trend.

[0054] ⑶ Summarize the quantity and shape information of charcoal particles and Lycopodium spores in the sub-map, calculate the statistical indicators of charcoal, and provide accurate data support for the quantitative reconstruction of fire events.

[0055] It should be noted that the above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make replacements or modifications without departing from the scope of protection of the claims of the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A deep learning driven method for automatic detection and quantitative analysis of carbon dust, comprising the following steps: ⑴Digital image acquisition: The experimentally treated charcoal samples were prepared into microscope slides and digital images were generated using a high-resolution slide scanner; ⑵Establish annotation database: The acquired digital images are converted and annotated to clearly mark the outlines of charcoal dust, lycopodium and other impurities in the images, thereby establishing an annotation database; ⑶ Deep learning model training: The labeled database is divided into training set, validation set and test set in the ratio of 70%, 15% and 15%, respectively. The YOLOv8 deep learning target detection model is introduced. The training set and validation set are used to train and preliminarily evaluate the deep learning model. The test set is used to test the accuracy of the deep learning algorithm to evaluate the performance of the model. ⑷Automatic identification and quantitative analysis of carbon chips: A deep learning model with verified accuracy is used to automatically identify the digital images of the samples to be identified, count the number of charcoal and Lycopodium spores, calculate statistical indicators including the concentration, aspect ratio, and size ratio of the charcoal, and further perform quantitative reconstruction and analysis of the fire event.

2. The deep learning driven carbon dust automatic detection and quantitative analysis method according to claim 1, characterized in that: In step (2), the annotation database is established in the following manner: ① Collect scanned images of carbon dust samples of various deposition types in TIFF format; ②Convert the scanned image in TIFF format to JPG or PNG format and crop it into multiple sub-images according to the window size of 1000×1000 pixels; ③ Label all charcoal, Lycopodium spores and impurities using Labelme software, clearly marking their categories and outlines; ④ Summarize all sub-graphs and their corresponding annotation information to establish the annotation database required for deep learning; ⑤ Perform random sampling checks on the annotation database to verify the consistency between the image and the label; if it does not meet the accuracy requirements, the non-matching sub-images need to be re-annotated.

3. The deep learning driven carbon dust automatic detection and quantitative analysis method according to claim 2, characterized in that: The charcoal samples of various sedimentary types in step ① refer to different types of samples including topsoil, air pollen, loess, strata, lakes and oceans.

4. The deep learning driven carbon dust automatic detection and quantitative analysis method according to claim 2, characterized in that: In the step ④, the number of charcoal scraps marked in the marking database is not less than 1,500, and the number of Lycopodium spores is not less than 200.

5. The deep learning driven carbon dust automatic detection and quantitative analysis method according to claim 1, characterized in that: In step (4), the automatic identification and quantitative analysis of carbon chips are carried out according to the following method: ⅰAfter obtaining the model that has passed the accuracy test, the digital image of the sample to be identified is input into the model, and the model outputs the category, confidence level and contour coordinates of all objects in the image; ii) Screen out the charcoal and Lycopodium spores with a confidence level greater than 0.5, and obtain their quantity and shape parameters; ⅲ Summarize the quantity and shape information of charcoal particles and Lycopodium spores in the sub-map, and reconstruct the fire event quantitatively based on the principle of charcoal science.

Images