An Automatic Detection Method for Nanoparticles Based on Deep Learning and AFM Images

Abstract

The present invention discloses an automatic detection method for nanoparticles based on deep learning and AFM images. The method includes: collecting the original AFM images of nanoparticles and processing them to obtain a normalized height map; establishing a deep learning model and training the model; collecting the original AFM detection images of the nanoparticles to be detected and processing them; inputting the detected normalized height map into the trained model, and outputting the regions, aggregation information, and confidence information of the nanoparticles to be displayed in the detected visualization height map; obtaining the pixel-size factor, size information, size distribution information, and aggregation ratio of the nanoparticles according to the detected original images and detected height maps, and finally realizing the automatic detection of nanoparticles. The present invention can automatically segment the nanoparticles in the AFM images and further analyze the nanoparticle information, solving the problems of low accuracy, weak applicability, and semi-automation existing in the current nanoparticle analysis based on traditional machine vision methods.

Description

Technical Field

[0001] The present invention relates to a method for automatically detecting nanoparticles, belonging to the field of microscopic image analysis, and particularly to a method for automatically detecting nanoparticles based on deep learning and AFM images. Background Art

[0002] In recent years, due to their physical, chemical, and biological properties such as small size, high specific surface area, and surface effect, nanoparticles have been widely used in fields such as medicine, electronics, energy, and materials. Atomic Force Microscopy (AFM) is the main means for characterizing the morphology, size, aggregation, etc. of nanoparticles.

[0003] Currently, the analysis of nanoparticles based on AFM images mostly relies on manual judgment of nanoparticles in the images, recording the number of nanoparticles, and obtaining the height information of individual nanoparticles by pulling and analyzing the truncation line, which has the disadvantages of time-consuming and laborious and being subject to subjective influence; in addition, the existing Nanoscope Analysis software has a particle analysis function, which defines the area of nanoparticles according to the height of pixels and can only distinguish isolated nanoparticles on a flat substrate without other impurities, having problems such as a small applicable range, the need for manual participation, and the inability to judge particle aggregation.

[0004] With the continuous expansion of the demand for nanoparticle characterization, an efficient and reliable method for automatically analyzing AFM image nanoparticles is particularly important. Summary of the Invention

[0005] In order to solve the problems in the background art, the present invention provides a method for automatically detecting nanoparticles based on deep learning and AFM images, which can efficiently, accurately, and automatically extract nanoparticle individuals and statistical information in AFM images.

[0006] The technical solution adopted by the present invention is as follows:

[0007] The method for automatically detecting nanoparticles based on deep learning and AFM images of the present invention includes the following steps:

[0008] 1) Collect atomic force microscope (AFM) original images of several nanoparticles, perform image processing and annotation, and obtain the AFM normalized height map and annotation information of each AFM original image, thereby establishing an AFM image database for nanoparticle analysis.

[0009] 2) Establish a deep learning model for nanoparticle detection and segmentation.

[0010] 3) Divide each AFM normalized height map into a training set and a validation set according to a preset ratio, and input the training set and the validation set into the deep learning model for nanoparticle detection and segmentation to be trained according to the annotation information to obtain the trained deep learning model for nanoparticle detection and segmentation; in actual operation, divide it into a training set, a validation set and a test set according to the ratio of 7:2:1, and use the test set to evaluate the performance of the trained deep learning model for nanoparticle detection and segmentation.

[0011] 4) Collect the original atomic force microscope (AFM) detection images of the nanoparticles to be detected, and obtain the AFM detection height map, the AFM detection visualization height map, and the AFM detection normalized height map after image processing; input the AFM detection normalized height map into the trained deep learning model for nanoparticle detection and segmentation, and the trained deep learning model for nanoparticle detection and segmentation outputs the region information, aggregation information, and confidence of each nanoparticle in the original AFM detection image, and then displays them in the AFM detection visualization height map.

[0012] 5) Calculate the pixel-size factor based on the original AFM detection image and the AFM detection height map, and then calculate the size information of each nanoparticle, so as to obtain the size distribution information and aggregation ratio of all nanoparticles in the original AFM detection image, and finally realize the automatic detection of nanoparticles.

[0013] In the above step 1), the original AFM images of several nanoparticles collected include the original AFM images of nanoparticles with different morphologies, sizes, concentrations, aggregation degrees, and substrates. The morphologies specifically include spherical and flaky, and the substrates are specifically the substrates for fixing nanoparticles, including silicon wafers, mica sheets, and glass slides.

[0014] In the above step 1), collect the original AFM images of several nanoparticles and perform image processing. For each original AFM image, specifically as follows:

[0015] Extract the AFM height sensor data from the original AFM image, perform smoothing processing on the AFM height sensor data, or perform smoothing processing and plane fitting in sequence to obtain the AFM height map; map the height values in the AFM height map to color pixels pixel by pixel to obtain the AFM visualization height map; mark the annotation information of each nanoparticle on the AFM visualization height map, and the annotation information includes region information and aggregation information. The region information includes position and contour, and the aggregation information is aggregation and non-aggregation; perform maximum-minimum normalization on the AFM height map to obtain the AFM normalized height map.

[0016] In the aforementioned step 2), the specific establishment of the deep learning model for nanoparticle detection and segmentation is as follows:

[0017] Based on the SOLOv2 real-time instance segmentation model, use a ResNet model with a depth of 10 for the backbone network in the SOLOv2 real-time instance segmentation model, and change the number of channels in the backbone network, neck network, and head network of the SOLOv2 real-time instance segmentation model to 0.5 times the original number of channels, thereby establishing a lightweight deep learning model for nanoparticle detection and segmentation.

[0018] In the aforementioned step 3), input the training set and the validation set into the deep learning model for nanoparticle detection and segmentation for training, specifically as follows:

[0019] Step 3.1) For each AFM normalized height map in the training set, perform data augmentation processing on the AFM normalized height map and the annotation information on its corresponding AFM visualized height map. The data augmentation processing includes using one or several processing methods such as random horizontal flipping, random vertical flipping, random diagonal flipping, and random size scaling for processing. After each processing, a data-augmented map is obtained.

[0020] Step 3.2) Input the training set and each data-augmented map into the deep learning model for nanoparticle detection and segmentation for training. Each time training is performed, the updated model parameters of the deep learning model for nanoparticle detection and segmentation are obtained as the model parameters for the next training.

[0021] Step 3.3) Repeat steps 3.1)-3.2). After each repeated training, the currently updated deep learning model for nanoparticle detection and segmentation is obtained. Input the validation set into the currently updated deep learning model for nanoparticle detection and segmentation to obtain the average precision (AP) value of the deep learning model for nanoparticle detection and segmentation. Repeat until the average precision (AP) value obtained after several trainings approaches a fixed value, complete the training, and obtain the trained deep learning model for nanoparticle detection and segmentation.

[0022] Use the test set to evaluate the performance of the trained deep learning model. Judge whether the model performance meets the requirements according to the preset experience. If it does not meet the requirements, repeat step 1) to increase the data volume of the AFM image database, and repeat steps 3.1)-3.3) until the performance meets the requirements to obtain the final deep learning model for nanoparticle detection and segmentation.

[0023] In the aforementioned step 4), according to the confidence of each nanoparticle in the original AFM detection image, display the region information and aggregation information of the nanoparticles with a confidence greater than the preset confidence threshold in the AFM detection visualized height map.

[0024] In step 5) described above, a pixel-size factor is calculated based on the original AFM image and the AFM height map, and then the size information of each nanoparticle is calculated as follows:

[0025] Step 5.1) Extract the AFM scan size from the original AFM image and the pixel size of the AFM height map from the AFM height map, and divide the AFM scan size by the AFM height map size to obtain the pixel-size factor.

[0026] Step 5.2) Calculate the mask map of each nanoparticle according to the region information of each nanoparticle in the original AFM image obtained in step 4). The value of the mask map inside the region of each nanoparticle is true, and the value of the mask map outside the region of each nanoparticle is false; perform an AND operation on the AFM height map and the mask of each nanoparticle to obtain the height map of each nanoparticle. The valid pixels in the height map of each nanoparticle are the pixels where the mask map of each nanoparticle is true.

[0027] Step 5.3) Calculate the number of valid pixels in the height map of each nanoparticle in the original AFM image, and calculate the product of the number of valid pixels and the pixel-size factor to obtain the area of each nanoparticle.

[0028] Step 5.4) Extract the values of the valid pixels in the height map of each nanoparticle in the original AFM image, and the maximum value of the values of all valid pixels is the height of each nanoparticle.

[0029] Step 5.5) Calculate the product of the sum of the values of all valid pixels and the pixel-size factor to obtain the volume of each nanoparticle; the area, height, and volume of each nanoparticle are the size information of each nanoparticle.

[0030] In step 5) described above, based on the size information of each nanoparticle, the size distribution information and aggregation ratio of all nanoparticles in the original AFM image are obtained as follows:

[0031] Based on the aggregation information of each nanoparticle in the original AFM detection image obtained in step 4), obtain the number of non-aggregated nanoparticles and the number of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and area of each nanoparticle in the original AFM detection image, obtain the average area and area distribution of non-aggregated nanoparticles and the area distribution of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and height of each nanoparticle in the original AFM detection image, obtain the average height and height distribution of non-aggregated nanoparticles and the height distribution of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and volume of each nanoparticle in the original AFM detection image, obtain the volume distribution of non-aggregated nanoparticles and the volume distribution of aggregated nanoparticles in the original AFM detection image; the number, average area, area distribution, average height, height distribution, and volume distribution of non-aggregated nanoparticles and the number, area distribution, height distribution, and volume distribution of aggregated nanoparticles in the original AFM detection image constitute the size distribution information of all nanoparticles in the original AFM detection image.

[0032] Calculate the quotient of the number of aggregated nanoparticles in the original AFM detection image and the number of all nanoparticles to obtain the aggregation ratio.

[0033] The beneficial effects of the present invention are:

[0034] The nanoparticle detection and segmentation deep learning model of the present invention can efficiently and accurately obtain the region and aggregation information of each nanoparticle in the AFM image; it can automatically obtain the number, area distribution, height distribution, volume distribution, and aggregation ratio information of nanoparticles in the AFM image by computer without or with little manual participation, and has the advantages of high efficiency, accuracy, and wide application range. The present invention can automatically segment nanoparticles in the AFM image and further analyze nanoparticle information, solving the problems of low accuracy, weak applicability, and semi-automation existing in the current nanoparticle analysis based on traditional machine vision methods. Description of the Drawings

[0035] Figure 1 It is a flowchart of the method of the present invention;

[0036] Figure 2 It is the AFM visualization height map in the embodiment;

[0037] Figure 3 It is the region of the nanoparticles detected in the embodiment and shown in the AFM visualization height map;

[0038] Figure 4 The area of the nanoparticles detected by the particle analysis function of the Nanoscope Analysis software used for comparison in the embodiments. Detailed implementation manners

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0040] As Figure 1 shown, the method for automatically detecting nanoparticles based on deep learning and AFM images of the present invention includes the following steps:

[0041] 1) Collect the original AFM images of a number of nanoparticles, perform image processing and annotation on them, and obtain the AFM normalized height map and annotation information of each original AFM image, so as to establish an AFM image database for nanoparticle analysis.

[0042] In step 1), the original AFM images of a number of nanoparticles collected include the original AFM images of nanoparticles with different morphologies, sizes, concentrations, aggregation degrees and substrates. The morphologies specifically include spherical and flaky, and the substrates are specifically the substrates for fixing nanoparticles, including silicon wafers, mica sheets and glass slides.

[0043] In step 1), collect the original AFM images of a number of nanoparticles and perform image processing. For each original AFM image, specifically as follows:

[0044] Extract the AFM height sensor data from the original AFM image, perform smoothing processing on the AFM height sensor data or perform smoothing processing and plane fitting in sequence to obtain the AFM height map; map the height values in the AFM height map to color pixels pixel by pixel to obtain the AFM visual height map; mark the annotation information of each nanoparticle on the AFM visual height map, and the annotation information includes region information and aggregation information. The region information includes position and contour, and the aggregation information is aggregation and non-aggregation; perform maximum-minimum normalization on the AFM height map to obtain the AFM normalized height map.

[0045] In the specific implementation manner of the present invention, the specific steps of step 1) are further defined as follows:

[0046] Collect the original AFM images containing nanoparticles: In this example, the specific steps for collecting the original AFM images containing nanoparticles are as follows: collect nanoparticles, prepare scanning samples, and perform AFM scanning and imaging; the types of nanoparticles collected include different types of nanoparticles such as zinc oxide nanoparticles, titanium dioxide nanoparticles, silicon dioxide nanoparticles, cerium oxide nanoparticles, iron(III) oxide nanoparticles, iron(II,III) oxide nanoparticles, zinc ferrite nanoparticles, gold nanoparticles, molybdenum disulfide nanosheets, MXene flakes, etc., or the same type of nanoparticles from different sources, with different morphologies, sizes, and degrees of aggregation.

[0047] The steps for preparing scanning samples are as follows: for each type of nanoparticle, prepare 3 concentrations of nanoparticle suspensions with water as the solution; for each nanoparticle suspension, use a pipette to aspirate 50 μL of the nanoparticle suspension and drop it onto the substrate; place the substrate with the dropped nanoparticle suspension in a dust-free environment and let it dry naturally until the surface is free of water; the values of the concentrations of the 3 nanoparticle suspensions increase in a 5-fold geometric progression; the setting of the maximum concentration depends on the nanoparticles, and the principle is to ensure that at this concentration, the nanoparticles in the AFM image are evenly distributed, tightly packed but do not show a large number of stacked nanoparticles in the field of view; to increase the data richness of the AFM image database for nanoparticle analysis, the substrate is randomly selected during sample preparation, and the optional substrates include mica sheets, silicon wafers, and glass slides.

[0048] In step 1), obtain the AFM height map, AFM visualized height map, and AFM normalized height map from the original AFM image; the specific steps for obtaining the AFM height map include: using Python software and the nanoscope software package, extract the AFM height sensor, i.e., ZSensor data, from the original AFM image; perform smoothing processing, i.e., flattening, on the AFM height sensor data, and the order of the smoothing processing is 1; in this embodiment, to increase the database richness, plane fitting is not performed on the smoothed AFM height sensor data; the specific steps for obtaining the AFM visualized height map include: using Python software and the nanoscope software package, colorize the AFM height map, i.e., map the height values in the AFM height map to colored pixels; the specific steps for obtaining the AFM normalized height map include: using Python software, extract the maximum value H max and the minimum value H min , and normalize the AFM height map pixel by pixel; for the normalization of a single pixel, the formula for calculating the normalized height value V from the height value H is V = (H - H min ) / (H max - H min) Mark the positions and contours of the nanoparticles in the AFM visualization image. The specific steps are as follows: Personnel with experience in analyzing nanoparticles in AFM images mark the area of each nanoparticle and whether there is agglomeration in the AFM visualization image. The areas of the nanoparticles are all marked along the contours of the nanoparticles.

[0049] 2) Establish a deep learning model for nanoparticle detection and segmentation.

[0050] In step 2), the established deep learning model for nanoparticle detection and segmentation is as follows:

[0051] Based on the SOLOv2 real-time instance segmentation model, use a ResNet model with a depth of 10 for the backbone network in the SOLOv2 real-time instance segmentation model, and change the number of channels in the backbone network, neck network, and head network of the SOLOv2 real-time instance segmentation model to 0.5 times the original number of channels, so as to establish a lightweight deep learning model for nanoparticle detection and segmentation.

[0052] 3) Divide each AFM normalized height map into a training set and a validation set according to a preset ratio, and input the training set and the validation set into the deep learning model for nanoparticle detection and segmentation to be trained according to the annotation information to obtain a trained deep learning model for nanoparticle detection and segmentation; in actual operation, divide it into a training set, a validation set, and a test set according to the ratio of 7:2:1, and use the test set to evaluate the performance of the trained deep learning model for nanoparticle detection and segmentation.

[0053] In step 3), input the training set and the validation set into the deep learning model for nanoparticle detection and segmentation for training, as follows:

[0054] Step 3.1) For each AFM normalized height map in the training set, perform data augmentation on the AFM normalized height map and the annotation information on its corresponding AFM visualization height map. The data augmentation includes using one or several processing methods such as random horizontal flipping, random vertical flipping, random diagonal flipping, and random size scaling for processing. Each time after processing, a data-augmented image is obtained.

[0055] In step 3.1), the specific data augmentation methods include random flipping and random size scaling. The probability of random flipping is 0.5, and the flipping direction is randomly selected from vertical, horizontal, and diagonal; random size scaling is randomly selecting a size from the optional sizes for scaling, and the optional sizes include 128×128 and 256×256.

[0056] Step 3.2) Input the training set and each data augmentation image into the deep learning model for nanoparticle detection and segmentation for training. Each time training is performed, the updated model parameters of the deep learning model for nanoparticle detection and segmentation are obtained as the model parameters for the next training.

[0057] Step 3.3) Repeat Step 3.1)-3.2). After each repeated training, the currently updated deep learning model for nanoparticle detection and segmentation is obtained. Input the validation set into the currently updated deep learning model for nanoparticle detection and segmentation to obtain the average precision (AP) value of the deep learning model for nanoparticle detection and segmentation. Repeat this until the AP value obtained after several trainings approaches a fixed value, completing the training and obtaining the trained deep learning model for nanoparticle detection and segmentation.

[0058] Use the test set to evaluate the performance of the trained deep learning model. Judge whether the model performance meets the requirements according to preset experience. If it does not meet the requirements, repeat Step 1) to increase the data volume of the AFM image database and repeat Step 3.1)-3.3) until the performance meets the requirements to obtain the final deep learning model for nanoparticle detection and segmentation.

[0059] 4) Collect the original AFM detection image of the nanoparticle to be detected and perform image processing to obtain the AFM detection height map, the AFM detection visualization height map, and the AFM detection normalized height map. Input the AFM detection normalized height map into the trained deep learning model for nanoparticle detection and segmentation. The trained deep learning model for nanoparticle detection and segmentation outputs the region information, aggregation information, and confidence of each nanoparticle in the original AFM detection image, and then displays them in the AFM detection visualization height map.

[0060] In Step 4), according to the confidence of each nanoparticle in the original AFM detection image, display the region information and aggregation information of the nanoparticles with a confidence greater than the preset confidence threshold in the AFM detection visualization height map.

[0061] 5) Calculate the pixel-size factor based on the original AFM detection image and the AFM detection height map, and then calculate the size information of each nanoparticle, so as to obtain the size distribution information and aggregation ratio of all nanoparticles in the original AFM detection image, and finally realize the automatic detection of nanoparticles.

[0062] In Step 5), calculate the pixel-size factor based on the original AFM detection image and the AFM detection height map, and then calculate the size information of each nanoparticle, specifically as follows:

[0063] Step 5.1) Extract the AFM scan size from the original AFM detection image, extract the AFM height image pixel size from the AFM detection height map, and divide the AFM scan size by the AFM height map size to obtain the pixel-size factor.

[0064] Step 5.2) Calculate the mask map for each nanoparticle in the original AFM detection image obtained in step 4). The value of the mask map inside the region of each nanoparticle is true, and the value of the mask map outside the region of each nanoparticle is false; perform an AND operation on the AFM detection height map and the mask of each nanoparticle to obtain the height map of each nanoparticle. The valid pixels in the height map of each nanoparticle are the pixels where the mask map of each nanoparticle is true.

[0065] Step 5.3) Calculate the number of valid pixels in the height map of each nanoparticle in the original AFM detection image, and calculate the product of the number of valid pixels and the pixel-size factor to obtain the area of each nanoparticle.

[0066] Step 5.4) Extract the values of the valid pixels in the height map of each nanoparticle in the original AFM detection image. The maximum value of the values of all valid pixels is the height of each nanoparticle.

[0067] Step 5.5) Calculate the product of the sum of the values of all valid pixels and the pixel-size factor to obtain the volume of each nanoparticle; the area, height, and volume of each nanoparticle are the size information of each nanoparticle.

[0068] In step 5), according to the size information of each nanoparticle, the size distribution information and aggregation ratio of all nanoparticles in the original AFM detection image are obtained as follows:

[0069] Based on the aggregation information of each nanoparticle in the original AFM detection image obtained in step 4), obtain the number of non-aggregated nanoparticles and the number of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and area of each nanoparticle in the original AFM detection image, obtain the average area and area distribution of non-aggregated nanoparticles and the area distribution of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and height of each nanoparticle in the original AFM detection image, obtain the average height and height distribution of non-aggregated nanoparticles and the height distribution of aggregated nanoparticles in the original AFM detection image; based on the aggregation information and volume of each nanoparticle in the original AFM detection image, obtain the volume distribution of non-aggregated nanoparticles and the volume distribution of aggregated nanoparticles in the original AFM detection image; the number, average area, area distribution, average height, height distribution, and volume distribution of non-aggregated nanoparticles in the original AFM detection image, as well as the number, area distribution, height distribution, and volume distribution of aggregated nanoparticles, constitute the size distribution information of all nanoparticles in the original AFM detection image.

[0070] Calculate the quotient of the number of aggregated nanoparticles in the original AFM detection image and the number of all nanoparticles to obtain the aggregation ratio.

[0071] To show the beneficial effects of the present invention, by performing steps 4)-5) of the present invention, particle analysis is performed on the AFM images for the examples, and the obtained AFM visualization height map is as Figure 2 shown. Through step 4), the region, aggregation, and confidence information of each nanoparticle are obtained. The result of displaying the region of each nanoparticle on the AFM visualization height map is as Figure 3 shown. The confidence threshold used in this example is 0.5; further, through step 5), the number, area distribution, height distribution, volume distribution, and aggregation ratio information of the nanoparticles in this example are obtained.

[0072] Furthermore, the method of the present invention is compared with two methods, manual measurement and the particle analysis function of Nanoscope Analysis software. The detection result of the region of nanoparticles by the particle analysis function of Nanoscope Analysis software is as Figure 4 shown, and the comparison results are shown in Table 1.

[0073] Table 1

[0074]

[0075] As Figure 4As shown, the particle analysis function of the Nanoscope Analysis software for comparison regards the mutually contacting nanoparticles as one nanoparticle, resulting in a smaller number of nanoparticles than that of manual measurement and the method of the present invention. The result of the method of the present invention is similar to that of manual measurement. Therefore, the technical effect of the present invention is outstanding and significant.

[0076] The above specific embodiments are used to explain and illustrate the present invention, rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. An automatic detection method for nanoparticles based on deep learning and AFM images, characterized in that: The method includes the following steps: 1) Collect the atomic force microscope (AFM) original images of a number of nanoparticles, perform image processing and annotation on them, and obtain the AFM normalized height map and annotation information of each AFM original image; 2) Establish a deep learning model for nanoparticle detection and segmentation; 3) Divide each AFM normalized height map into a training set and a validation set according to a preset ratio, input the training set and the validation set into the deep learning model for nanoparticle detection and segmentation, and train according to the annotation information to obtain a trained deep learning model for nanoparticle detection and segmentation; 4) Collect the AFM detection original image of the nanoparticles to be detected, perform image processing on it, and obtain the AFM detection height map, the AFM detection visualization height map, and the AFM detection normalized height map; input the AFM detection normalized height map into the trained deep learning model for nanoparticle detection and segmentation, and the trained deep learning model for nanoparticle detection and segmentation outputs the region information, aggregation information, and confidence of each nanoparticle in the AFM detection original image, and then displays them in the AFM detection visualization height map; 5) Calculate the pixel-size factor based on the AFM detection original image and the AFM detection height map, and then calculate the size information of each nanoparticle, so as to obtain the size distribution information and aggregation ratio of all nanoparticles in the AFM detection original image, and finally realize the automatic detection of nanoparticles.

2. The automatic detection method for nanoparticles based on deep learning and AFM images according to claim 1, characterized in that: In the step 1), the collected AFM original images of a number of nanoparticles include AFM original images of nanoparticles with different morphologies, sizes, concentrations, aggregation degrees, and substrates. The morphologies specifically include spherical and flaky, and the substrate is specifically the substrate for fixing nanoparticles, including silicon wafers, mica sheets, and glass slides.

3. The automatic detection method for nanoparticles based on deep learning and AFM images according to claim 1, characterized in that: In the step 1), collect the AFM original images of a number of nanoparticles and perform image processing. For each AFM original image, specifically as follows: Extract the AFM height sensor data from the AFM original image, perform smoothing processing on the AFM height sensor data, or perform smoothing processing and plane fitting in sequence to obtain the AFM height map; map the height values in the AFM height map to color pixels pixel by pixel to obtain the AFM visualization height map; mark the annotation information of each nanoparticle on the AFM visualization height map, and the annotation information includes region information and aggregation information. The region information includes position and contour, and the aggregation information is aggregation and non-aggregation; perform maximum-minimum normalization on the AFM height map to obtain the AFM normalized height map.

4. The automatic detection method of nanoparticles based on deep learning and AFM images according to claim 1, characterized in that: In step 2), the deep learning model for nanoparticle detection and segmentation is established as follows: Based on the SOLOv2 real-time instance segmentation model, the backbone network in the SOLOv2 real-time instance segmentation model uses a ResNet model with a depth of 10, and the number of channels of the backbone network, neck network, and head network in the SOLOv2 real-time instance segmentation model is changed to 0.5 times the original number of channels, thereby establishing a lightweight deep learning model for nanoparticle detection and segmentation.

5. The automatic detection method of nanoparticles based on deep learning and AFM images according to claim 3, characterized in that: In step 3), the training set and the validation set are input into the deep learning model for nanoparticle detection and segmentation for training, specifically as follows: Step 3.1) For each AFM normalized height map in the training set, the annotation information on the AFM normalized height map and its corresponding AFM visualized height map is subjected to data augmentation processing. The data augmentation processing includes using one or several of the processing methods of random horizontal flipping, random vertical flipping, random diagonal flipping, and random size scaling for processing, and a data-augmented map is obtained after each processing; Step 3.2) The training set and each data-augmented map are input into the deep learning model for nanoparticle detection and segmentation for training. The updated model parameters of the deep learning model for nanoparticle detection and segmentation are obtained each time of training and used as the model parameters for the next training; Step 3.3) Repeat steps 3.1)-3.2). After each repeated training, the currently updated deep learning model for nanoparticle detection and segmentation is obtained. The validation set is input into the currently updated deep learning model for nanoparticle detection and segmentation to obtain the average precision (AP) value of the deep learning model for nanoparticle detection and segmentation until the average precision (AP) value obtained after several trainings approaches a fixed value, and the training is completed to obtain the trained deep learning model for nanoparticle detection and segmentation.

6. The automatic detection method of nanoparticles based on deep learning and AFM images according to claim 1, characterized in that: In step 4), according to the confidence of each nanoparticle in the original AFM detection image, the region information and aggregation information of the nanoparticles with a confidence greater than the preset confidence threshold are displayed in the AFM detection visualized height map.

7. The automatic detection method of nanoparticles based on deep learning and AFM images according to claim 1, characterized in that: In step 5), the pixel-size factor is calculated based on the original AFM detection image and the AFM detection height map, and then the size information of each nanoparticle is calculated, specifically as follows: Step 5.1) Extract the AFM scan size from the original AFM detection image, extract the AFM height image pixel size from the AFM detection height map, and divide the AFM scan size by the AFM height map size to obtain the pixel-size factor; Step 5.2) Calculate the mask map of each nanoparticle according to the area information of each nanoparticle in the original AFM detection image obtained in step 4). The value of the mask map inside the area of each nanoparticle is true, and the value of the mask map outside the area of each nanoparticle is false; perform an AND operation on the AFM detection height map and the mask of each nanoparticle to obtain the height map of each nanoparticle. The valid pixels in the height map of each nanoparticle are the pixels where the mask map of each nanoparticle is true. Step 5.3) Calculate the number of valid pixels in the height map of each nanoparticle in the original AFM detection image, and calculate the product of the number of valid pixels and the pixel-size factor to obtain the area of each nanoparticle. Step 5.4) Extract the values of the valid pixels in the height map of each nanoparticle in the original AFM detection image. The maximum value of the values of all valid pixels is the height of each nanoparticle. Step 5.5) Calculate the product of the sum of the values of all valid pixels and the pixel-size factor to obtain the volume of each nanoparticle; the area, height, and volume of each nanoparticle are the size information of each nanoparticle.

8. A method for automatic detection of nanoparticles based on deep learning and AFM images according to claim 7, characterized in that: in the said step 5), according to the size information of each nanoparticle, the size distribution information and aggregation ratio of all nanoparticles in the original AFM detection image are obtained, specifically as follows: According to the aggregation information of each nanoparticle in the original AFM detection image obtained in step 4), obtain the number of non-aggregated nanoparticles and the number of aggregated nanoparticles in the original AFM detection image; According to the aggregation information and area of each nanoparticle in the original AFM detection image, obtain the average area and area distribution of non-aggregated nanoparticles and the area distribution of aggregated nanoparticles in the original AFM detection image; according to the aggregation information and height of each nanoparticle in the original AFM detection image, obtain the average height and height distribution of non-aggregated nanoparticles and the height distribution of aggregated nanoparticles in the original AFM detection image; according to the aggregation information and volume of each nanoparticle in the original AFM detection image, obtain the volume distribution of non-aggregated nanoparticles and the volume distribution of aggregated nanoparticles in the original AFM detection image; the number, average area, area distribution, average height, height distribution, and volume distribution of non-aggregated nanoparticles and the number, area distribution, height distribution, and volume distribution of aggregated nanoparticles in the original AFM detection image constitute the size distribution information of all nanoparticles in the original AFM detection image; Calculate the quotient of the number of aggregated nanoparticles in the original AFM detection image and the number of all nanoparticles to obtain the aggregation ratio.

Images