Nanoparticle topological structure segmentation method based on SwinU-Net* model

Abstract

The invention provides a nanoparticle topological structure segmentation method based on a SwinU-Net * model, and relates to data processing in the field of materials. The method comprises the following steps: acquiring a nanoparticle topological structure image set; marking a circular ring structure on each nanoparticle topological structure image, performing preprocessing, and then dividing the nanoparticle topological structure image set into a training set and a test set according to a preset proportion; the method comprises the following steps: by taking a SwinUnet model as a basic framework, adding a next-generation convolution block NCB and a space attention module SAM on the SwinUnet model, and constructing a SwinU-Net * model; inputting the training set into a SwinU-Net * model to carry out model training; inputting the test set into the trained SwinU-Net * model, and evaluating the segmentation performance of the SwinU-Net * model; and inputting a to-be-segmented nanoparticle topological structure image into the trained SwinU-Net * model, and segmenting a circular ring structure in the image. According to the method, in the SwinU-Net model, a next generation convolution block (NCB) and a space attention module (SAM) are introduced for feature extraction, so that the segmentation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing in the field of materials, and specifically to a nanoparticle topological structure segmentation method based on the SwinU-Net* model. Background Art

[0002] Nanomaterials refer to materials with three-dimensional spatial dimensions in at least one dimension at the nanoscale (1 to 100 nanometers). They are a new generation of materials composed of nanoparticles ranging from atoms, molecules to macroscopic objects. Accurately measuring the particle size and particle morphology distribution of these materials plays a key role in different fields such as medicine, electronics, semiconductors, optics and biology. In the medical field, in 2024, Yijie Zho et al. found that neutrophil extracellular network (NET) plays a key role in thrombosis, and developed Prussian blue nanoparticles (PB NPs) with surface modification of neutrophil elastase (NE) binding peptides, which specifically target activated neutrophils, remove abnormal reactive oxygen species (ROS), and destroy the fibrin network through photothermal therapy (PTT). This non-invasive treatment is safe for mothers and fetuses and has a significant effect on alleviating intrauterine growth restriction. In the semiconductor field, in 2021, Marwa M. Tharwat et al. studied a new GaAs thin-film solar cell, which introduced a randomly distributed aluminum plasma nanoparticle array on the top surface to improve the ability to collect sunlight; in the biology field, in 2022, Da Zou et al. explored the application of cell membrane-wrapped nanoparticles as new nanomaterials in targeted cancer treatment. By comparing silicon nanoparticles wrapped in two mesenchymal stem cell membranes with different hardness, they found that softer nanoparticles were less taken up by macrophages but more easily absorbed by cancer cells.

[0003] With the rapid development of deep learning, many deep learning models have emerged for the segmentation of nanoparticles, which show strong generalization and robustness. For example, ZhenZhou Wang et al. proposed a novel method for the segmentation and quantification of cells and nanoparticles in microscopic imaging. The method is divided into five steps: automatic creation of gradient images, automatic threshold selection, manual threshold calibration of each type of image, manual segmentation setting, and automatic quantification of iterative morphological erosion. After the initial manual calibration, the subsequent processing is automated. The test results prove its effectiveness on different samples. Some reports also proposed an automatic particle picking device based on the concept of variance mixed mean local threshold to improve the automatic and accurate segmentation of nanoparticles in transmission electron microscopy (TEM) images. The new model was verified by using a program written in Matlab on a database of 150 bright field TEM images containing about 2000 nanoparticles. Compared with the global threshold method, the local threshold method and manual segmentation, the new model significantly reduced false positive and false negative results, and detected more single particles in the overlapping area of ​​particles. There are also reports on the development of a new imaging workflow to analyze the distribution of nanoparticles in micrometastases at single-cell resolution. This technology combines tissue transparency, three-dimensional microscopic imaging, and machine learning analysis, and found that nanoparticles have a higher access rate in micrometastases, which helps target these areas to prevent tumor growth. By analyzing more than 1,300 micrometastases, this technology can also predict the distribution of nanoparticles in different micrometastases, providing the possibility for personalized treatment.

[0004] At present, most of the research in the nano field is aimed at nanoparticles themselves, while ignoring the importance of nanoparticle structure. Nanoparticle structure refers to a more complex structure formed by specific nanoparticles organized in a certain form. Due to its unique physical, chemical and biological properties, nanoparticle structures can be created by various methods such as self-assembly, photolithography, electron beam etching, chemical synthesis, etc., and the specific application depends on their composition, size, morphology, and surface functionalization. Although a few reports have mentioned the research on nanoparticle image segmentation, most of the research on nanoparticle image segmentation is based on convolution. However, the convolution operation is local in nature, and each convolution can only obtain part of the information of the input image. Some studies have expanded the receptive field by increasing the number of convolutional layers, but this will significantly increase the computational complexity. Although the pooling operation can reduce the amount of calculation and the number of parameters, it will lead to the loss of detailed information. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a nanoparticle topological structure segmentation method based on the SwinU-Net* model. The present invention introduces an innovative next-generation convolutional block (NCB) and spatial attention module (SAM), proposes a hybrid strategy that combines the advantages of CNN and Transformer, and adopts a synthetic data method to make up for the lack of data in the field of nanomaterials when constructing a data set. The performance of the SwinU-Net* model proposed in the present invention has been significantly improved, with a Dice coefficient of 0.8290, an average intersection over union (MIoU) of 0.7131, and a Kappa coefficient and accuracy (Acc) of 0.9949. Compared with the original SwinU-Net model, the Dice coefficient is increased by 5.67%. This result confirms that the improved strategy of the present invention greatly enhances the model's ability to capture nanoscale spatial relationships and significantly improves accuracy and efficiency when processing complex visual tasks.

[0006] Specifically, the present invention provides a nanoparticle topological structure segmentation method based on the SwinU-Net* model, comprising:

[0007] Acquire a nanoparticle topological structure image set, wherein the nanoparticle topological structure image comprises a synthetic image set and a real image set;

[0008] After marking each of the nanoparticle topological structure images with a ring structure and preprocessing, the nanoparticle topological structure image set is divided into a training set and a test set according to a preset ratio;

[0009] The SwinUnet model is used as the basic architecture, and the next generation convolutional block NCB and spatial attention module SAM are added to the SwinUnet model to build the SwinU-Net* model;

[0010] Input the training set into the SwinU-Net* model to perform model training;

[0011] Inputting the test set into the trained SwinU-Net* model to evaluate the segmentation performance of the SwinU-Net* model;

[0012] The nanoparticle topological structure image to be segmented is input into the trained SwinU-Net* model to segment the circular ring structure in the image.

[0013] As a further illustration of the present invention, a model design software is used to create a synthetic image of a nanoparticle topological structure that is close to the actual nanoparticle topological structure; and a scanning electron microscope imaging technique is used to obtain an actual image of the nanoparticle topological structure.

[0014] As a further illustration of the present invention, the preprocessing process of the nanoparticle topological structure image includes: converting the image data from PNG format to NPY format, and adjusting the image size.

[0015] As a further illustration of the present invention, the construction process of the SwinU-Net* model specifically includes:

[0016] The SwinUnet model is used as the basic architecture, and the NCB module is added before it divides the small windows;

[0017] In the SwinUnet model, the SAM module is added before the window-based multi-head attention module W-MSA and the moving window-based multi-head attention module SW-MSA of the Swin transformer block to form the Swin transformerblock*.

[0018] As a further illustration of the present invention, each of the Swin transformer block* is composed of an LN layer, a SAM module, a multi-head attention module, a residual connection and a two-layer MLP with GELU nonlinearity, wherein two consecutive Swin transformer blocks* respectively apply a window-based multi-head attention module W-MSA and a moving window-based multi-head attention module SW-MSA.

[0019] As a further illustration of the present invention, the expression of the Swin transformer block* is: (1) (2) (3) (4)

[0020] in, represents the output of the Lth W-MSA module, Represents the output of the Lth MLP module;

[0021] The calculation formula of the multi-head attention module is:

[0022] Attention(Q, K, V)=SoftMax( )V

[0023] Among them, Q, K, V , represent query, key and value matrices respectively, represents the number of patches in the window, d represents the dimension of the query or key, and the value of B is taken from the bias matrix .

[0024] As a further illustration of the present invention, the NCB module expression is: (6) (7)

[0025] in, is the input of the l-1 block, and are the outputs of the lth block MHCA and BCN respectively;

[0026] The definition of the multi-head convolutional attention module MHCA is as follows: (8)

[0027] Among them, CA is a single convolutional attention, z=[ , , ..., ] means that the input feature z is divided into h parallel subspaces in the channel dimension to capture features. In order to make the information captured by multiple parallel subspaces interact, a projection layer is equipped ;

[0028] The expression of CA is: (9)

[0029] in, and is the neighboring tokens in the input feature z, O is a vector with trainable parameters W and input tokens Through the inner product operation of , CA can learn the affinity between different tags in the local receptive field by iteratively optimizing the trainable parameters W.

[0030] As a further illustration of the present invention, the specific implementation process of the SwinUnet* model is as follows:

[0031] The input image first enters the NCB module for feature extraction, and then the encoder part starts by splitting the initial data processed by the NCB module into non-overlapping blocks of 4*4 blocks. The feature dimension of each block is expanded to 4*4*3. These blocks are then linearly converted into sequence embeddings, and the linear embedding layer projects these feature dimensions to higher or required dimensions;

[0032] The dimensionally projected non-overlapping blocks are processed through several STB* and patch merging layers to produce a stacked feature representation, where STB* is responsible for deep feature representation learning and the patch merging layer increases the feature dimension while downsampling to provide a rich source of information for subsequent processing;

[0033] The decoder part is a symmetrical structure of the encoder, consisting of several STBs* and patch expansion layers, in which skip connections are used to effectively fuse the context features extracted in the decoder with the features extracted by the encoder at multiple scales; the patch expansion layer plays the role of upsampling, reshaping the adjacent dimensions of the feature map into a feature map with twice the resolution to gradually restore the image scale;

[0034] At the end of the model, the feature map of the last layer is upsampled by a factor of 4 to restore the resolution of the feature map to the same size as the input image, and then the final pixel-level segmentation prediction is generated through a linear projection layer;

[0035] A bottleneck is introduced into the model, which is composed of two consecutive STB*. In this area, the resolution of the feature is kept unchanged.

[0036] As a further illustration of the present invention, during the model training process, the training iteration cycle is 50,000, the initial learning rate is set to 0.05, SGD is used as the optimizer, DiceLoss is used as the loss function, the batch_size is 8, and the initialization method is random initialization.

[0037] Compared with the prior art, the present invention has the following beneficial technical effects:

[0038] (1) The present invention constructs a data set by using a small amount of real data and a large amount of artificially synthesized data to achieve sufficient sample size and diversity required for the experiment of segmenting nanoparticle structure.

[0039] (2) In the SwinU-Net model, the present invention introduces an innovative next-generation convolutional block (NCB) and spatial attention module (SAM) for feature extraction, thereby improving the accuracy of segmentation.

[0040] (3) This paper develops an innovative CNN and Transformer hybrid strategy, aiming to improve both the performance and efficiency of the system.

[0041] Other features and advantages of the technical solution will be described in the subsequent description, and partly become apparent from the description, or understood by implementing the technical solution. The purpose and other advantages of the technical solution can be achieved and obtained through the structures specifically pointed out in the written description and the drawings.

[0042] The technical solution of the present technical solution is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the technical solution and constitute a part of the specification. Together with the embodiments of the technical solution, they are used to explain the technical solution and do not constitute a limitation of the technical solution. In the accompanying drawings:

[0044] Figure 1 1 is an overall structural diagram of the SwinUnet* model constructed in an embodiment of the present invention.

[0045] Figure 2 2 is a structural diagram of STB and STB* in an embodiment of the present invention.

[0046] Figure 3 2 is a structural diagram of a next generation convolutional block (NCB) in an embodiment of the present invention.

[0047] Figure 4 2 is a structural diagram of a spatial attention module (SAM) in an embodiment of the present invention.

[0048] Figure 5 are examples of original images of real and synthetic data and corresponding annotated images in an embodiment of the present invention, wherein Figure 5 (a) is the original SEM image. Figure 5 (b) is the corresponding annotated image.

[0049] Figure 6 Result of comparison between the original image, label, and predicted image in the embodiment of the present invention. DETAILED DESCRIPTION

[0050] The preferred embodiments of the present technical solution are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present technical solution, and are not used to limit the present technical solution.

[0051] In the field of functional catalytic nanomaterials, nanoparticles on the surface of catalysts show non-random, spatially ordered arrangements to form specific geometric patterns. In order to accurately identify and segment those nanoparticle structures that are regular in shape but have variable arrangements, this paper proposes a nanoparticle topological structure segmentation method based on the SwinU-Net* model, introduces an innovative next-generation convolutional block (NCB) and spatial attention module (SAM), and proposes a hybrid strategy that combines the advantages of CNN and Transformer. When constructing the dataset, the method of synthetic data is used to make up for the lack of data in the field of nanomaterials. The performance of the SwinU-Net* model proposed in this paper has been significantly improved, with a Dice coefficient of 0.8290, an average intersection over union (MIoU) of 0.7131, and a Kappa coefficient and accuracy (Acc) of 0.9949. Compared with the original SwinU-Net model, the Dice coefficient is increased by 5.67%. This result confirms that the improved strategy of the present invention greatly enhances the model's ability to capture nanoscale spatial relationships and significantly improves accuracy and efficiency when processing complex visual tasks.

[0052] Specifically, an embodiment of the present invention provides a nanoparticle topological structure segmentation method based on the SwinU-Net* model, comprising the following steps:

[0053] Step 100: Acquire a nanoparticle topological structure image set, wherein the nanoparticle topological structure image includes a synthetic image set and a real image set.

[0054] Specifically, a model design software is used to create a synthetic image of a nanoparticle topological structure that is close to the actual nanoparticle topological structure; that is, a 3D model design software K-3D is used as a rendering program to create a synthetic image of a nanoparticle topological structure that is close to the actual nanoparticle topological structure.

[0055] Scanning electron microscopy imaging technology is used to obtain realistic images of the nanoparticle topology.

[0056] In the present invention, the data set used is composed of the above-mentioned actual collected data and the generated synthetic data. For the actual collected part, the data comes from the scanning electron microscopy (SEM) imaging technology, showing the image of palladium (Pd) nanoparticles deposited on the surface of the carbon substrate. The preparation process of these nanoparticles includes mixing Pd2dba3·CHCl3 with graphite powder and chloroform (CHCl3), and stirring at 50 degrees Celsius for 1 hour to complete the reaction. Subsequently, the carbon support is separated from the reaction solution by filter paper and dried, and finally the dried carbon support is washed with acetone to eliminate the diphenyl acrylonitrile (dba) residue.

[0057] Step 200: After marking each nanoparticle topological structure image with a circular ring structure and performing preprocessing, the nanoparticle topological structure image set is divided into a training set and a test set according to a preset ratio.

[0058] Specifically, the preprocessing process of the nanoparticle topological structure image includes: converting the image data from PNG format to NPY format, and adjusting the image size.

[0059] In the subsequent model training experiments, the optimal ratio dataset containing 15% real data and 85% synthetic data was used to train and verify the deep learning model. During the experiment, the image data was converted from PNG format to NPY format to meet the processing requirements, and the input image resolution was 321x321 pixels. Figure 5 Examples of original images of real and synthetic data and their corresponding annotated images are shown in: Figure 5 (a) shows the original SEM image, while Figure 5 (b) shows the corresponding annotated image, which is used to guide the model to accurately detect and segment nanoparticles.

[0060] Step 300: Use the SwinUnet model as the basic architecture, and add the next generation convolution block NCB and the spatial attention module SAM to the SwinUnet model to build the SwinU-Net* model.

[0061] like Figure 1 As shown in the figure, the construction process of the SwinU-Net* model includes:

[0062] The SwinUnet model is used as the basic architecture, and the NCB module is added before it is divided into small windows.

[0063] The present invention adopts the SwinUnet model as the basic architecture, which inherits the purity of the Transformer model and is built with its unique encoder, decoder, bottleneck structure and skip link mechanism. Then NCB is added before dividing the small window, which captures the short-term dependency information in the visual data through a new deployment-friendly multi-head convolutional attention (MHCA), and uses it to perform shallow feature extraction on the input feature map.

[0064] Figure 3 is a structural diagram of the next generation convolution block (NCB) in an embodiment of the present invention, such as Figure 3As shown in the figure, the NCB block combines the design paradigm of MetaFormer with the structural advantages of the multi-layer perceptron (MLP) layer, and introduces an efficient token mixer called Multi-Head Convolutional Attention (MHCA). The design of the NCB block allows for performance comparable to that of the Transformer block through the MHCA component while retaining the hardware deployment convenience of the BottleNeck block. This design effectively balances the computational resource constraints and processing speed requirements, making NCB a network building block suitable for modern deep learning vision tasks and capable of performing well in a variety of deployment environments.

[0065] The NCB module expression is: (6) (7)

[0066] in, is the input of the l-1 block, and are the outputs of the lth block MHCA and BCN respectively;

[0067] MHCA acts as a key component. While taking into account the friendliness of network deployment, it provides functions similar to the continuous attention mechanism, and performs efficient interaction and information exchange between features through convolution operations. This attention mechanism adopts a multi-head strategy to parallelize the processing process, capture information in different subspaces, and implement it through convolution operations to ensure operational efficiency and processing speed. Convolutional attention (CA) is created based on the multi-head paradigm, which can jointly focus on feature information in different subspaces at different locations for effective local learning. MHCA is implemented through multiple convolutional attentions (CA) and point convolutions.

[0068] The definition of the multi-head convolutional attention module MHCA is as follows: (8)

[0069] Among them, CA is a single convolutional attention, z=[ , , ..., ] means that the input feature z is divided into h parallel subspaces in the channel dimension to capture features. In order to make the information captured by multiple parallel subspaces interact, a projection layer is equipped .

[0070] The expression of CA is: (9)

[0071] in, and is the neighboring tokens in the input feature z, O is a vector with trainable parameters W and input tokens Through the inner product operation of , CA can learn the affinity between different tags in the local receptive field by iteratively optimizing the trainable parameters W.

[0072] Then, in the SwinUnet model, a SAM module is added before the window-based multi-head attention module W-MSA and the moving window-based multi-head attention module SW-MSA of the Swin transformer block (STB) to form a Swin transformer block*.

[0073] SAM is an attention mechanism that imitates human visual attention. It can dynamically focus on the feature part of the input feature map, significantly improve the model's sensitivity to specific areas, and make the model more focused on the topological structure of nanoparticles. The modified Swin Transformer block is called Swin transformer block* (STB*), and its structure is as follows: Figure 2 As shown in the figure, each Swin transformer block* consists of an LN layer, a SAM module, a multi-head attention module, a residual connection, and a two-layer MLP with GELU nonlinearity, where two consecutive Swin transformer blocks* apply a window-based multi-head attention module W-MSA and a moving window-based multi-head attention module SW-MSA, respectively.

[0074] The expression for the Swin transformer block* is: (1) (2) (3) (4)

[0075] in, represents the output of the Lth W-MSA module, Represents the output of the Lth MLP module.

[0076] The calculation formula of the multi-head attention module is:

[0077] Attention(Q, K, V)=SoftMax( )V

[0078] Among them, Q, K, V , represent query, key and value matrices respectively, represents the number of patches in the window, d represents the dimension of the query or key, and the value of B is taken from the bias matrix .

[0079] SAM is a mechanism that focuses on strengthening the convolutional neural network's ability to process spatial information in feature maps. Its core goal is to determine the specific location of effective information in the feature map, prompting the network to pay more attention to more critical and relevant areas. The working principle of SAM is to give different weights to each spatial position on the input feature map by introducing a set of learnable parameters. This mechanism allows the network to dynamically adjust its attention during the learning process, autonomously identifying and focusing on areas containing key visual cues. In this way, SAM is able to capture the spatial structural details of the input data more accurately, thereby improving the recognition and analysis capabilities of the overall model. This spatial attention-based method shows significant advantages in feature selection and feature representation optimization, especially in complex scene parsing and refined segmentation tasks. The structural diagram is shown in Figure 4 shown.

[0080] In summary, the specific implementation process of the above SwinUnet* model is as follows:

[0081] The input image first enters the NCB module for feature extraction, and then the encoder part starts by dividing the initial data processed by the NCB module into non-overlapping blocks of 4*4 size. The feature dimension of each block is expanded to 48 (4*4*3). These blocks are then linearly converted into sequence embeddings, and the linear embedding layer projects these feature dimensions to a higher or required dimension (denoted as C); the non-overlapping blocks after dimension projection are processed by several STB* and patch merging layers to produce a stacked feature representation, among which STB* is responsible for deep feature representation learning, and the patch merging layer increases the feature dimension while downsampling. It provides a rich source of information for subsequent processing; the decoder part is a symmetrical structure of the encoder, consisting of several STB* and patch expansion layers, in which skip links are used to effectively fuse the contextual features extracted from the decoder with the features extracted from the encoder at multiple scales; the patch expansion layer plays the role of upsampling, reshaping the adjacent dimensions of the feature map into a feature map with twice the resolution to gradually restore the image scale; at the end of the model, a 4-fold upsampling operation is performed on the last layer of feature maps to restore the resolution of the feature map to the same size as the input image, and then the final pixel-level segmentation prediction is generated through the linear projection layer. The bottleneck part is introduced in the model to solve the problem of difficulty in convergence due to the deep Transformer structure. The bottleneck consists of two consecutive STB*, in which the resolution of the features is kept unchanged to ensure that the depth and complexity of the model can be effectively managed without losing the stability and accuracy of the model.

[0082] Table 1 Detailed configuration of the above SwinUnet* model

[0083] In the table, heads is the number of heads in different layers, k is the size of the convolution kernel, s is the step size, and c is the number of channels.

[0084] Step 400: input the training set into the SwinU-Net* model for model training;

[0085] During the model training process, the training iteration cycle is 50,000, the initial learning rate is set to 0.05, SGD is used as the optimizer, DiceLoss is used as the loss function, batch_size is 8, and the initialization method is random initialization. This experiment selected Baidu PaddlePaddle as the deep learning platform. The CPU type is 4 Cores, and the GPU type is Tesla V100\32GB.

[0086] Step 500: Input the test set into the trained SwinU-Net* model to evaluate the segmentation performance of the SwinU-Net* model;

[0087] When processing images of nanoparticle ring structures, the goal is to accurately identify and segment each circular ring structure in the image through instance segmentation technology. After completing this segmentation, the present invention will collect and evaluate the performance of the segmented nanoparticle ring structures, aiming to explore the interaction between their microstructural characteristics and macroscopic material properties. However, various types of noise introduced during the sample preparation process may pose a great challenge to the accuracy of identifying nanoparticle ring structures, thereby affecting subsequent performance analysis. In order to effectively evaluate the performance of the algorithm, the present invention uses several indicators, including (1) Dice similarity index, (2) Kappa coefficient (Kappa), (3) Accuracy (Acc) and (4) Mean intersection over union (MIoU).

[0088] (1) Dice similarity index is a set similarity measurement function used to calculate the similarity between two samples. It is often used to evaluate the quality of segmentation algorithms. The best segmentation result is 1 and the worst result is 0. The following is its formula: (13)

[0089] Where pred is the set of predicted values, true is the set of true values, and the numerator is the intersection of pred and true. The result is multiplied by 2 because the denominator repeatedly counts the common elements between pred and true. The denominator is the union of pred and true.

[0090] (2) Kappa coefficient is an indicator used for consistency testing and can also be used to measure the effectiveness of classification. The so-called consistency refers to whether the model prediction results are consistent with the actual classification results. The value range of Kappa coefficient is -1 to 1, where 1 indicates complete consistency, 0 indicates consistency with the random model, and -1 indicates complete inconsistency. Generally, for most tasks, Kappa coefficients between 0.4 and 0.6 are considered to be moderately consistent, and above 0.6 are considered to be good consistency. The following is its formula: (11)

[0091] in, is the proportion of observed exact agreement, that is, the proportion of samples for which the model predictions are consistent with the actual observations. is the proportion of random consistency, that is, the expected proportion of consistency between the model's predictions and actual observations if they were completely random. In a two-classification problem, It can be obtained by calculating the sum of the products of the marginal probabilities of the two categories.

[0092] (3) Acc is an indicator used to evaluate the performance of a classification model, that is, the proportion of the model's correct predictions to the total number. The higher the accuracy, the better the classifier. The following is its formula: (12)

[0093] Among them, pred is the number of samples correctly classified by the model. All is the total number of sample predictions made by the model.

[0094] (4) Miou is a standard metric for semantic segmentation, which calculates the average ratio of the intersection and union of all categories. Here is its formula: (10)

[0095] Among them, i represents the true value, j represents the predicted value, It represents the number of pixels that predict i to be j. That is, the numerator is the intersection of the true label and the predicted result, and the denominator is their union.

[0096] Table 2 shows the experimental results of SwinUnet*, SwinUnet and adding SAM and NCB respectively. Among them, the Dice of the SwinUnet* model reaches 0.8290, which is 0.0567 higher than that of SwinUnet, 0.0431 higher than that of SwinUnet with SAM added, and 0.0260 higher than that of SwinUnet with NCB added.

[0097] Table 2 Results of various evaluation indicators of adding NCB and adding spatial attention mechanism to swinUnet

[0098] Table 3 shows the comparison results of more than ten models including SwinUnet* model. The comparison models include Setr, SegFormer, SegMenter, Rtformer, Vit_adapter, Topformer, Efficientformerv2, Hrformer and Lpsnet. Among them, only Lpsnet is a convolutional model, and the others are Transformer models.

[0099] Table 3 Segmentation evaluation results of various comparison models

[0100] Figure 6 The prediction results of the model designed by the present invention are shown. In this figure, the prediction effects of various image backgrounds and structures can be observed, as follows:

[0101] Figure 6Parts (a) and (b) in the figure show the prediction effect of images with relatively simple backgrounds. Although the image background of part (b) is more complex and messy than that of part (a), the model provided by the present invention still shows excellent prediction ability, accurately segmenting and predicting specific structures in the image. This result reflects the robustness and prediction accuracy of the model for different background complexities. In parts (b) and (c), we further explored the prediction effect of circular ring structures that occupy prominent positions and relatively small positions in the image. Comparing the correspondence between the two and the corresponding labels, it can be found that the prediction results of the model of the present invention in these cases match the original image to a higher degree, showing its high sensitivity and accuracy in identifying the main and subtle features in the image. Particularly noteworthy is part (d), which shows a scene with high structural complexity, that is, a small ring is embedded in a larger ring. Although a scattered nanoparticle is presented in the prediction image, the model provided by the present invention successfully and accurately segments this embedded nanoparticle structure, once again proving the superior performance of the SwinUnet* model in dealing with highly complex structures. Although the processing performance of such complex scenes may be slightly insufficient compared to images showing single nanoparticle structures, the segmentation results still show considerable accuracy and reliability.

[0102] In summary, Figure 6 The prediction results highlight the high applicability and accuracy of the SwinUnet* model in dealing with backgrounds and structures of different complexity. These results not only demonstrate the model's sensitivity to different structural features in the image, but also verify its effectiveness and robustness in dealing with highly complex image segmentation tasks.

[0103] Step 600: Input the nanoparticle topological structure image to be segmented into the trained SwinU-Net* model to segment the circular ring structure in the image.

[0104] Obviously, those skilled in the art can make various changes and modifications to the technical solution without departing from the spirit and scope of the technical solution. Thus, if these modifications and variations of the technical solution fall within the scope of the claims of the technical solution and their equivalents, the technical solution is also intended to include these modifications and variations.

Claims

1. A nanoparticle topological structure segmentation method based on the SwinU-Net* model, characterized in that: include: Acquire a nanoparticle topological structure image set, wherein the nanoparticle topological structure image comprises a synthetic image set and a real image set; After marking each of the nanoparticle topological structure images with a ring structure and preprocessing, the nanoparticle topological structure image set is divided into a training set and a test set according to a preset ratio; The SwinUnet model is used as the basic architecture, and the next generation convolutional block NCB and spatial attention module SAM are added to the SwinUnet model to build the SwinU-Net* model; Input the training set into the SwinU-Net* model to perform model training; Inputting the test set into the trained SwinU-Net* model to evaluate the segmentation performance of the SwinU-Net* model; The nanoparticle topological structure image to be segmented is input into the trained SwinU-Net* model to segment the circular ring structure in the image.

2. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 1, characterized in that: Model design software is used to create a synthetic image of the nanoparticle topology that is close to the actual nanoparticle topology; scanning electron microscopy imaging technology is used to obtain a real image of the nanoparticle topology.

3. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 1, characterized in that: The preprocessing process of the nanoparticle topological structure image includes: converting the image data from PNG format to NPY format, and adjusting the image size.

4. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 1, characterized in that: The construction process of the SwinU-Net* model specifically includes: The SwinUnet model is used as the basic architecture, and the NCB module is added before it divides the small windows; In the SwinUnet model, a SAM module is added before the window-based multi-head attention module W-MSA and the moving window-based multi-head attention module SW-MSA of the Swin transformer block to form a Swin transformer block*.

5. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 4, characterized in that: Each of the Swin transformer block* consists of an LN layer, a SAM module, a multi-head attention module, a residual connection and a two-layer MLP with GELU nonlinearity, wherein two consecutive Swin transformer blocks* respectively apply a window-based multi-head attention module W-MSA and a moving window-based multi-head attention module SW-MSA.

6. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 5, characterized in that: The expression of the Swin transformer block* is: （1） （2） （3） （4） in, represents the output of the Lth W-MSA module, Represents the output of the Lth MLP module; The calculation formula of the multi-head attention module is: Attention(Q, K, V)=SoftMax( )V (5) Among them, Q, K, V , represent query, key and value matrices respectively, represents the number of patches in the window, d represents the dimension of the query or key, and the value of B is taken from the bias matrix .

7. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 5, characterized in that: The NCB module expression is: （6） （7） in, is the input of the l-1 block, and are the outputs of the lth block MHCA and BCN respectively; The definition of the multi-head convolutional attention module MHCA is as follows: （8） Among them, CA is a single convolutional attention, z=[ , , ..., ] means that the input feature z is divided into h parallel subspaces in the channel dimension to capture features. In order to make the information captured by multiple parallel subspaces interact, a projection layer is equipped ; The expression of CA is: （9） in, and is the neighboring tokens in the input feature z, O is a vector with trainable parameters W and input tokens Through the inner product operation of , CA can learn the affinity between different tags in the local receptive field by iteratively optimizing the trainable parameters W.

8. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 1, characterized in that: The specific implementation process of the SwinUnet* model is as follows: The input image first enters the NCB module for feature extraction, and then the encoder part starts by splitting the initial data processed by the NCB module into non-overlapping blocks of 4*4 blocks. The feature dimension of each block is expanded to 4*4*3. These blocks are then linearly converted into sequence embeddings, and the linear embedding layer projects these feature dimensions to higher or required dimensions; The dimensionally projected non-overlapping blocks are processed through several STB* and patch merging layers to produce a stacked feature representation, where STB* is responsible for deep feature representation learning and the patch merging layer increases the feature dimension while downsampling to provide a rich source of information for subsequent processing; The decoder part is a symmetrical structure of the encoder, consisting of several STBs* and patch expansion layers, in which skip connections are used to effectively fuse the context features extracted in the decoder with the features extracted by the encoder at multiple scales; the patch expansion layer plays the role of upsampling, reshaping the adjacent dimensions of the feature map into a feature map with twice the resolution to gradually restore the image scale; At the end of the model, the feature map of the last layer is upsampled by a factor of 4 to restore the resolution of the feature map to the same size as the input image, and then the final pixel-level segmentation prediction is generated through a linear projection layer; A bottleneck is introduced into the model, which is composed of two consecutive STB*. In this area, the resolution of the feature is kept unchanged.

9. The nanoparticle topological structure segmentation method based on the SwinU-Net* model as claimed in claim 1, characterized in that: During the model training process, the training iteration cycle was 50,000, the initial learning rate was set to 0.05, SGD was used as the optimizer, DiceLoss was used as the loss function, batch_size was 8, and the initialization method was random initialization.

Images