CA-DeepLabv3 +-based natural fiber microstructure detection method

Abstract

The invention relates to the technical field of natural fiber microstructure detection, in particular to a natural fiber microstructure detection method based on CA-DeepLabv < 3 + >. According to the method, an attention mechanism is introduced into a DeepLabv3 + framework, cascade optimization is carried out on a cavity space pyramid pooling module, a natural fiber microstructure image is obtained through a scanning electron microscope, a training model segments the microstructure image, segmentation results under different deep learning network models are compared, and the natural fiber microstructure image is obtained. Good performance of the method in natural fiber microstructure image segmentation is verified, the natural fiber microstructure image extraction accuracy is improved, and a foundation is laid for improving the accuracy of measuring geometric attributes such as the area, the perimeter and the degree of irregularity (DOI) of the cross section of the natural fiber. According to the method, the MobileNetV2 is adopted as a feature extraction backbone network, so that the calculation complexity is reduced, and the model operation speed is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of natural fiber microstructure detection, and in particular to a natural fiber microstructure detection method based on CA-DeepLabv3+. Background Art

[0002] Synthetic fibers, such as glass fibers, carbon fibers, and aramid fibers, are the reinforcement materials for most polymer composites produced today. However, environmental issues associated with synthetic fibers and uncertainty in oil prices have prompted in-depth research into the suitability of natural fibers as reinforcement materials. Research results show that natural fiber reinforced composites show great potential in a variety of fields, including automotive interior panels and building materials. Because panels made of natural fiber reinforced composites not only have considerable strength and lightweight properties, but the material is renewable, it meets the requirements of modern sustainable development.

[0003] Fiber-reinforced composites are mainly composed of two parts: fibers and matrix. Fibers provide strength and hardness to the material, while the matrix holds the fibers together and protects them. The connection area between the fiber and the matrix is ​​called the interface, which is crucial to the performance of the composite. The interface is like the "glue" between the fiber and the matrix, helping to transfer force so that the entire material can withstand external pressure. The tensile strength and stiffness of the fiber and the shear strength of the interface play a key role in controlling the mechanical properties of the composite. The tensile strength and stiffness values ​​of the fiber depend on the cross-sectional area of ​​the fiber, while the interface shear strength value depends on the interface area. The interface area is the product of the circumference of the fiber cross section and the fiber embedding length, assuming that the fiber has a uniform cross section along its embedding length. Therefore, accurate measurement of the area and circumference of the fiber cross section provides the necessary theoretical support for the use of natural fibers as reinforcement materials in composites.

[0004] However, the cross-sectional shape and size of natural fibers vary greatly, which makes it complicated to accurately measure their area and perimeter. Many scholars use a simplified method and assume that the fiber is cylindrical for calculation. In recent years, with the advancement of technology, researchers have begun to use more advanced methods to accurately measure the cross-sectional characteristics of natural fibers, such as scanning electron microscopy (SEM) and X-ray micro-computed tomography (Micro-CT). These technologies can provide detailed three-dimensional structural information of fibers at the nanoscale, greatly improving the accuracy of measurements, but their equipment is very expensive and requires professional operation. Therefore, in many studies on natural fibers, it is still common to use the average diameter to characterize the fiber size. Although this simplification is convenient, it may not fully reflect the true geometric properties of natural fibers, especially considering the complex structure revealed by advanced measurement techniques. Summary of the invention

[0005] In view of the technical problems in the above-mentioned prior art, the present invention proposes a natural fiber microstructure detection method based on CA-DeepLabv3+.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0007] A natural fiber microstructure detection method based on CA-DeepLabv3+, comprising the following steps:

[0008] Step 1: Replace the DeepLabv3+ backbone network Xception with the lightweight MobileNetV2;

[0009] Step 2: In the Atrous Spatial Pyramid Pooling ASPP module of DeepLabv3+, the branches are operated in parallel. By fusing the feature layers between different branches, each branch shares the multi-level information of the fiber structure, thus expanding the receptive field of the network model.

[0010] Step 3: Embed the EMA attention mechanism module in the DeepLabv3+ network model;

[0011] Step 4: Add the output feature maps in each group to generate two sets of spatial attention weights, and then use the Sigmoid function and simple multiplication operations to make the final output of EMA the same size as the input feature map X; the multi-scale processing capability of the EMA module can capture structural information of different scales at the same time.

[0012] Further, as a preferred technical solution of the present invention, the lightweight MobileNetV2 in step 1 is a lightweight convolutional neural network based on Inverted ResBlock; wherein Inverted ResBlock is divided into two parts, the backbone part performs dimensionality increase processing on the features through a 1×1 convolution operation, and then extracts the features using a 3×3 depthwise separable convolution, and then uses a 1×1 convolution for dimensionality reduction; and the residual edge part as the second part directly connects the input and the output.

[0013] Further, as a preferred technical solution of the present invention, the calculation of the receptive field in step 2 is shown in formula (1):

[0014] RF=(E r -1)×(k-1)+k (1)

[0015] Among them: RF represents the receptive field, E r represents the dilation rate, and k represents the convolution kernel size.

[0016] According to formula (2), the maximum receptive field RF can be obtained max for:

[0017]

[0018] in: It represents the receptive field of the dilated convolution when the convolution kernel size is 3×3 and the dilation rate is 6.

[0019] Further, as a preferred technical solution of the present invention, in the EMA attention mechanism module, the input feature map X∈R C×H×W First, they are grouped and then processed by different branches; two parallel branches perform two one-dimensional global average pooling operations on the channels along two spatial directions respectively through 1×1 convolution. After being processed by the Sigmoid activation function, the two channel attention maps are aggregated together using matrix dot product operations; the third branch uses 3×3 convolution to capture multi-scale features; then, for the two tensors output by the 1×1 branch and the 3×3 branch, the output of the 1×1 branch is encoded with global spatial information using two-dimensional global average pooling. Before the channel feature aggregation activation mechanism, the output of the smallest branch is directly converted into the corresponding dimensional shape, that is, The dimensional shape of the 3×3 branch output is

[0020] The natural fiber microstructure detection method based on CA-DeepLabv3+ described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0021] (1) The present invention adopts MobileNetV2 as the feature extraction backbone network, which reduces the computational complexity and improves the model running speed.

[0022] (2) The present invention performs cascade optimization on the hollow space pyramid pooling ASPP module, thereby enhancing the model's ability to extract multi-scale features of natural fibers.

[0023] (3) The present invention embeds an efficient multi-scale attention mechanism EMA, which improves the model's ability to perceive the microstructural details of natural fibers. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the improved DeepLabv3+ network structure in an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a MobileNetV2 model in an embodiment of the present invention;

[0026] Figure 3 This is a structural diagram of an EMA module in an embodiment of the present invention;

[0027] Figure 4An example diagram of the microstructure of natural fibers and its corresponding mask diagram are provided in accordance with an embodiment of the present invention;

[0028] FIG5( a ) is a schematic diagram of a visualization comparison result of the mean intersection-over-union (MIoU) ratio in an embodiment of the present invention;

[0029] FIG5( b ) is a schematic diagram of a loss function visualization comparison result in an embodiment of the present invention;

[0030] Figure 6 Schematic diagram of the microstructure of 6 natural fibers with different networks in the embodiments of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further explained below in detail with reference to the accompanying drawings so that those skilled in the art can more deeply understand the present invention and be able to implement it. However, the following reference examples are only used to explain the present invention and are not intended to limit the present invention.

[0032] A natural fiber microstructure detection method based on CA-DeepLabv3+, comprising the following steps:

[0033] Step 1: Replace the DeepLabv3+ backbone network Xception with the lightweight MobileNetV2;

[0034] Step 2: In the Atrous Spatial Pyramid Pooling ASPP module of DeepLabv3+, the branches are operated in parallel. By fusing the feature layers between different branches, each branch shares the multi-level information of the fiber structure, thus expanding the receptive field of the network model.

[0035] Step 3: Embed the EMA attention mechanism module in the DeepLabv3+ network model;

[0036] Step 4: Add the output feature maps in each group to generate two sets of spatial attention weights, and then use the Sigmoid function and simple multiplication operations to make the final output of EMA the same size as the input feature map X; the multi-scale processing capability of the EMA module can capture structural information of different scales at the same time.

[0037] The Xception backbone network of the standard DeepLabv3+ network improves the accuracy of the network model by expanding the network depth and breadth, but the computational complexity of the Xception backbone network is relatively large and difficult to deploy in embedded systems. The present invention uses the MobileNetV2 backbone network to replace the original Xception backbone network, reduce network parameters, and improve the model running speed; at the same time, since the morphology and size of the natural fiber microstructure will vary greatly in different images, in the original ASPP module of DeepLabv3+, the branches run in parallel, and the microstructure feature information cannot interact. Therefore, the present invention cascades and optimizes the original ASPP module of the DeepLabv3+ network, and improves the accuracy and robustness of microstructure feature extraction by extracting and fusing multi-scale information of the microstructure step by step; in view of the interference of the complex network structure of natural fibers, the interweaving characteristics between fibers, and the surface morphology changes on the segmentation of the natural fiber microstructure, the present invention embeds the EMA attention mechanism in the DeepLabv3+ network model, makes full use of the multi-scale information, improves the extraction effect of deep features and shallow features, and enhances the ability to extract features of natural fiber microstructure images. The CA-DeepLabv3+ network structure is as follows: Figure 1 As shown in the figure, ⊕ represents the splicing operation.

[0038] Since the backbone network Xception in the original DeepLabv3+ network has a complex structure and a large number of parameters, for the complex and changeable goal of extracting the microstructure of natural fibers, the network may lose the fine structure and surface features of the fiber when extracting fiber details, resulting in inaccurate or discontinuous segmentation results. The microstructure of natural fibers usually presents a complex network morphology, with interweaving and overlapping between fibers, and the surface morphology also varies greatly. The present invention replaces the original DeepLabv3+ backbone network Xception with a lightweight MobileNetV2, so that the improved network can extract features in natural fiber microscopic images faster, and at the same time, make it easier for the model to capture fiber details, interweaving structures and surface morphology. This improvement helps to improve the accuracy and efficiency of natural fiber microstructure segmentation, and provides a more reliable basis for subsequent fiber performance analysis.

[0039] The lightweight MobileNetV2 is a lightweight convolutional neural network based on InvertedResBlock, which reduces model parameters while ensuring the segmentation accuracy of the model. The Inverted ResBlock is divided into two parts. The main part uses a 1×1 convolution operation to increase the dimension of the feature, then uses a 3×3 depth-separable convolution to extract the feature, and then uses a 1×1 convolution to reduce the dimension. The residual edge part is the second part that directly connects the input and output. The MobileNetV2 model is as follows: Figure 2 shown.

[0040] In the original ASPP module of DeepLabv3+, the branches run in parallel, which makes it difficult to fully capture the multi-scale features and complex spatial relationships of the natural fiber microstructure. By fusing the feature layers between different branches, each branch can share the multi-level information of the fiber structure, effectively expanding the receptive field of the network model. This enables the network to better extract the complex structure and morphological information in the natural fiber image. The receptive field (RF) refers to the size of the input image area corresponding to an output unit in a convolutional neural network, which affects the model's ability to capture multi-scale features. The calculation of the receptive field is shown in formula (1):

[0041] RF=(E r -1)×(k-1)+k (1)

[0042] Among them: RF represents the receptive field, E r represents the dilation rate, and k represents the convolution kernel size.

[0043] According to formula (2), the maximum receptive field RF can be obtained max for:

[0044]

[0045] in: It represents the receptive field of the dilated convolution when the convolution kernel size is 3×3 and the dilation rate is 6.

[0046] By connecting the feature layers in the ASPP module, the model can obtain a larger receptive field, thereby improving the model's utilization of complex structural information in natural fiber images, and ultimately improving the overall accuracy of the model's fiber structure recognition. In particular, for natural fibers such as flax, hemp or bamboo fibers, their microstructures often show a high degree of irregularity and variability. The improved model can better adapt to this complexity and capture the subtle features of the fiber, such as microfibril angles, shape changes in the fiber cross section, and the layered structure of the cell wall, which are crucial for understanding and predicting the mechanical properties and interface characteristics of the fiber.

[0047] In order to enhance the feature extraction capability of deep neural networks for natural fiber microstructures and suppress background noise, this paper integrates a cross-space learning efficient multi-scale attention (EMA) mechanism into the DeepLabv3+ framework, which significantly improves the model's feature expression capability and spatial perception accuracy. The EMA module uses a parallel substructure to reduce the network depth and achieves fine pixel-level attention allocation to feature maps without reducing the channel dimension. The EMA module structure is shown in Figure 2. Figure 3 shown.

[0048] In the EMA attention mechanism module, the input feature map X∈R C×H×W First, they are grouped and then processed by different branches; two parallel branches perform two one-dimensional global average pooling operations on the channels along two spatial directions respectively through 1×1 convolution. After being processed by the Sigmoid activation function, the two channel attention maps are aggregated together using matrix dot product operations; the third branch uses 3×3 convolution to capture multi-scale features; then, for the two tensors output by the 1×1 branch and the 3×3 branch, the output of the 1×1 branch is encoded with global spatial information using two-dimensional global average pooling. Before the channel feature aggregation activation mechanism, the output of the smallest branch is directly converted into the corresponding dimensional shape, that is, The dimensional shape of the 3×3 branch output is

[0049] Finally, the output feature maps in each group are added to generate two sets of spatial attention weights, and then the Sigmoid function and simple multiplication operation are used to make the final output of EMA the same size as the input feature map X. The EMA attention mechanism improves the model's ability to focus on target features, allowing the model to obtain more natural fiber microstructure feature information.

[0050] This improvement is particularly effective for analyzing the complex microstructure of natural fibers. Natural fibers, such as flax, hemp or bamboo fibers, often exhibit multi-scale structural features. The multi-scale processing capabilities of the EMA module can simultaneously capture structural information at these different scales, such as the cross-sectional shape of the fiber, the layered structure of the cell wall, the pore distribution, etc.

[0051] Specific experimental analysis: In order to train and improve the DeepLabv3+ image segmentation model, we first need to establish an image dataset. The image set selected in this article uses an Olympus BX61 electric microscope to take digital images of natural fibers. This dataset captures microstructure images of bamboo fiber, hemp fiber, and abaca fiber. 500 original images were collected, and after flipping, rotating, translating, brightness conversion, adding noise, mirroring, and splicing, the dataset was expanded to 3,500 images. Subsequently, the annotation software Labelme 3.16.7 was used to annotate the microstructure of natural fibers and generate mask images. Some examples of natural fiber microstructure annotations and their corresponding mask images are shown below. Figure 4 As shown. Subsequently, the enhanced natural fiber microstructure images were randomly generated into training sets and validation sets in a ratio of 9:1;

[0052] The experimental environment is as follows: the operating system is Windows 11; the GPU is NVIDIA GeForce RTX 3060LaptopGPU and CUDA parallel computing platform for GPU programming acceleration; the CPU is 11th Gen Intel(R)Core(TM)i7-11800H@2.30GHz, 32GB memory. Use Python 3.10 version for programming; use PyCharm Community Edition 2023 for network deployment and debugging; the deep learning framework used is Pytorch 2.0.1, CUDA Version 11.7; Input_shape is 512×512; Batch_size is 8, Num_workers is 4, Epoch is 300, and the initial learning rate is 0.007.

[0053] In order to verify the superiority of the natural fiber microstructure detection method based on improved DeepLabv3+ proposed in the present invention, the segmentation performance indicators of the proposed algorithm CA-DeepLabv3+ are compared with those of the current mainstream image segmentation algorithms such as U-Net, PSPNet (Pyramid Scene Parsing Network) and standard Deeplabv3+ network model. Under the same training parameter conditions, the segmentation performance indicators of different methods for microstructures are compared as shown in Table 1. When different networks are trained, mIoU is recorded every 5 epochs, and the visual comparison of the training results of different networks is shown in Figures 5(a) and 5(b).

[0054] Table 1 Comparison of segmentation performance of different methods

[0055]

[0056] Where, F1 = (2*Precision*Recall) / (Precision+Recall); the harmonic mean of Precision and Recall. Purpose: Comprehensively evaluate the performance of the model, especially in the case of class imbalance: Reflection: The model's ability to balance between precision and recall.

[0057] The improved model proposed in the present invention is superior to the comparison algorithm in terms of segmentation accuracy and detection rate. The detection accuracy Precision is improved by 1.36, 10.50, and 17.85 percentage points respectively compared with the standard DeepLabv3+, U-Net, and PSPNet network models, and the recall rate is improved by 0.93, 17.45, and 21.82 percentage points respectively. The average pixel accuracy mPA is improved by 0.9%, 17.5%, and 21.7% respectively, and the average intersection over union ratio mIoU is improved by 2.1%, 18.5%, and 23.9% respectively. In addition, the method proposed in the present invention is more accurate in the segmentation of natural fiber microstructure images. Compared with the standard DeepLabv3+, U-Net, and PSPNet network models, the F1 score of the present invention is higher than the comparison algorithm and can meet the real-time detection requirements.

[0058] In order to verify the reliability and detection accuracy of the method proposed in this invention, a detection accuracy comparison test was carried out. Figure 6 The 6 natural fiber microstructure images in the test set are used as part of the test set. After the DeepLabv3+ network model and the CE-DeepLabv3+ network model proposed in this invention predict the test set, the visualization results of the microstructure prediction of different networks are shown in Figure 2. Figure 6 shown.

[0059] according to Figure 6 It can be seen that the CA-DeepLabv3+ algorithm proposed in the present invention has a significantly better segmentation effect than the comparison algorithm when segmenting the natural fiber microstructure. It has a higher segmentation accuracy for most natural fiber microstructure images, and the edges are more closely aligned with the actual contours. It can still maintain a good segmentation effect in the face of complex backgrounds in the image.

[0060] Experimental results show that compared with mainstream image segmentation algorithms such as standard DeepLabv3+, U-Net and PSPNet, the CA-DeepLabv3+ algorithm proposed in the present invention has significant advantages in natural fiber microstructure detection: detection accuracy (Precision) is improved by 1.36%-17.85%, recall rate (Recall) is improved by 0.93%-21.82%, mean pixel accuracy (mPA) is improved by 0.9%-21.7%, mean intersection over union (mIoU) is improved by 2.1%-23.9%, and the F1 score is higher than that of the comparison algorithm.

[0061] The improved algorithm not only improves the segmentation accuracy of natural fiber microstructure, but also can better handle fiber edges under complex backgrounds, maintaining high detection efficiency. This method provides a reliable basis for accurately measuring geometric properties such as area, perimeter and DOI of natural fiber cross-sections, which is of great significance for in-depth understanding of the microstructural characteristics of natural fibers and their relationship with macroscopic properties.

[0062] The present invention introduces the attention mechanism into the DeepLabv3+ framework, and performs cascade optimization on the void space pyramid pooling module. The natural fiber microstructure image is acquired through a scanning electron microscope, and the model is trained to segment the microstructure image. The segmentation results under different deep learning network models are compared, which verifies its good performance in the segmentation of natural fiber microstructure images and improves the accuracy of natural fiber microstructure image extraction, laying a foundation for improving the accuracy of measuring geometric properties such as the area, perimeter and irregularity DOI of the natural fiber cross section.

[0063] The specific implementation scheme described above further describes in detail the purpose, technical scheme and beneficial effects of the present invention. It should be understood that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes and modifications made by any technician in the field without departing from the concept and principle of the present invention should fall within the scope of protection of the present invention.

Claims

1. A natural fiber microstructure detection method based on CA-DeepLabv3+, characterized in that: The following steps are involved: Step 1: Replace the DeepLabv3+ backbone network Xception with the lightweight MobileNetV2; Step 2: In the Atrous Spatial Pyramid Pooling ASPP module of DeepLabv3+, the branches are operated in parallel. By fusing the feature layers between different branches, each branch shares the multi-level information of the fiber structure, thus expanding the receptive field of the network model. Step 3: Embed the EMA attention mechanism module in the DeepLabv3+ network model; Step 4: Add the output feature maps in each group to generate two sets of spatial attention weights, and then use the Sigmoid function and simple multiplication operations to make the final output of EMA the same size as the input feature map X; the multi-scale processing capability of the EMA module can capture structural information of different scales at the same time.

2. A natural fiber microstructure detection method based on CA-DeepLabv3+ according to claim 1, characterized in that: The lightweight MobileNetV2 in step 1 is a lightweight convolutional neural network based on Inverted ResBlock; wherein Inverted ResBlock is divided into two parts, the backbone part performs dimensionality increase processing on the features through a 1×1 convolution operation, then extracts the features through a 3×3 depthwise separable convolution, and then reduces the dimensionality through a 1×1 convolution; and the residual edge part as the second part directly connects the input and the output.

3. A natural fiber microstructure detection method based on CA-DeepLabv3+ according to claim 2, characterized in that: The calculation of the receptive field in step 2 is shown in formula (1): RF=(E r -1)×(k-1)+k (1) Among them: RF represents the receptive field, E r represents the dilation rate, and k represents the convolution kernel size. According to formula (2), the maximum receptive field RF can be obtained max for: in: It represents the receptive field of the dilated convolution when the convolution kernel size is 3×3 and the dilation rate is 6.

4. A natural fiber microstructure detection method based on CA-DeepLabv3+ according to claim 3, characterized in that: In the EMA attention mechanism module, the input feature map X∈R C×H×W First they are grouped and then processed through different branches; Two parallel branches perform two one-dimensional global average pooling operations on the channels along two spatial directions through 1×1 convolutions. After being processed by the Sigmoid activation function, the two channel attention maps are aggregated together using matrix dot product operations. The third branch uses 3×3 convolutions to capture multi-scale features. Subsequently, the two tensors of the 1×1 branch and the 3×3 branch output are subjected to global spatial information encoding using two-dimensional global average pooling. Before the channel feature aggregation activation mechanism, the output of the smallest branch is directly converted into the corresponding dimensional shape, i.e. The dimensional shape of the 3×3 branch output is

Images