Shape memory polymer printing defect detection method, device and system and medium

Abstract

The embodiment of the invention discloses a shape memory polymer printing defect detection method, device and system and a medium, and the method comprises the steps: improving a C2f module in a YOLOv8 model backbone network through DCNv2 deformable convolution, and carrying out the feature extraction of an original defect image through a self-adaptive deformation convolution core; introducing a multi-scale space attention module into the neck network of the YOLOv8 model; and meanwhile, a down-sampling module is introduced into the backbone network, so that the calculation complexity of the model is reduced. The DCNv2 can adaptively adjust the convolution kernel by using offset and modulation scalar according to actual printing defects to form an adaptive deformation convolution kernel, can adapt to various different types of printing defects with large shape and size differences, and can perform weighting on printing defect parts in a feature fusion stage through a multi-scale space attention module, so as to obtain the adaptive deformation convolution kernel. The model has relatively high detection accuracy; by introducing the down-sampling module into the backbone network, the calculation complexity of the model is reduced, and the detection speed of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of printing defect detection, and in particular to a shape memory polymer printing defect detection method, device, system and medium. Background Art

[0002] Shape memory polymer (SMP) is a new deformable material that can be used as a raw material for 3D printing. Due to its high strength, stiffness, excellent anti-fatigue performance, shape memory effect and deformation ability, it is widely used in aerospace and other fields after 3D printing. In the process of 3D printing, due to the characteristics of shape memory polymers themselves, multiple factors such as appropriate printing speed, extrusion rate and temperature are required to work together to obtain an ideal printed product. However, in the actual printing process, printing defects may occur during the printing process due to factors such as the motion error of the nozzle, uneven distribution of temperature and stress fields, and too fast or too slow extrusion rate. It is necessary to detect the printing defects to determine whether the printed product is qualified.

[0003] Using image recognition models to identify images collected during the shape memory polymer 3D printing process can determine whether there are printing defects during the printing process. However, due to the many influencing factors involved in the 3D printing process, the printing defects generated during the printing process may have large differences in shape and size, and the features formed also have large differences. It is impossible to obtain accurate recognition results using traditional models. Summary of the invention

[0004] Embodiments of the present invention provide a shape memory polymer printing defect detection method, device, system and medium to solve the technical problem of accurately detecting printing defects with large differences in the shape memory polymer printing process.

[0005] In a first aspect, an embodiment of the present invention provides a shape memory polymer printing defect detection method, comprising:

[0006] Obtaining original printing defect images collected during the 3D printing process;

[0007] Input the original printing defect image into the trained improved YOLOv8 model, and use the backbone network of the improved YOLOv8 model to extract features of the original defect image to obtain a printing defect feature map, wherein the backbone network of the improved YOLOv8 model includes a deformable convolution C2f module;

[0008] The deformable convolution C2f module includes a deformable convolution Bottleneck unit, and the deformable convolution Bottleneck unit includes a DCNv2 layer. The DCNv2 layer first processes the original defect image through an offset convolution sublayer to calculate the offset of each sampling point, and processes the original defect image through a modulation scalar convolution sublayer to calculate the modulation scalar of each sampling point, and at the same time uses an activation function to determine the value range of the modulation scalar, and uses the offset and modulation scalar of each sampling point and the convolution kernel to form an adaptive deformable convolution kernel, and uses the adaptive deformable convolution kernel to perform convolution processing on the original defect image to obtain a printing defect feature map;

[0009] Using the neck network of the improved YOLOv8 model to perform feature fusion on the printing defect feature map to obtain a fused printing defect feature map, wherein the neck network of the improved YOLOv8 model includes a multi-scale spatial attention module;

[0010] The head network of the improved YOLOv8 model is used to identify printing defects by fusion printing defect feature maps to determine the damage of the printed products.

[0011] Furthermore, the method of using the offset and modulation scalar of each sampling point and the convolution kernel to form an adaptive deformation convolution kernel includes:

[0012] The offset of each sampling point is summed with the corresponding sampling point in the convolution kernel to obtain all offset sampling points of the convolution kernel;

[0013] The modulation scalar of each sampling point is used to weight the corresponding offset sampling point to form an adaptive deformation convolution kernel.

[0014] Furthermore, the multi-scale spatial attention module includes: a spatial attention unit and a channel attention unit. The spatial attention unit performs weight allocation on the printing defect feature map based on the spatial attention mechanism to obtain a spatial attention weighted feature map. The channel attention unit performs weight allocation on the printing defect feature map based on the channel attention mechanism to obtain a channel attention weighted feature map. The spatial attention weighted feature map and the channel attention weighted feature map are used to form a fused printing defect feature map.

[0015] Furthermore, the weight allocation of the printing defect feature map based on the spatial attention mechanism to obtain the spatial attention weighted feature map includes:

[0016] Perform global average pooling and maximum pooling on the printing defect feature map to obtain average pooling features and maximum pooling features;

[0017] Perform convolution on the global pooling features formed by the average pooling features and the maximum pooling features to obtain a spatial attention map;

[0018] The spatial attention map and the printing defect feature map are used to form a spatial attention weighted feature map.

[0019] Furthermore, the method of weighting the printing defect feature map based on the channel attention mechanism to obtain the channel attention weighted feature map includes:

[0020] Perform global average pooling and maximum pooling on the printing defect feature map to obtain average pooling features and maximum pooling features;

[0021] The average pooling features and the maximum pooling features are processed by the fully connected layer to obtain the fully connected average pooling features and the fully connected maximum pooling features;

[0022] The channel attention weights are formed using the fully connected average pooling features and the fully connected maximum pooling features;

[0023] The channel attention weight and the printing defect feature map are used to form a channel attention weighted feature map.

[0024] Furthermore, the backbone network of the improved YOLOv8 also includes a downsampling module, which performs convolution processing on the printing defect feature map through multiple branch paths respectively to obtain a branch feature map of each branch path, and uses spatial attention and channel attention to weight the branch feature map of each branch path respectively to obtain a branch weighted feature map of each branch path, and performs feature fusion on the branch weighted feature maps of all branch paths to obtain a printing defect feature map after downsampling.

[0025] Furthermore, the multi-scale spatial attention module is an MSDA module, and the head network is connected to the neck network through the MSDA module.

[0026] In a second aspect, an embodiment of the present invention further provides a shape memory polymer printing defect detection device, comprising:

[0027] Printing image acquisition module, used to acquire original printing defect images during 3D printing;

[0028] A feature extraction module, used for performing feature extraction including deformable convolution on the original printing defect image;

[0029] A feature fusion module, used for performing feature fusion including multi-scale spatial attention on the printing defect feature map;

[0030] The defect recognition module is used to recognize printing defects based on the fused printing defect feature map.

[0031] In a third aspect, an embodiment of the present invention further provides a computer system, including:

[0032] one or more processors;

[0033] a storage device for storing one or more programs,

[0034] When the one or more programs are executed by the one or more processors, the one or more processors implement the shape memory polymer printing defect detection method provided in the above embodiment.

[0035] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the shape memory polymer printing defect detection method provided in the above embodiment.

[0036] The embodiments of the present invention provide a shape memory polymer printing defect detection method, device, system and medium. By improving the DCNv2 layer in the backbone network of the YOLOv8 model, an offset and a modulation scalar are calculated according to the original printing defect image to form an adaptive deformation convolution kernel. The adaptive deformation convolution kernel can make adaptive adjustments to the convolution kernel according to the actual printing defect, can adapt to a variety of different types of printing defects with large shape and size differences, and has high accuracy when extracting features of the printing defects; at the same time, by setting a multi-scale spatial attention module in the neck network of the improved YOLOv8 model, the printing defect part is weighted in the feature fusion stage, so that the attention of the printing defect part is improved, and the detection accuracy of the model is further improved; in addition, by introducing a downsampling module in the backbone network of the improved YOLOv8 model, the calculation complexity of the model is reduced, the detection speed of the model is improved, and the model can better monitor the defects of 3D printing in real time, stop the printing process that produces defects in time, and reduce resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0038] Figure 1 This is a flow chart of a shape memory polymer printing defect detection method according to Embodiment 1 of the present invention;

[0039] Figure 2 This is a schematic diagram of printing defects according to the first embodiment of the present invention;

[0040] Figure 3 This is a model architecture diagram of the improved YOLOv8 model described in Example 1 of the present invention;

[0041] Figure 4This is a flow chart of a shape memory polymer printing defect detection method according to Embodiment 2 of the present invention;

[0042] Figure 5 This is a flow chart of a shape memory polymer printing defect detection method according to Embodiment 3 of the present invention;

[0043] Figure 6 This is a schematic structural diagram of a shape memory polymer printing defect detection device according to Embodiment 5 of the present invention;

[0044] Figure 7 This is a schematic diagram of the structure of a computer system described in Example 6 of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0046] Embodiment 1

[0047] Figure 1 This is a flow chart of a shape memory polymer printing defect detection method according to Embodiment 1 of the present invention. This embodiment introduces deformable convolution DCNv2 into the backbone network of YOLOv8, uses DCNv2 to calculate offset and modulation scalar according to printing defects, forms an adaptive deformable convolution kernel, and makes the shape and size of the convolution kernel adapt to the shape and size of printing defects, so as to accurately detect printing defects. The method specifically includes the following steps:

[0048] When using shape memory polymer (SMP) as raw material for 3D printing, the formation of the ideal SMP molding structure requires the perfect coordination of multiple factors such as printing speed, extrusion rate, and temperature. However, factors such as nozzle motion error, uneven distribution of temperature and stress fields, and too fast or too slow extrusion rates in the actual printing process can easily lead to printing defects. Printing defects are often divided into six categories: under-extrusion, over-extrusion, cracks, blobs, spaghetti, and stringing. Figure 2 As shown, Figure 2(a) is under-extrusion, (b) is over-extrusion, (c) is crack, (d) is spot, (e) is spaghetti, and (f) is wiredrawing. Due to the large differences in the characteristics of the six defects and the characteristics of SMP materials, the printing defects generated during the printing process may change greatly over time due to thermal or mechanical factors, and even become different types of printing defects. These defects often present non-uniform, curved or patterns with strong local features, and the formation characteristics are quite different. Therefore, it is necessary to flexibly adapt to printing defects with large differences in shape and size to improve detection accuracy.

[0049] S110, obtaining an original printing defect image collected during the 3D printing process.

[0050] In the process of 3D printing, an image acquisition device is used to continuously acquire a video of the printing process, and a video segmentation code is used to segment the acquired video of the printing process into images by frame. For example, the printer used in this embodiment is an FDM 3D printer, a camera is used to acquire a video of the printing process, and a microprocessor is a Raspberry Pi 4B used to process the acquired video image and output an original defect image.

[0051] S120, input the original printing defect image into the trained improved YOLOv8 model, use the backbone network of the improved YOLOv8 model to extract features of the original defect image, and obtain a printing defect feature map, the backbone network of the improved YOLOv8 model includes a deformable convolution C2f module; the deformable convolution C2f module includes a deformable convolution Bottleneck unit, and the deformable convolution Bottleneck unit includes a DCNv2 layer.

[0052] The YOLOv8 model can be used for image target detection. When using YOLOv8 to detect printing defects in the original printing defect image, in order to adapt to the large feature differences of printing defects, the YOLOv8 model is improved, including improving the C2f module in the YOLOv8 backbone network to a deformable convolution C2f (C2f_DCNv2) module. Specifically, the CBS convolution structure of the Bottleneck module in the C2f module is improved. The CBS convolution structure includes a standard convolution Conv2d layer, a BatchNorm2d layer, and a SiLu layer. The standard convolution Conv2d layer is replaced with a DCNv2 deformable convolution layer to construct a DBS convolution structure including a deformable convolution DCNv2 layer, a BatchNorm2d layer, and a SiLu layer. Correspondingly, according to the architecture of the YOLOv8 model, in order to adapt to the improvement of the C2f module in the backbone network, the C2f module in the neck network is also replaced in the same way. The model architecture of the improved YOLOv8 model is as follows: Figure 3shown.

[0053] Specifically, S120 includes:

[0054] S121, the DCNv2 layer first processes the input feature map of the DCNv2 layer through the offset convolution sublayer, calculates the offset of each sampling point, and processes the input feature map of the DCNv2 layer through the modulation scalar convolution sublayer, calculates the modulation scalar of each sampling point, and uses the activation function to determine the value range of the modulation scalar. DCNv2 can use the offset convolution sublayer to learn the offset Δ of each sampling point through the input feature map of the DCNv2 layer. pk , the calculation formula is as follows:

[0055] {Δ pk}=Conv offset (X)

[0056] Among them, Conv offset is the offset convolution sublayer, and X is the input feature map of the DCNv2 sublayer.

[0057] The modulation scalar convolution sublayer of DCNv2 can learn the modulation scalar Δ of each sampling point by inputting the input feature map of the DCNv2 layer. mk , the calculation formula is as follows:

[0058] Δ mk =σ(Conv mod (X)

[0059] Among them, Conv mod (X) is the modulated scalar convolution sublayer, and σ is the Sigmoid activation function, which is used to ensure that Δ mk ∈[0,1].

[0060] S122, using the offset and modulation scalar of each sampling point and the convolution kernel to form an adaptive deformation convolution kernel, and using the adaptive deformation convolution kernel to perform convolution processing on the input feature map of the DCNv2 layer to obtain the output feature map of the DCNv2 layer. The offset can change the sampling point position of the convolution kernel, and then change the shape and size of the convolution kernel. Through the learnable offset, the convolution kernel can be adaptively deformed to different types of printing defects. At the same time, the modulation scalar is used to weight the sampling points of the convolution kernel, increase the ability of defect feature extraction receptive field, reduce the weight of uninterested areas, and enable the adaptive deformation convolution kernel to adjust according to the target size, thereby accurately extracting printing defect features.

[0061] S130, using the neck network of the improved YOLOv8 model to perform feature fusion on the printing defect feature map to obtain a fused printing defect feature map, the neck network of the improved YOLOv8 model includes a multi-scale spatial attention module.

[0062] The improved YOLOv8 model uses the deformable convolution C2f (C2f_DCNv2) module in the backbone network to extract multi-scale features of the original printing defect image, and then uses the neck network to fuse the multi-scale features. In the feature fusion stage, the multi-scale spatial attention is used to weight the printing defect features. By assigning different weights to the printing defect parts and other parts in the image, the model can increase its attention to printing defects, thereby improving the detection accuracy of printing defects.

[0063] S140, using the head network of the improved YOLOv8 model to identify printing defects on the fused printing defect feature map, and determine the damage of the printed product.

[0064] The head network of the improved YOLOv8 model uses the fused printing defect feature image weighted by the multi-scale spatial attention module to classify the features in the image, determine whether there are printing defects in the image, and the size, accuracy and other information of the printing defects. Based on the obtained information, the existence of printing defects can be determined, and the 3D printing process can be stopped in time to reduce the waste of resources. After printing is completed, the printing defects can also be reviewed and traced to determine the qualified rate of 3D printed products.

[0065] When training the improved YOLOv8 model, all the printing process video images collected during the complete historical 3D printing process can be used to generate original printing defect images. In order to expand the data samples and enhance the training effect of the model, the original printing defect images can be randomly flipped, rotated, noisy, distorted, etc. to perform data enhancement, build an original defect image data set, and divide the data set into training set, validation set and test set to train and adjust the improved YOLOv8 model.

[0066] This embodiment uses deformable convolution DCNv2 to improve the CBS convolution structure of the Bottleneck unit in the C2f module into a DBS convolution structure. DCNv2 adds an offset and a modulation scalar to the convolution kernel to form an adaptive deformable convolution kernel, which can adaptively adjust different types of printing defects with large differences in shape and size, thereby extracting printing defect features with higher precision and improving the recognition accuracy of the model.

[0067] Embodiment 2

[0068] Figure 4 This is a flow chart of a shape memory polymer printing defect detection method according to the second embodiment of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the offset and modulation scalar of each sampling point are used with the convolution kernel to form an adaptive deformation convolution kernel. The specific optimization is as follows:

[0069] The offset of each sampling point is summed with the corresponding sampling point in the convolution kernel to obtain all offset sampling points of the convolution kernel;

[0070] The modulation scalar of each sampling point is used to weight the corresponding offset sampling point to form an adaptive deformation convolution kernel.

[0071] Accordingly, the shape memory polymer printing defect detection method provided in this embodiment specifically includes:

[0072] S210, obtaining an original printing defect image collected during the 3D printing process.

[0073] S220, input the original printing defect image into the trained improved YOLOv8 model, use the backbone network of the improved YOLOv8 model to extract features of the original defect image, and obtain a printing defect feature map, the backbone network of the improved YOLOv8 model includes a deformable convolution C2f module; the deformable convolution C2f module includes a deformable convolution Bottleneck unit, and the deformable convolution Bottleneck unit includes a DCNv2 layer.

[0074] Specifically, S220 includes:

[0075] S221, the DCNv2 layer first processes the input feature map of the DCNv2 layer through the offset convolution sublayer, calculates the offset of each sampling point, and processes the input feature map of the DCNv2 layer through the modulation scalar convolution sublayer, calculates the modulation scalar of each sampling point, and uses the activation function to determine the value range of the modulation scalar.

[0076] S222: Sum the offset of each sampling point and the corresponding sampling point in the convolution kernel to obtain all offset sampling points of the convolution kernel. Use the modulation scalar of each sampling point to weight the corresponding offset sampling point to form an adaptive deformation convolution kernel. Use the adaptive deformation convolution kernel to convolve the input feature map of the input DCNv2 layer to obtain the output feature map of the input DCNv2 layer.

[0077] The position of the sampling point is determined by summing the offset and the corresponding sampling point to form an irregular convolution kernel that can adapt to the characteristics of printing defects. The modulation scalar and the corresponding sampling point are multiplied to calculate the sampling point allocation weight to form an adaptive deformation convolution kernel, and the input feature map of the input DCNv2 layer is convolved for feature extraction. The calculation formula is as follows:

[0078]

[0079] Among them, Y(i,j) is the output feature map of the input DCNv2 layer, ωk is the convolution kernel weight, determined by model training, Δ mk is the modulation scalar, Δ pk is the offset, p k is the standard sampling position of the convolution kernel.

[0080] S230, using the neck network of the improved YOLOv8 model to perform feature fusion on the printing defect feature map to obtain a fused printing defect feature map, the neck network of the improved YOLOv8 model includes a multi-scale spatial attention module.

[0081] S240, using the head network of the improved YOLOv8 model to identify printing defects on the fused printing defect feature map, and determine the damage of the printed product.

[0082] This embodiment uses the calculated offset to add a corresponding offset to each sampling point on the sampling point of the standard convolution kernel to form an irregular convolution kernel, and uses the calculated modulation scalar to assign a corresponding weight to each sampling point to form an adaptive deformation convolution kernel that can adapt to the characteristics of printing defects. When using the adaptive deformation convolution kernel to extract features of printing defects, higher accuracy can be achieved.

[0083] Embodiment 3

[0084] Figure 5 This is a flow chart of a shape memory polymer printing defect detection method described in Example 3 of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the multi-scale spatial attention module includes: a spatial attention unit and a channel attention unit. The spatial attention unit assigns weights to the printing defect feature map based on the spatial attention mechanism to obtain a spatial attention weighted feature map; the channel attention unit assigns weights to the printing defect feature map based on the channel attention mechanism to obtain a channel attention weighted feature map; the spatial attention weighted feature map and the channel attention weighted feature map are used to form a fused printing defect feature map.

[0085] Accordingly, the shape memory polymer printing defect detection method provided in this embodiment specifically includes:

[0086] S310, obtaining an original printing defect image collected during the 3D printing process.

[0087] S320, input the original printing defect image into the trained improved YOLOv8 model, use the backbone network of the improved YOLOv8 model to extract features of the original defect image, and obtain a printing defect feature map, the backbone network of the improved YOLOv8 model includes a deformable convolution C2f module; the deformable convolution C2f module includes a deformable convolution Bottleneck unit, and the deformable convolution Bottleneck unit includes a DCNv2 layer.

[0088] S330, using the neck network of the improved YOLOv8 model to perform feature fusion on the printing defect feature map to obtain a fused printing defect feature map, wherein the neck network of the improved YOLOv8 model includes a multi-scale spatial attention module.

[0089] Specifically, S330 includes:

[0090] S331, the spatial attention unit generates a spatial feature map about the input feature map of the multi-scale spatial attention module by globally pooling the input feature map of the multi-scale spatial attention module, and then convolves the spatial feature map to obtain a spatial attention map, and uses the spatial attention map to weight the input feature map of the multi-scale spatial attention module to obtain a spatial attention weighted feature map.

[0091] S332, the channel attention unit generates a channel feature vector about the input feature map of the multi-scale spatial attention module by globally pooling the input feature map of the multi-scale spatial attention module, and then processes the channel attention vector using the fully connected layer to obtain the channel attention weight, and uses the channel attention weight to weight the input feature map of the multi-scale spatial attention module to obtain the channel attention weighted feature map.

[0092] S333, summing the spatial attention weighted feature map and the channel attention weighted feature map to obtain a fused printing defect feature map, the formula is as follows:

[0093] X MSDA =X spatial +X channel

[0094] Among them, X MSDA To fuse the print defect feature map, X spatial is the spatial attention weighted feature map, X channel Weighted feature map for channel attention.

[0095] Spatial attention and channel attention are used for weighting in the feature fusion stage respectively, so that the printing defect parts in the fused printing defect feature map output by the neck network of the improved YOLOv8 model gain higher attention and are transmitted to the head network of the improved YOLOv8 model for printing defect recognition, which is beneficial to improve the recognition accuracy of the head network.

[0096] S340, using the head network of the improved YOLOv8 model to identify printing defects on the fused printing defect feature map, and determine the damage of the printed product.

[0097] Specifically, the multi-scale spatial attention module is an MSDA module, and the head network is connected to the neck network through the MSDA module. The MSDA module is located at the connection between the neck network and the head network, such as Figure 3 As shown in the figure, the MSDA module receives the feature map fused by the neck network and performs multi-scale spatial attention weighting before outputting it to the head network. MSDA can effectively aggregate semantic information of different scales within the concerned receptive field and effectively reduce the redundancy of the self-attention mechanism without complex operations and additional computational costs.

[0098] An optional implementation of this embodiment is to perform weight allocation on the printing defect feature map based on the spatial attention mechanism to obtain a spatial attention weighted feature map, including:

[0099] Perform global average pooling and maximum pooling on the input feature map of the MSDA module to obtain average pooling features and maximum pooling features; perform feature splicing on the average pooling features and the maximum pooling features to form global pooling features, perform convolution processing on the global pooling features, and use the activation function to determine the value range of spatial attention to obtain a spatial attention map;

[0100] S = σ(Conv 7×7 ([GAP(X); GMP(X)]))

[0101] Among them, S is the spatial attention map, σ is the Sigmoid activation function, Conv 7×7 represents convolution processing, GAP(X) is the average pooling feature, GMP(X) is the maximum pooling feature, and X is the input feature map of the MSDA module.

[0102] The spatial attention map and the printing defect feature map are used to form a spatial attention weighted feature map.

[0103] X spatial =X⊙S

[0104] Among them, X spatialis the spatial attention weighted feature map, X is the input feature map of the MSDA module, S is the spatial attention map, and ⊙ represents element-by-element multiplication.

[0105] Optionally, the weighting of the printing defect feature map is performed based on the channel attention mechanism to obtain a channel attention weighted feature map, including:

[0106] Perform global average pooling and maximum pooling on the print defect feature map to obtain average pooling features and maximum pooling features; use the fully connected layer to process the average pooling features and maximum pooling features respectively to obtain fully connected average pooling features and fully connected maximum pooling features; use the fully connected average pooling features and fully connected maximum pooling features to form channel attention weights;

[0107] C = σ(FC(GAP(X))+FC(GMP(X)))

[0108] Among them, C is the channel attention weight, σ is the Sigmoid activation function, FC is the fully connected layer, GAP(X) is the average pooling feature, and GMP(X) is the maximum pooling feature.

[0109] The channel attention weight and the printing defect feature map are used to form a channel attention weighted feature map.

[0110] X channel =X⊙C

[0111] Among them, X channel is the channel attention weighted feature map, X is the input feature map of the MSDA module, C is the channel attention weight, and ⊙ represents element-by-element multiplication.

[0112] This embodiment combines spatial attention and channel attention to weight the printing defect features, improves the attention to printing defects in the feature fusion stage of the improved YOLOv8 model, helps the improved YOLOv8 model to better adapt to the unique deformation characteristics, irregular shapes and multi-scale features of printing defects, and enables the head network of the improved YOLOv8 model to obtain higher accuracy when performing feature recognition, classification or accuracy judgment, thereby improving the recognition accuracy of printing defects.

[0113] Embodiment 4

[0114] This embodiment is optimized based on the above embodiment. In this embodiment, the backbone network of the improved YOLOv8 also includes a downsampling module. The ADown downsampling module is used to replace the ordinary convolution Conv module in the YOLOv8 backbone network and the neck network, such as Figure 3 As shown, the ADown downsampling module performs different convolution processing on the printing defect feature map through multiple branch paths to obtain a branch feature map of each branch path.

[0115] X i =f i (X)

[0116] Among them, X i For the output of each branch path, f i is the downsampling operation of the i-th branch, and X is.

[0117] The branch feature map of each branch path is weighted by spatial attention and channel attention respectively to obtain the branch weighted feature map of each branch path. The branch weighted feature maps of all branch paths are fused to obtain the downsampled printing defect feature map.

[0118]

[0119] Among them, Y is the downsampled output feature map, α i is the weight coefficient of each branch path, determined by model training, X i The output for each branch path.

[0120] In this embodiment, the resolution of the feature map is reduced by the ADown downsampling module, thereby reducing the amount of calculation and storage requirements, and retaining the features required for printing defect detection as much as possible. Since the introduction of deformable convolution DCNv2 and MSDA modules in the improved YOLOv8 model increases the amount of calculation of the model, in order to meet the real-time detection capability of printing defects, the Adown module is used to reduce the amount of calculation of the improved YOLOv8 model and improve the detection speed of the model, so that the improved YOLOv8 model can be deployed on embedded devices for real-time 3D printing defect detection.

[0121] Embodiment 5

[0122] Figure 6 FIG. 1 is a schematic diagram of the structure of a shape memory polymer printing defect detection device according to Embodiment 5 of the present invention. Figure 6 As shown, the system comprises:

[0123] The printing image acquisition module 810 is used to acquire original printing defect images during the 3D printing process.

[0124] A feature extraction module 820 is used to perform feature extraction including deformable convolution on the original printing defect image to generate a printing defect feature map;

[0125] A feature fusion module 830 is used to perform feature fusion including multi-scale spatial attention on the print defect feature map to generate a fused print defect feature map;

[0126] The defect recognition module 840 is used to perform printing defect recognition on the fused printing defect feature map.

[0127] The shape memory polymer printing defect detection device described in this embodiment collects video images during the 3D printing process through a printing image acquisition module, and performs frame-by-frame segmentation to generate an original printing defect image, uses a feature extraction module to extract features from the original printing defect image, uses a feature fusion module to perform feature fusion on a printing defect feature map, and uses a defect recognition module to recognize the fused printing feature map, recognizes the printing defects generated during the 3D printing process, can perform feature extraction of adaptive deformation convolution according to the characteristics of the printing defects, can adapt to the recognition of a variety of printing defects with large differences, and has high recognition accuracy.

[0128] The shape memory polymer printing defect detection device provided in the embodiment of the present invention can execute the shape memory polymer printing defect detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0129] Embodiment 6

[0130] Figure 7 This is a schematic diagram of the structure of a computer system described in Example 6 of the present invention. Figure 7 A block diagram of an exemplary computer system 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The computer system 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0131] like Figure 7 As shown, computer system 12 is represented in the form of a general-purpose computing device. Components of computer system 12 may include, but are not limited to: one or more processors or processing units;

[0132] System memory, bus 18 that connects various system components including system memory 28 and processing unit 16.

[0133] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0134] The computer system 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer system 12, including volatile and non-volatile media, removable and non-removable media.

[0135] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer system 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0136] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0137] The computer system 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the device / server / computer system 12, and / or any device that enables the computer system 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the computer system 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer system 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer system 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0138] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the shape memory polymer printing defect detection method provided in the embodiment of the present invention.

[0139] Embodiment 7

[0140] Embodiment 7 of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the shape memory polymer printing defect detection method provided in the above embodiment.

[0141] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0142] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0143] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0144] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A shape memory polymer printing defect detection method, characterized in that: include: Obtaining original printing defect images collected during the 3D printing process; Input the original printing defect image into the trained improved YOLOv8 model, and use the backbone network of the improved YOLOv8 model to extract features of the original defect image to obtain a printing defect feature map, wherein the backbone network of the improved YOLOv8 model includes a deformable convolution C2f module; The deformable convolution C2f module includes a deformable convolution Bottleneck unit, and the deformable convolution Bottleneck unit includes a DCNv2 layer. The DCNv2 layer first processes the original defect image through an offset convolution sublayer to calculate the offset of each sampling point, and processes the original defect image through a modulation scalar convolution sublayer to calculate the modulation scalar of each sampling point, and at the same time uses an activation function to determine the value range of the modulation scalar, and uses the offset and modulation scalar of each sampling point and the convolution kernel to form an adaptive deformable convolution kernel, and uses the adaptive deformable convolution kernel to perform convolution processing on the original defect image to obtain a printing defect feature map; Using the neck network of the improved YOLOv8 model to perform feature fusion on the printing defect feature map to obtain a fused printing defect feature map, wherein the neck network of the improved YOLOv8 model includes a multi-scale spatial attention module; The head network of the improved YOLOv8 model is used to identify printing defects by fusion printing defect feature maps to determine the damage of the printed products.

2. The method according to claim 1, characterized in that: The method uses the offset and modulation scalar of each sampling point and the convolution kernel to form an adaptive deformation convolution kernel, including: The offset of each sampling point is summed with the corresponding sampling point in the convolution kernel to obtain all offset sampling points of the convolution kernel; The modulation scalar of each sampling point is used to weight the corresponding offset sampling point to form an adaptive deformation convolution kernel.

3. The method according to claim 1, characterized in that The multi-scale spatial attention module includes: a spatial attention unit and a channel attention unit. The spatial attention unit performs weight allocation on the printing defect feature map based on the spatial attention mechanism to obtain a spatial attention weighted feature map. The channel attention unit performs weight allocation on the printing defect feature map based on the channel attention mechanism to obtain a channel attention weighted feature map. The spatial attention weighted feature map and the channel attention weighted feature map are used to form a fused printing defect feature map.

4. The method according to claim 3, characterized in that: The method of weighting the printing defect feature map based on the spatial attention mechanism to obtain the spatial attention weighted feature map includes: Perform global average pooling and maximum pooling on the printing defect feature map to obtain average pooling features and maximum pooling features; Perform convolution on the global pooling features formed by the average pooling features and the maximum pooling features to obtain a spatial attention map; The spatial attention map and the printing defect feature map are used to form a spatial attention weighted feature map.

5. The method according to claim 3, characterized in that: The method of weighting the printing defect feature map based on the channel attention mechanism to obtain the channel attention weighted feature map includes: Perform global average pooling and maximum pooling on the printing defect feature map to obtain average pooling features and maximum pooling features; The average pooling features and the maximum pooling features are processed by the fully connected layer to obtain the fully connected average pooling features and the fully connected maximum pooling features; The channel attention weights are formed using the fully connected average pooling features and the fully connected maximum pooling features; The channel attention weight and the printing defect feature map are used to form a channel attention weighted feature map.

6. The method according to claim 1, characterized in that: The improved YOLOv8 backbone network also includes a downsampling module, which performs convolution processing on the printing defect feature map through multiple branch paths to obtain a branch feature map of each branch path, and uses spatial attention and channel attention to weight the branch feature map of each branch path to obtain a branch weighted feature map of each branch path. The branch weighted feature maps of all branch paths are feature fused to obtain a printing defect feature map after downsampling.

7. The method according to claim 1, characterized in that: The multi-scale spatial attention module is an MSDA module, and the head network is connected to the neck network through the MSDA module.

8. A shape memory polymer printing defect detection device, characterized in that: include: Printing image acquisition module, used to acquire original printing defect images during 3D printing; A feature extraction module, used for performing feature extraction including deformable convolution on the original printing defect image; A feature fusion module, used for performing feature fusion including multi-scale spatial attention on the printing defect feature map; The defect recognition module is used to recognize printing defects based on the fused printing defect feature map.

9. A computer system, characterized in that: The system comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the shape memory polymer printing defect detection method as described in any one of claims 1-7.

10. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the shape memory polymer printing defect detection method according to any one of claims 1 to 7 when executed by a computer processor.

Images