A digital printing defect detection method and device based on a multi-head cooperative mechanism

Abstract

The application belongs to the technical field of computer vision, and discloses a digital printing defect detection method and device based on a multi-head cooperative mechanism, which comprises the following steps: acquiring a digital printing image slice to be detected; inputting the digital printing image slice to be detected into a pre-trained digital printing defect detection model based on multi-head cooperation to obtain a defect detection result; the digital printing defect detection model comprises a backbone network, an encoder module, a decoder module and an output module; wherein the backbone network is used for extracting a multi-scale feature map of the digital printing image slice to be detected, and is an improved network in which multi-head attention, depth separable convolution, deformable attention, local attention and sliding window attention are introduced into a ResNet network. The application significantly enhances the detection accuracy of the digital printing defect detection model for microscopic defects and small texture defects, and reduces the false detection rate and the missed detection rate in complex background interference and irregular pattern deformation scenes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a digital printing defect detection method and device based on a multi-head cooperative mechanism. BACKGROUND

[0002] In the production process of digital printing fabrics, factors such as mechanical failure and nozzle failure can cause defects on the surface of the fabric, which can significantly affect the quality of the final product. Traditional defect detection methods based on statistics and filtering techniques mainly focus on solid color fabrics. Such methods are difficult to distinguish between complex textures and small defects, and therefore are difficult to generalize under different materials or lighting conditions, and cannot accurately and efficiently detect defects in digital printing fabrics with rich textures and bright colors.

[0003] With the rapid development of deep learning technology, Convolutional Neural Network (CNN) has become an important tool in object detection, especially in surface defect detection. Object detection algorithms based on CNN are mainly divided into two-stage and single-stage methods. Among them, the two-stage detection method first extracts candidate frames from the image, and then obtains the detection result through secondary correction based on the candidate region; the single-stage detection method directly calculates the detection result on the image, which has fast detection speed, but its detection accuracy is relatively low.

[0004] The target detection method based on Transformer and CNN is a current mainstream defect detection scheme (such as the collaborative DEtection TRansformer model), which uses CNN for feature extraction and uses Transformer structure to model global information. However, the existing target detection method based on Transformer and CNN still has limitations when applied to digital printing defect detection, including:

[0005] High false detection rate: the existing scheme is difficult to effectively distinguish defects from background in complex printing texture background, and is prone to misjudging texture details as defects, resulting in high false detection rate, which not only affects the accuracy of detection, but also increases the workload of subsequent manual review, and reduces the overall system efficiency;

[0006] Serious missing detection: due to the diversity of digital printing defect shapes and the existence of scale difference, the existing target detection method based on Transformer and CNN shows obvious shortcomings in processing small defects or irregular defects, resulting in some key defects being missed, which seriously affects product quality control and brings potential economic losses and reputation risks;

[0007] Poor multi-scale defect adaptability: although the existing method is sensitive to scale changes, it is difficult to simultaneously consider the defects of micro details and macro structures, resulting in that part of defects of different sizes cannot be comprehensively and accurately detected;

[0008] In addition, the existing scheme usually requires a large amount of computing resources, especially under high-resolution image processing and complex network structure, which leads to difficulty in real-time deployment in resource-limited industrial environments. SUMMARY

[0009] In order to solve the above problems existing in the prior art, the present application provides a digital printing defect detection method based on a multi-head cooperative mechanism.

[0010] The technical problem to be solved by the present application is solved by the following technical scheme:

[0011] In a first aspect, the present application provides a digital printing defect detection method based on a multi-head cooperative mechanism, comprising:

[0012] Obtaining a digital printing image and pre-processing the digital printing image to obtain a to-be-detected digital printing image slice;

[0013] Inputting the to-be-detected digital printing image slice into a pre-trained digital printing defect detection model based on multi-head cooperation to obtain a defect detection result; the digital printing defect detection model based on multi-head cooperation comprises a backbone network, an encoder module, a decoder module and an output module;

[0014] The backbone network is used to extract multi-scale feature maps of the to-be-detected digital printing image slice; the backbone network is an improved network in which multi-head attention, depth separable convolution, deformable attention, local attention and sliding window attention are introduced into the ResNet network; the multi-head attention is applied to the STAGE0 module of the ResNet network, and is used to capture local microscopic details and global structure information of the to-be-detected digital printing image slice through a multi-head attention parallel mechanism; the depth separable convolution is applied to the STAGE0 module of the ResNet network, and is used to reduce the parameter size and computational complexity of the digital printing defect detection model; the local attention and the sliding window attention are sequentially applied to the residual blocks of the STAGE1 module and the STAGE2 module of the ResNet network, and the local attention and the deformable attention are sequentially applied to the residual blocks of the STAGE3 module and the STAGE4 module of the ResNet network;

[0015] The encoder module is used to obtain multi-scale enhanced features based on a self-attention mechanism according to the multi-scale feature maps output by the backbone network;

[0016] The decoder module is configured to output query features by using a multi-head attention mechanism according to the multi-scale enhanced features.

[0017] The output module is configured to output a defect bounding box and a category label according to the query features.

[0018] Optionally, the STAGE0 module comprises a multi-head attention sub-module, a convolution sub-module and a depth separable convolution sub-module connected in sequence.

[0019] The multi-head attention sub-module is configured to extract local features of the to-be-detected digital printing image slice by a local attention head and calculate local attention weights, extract global features of the to-be-detected digital printing image slice by a global attention head and calculate global attention weights, generate normalized weights according to the local attention weights and the global attention weights, and calculate fused features according to the normalized weights, the local features and the global features.

[0020] The convolution sub-module comprises a convolution layer, a batch normalization layer and an activation function layer connected in sequence, and is configured to perform convolution operation on the fused features and output a feature map.

[0021] The depth separable convolution sub-module is configured to sequentially perform depth convolution and point-by-point convolution on the feature map output by the convolution sub-module to obtain a new feature map.

[0022] Optionally, sequentially performing depth convolution and point-by-point convolution on the feature map output by the convolution sub-module to obtain a new feature map comprises:

[0023] performing spatial convolution on each channel of the feature map output by the convolution sub-module to extract spatial dimension features of the feature map;

[0024] performing channel linear combination on the spatial dimension features by using 1*1 convolution to obtain a new feature map.

[0025] Optionally, the backbone network specifically comprises an INPUT module, a STAGE0 module, a STAGE1 module, a STAGE2 module, a STAGE3 module, a STAGE4 module and an OUTPUT module.

[0026] The STAGE1 module comprises one first residual block and two second residual blocks connected in sequence; the STAGE2 module comprises one first residual block and three second residual blocks connected in sequence; the STAGE3 module comprises one first residual block, four second residual blocks and one first residual block connected in sequence; and the STAGE4 module comprises one first residual block and two second residual blocks connected in sequence.

[0027] The first residual block includes a first convolutional structure, a second convolutional structure, a third convolutional structure, a fourth convolutional structure, an attention structure, and a first activation function. The feature map of the first residual block is input as the input to the first and fourth convolutional structures. The first, second, and attention structures are sequentially connected. The output of the fourth convolutional structure is connected to the input of the first activation function. The outputs of the attention structure and the first activation function are both connected to the input of the third convolutional structure. The output of the third convolutional structure is used as the output of the first residual block. In the STAGE1 and STAGE2 modules, the attention structure includes sequentially applied local attention and sliding window attention. In the STAGE3 and STAGE4 modules, the attention structure includes sequentially applied local attention and deformable attention.

[0028] The second residual block includes a fifth convolutional structure, a sixth convolutional structure, a seventh convolutional structure, and a second activation function; the feature map of the second residual block is input as the input of the fifth convolutional structure and the second activation function, the output of the fifth convolutional structure is connected to the input of the sixth convolutional structure, the outputs of the sixth convolutional structure and the second activation function are both connected to the input of the seventh convolutional structure, and the output of the seventh convolutional structure is used as the output of the second residual block;

[0029] The first convolutional structure, the second convolutional structure, the third convolutional structure, the fifth convolutional structure, and the sixth convolutional structure all include sequentially connected convolutional layers, batch normalization layers, and activation function layers; the fourth convolutional structure and the seventh convolutional structure both include sequentially connected convolutional layers and batch normalization layers.

[0030] Optionally, the deformable attention is achieved by the following formula:

[0031] ;

[0032] in, This represents the query vector for which deformable attention needs to be computed. The first feature map in the input represents the... The key vector of a dynamic sampling point This represents the learnable query projection matrix, used to project the query vector. Mapped to attention space, This represents the learnable key projection matrix, used to project key vectors. Mapped to attention space, This represents the normalization operation. This represents the projected query vector. This represents the projected key vector. denotes a fusion feature vector after deformable attention weighting, denotes the number of dynamic sampling points.

[0033] Optionally, the local attention weight is represented as:

[0034] ;

[0035] wherein, denotes the attention weight map of the th local attention head, denotes the local query vector corresponding to the th local attention head, denotes the key vector corresponding to the th local attention head, and denotes an element-wise multiplication operation, denotes a 1x1 convolution operation, denotes a 3x3 convolution operation, denotes a function operation.

[0036] In a second aspect, the present application provides a digital printing defect detection device based on a multi-head cooperative mechanism, comprising:

[0037] An acquisition module is configured to acquire a digital printing image, pre-process the digital printing image, and obtain a digital printing image slice to be detected.

[0038] A detection module is configured to input the digital printing image slice to be detected into a pre-trained digital printing defect detection model based on multi-head cooperation, and obtain a defect detection result. The digital printing defect detection model based on multi-head cooperation comprises a backbone network, an encoder module, a decoder module, and an output module.

[0039] The backbone network is configured to extract a multi-scale feature map of the digital printing image slice to be detected. The backbone network is an improved network in which multi-head attention, deep separable convolution, deformable attention, local attention, and sliding window attention are introduced into a ResNet network. The multi-head attention is applied to a STAGE0 module of the ResNet network, and is configured to capture local microscopic details and global structural information of the digital printing image slice to be detected through a multi-head attention parallel mechanism. The deep separable convolution is applied to the STAGE0 module of the ResNet network, and is configured to reduce the parameter size and computational complexity of the digital printing defect detection model. The residual blocks of a STAGE1 module and a STAGE2 module of the ResNet network both sequentially apply local attention and sliding window attention, and the residual blocks of a STAGE3 module and a STAGE4 module both sequentially apply local attention and deformable attention.

[0040] The encoder module is configured to obtain multi-scale enhanced features based on a self-attention mechanism according to the multi-scale feature maps output by the backbone network.

[0041] The decoder module is configured to output query features by using a multi-head attention mechanism according to the multi-scale enhanced features.

[0042] The output module is configured to output a defect bounding box and a class label according to the query features.

[0043] The present application provides a digital printing defect detection method based on a multi-head collaborative mechanism, which captures local microscopic details and global structural information of a digital printing image through parallel local attention heads and global attention heads, significantly improving the sensitivity of the digital printing defect detection model based on multi-head collaboration to fine textures and global structures; by integrating local attention and sliding window attention in the residual blocks of the STAGE1 module and the STAGE2 module of the ResNet network, and integrating local attention and deformable attention in the residual blocks of the STAGE3 module and the STAGE4 module, the digital printing defect detection model based on multi-head collaboration can accurately capture the local texture features of microscopic defects such as nozzle ink breakage and fine stains, especially through deformable attention to achieve precise attention and reinforcement of key defect areas, thereby significantly enhancing the adaptability and robustness of the digital printing defect detection model based on multi-head collaboration in complex background interference, irregular pattern deformation and other scenarios, significantly reducing the false detection rate and the missed detection rate, and improving the detection accuracy; in addition, the application of depth separable convolution effectively reduces the computational overhead and the number of parameters, improves the model inference speed, and meets the lightweight requirements of high-frequency, real-time defect detection on the digital printing production line.

[0044] The present application will be further described in detail below with reference to the accompanying drawings and the present application. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of a digital printing defect detection method based on a multi-head collaborative mechanism provided by an embodiment of the present application;

[0046] Figure 2 is a structural diagram of a backbone network in a digital printing defect detection method based on a multi-head collaborative mechanism provided by an embodiment of the present application;

[0047] Figure 3 is a structural diagram of a STAGE0 module-STAGE4 module in a backbone network of a digital printing defect detection method based on a multi-head collaborative mechanism provided by an embodiment of the present application;

[0048] Figure 4is a structural schematic diagram of a first residual block and a second residual block in a backbone network of a digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application.

[0049] Figure 5 is a structural schematic diagram of a multi-head attention sub-module in a digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application.

[0050] Figure 6 is a schematic diagram of a depth separable convolution in a digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application.

[0051] Figure 7 is a schematic diagram of an attention structure of a digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application.

[0052] Figure 8 is an effect comparison diagram of a digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application and an existing commonly used defect detection method. DETAILED DESCRIPTION

[0053] The present application will be further described in detail below in combination with specific embodiments, but the embodiments of the present application are not limited thereto.

[0054] In order to overcome the problems of high false detection rate, serious missed detection phenomenon and insufficient multi-scale defect adaptation ability of the existing scheme, and reduce the calculation cost and the number of parameters, and meet the lightweight demand of high-frequency and real-time defect detection on the digital printing production line, the present application provides a digital printing defect detection method and device based on a multi-head cooperative mechanism.

[0055] First, the digital printing defect detection method based on a multi-head cooperative mechanism provided by the embodiment of the present application is described in detail, referring to Figure 1 The method comprises the following steps:

[0056] S10, acquiring a digital printing image and pre-processing the digital printing image to obtain a to-be-detected digital printing image slice.

[0057] Specifically, a Balser industrial camera or other means is used to collect high-resolution images of the digital printed fabric, ensuring the completeness of the details and the degree of color restoration of the collected digital printed images. Then, the collected digital printed images are systematically preprocessed, including but not limited to: eliminating pixel fluctuations caused by light intensity, exposure differences, etc. in the digital printed images through pixel-level normalization processing; performing color space conversion and format standardization processing on the collected digital printed images to ensure the consistency of data collected by different batches and different devices, providing high-quality input for subsequent digital printed defect detection. Then, the preprocessed digital printed images are sliced to obtain digital printed image slices to be detected.

[0058] S20, inputting the digital printed image slice to be detected into the multi-head cooperative based digital printed defect detection model pre-trained to obtain a defect detection result.

[0059] Here, the multi-head cooperative based digital printed defect detection model includes a backbone network, an encoder module, a decoder module, and an output module. The backbone network is used to extract multi-scale feature maps of the digital printed image slice to be detected. The backbone network is an improved network in which multi-head attention, depthwise separable convolution, deformable attention, local attention, and shift-window attention are introduced into the ResNet network. The multi-head attention is applied to the STAGE0 module of the ResNet network, which is used to capture local microscopic details and global structural information of the digital printed image slice to be detected through the multi-head attention parallel mechanism. The depthwise separable convolution is applied to the STAGE0 module of the ResNet network, specifically to the last of the STAGE0 module, which is used to reduce the parameter size and computational complexity of the digital printed defect detection model. The local attention and the shift-window attention are sequentially applied to the residual blocks of the STAGE1 module and the STAGE2 module of the ResNet network, and the local attention and the deformable attention are sequentially applied to the residual blocks of the STAGE3 module and the STAGE4 module of the ResNet network.

[0060] The encoder module is used to obtain multi-scale enhanced features based on the self-attention mechanism according to the multi-scale feature maps output by the backbone network.

[0061] The decoder module is used to output query features using the multi-head attention mechanism according to the multi-scale enhanced features.

[0062] The output module is used to output the defect bounding box and the class label according to the query features.

[0063] Specifically, the digital printing defect detection model is used to detect defects existing in the digital printing image and determine the defect category, so that the digital printing image to be detected is cut into slices and input into the digital printing defect detection model, and the digital printing defect detection model can output the position of the defect existing in the digital printing image and the defect category, that is, the defect position is framed in the digital printing image by using the defect bounding box, and the category of the defect is identified, that is, the defect detection result.

[0064] Next, each part of the digital printing defect detection model based on multi-head cooperation will be described in detail.

[0065] 1. Backbone network:

[0066] Specifically, after the digital printing image to be detected is input into the backbone network, the backbone network extracts four different scales of features of the digital printing image to be detected, and outputs them to the encoder module for subsequent defect detection.

[0067] In the present application, the backbone network is an improved network in which multi-head attention, depthwise separable convolution, deformable attention, local attention and sliding window attention are introduced into the ResNet network. For example, the ResNet network can be a ResNet50 network. Taking the ResNet50 network as an example, as shown in Figure 2 The backbone network includes an INPUT module, a STAGE0 module, a STAGE1 module, a STAGE2 module, a STAGE3 module, a STAGE4 module and an OUTPUT module.

[0068] The INPUT module is used to perform basic standardization operations such as size adaptation and pixel value processing on the input digital printing image to be detected, so as to ensure that the data format, scale and distribution are uniform.

[0069] The STAGE0 module is used to perform preliminary feature extraction and down-sampling on the digital printing image to be detected output by the INPUT module, and increase the number of channels to prepare for subsequent processing. Referring to Figure 3 The STAGE0 module includes a multi-head attention (MHA) submodule, a convolution submodule and a depthwise separable convolution (Depthwise Separable Conv) submodule connected in sequence.

[0070] Figure 5A schematic diagram of the multi-head attention submodule is shown, where the specific structures of a single global attention head (right) and a single local attention head (left) are exemplarily illustrated within the dashed boxes. This multi-head attention submodule is used to extract local features of the digital print image slice to be detected using the local attention head and calculate local attention weights, extract global features of the digital print image slice to be detected using the global attention head and calculate global attention weights, generate normalized weights based on the local and global attention weights, and calculate fused features based on the normalized weights, local features, and global features.

[0071] In this embodiment, the multi-head attention submodule includes multiple local attention heads and multiple global attention heads operating in parallel. See also... Figure 5 In the local branch, each local attention head generates a basic feature representation by performing initial channel adjustment through a 1×1 convolution (CONV(1×1)). Then, the basic feature representation is enhanced at multiple scales through a 5×5 convolutional layer (CONV(5×5)) and a 3×3 convolutional layer (CONV(3×3)) set layer by layer, thereby expanding the receptive field and focusing on the small and complex defects in the digital printing pattern. Then, the multi-scale enhanced features are input into the activation mapping module. In the activation mapping module, a local query vector, a local key vector, and a local value vector are generated through a 1×1 convolution (CONV(1×1)), and the local attention weight corresponding to each local attention head is calculated.

[0072] For example, the calculation method for local attention weights can be expressed as follows:

[0073] ;

[0074] in, Indicates the first Attention weight map of each local attention head. Indicates the first The local query vector corresponding to each local attention head. Indicates the first The key vectors corresponding to the local attention heads, where ⊙ represents element-wise multiplication. This represents a 1×1 convolution operation. This represents a 3×3 convolution operation. express Function operations.

[0075] See Figure 5In the global branch, a lightweight windowed self-attention mechanism is used to model long-range dependencies. Specifically, the input digital printing image slice to be detected is first adjusted in channel number and global feature map is extracted through 1x1 convolution (CONV (1x1)), and then the global feature map is windowed and enhanced, that is, the global feature map is divided into several windows (such as non-overlapping 4x4 windows), and each window is averaged to compress the spatial size and extract the global context. Then, the global feature map after windowed and enhanced is subjected to 1x1 convolution (CONV (1x1)) to generate the corresponding global query vector, global key vector and global value vector, and the global attention weight of each global attention head is calculated. It can be understood that the global attention weight can be upsampled to have the same size as the local attention weight. Here, the global attention weight uses the same calculation method as the local attention weight, so it is not described again.

[0076] Then, the obtained multiple local attention weights and multiple global attention weights are spliced in the channel dimension, and the weight of each spatial position is independently normalized by the softmax module to generate a normalized weight, realizing the adaptive matching of local and global weights. Next, according to the normalized weight, the local value vector and the global value vector, the weighted fusion feature is calculated, and then two layers of 1x1 convolution (CONV (1x1)) are used for channel mapping and regularization processing to enhance the robustness of the digital printing defect detection model and suppress overfitting, and finally the fusion feature is obtained.

[0077] In the present application, the multi-head attention sub-module enhances the features of the digital printing image slice to be detected output by the INPUT module. The multi-head attention sub-module uses a multi-head attention parallel mechanism to aggregate information in multiple scales and multiple perspectives in spatial and channel dimensions, and significantly improves the sensitivity of the digital printing defect detection model to small textures and global structures through dynamic allocation of local and global weights. Specifically, the local branch in the multi-head attention sub-module focuses on capturing local defects such as detail changes, texture cracks and color point pollution in the digital printing image slice to be detected, which can significantly enhance the recognition ability of the digital printing defect detection model in low-level detail textures. The global branch is committed to modeling the overall structure, long-range dependencies and context information in the digital printing image slice to be detected, which is beneficial to identifying large-scale defects such as pattern misplacement and fuzzy boundary. Through this way of cooperative modeling of local attention and global attention, the expression ability of the input features can be effectively enhanced, so that the digital printing defect detection model has both fine-grained perception ability and macro-structure understanding ability, thereby significantly enhancing the recognition ability of the digital printing defect detection model to multi-scale and multi-type defects in complex printing backgrounds.

[0078] Referring to Figure 3The convolution submodule includes a convolution layer (CONV(7x7)), a batch normalization layer (BN), and an activation function layer (RELU) connected in sequence, and is configured to perform convolution operation on the fused features and output a feature map.

[0079] Specifically, the convolution submodule is configured to reduce the resolution of the fused features output by the multi-head attention submodule and increase the number of channels, so as to prepare for subsequent processing. For example, the convolution kernel window size of the convolution layer is 7x7, the sliding step is 2, the output channel number is 64, and the activation function layer uses a ReLu function.

[0080] The depthwise separable convolution submodule is configured to sequentially perform depthwise convolution and pointwise convolution on the feature map output by the convolution submodule to obtain a new feature map.

[0081] Figure 6 A schematic diagram showing that the standard convolution is decomposed into depthwise convolution and pointwise convolution is shown, where the upper part is the standard convolution operation, the lower left part is the depthwise convolution operation in the depthwise separable convolution, and the lower right part is the pointwise convolution operation in the depthwise separable convolution. Here, the depthwise separable convolution submodule is used to decompose the standard convolution operation into two stages of depthwise convolution and pointwise convolution, respectively, to complete spatial feature extraction and channel fusion, thereby realizing decoupling processing of spatial modeling and channel fusion. This design significantly reduces the amount of calculation and the number of parameters while maintaining good expression capability for texture features in the input feature map. Specifically, the depthwise separable convolution submodule performs depthwise convolution and pointwise convolution on the feature map output by the convolution submodule to obtain a new feature map, including: first, performing spatial convolution on each channel of the feature map output by the convolution submodule to extract spatial dimension features of the feature map; and second, using 1x1 convolution to linearly combine the spatial dimension features to obtain a new feature map.

[0082] For example, the depthwise convolution separately performs spatial convolution on each input channel of the input feature map to extract spatial dimension features, where the size of the convolution kernel is 3x3, and each convolution kernel only acts on one channel. No interaction between different channels is involved, so the amount of calculation of the depthwise convolution is: wherein represents the height of the input feature map, represents the width of the input feature map, represents the number of channels of the input feature map. The pointwise convolution uses a 1x1 convolution kernel to linearly combine the spatial dimension features output by the depthwise convolution in the channel dimension to obtain a new feature map. Here, the pointwise convolution is responsible for information interaction between channels and does not change the spatial resolution, and the amount of calculation thereof is wherein represents the output channel number.

[0083] It can be understood that, referring to Figure 6 , when the input feature map is processed by the standard convolution, the spatial dimension and the channel dimension are fused at the same time, that is, the feature of each output channel is the result of summing the multiplication of all input channels and the corresponding convolution kernel elements, and the calculation amount is Therefore, compared with the standard convolution, the calculation amount of the depth separable convolution is reduced by about Therefore, the application decouples the spatial convolution and the channel convolution by the depth separable convolution, not only reduces the resource consumption in the feature extraction process, greatly reduces the parameter size and the calculation complexity, ensures the efficient primary feature extraction, but also ensures the sufficient reservation of the spatial information and the channel feature of the input feature map, so as to realize the significant improvement of the detection speed on the basis of maintaining the detection accuracy, and make the application more suitable for the lightweight demand of the high-frequency, low-delay defect detection system in the digital printing industry.

[0084] Continuing to refer to Figure 2 and Figure 3 , the STAGE1 module, the STAGE2 module, the STAGE3 module and the STAGE4 module in the backbone network are sequentially connected and used for extracting features of four different scales, and the specific structure is as follows: the STAGE1 module includes one first residual block BTNK1 and two second residual blocks BTNK2 which are sequentially connected; the STAGE2 module includes one first residual block BTNK1 and three second residual blocks BTNK2 which are sequentially connected; the STAGE3 module includes one first residual block BTNK1, four second residual blocks BTNK2 and one first residual block BTNK1 which are sequentially connected; and the STAGE4 module includes one first residual block BTNK1 and two second residual blocks BTNK2 which are sequentially connected.

[0085] Among them, referring to Figure 4 , the first residual block BTNK1 includes a first convolution structure, a second convolution structure, a third convolution structure, a fourth convolution structure, an attention structure and a first activation function. Wherein, the feature map input into the first residual block BTNK1 is taken as the input of the first convolution structure and the fourth convolution structure, the first convolution structure, the second convolution structure and the attention structure are sequentially connected, the output of the fourth convolution structure is connected to the input of the first activation function, the output of the attention structure and the first activation function are both connected to the input of the third convolution structure, and the output of the third convolution structure is taken as the output of the first residual block BTNK1. In the STAGE1 module and the STAGE2 module, the attention structure includes local attention and sliding window attention which are applied in sequence, and in the STAGE3 module and the STAGE4 module, the attention structure includes local attention and deformable attention which are applied in sequence.

[0086] Referring to Figure 4 The second residual block BTNK2 includes a fifth convolutional structure, a sixth convolutional structure, a seventh convolutional structure, and a second activation function. The feature map input to the second residual block BTNK2 is input to the fifth convolutional structure and the second activation function, the output of the fifth convolutional structure is connected to the input of the sixth convolutional structure, the outputs of the sixth convolutional structure and the second activation function are both connected to the input of the seventh convolutional structure, and the output of the seventh convolutional structure is the output of the second residual block.

[0087] Here, the first convolutional structure, the second convolutional structure, the third convolutional structure, the fifth convolutional structure, and the sixth convolutional structure each include a convolutional layer, a batch normalization layer (BN), and an activation function layer connected in sequence, and the fourth convolutional structure and the seventh convolutional structure each include a convolutional layer and a batch normalization layer (BN) connected in sequence.

[0088] For example, in the first convolutional structure, the convolutional kernel window size of the convolutional layer is 1x1, the sliding step is S, the output channel number is C1, and the activation function layer uses the RELU function, where C1 is the channel number of the feature map input to the first convolutional structure, and S is a variable parameter used to control whether each convolutional structure performs down-sampling on the input feature map. In the STAGE1 module, S=1, i.e., the first convolutional structure does not perform down-sampling on the input feature map, and in the STAGE2-STAGE4 module, S=2, i.e., the first convolutional structure performs down-sampling on the input feature map. In the second convolutional structure, the convolutional kernel window size of the convolutional layer is 3x3, the sliding step is 1, the output channel number is C1, and the activation function layer uses the RELU function. In the third convolutional structure, the convolutional kernel window size of the convolutional layer is 3x3, the sliding step is 1, the output channel number is C1x4, and the activation function layer uses the RELU function. In the fourth convolutional structure, the convolutional kernel window size of the convolutional layer is 1x1, the sliding step is S, and the output channel number is C1, where S is the same as S in the first convolutional structure. In the fifth convolutional structure, the convolutional kernel window size of the convolutional layer is 1x1, the sliding step is 1, the output channel number is C1 / 4, and the activation function layer uses the RELU function. In the sixth convolutional structure, the convolutional kernel window size of the convolutional layer is 3x3, the sliding step is 1, the output channel number is C1 / 4, and the activation function layer uses the RELU function. In the seventh convolutional structure, the convolutional kernel window size of the convolutional layer is 1x1, the sliding step is 1, and the output channel number is C. Specifically, in the STAGE1 module, C=C1, and in the STAGE2-STAGE4 module, C is twice C1.

[0089] The present application introduces an attention structure in the first residual block BTNK1, referring to Figure 7 , Figure 7The specific structure of the attention structure in the STAGE1 module-STAGE4 module is shown. In each attention structure, the two-dimensional feature map is first divided into fixed-size image blocks by the image block embedding module, and is converted into a uniform-dimensional embedding representation by linear mapping. This structured processing helps the attention structure to efficiently model the local and global dependency in the spatial and channel dimensions, while maintaining the integrity and consistency of the feature representation.

[0090] In the STAGE1 module and the STAGE2 module, the digital printing defect detection model of the application introduces an enhanced module composed of fused local attention and sliding window attention, wherein the local attention is realized by fixed neighborhood attention, so that the backbone network can accurately capture the local texture features of micro defects such as nozzle ink breakage and small stains, and the sliding window attention realizes continuous perception of local information by sliding operation of the window on the input feature map, thereby relieving the problem of receptive field fragmentation caused by fixed window boundaries and improving the coherence and integrity of spatial perception. The reason for using joint modeling of local attention and sliding window attention at this stage is that the resolution of the input feature map is higher at this time, which is more suitable for fine modeling of fine-grained textures. Through the cooperation of local perception and continuous modeling, the recognition ability of the digital printing defect detection model for micro defects can be effectively improved. Here, the way of extracting local features of the input feature map by local attention and processing the local features by sliding window attention can be referred to related prior art, which will not be described here.

[0091] In the attention structure of the STAGE3 module and the STAGE4 module, a composite mechanism combining local attention and deformable attention is introduced, wherein the local attention is also used to improve the sensitivity of the digital printing defect detection model to detailed textures, and the deformable attention is a dynamic adaptive attention mechanism that breaks through the fixed sampling mode of traditional attention mechanisms, dynamically generates a sampling offset according to the input feature map, so that the query vector in the deformable attention only focuses on a small number of key regions in the input feature map rather than the global or fixed window, and can adaptively focus on irregular defect regions, thereby effectively dealing with macro-structure abnormalities such as pattern distortion, boundary misplacement, and line breakage, and realizing accurate attention and strengthening of key defect regions. Moreover, the deformable attention can dynamically adjust the feature extraction range, improve the detection capability of the digital printing defect detection model in complex scenes and deformed targets, thereby significantly improving the detection accuracy and reducing the false detection rate. Here, the way of extracting local features of the input feature map by local attention and processing the local features by deformable attention can be referred to related prior art, which will not be described here.

[0092] Exemplarily, the deformable attention is realized by the following formula:

[0093] ;

[0094] in, This represents the query vector for which deformable attention needs to be computed. The first feature in the input feature map The key vector of each dynamic sampling point This represents the learnable query projection matrix, used to project the query vector. Mapped to attention space, This represents the learnable key projection matrix, used to project key vectors. Mapped to attention space, This represents the normalization operation. This represents the projected query vector. This represents the projected key vector. This represents the fused feature vector after applying deformable attention weighting. This indicates the number of dynamic sampling points.

[0095] In practice, digital printing often suffers from defects such as blurred breaks, local misalignment, pattern distortion, and broken lines due to changes in printing tension or dye diffusion during the digital printing production process. These defects are often irregular in shape and have unclear boundaries. The digital printing defect detection model of this invention introduces an attention structure, which enables the digital printing defect detection model to have the ability to perceive micro-details and understand macro-structures at different levels, thereby significantly improving the robustness of the digital printing defect detection model in scenarios such as small-sized targets, complex background interference, and blurred boundary transitions.

[0096] 2. Encoder module:

[0097] Specifically, the multi-scale feature maps extracted by the backbone network are input to the encoder module, which is a Transformer-based encoder. This module establishes long-distance dependencies between features through a self-attention mechanism, capturing cross-regional topological relationships and complex spatial structure information in digital printing images. It coordinates shallow details (such as edges and textures) with deep semantics (defect categories / distribution patterns), enhancing the multi-scale integration capability of the digital printing defect detection model for defect distribution features. This outputs multi-scale enhanced features, significantly improving the detection accuracy of heterogeneous defects such as point defects, linear cracks, and regional stains, while simultaneously optimizing the detection rate of small-scale defects and large-scale structural anomalies. The encoder module comprises multiple encoder layers, each implemented using a self-attention layer, a feedforward neural network, residual connections, and layer normalization. Specific implementation details can be found in existing technologies and will not be elaborated here.

[0098] 3. Decoder module:

[0099] Specifically, the decoder module is a Transformer-based decoder, which is responsible for converting the multi-scale enhanced features output by the encoder module into refined query features. Here, a set of learnable defect query vectors are adopted in the decoder module, which represent the digital textile defect detection model's preliminary assumptions about potential defect instances. Each defect query vector interacts with the multi-scale enhanced features output by the encoder through a multi-head attention mechanism during the decoding process, achieving dynamic focusing and feature fusion of potential defect regions. Specifically, the defect query vectors first exchange information with each other through self-attention mechanisms to enhance the contextual relevance between the defect query vectors, thereby suppressing redundant detection and modeling the relationship between defects. Then, through cross-attention mechanisms, the defect query vectors are weighted and fused with the multi-scale enhanced features of the encoder, focusing on key defect regions and capturing key features of defects in space and semantics. Finally, the encoder module outputs query features rich in semantic and positional information.

[0100] Compared with traditional candidate box-based detection methods, the defect query vector mechanism of the encoder module avoids complex processes such as region proposal and subsequent non-maximum suppression (NMS), simplifying the detection process. At the same time, the multi-head attention mechanism in the encoder module ensures that the digital textile defect detection model can efficiently capture the long-distance dependency relationships and detailed information of diverse defects from the multi-scale enhanced features output by the encoder, thereby significantly improving the accuracy and inference speed of defect detection. This end-to-end design not only improves the structural simplicity of the digital textile defect detection model, but also greatly enhances the real-time response capability and stability of the system in an industrial environment.

[0101] Here, the decoder module includes multiple decoder layers, each of which is implemented by a self-attention layer, a cross-attention layer, a feedforward network layer, and a residual connection and layer normalization. For specific implementation, please refer to related prior art, which will not be described here.

[0102] 4. Output module:

[0103] Specifically, the output module outputs specific detection results, including defect bounding boxes and class labels, based on the query features rich in semantic and positional information output by the decoder module. Here, the output module includes a classification head and a regression head, and their specific implementation can be found in related prior art, which will not be described here.

[0104] Based on the structure of the digital textile defect detection model shown above, the training set is used to train it, and a trained multi-head collaborative digital textile defect detection model is obtained.

[0105] Specifically, the digital printing defect detection model of the present application is trained and tested on a self-constructed digital printing defect detection dataset (Digitally Printed Fabric Defect Dataset). The dataset contains eight common digital printing defects, including water marks, wire marks, ink leakage, lint, ghosting, slurry spots, track residues, and missing printing, covering typical problems in actual industrial scenarios such as small targets, low contrast, and complex backgrounds. The dataset is accurately labeled using the image labeling software LabelImg, and the labeled information includes the coordinates of the defect bounding box and the defect category. During the training process, the class loss function and the regression loss function are used to constrain the model training process, and the training is ended when the loss value of the loss function reaches the convergence state, and the trained digital printing defect detection model based on multi-head collaboration is obtained.

[0106] The digital printing defect detection method based on the multi-head collaboration mechanism provided by the present application is further described below through experiments.

[0107] The training and evaluation of the digital printing defect detection model based on multi-head collaboration in the present application are completed under the following configurations, including: the processor is Intel Core i7-13700K, the main frequency is 3.40GHz, the memory is 64GB, the graphics card is NVIDIA GeForce RTX 4090, the video memory is 24GB, and the deep learning framework uses PyTorch. In the experimental process, the hyperparameters of the model training are set as follows: the training period (Epochs) is 12, the initial learning rate is 0.0001, and the batch size (Batch Size) is 4. The digital printing defect detection model based on multi-head collaboration in the present application is trained and tested on a self-constructed digital printing defect detection dataset (Digitally Printed Fabric Defect Dataset). The dataset contains eight common digital printing defects, including water marks, wire marks, ink leakage, lint, ghosting, slurry spots, track residues, and missing printing, covering typical problems in actual industrial scenarios such as small targets, low contrast, and complex backgrounds. The dataset is divided into training set, validation set and test set according to the ratio of 8:1:1, and is accurately labeled using LabelImg to ensure the quality and repeatability of the model training. Here, LabelImg is an open-source graphical image labeling tool mainly used for labeling image data for computer vision tasks (such as target detection, image classification, etc.).

[0108] Firstly, the contribution of the introduced multi-head attention sub-module (i.e. MHA sub-module) and attention structure to the performance of the multi-head cooperative based digital printing defect detection model provided by the application is verified through a series of ablation experiments, and the experimental results are shown in Table 1. The base model specifically consists of a backbone network, an encoder module, a decoder module and an output module, wherein the backbone network is an unimproved ResNet network. MHA represents that the multi-head attention is introduced in the STAGE0 module of the ResNet network of the base model, and DF represents that the attention structure is further introduced in the STAGE1 module-STAGE4 module of the base model. As can be seen from Table 1, under the premise of keeping the base digital printing defect detection model and other parameters unchanged, the base model reaches 91.6% and 66.4% in mAP50 (average precision mean of IoU threshold value of 0.5) and mAP50:95 (average precision mean of IoU threshold value of 0.5 to 0.95) indicators respectively; when MHA is introduced alone, the mAP50 of the model is improved to 93.0%, and the mAP50:95 is improved to 69.1%, which shows that the introduction of multi-head attention can significantly enhance the recognition ability of the digital printing defect detection model in complex printing background for multi-scale and multi-type defects; when DF is introduced alone, the mAP50 is improved to 93.3%, and the mAP50:95 is improved to 69.4%, which shows that the introduction of attention structure can improve the focusing ability of the digital printing defect detection model for key regions in complex background; when MHA and DF are introduced at the same time, the performance of the digital printing defect detection model reaches the best, the mAP50 is improved to 94.2%, and the mAP50:95 is improved to 70.5%, therefore, through the synergistic effect of the introduced multi-head attention sub-module and attention structure, the global and local feature modeling capability is significantly enhanced, the false detection and missed detection rate is significantly reduced, and especially the performance is more stable and accurate in small target and medium target detection.

[0109] Table 1. Contribution of multi-head attention sub-module and attention structure to the performance of the application

[0110]

[0111] Secondly, the performance of introducing different attention mechanisms in the base model is analyzed, the introduced attention mechanisms include squeeze-excitation module (Squeeze-and-Excitation, SE), convolutional block attention module (Convolutional Block Attention Module, CBAM), non-local attention (Non-Local Attention, NLA) and standard attention (Standard Attention, SA). As shown in Table 2, wherein mAP represents the average precision mean, mAP75 represents the average precision mean of the IoU threshold of 0.75, mAPS represents the average precision mean of small targets, mAPM represents the average precision mean of medium targets, and mAPL represents the average precision mean of large targets. Here, mAP, mAPS, mAPM and mAPL are the average precision means of the IoU threshold of 0.5 to 0.95. As shown in Table 2, the improvement effect of these attention mechanisms in digital printing defect detection is limited, especially in small target detection. For example, when SE and NLA are introduced, the mAPS of the digital printing defect detection model not only does not improve, but also decreases. This is because these attention mechanisms use static channel or spatial attention methods, which lack dynamic adaptability to target shape, scale and position changes, making it difficult to handle dense, small and boundary fuzzy defect patterns in real industrial images. In contrast, the multi-head attention sub-module introduced in the present application realizes efficient modeling of multi-scale context by parallelly fusing local attention and global attention, so that the model can not only capture detailed textures, but also understand global image semantics. Moreover, the attention structure introduced in the present application adopts a learnable offset mechanism in the deformable attention, which can dynamically adjust the attention sampling position, so that the digital printing defect detection model can more flexibly focus on the defect area and improve the response ability to target spatial changes. Experimental results show that by introducing the multi-head attention sub-module and the attention structure, the digital printing defect detection model has significant improvement in multiple key indicators such as mAP50, mAP75 and mAPS. Especially when facing small defects in complex backgrounds, it shows higher recognition accuracy and detection stability.

[0112] Table 2 Performance analysis of introducing different attention mechanisms in the base model

[0113]

[0114] After that, in order to evaluate the actual effect of depth separable convolution on model compression and acceleration, the performance of standard convolution and depth separable convolution in total parameter amount, single calculation floating point operation amount (FLOPs) and memory usage in the inference stage is analyzed. The experimental results are shown in Table 3. As can be seen from Table 3, in terms of total parameter amount, the total parameter amount of depth separable convolution is 0.0135G, which is reduced by about 42.5% compared with 0.0235G of standard convolution, indicating that it has higher model compression efficiency. In terms of calculation complexity, the single calculation floating point operation amount of depth separable convolution is 20.07G, which is reduced by about 40% compared with 33.41G of standard convolution, significantly reducing the operation overhead of the model and being conducive to improving the inference speed. The storage space occupied by the parameters of depth separable convolution is only 0.05GB, further verifying its advantage in model lightweight.

[0115]

[0116] In addition, the method of the present application is compared with the commonly used digital printing defect detection model, which includes: Cascade Region-based Convolutional Neural Network (Cascade R-CNN), Faster Region-based Convolutional Neural Network (Faster R-CNN), Collaborative DETR model (Co-DETR), Aligned DETR model (Align-DETR), YOLOv11 model and Diffusion-based Object Detection (DiffusionDet), and the comparison results are shown in Table 4. The present application also shows significant advantages in mAP, mAP50, mAP75 and mAPM, which are 65.9%, 94.2%, 70.5% and 53.9% respectively, which are better than other comparison models. Among them, although the Faster R-CNN model has a slight advantage in mAPS, its overall accuracy is still significantly lower than the present application. Co-DETR performs well in large size target detection (i.e. high mAPL value), but has a missed detection problem in small and medium size target scenarios. The present application performs balanced in mAPS, mAPM and mAPL three scale dimensions, which further illustrates the strong generalization ability and robustness of the present application in complex background and multi-scale target scenarios. In general, the experimental results fully prove the effectiveness and practical value of the present application in improving the accuracy of digital printing defect detection, reducing the false detection rate and adapting to complex industrial scenarios.

[0117]

[0118] Finally, the detection effect of the method of the present application and the commonly used digital printing defect detection model is compared, see Figure 8 OURS is the method of the present application, Figure 8 FP represents false positive detection points (i.e. false detection points), and FN represents false negative detection points (i.e. missed detection points), which are Figure 8 It can be seen that different digital printing defect detection models have significant differences in complex digital printing images. The traditional Faster Region-based Convolutional Neural Network model (i.e. Faster RCNN in Figure 8 ) and Cascade Region-based Convolutional Neural Network model (i.e. Cascade R-CNN in Figure 8The Cascaded RCNN in the related art is prone to interference in a complex texture background, and false detection and missed detection phenomena exist in the detection result to varying degrees. Especially when dealing with fuzzy contours, low contrast or small size defects, problems such as unclear boundary positioning and complete missed detection often occur, which is mainly due to the difficulty of the structure based on fixed receptive field and candidate region generation to adapt to the high-density and high-complexity defect distribution in the print pattern. In contrast, the Align-DETR model based on the Transformer architecture has certain advantages in global feature modeling, but it still produces redundant detection boxes or false detection in images with fuzzy defect edges or strong background noise. However, the method of the present application exhibits stronger robustness and generalization ability in the above typical difficult scenarios, can more accurately identify small targets and edge fuzzy defect regions, effectively suppress background interference and reduce false detection and missed detection phenomena. In particular, in images with multiple small defect targets, the method of the present application still maintains clear and accurate positioning effect, further verifying its application potential and practicality in industrial defect detection tasks.

[0119] The present application proposes a digital printing defect detection method based on a multi-head cooperative mechanism. Through parallel local attention heads and global attention heads, local microscopic details and global structural information of digital printing images are captured, and through dynamic allocation of the obtained local attention weights and global attention weights, the sensitivity of the digital printing defect detection model based on multi-head cooperation to small textures and global structures is significantly improved. By incorporating local attention and sliding window attention in the residual blocks of the STAGE1 module and the STAGE2 module of the ResNet network, and incorporating local attention and deformable attention in the residual blocks of the STAGE3 module and the STAGE4 module, the digital printing defect detection model based on multi-head cooperation can accurately capture the local texture features of microscopic defects such as ink jet breakage and small stains, especially through deformable attention to achieve precise attention and reinforcement of key defect areas, thereby significantly enhancing the adaptability and robustness of the digital printing defect detection model based on multi-head cooperation in complex background interference, irregular pattern deformation and other scenarios, significantly reducing the false detection rate and the missed detection rate, and improving the detection accuracy. In addition, the application of depth separable convolution structure effectively reduces the computational overhead and the number of parameters, improves the model inference speed, and meets the lightweight requirements of high-frequency and real-time defect detection on digital printing production lines.

[0120] Corresponding to the digital printing defect detection method based on the multi-head cooperative mechanism described above, the present application embodiment further provides a digital printing defect detection device based on the multi-head cooperative mechanism, which can include:

[0121] The acquisition module is configured to acquire a digital printing image, pre-process the digital printing image, and obtain a digital printing image slice to be detected.

[0122] a detection module configured to input a to-be-detected digital printing image slice into a multi-head collaboration-based digital printing defect detection model that has been pre-trained to obtain a defect detection result; the multi-head collaboration-based digital printing defect detection model comprises a backbone network, an encoder module, a decoder module, and an output module;

[0123] The backbone network is configured to extract a multi-scale feature map of the to-be-detected digital printing image slice; the backbone network is an improved network in which multi-head attention, deep separable convolution, deformable attention, local attention, and sliding window attention are introduced into a ResNet network; the multi-head attention is applied to a STAGE0 module of the ResNet network, and is configured to capture local microscopic details and global structural information of the to-be-detected digital printing image slice through a multi-head attention parallel mechanism; the deep separable convolution is applied to the STAGE0 module of the ResNet network, and is configured to reduce the parameter size and computational complexity of the digital printing defect detection model; the local attention and the sliding window attention are sequentially applied to residual blocks of STAGE1 and STAGE2 modules of the ResNet network; and the local attention and the deformable attention are sequentially applied to residual blocks of STAGE3 and STAGE4 modules of the ResNet network.

[0124] The encoder module is configured to obtain a multi-scale enhanced feature based on a self-attention mechanism according to the multi-scale feature map output by the backbone network;

[0125] The decoder module is configured to output a query feature by using a multi-head attention mechanism according to the multi-scale enhanced feature;

[0126] The output module is configured to output a defect bounding box and a class label according to the query feature.

[0127] The application provides a digital printing defect detection device based on a multi-head cooperative mechanism, which captures local microscopic details and global structure information of a digital printing image through parallel local attention heads and global attention heads, and significantly improves the sensitivity of a digital printing defect detection model based on multi-head cooperation to small textures and global structures through dynamic distribution of obtained local attention weights and global attention weights; the digital printing defect detection model based on multi-head cooperation can accurately capture the local texture characteristics of microscopic defects such as nozzle ink breakage and small stains, especially through deformable attention to accurately focus on and strengthen key defect areas, thereby significantly enhancing the adaptability and robustness of the digital printing defect detection model based on multi-head cooperation in complex background interference, irregular pattern deformation and other scenes, significantly reducing the false detection rate and the missed detection rate, and improving the detection accuracy; in addition, the application of depth separable convolution structure effectively reduces the calculation overhead and the number of parameters, improves the model inference speed, and meets the lightweight demand of high-frequency and real-time defect detection on a digital printing production line.

[0128] It should be noted that the device is basically similar to the method embodiment, so the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0129] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0130] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present application.

[0131] Although the present application has been described in connection with certain embodiments, persons skilled in the art will understand and appreciate that many modifications can be made to the described embodiments and these modifications can be made without departing from the scope of the application in its broader aspects. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality, and the

[0132] The above description is further detailed in connection with specific preferred embodiments of the application, and it is not to be construed that the specific implementation of the application is limited to these descriptions. For those skilled in the art, without departing from the concept of the application, a number of simple deductions or substitutions can be made, and all of these should be considered as falling within the scope of protection of the application.

Claims

1. A digital printing defect detection method based on a multi-head collaborative mechanism, characterized in that, include: A digital printing image is acquired and preprocessed to obtain a slice of the digital printing image to be detected. The digital printing image slice to be detected is input into a pre-trained digital printing defect detection model based on multi-head collaboration to obtain the defect detection result; the digital printing defect detection model based on multi-head collaboration includes a backbone network, an encoder module, a decoder module and an output module. The backbone network is used to extract multi-scale feature maps of the digital printing image slices to be detected. The backbone network is an improved ResNet network that incorporates multi-head attention, depthwise separable convolution, deformable attention, local attention, and sliding window attention. Multi-head attention is applied to the STAGE0 module of the ResNet network to capture local microscopic details and global structural information of the digital print image slices to be detected through a parallel multi-head attention mechanism. Depthwise separable convolution is applied to the STAGE0 module of the ResNet network to reduce the parameter size and computational complexity of the digital print defect detection model. Local attention and sliding window attention are applied sequentially in the residual blocks of the STAGE1 and STAGE2 modules of the ResNet network, and local attention and deformable attention are applied sequentially in the residual blocks of the STAGE3 and STAGE4 modules of the ResNet network. The STAGE0 module includes a multi-head attention submodule, a convolution submodule, and a depth-separable convolution submodule connected in sequence. The multi-head attention submodule is used to extract local features of the digital print image slice to be detected and calculate local attention weights through a local attention head, extract global features of the digital print image slice to be detected and calculate global attention weights through a global attention head, generate normalized weights based on the local attention weights and the global attention weights, and calculate fusion features based on the normalized weights, local features and global features. The convolutional submodule includes sequentially connected convolutional layers, batch normalization layers, and activation function layers, used to perform convolution operations on the fused features and output a feature map; The depthwise separable convolutional submodule is used to sequentially perform depthwise convolution and pointwise convolution on the feature map output by the convolutional submodule to obtain a new feature map; The backbone network specifically includes: INPUT module, STAGE0 module, STAGE1 module, STAGE2 module, STAGE3 module, STAGE4 module and OUTPUT module; The STAGE1 module includes one first residual block and two second residual blocks connected in sequence; the STAGE2 module includes one first residual block and three second residual blocks connected in sequence; the STAGE3 module includes one first residual block, four second residual blocks and one first residual block connected in sequence; the STAGE4 module includes one first residual block and two second residual blocks connected in sequence. The first residual block includes a first convolutional structure, a second convolutional structure, a third convolutional structure, a fourth convolutional structure, an attention structure, and a first activation function. The feature map of the first residual block is input as the input to the first and fourth convolutional structures. The first, second, and attention structures are sequentially connected. The output of the fourth convolutional structure is connected to the input of the first activation function. The outputs of the attention structure and the first activation function are both connected to the input of the third convolutional structure. The output of the third convolutional structure is used as the output of the first residual block. In the STAGE1 and STAGE2 modules, the attention structure includes sequentially applied local attention and sliding window attention. In the STAGE3 and STAGE4 modules, the attention structure includes sequentially applied local attention and deformable attention. The second residual block includes a fifth convolutional structure, a sixth convolutional structure, a seventh convolutional structure, and a second activation function; the feature map of the second residual block is input as the input of the fifth convolutional structure and the second activation function, the output of the fifth convolutional structure is connected to the input of the sixth convolutional structure, the outputs of the sixth convolutional structure and the second activation function are both connected to the input of the seventh convolutional structure, and the output of the seventh convolutional structure is used as the output of the second residual block; The first convolutional structure, the second convolutional structure, the third convolutional structure, the fifth convolutional structure, and the sixth convolutional structure all include sequentially connected convolutional layers, batch normalization layers, and activation function layers; the fourth convolutional structure and the seventh convolutional structure both include sequentially connected convolutional layers and batch normalization layers. The encoder module is used to obtain multi-scale enhanced features based on the self-attention mechanism according to the multi-scale feature map output by the backbone network. The decoder module is used to output query features based on the multi-scale enhanced features using a multi-head attention mechanism; The output module is used to output defect bounding boxes and category labels based on the query features.

2. The digital printing defect detection method based on a multi-head collaborative mechanism according to claim 1, characterized in that, The process of sequentially performing depthwise convolution and pointwise convolution on the feature map output by the convolutional submodule to obtain a new feature map includes: Spatial convolution is performed on each channel of the feature map output by the convolutional submodule to extract the spatial dimension features of the feature map; A new feature map is obtained by linearly combining the spatial dimension features using a 1×1 convolution.

3. The digital printing defect detection method based on a multi-head collaborative mechanism according to claim 1, characterized in that, The deformable attention is achieved by the following formula: ； in, This represents the query vector for which deformable attention needs to be computed. The first feature in the input feature map The key vector of each dynamic sampling point This represents the learnable query projection matrix, used to project the query vector. Mapped to attention space, This represents the learnable key projection matrix, used to project key vectors. Mapped to attention space, This represents the normalization operation. This represents the projected query vector. This represents the projected key vector. This represents the fused feature vector after applying deformable attention weighting. This indicates the number of dynamic sampling points.

4. The digital printing defect detection method based on a multi-head collaborative mechanism according to claim 1, characterized in that, The local attention weights are represented as follows: ； in, Indicates the first Attention weight map of each local attention head. Indicates the first The local query vector corresponding to each local attention head. Indicates the first The key vectors corresponding to the local attention heads, where ⊙ represents element-wise multiplication. This represents a 1×1 convolution operation. This represents a 3×3 convolution operation. express Function operations.

5. A digital printing defect detection device based on a multi-head collaborative mechanism, characterized in that, The apparatus for implementing the digital printing defect detection method based on a multi-head collaborative mechanism as described in any one of claims 1 to 4 includes: The acquisition module is used to acquire digital printing images and preprocess the digital printing images to obtain digital printing image slices to be detected. The detection module is used to input the digital printing image slice to be detected into a pre-trained digital printing defect detection model based on multi-head collaboration to obtain the defect detection result; the digital printing defect detection model based on multi-head collaboration includes a backbone network, an encoder module, a decoder module and an output module. The backbone network is used to extract multi-scale feature maps of the digital printed image slices to be detected. The backbone network is an improved network that introduces multi-head attention, depthwise separable convolution, deformable attention, local attention, and sliding window attention into the ResNet network. The multi-head attention is applied to the STAGE0 module of the ResNet network to capture the local microscopic details and global structural information of the digital printed image slices to be detected through the parallel mechanism of multi-head attention. The depthwise separable convolution is applied to the STAGE0 module of the ResNet network to reduce the parameter scale and computational complexity of the digital printed defect detection model. Local attention and sliding window attention are applied sequentially in the residual blocks of the STAGE1 and STAGE2 modules of the ResNet network, and local attention and deformable attention are applied sequentially in the residual blocks of the STAGE3 and STAGE4 modules. The encoder module is used to obtain multi-scale enhanced features based on the self-attention mechanism according to the multi-scale feature map output by the backbone network. The decoder module is used to output query features based on the multi-scale enhanced features using a multi-head attention mechanism; The output module is used to output defect bounding boxes and category labels based on the query features.

Images