Carton printing defect detection method and system based on visual identification

Abstract

The invention discloses a carton printing defect detection method and system based on visual identification, and relates to the field of defect detection, and the method comprises the steps: obtaining a printing carton image set; constructing an edge enhancement detection model based on YOLov; taking the image set as input, and extracting a plurality of feature maps with different resolutions by using a backbone network; inputting a plurality of feature maps with different resolutions into a neck network, carrying out adaptive aggregation through a feature pyramid module to obtain an aggregated feature map, then obtaining a direction and an edge response weight through an edge sensing module, and carrying out edge reservation fusion to obtain an edge enhancement feature; and inputting the edge enhancement features into a detection network, and obtaining defect detection positions and types through multi-task training of position perception loss and direction enhancement classification loss. Aiming at the low detection precision of the printing defects of the carton, the detection precision of the ink dot and broken line defects is improved through an edge reservation fusion strategy, adaptive feature fusion and the like.

Description

Technical Field

[0001] The present application relates to the field of defect detection, and in particular to a method and system for detecting carton printing defects based on visual recognition. Background Art

[0002] Common defects in the carton printing process primarily fall into two categories: ink spots and broken lines. Ink spots typically appear as irregular patches with complex edges and varying sizes. Broken lines, on the other hand, appear as interruptions in a linear structure with distinct directional characteristics, narrow width, and variable length. Detecting these two types of defects presents the following technical challenges: The irregular shapes of ink spots and complex edges; and the variable length and direction of broken lines, making them difficult to describe with simple geometric features. Defect sizes range from millimeters to centimeters, requiring detection algorithms to possess robust multi-scale adaptability.

[0003] In recent years, the YOLO (You Only Look Once) series of algorithms has been widely used in the field of industrial defect detection due to its excellent balance between speed and accuracy. The evolution from YOLOv3 to YOLOv7 has significantly improved detection performance, especially in small target detection. However, the YOLOv series still faces the following problems when applied to carton printing defect detection: Although the standard YOLOv model adopts a feature pyramid structure, it lacks an adaptive aggregation mechanism for features of different resolutions, resulting in insufficient fusion of high- and low-level features. The standard convolution operation that YOLOv relies on is insensitive to directional information and cannot effectively capture the directional characteristics of broken lines. During the feature extraction and transmission process, edge information is easily smoothed, affecting the precise positioning of ink dot edges.

[0004] For example, the related technical document CN119624971A discloses a carton printing defect detection method and system based on visual recognition, including: obtaining a carton image through an image acquisition device, and calculating the grayscale feature of the pixel based on the grayscale value of each pixel in the image and the grayscale difference in its neighborhood, which is used to reflect the grayscale change in the local area; calculating the degree of change of the pixel based on the grayscale feature, and the degree of change characterizes the degree of clutter of the pixel, and constructing a degree of change co-occurrence matrix to describe the joint distribution characteristics between different degrees of change. Based on the degree of change co-occurrence matrix, the degree of defect of each pixel is calculated; the degree of defect of each pixel is calculated, and the pixel whose degree of defect is greater than the set threshold is regarded as a defective pixel. However, this solution has the following defects:

[0005] First, this solution uses traditional image processing techniques, relying solely on grayscale features and statistical distribution characteristics for defect detection, which has limited feature expression capabilities. Faced with the complex backgrounds and variable defect morphologies of carton printing, especially the irregular edges of ink dots and directional characteristics of broken lines, this simple feature cannot provide sufficient differentiation, limiting detection accuracy.

[0006] Furthermore, this solution's pixel-level detection method, based on a fixed threshold, lacks adaptability and struggles to adapt to different carton batches, different printing processes, and fluctuating ambient lighting conditions. Setting the threshold too high can lead to missed detections, while setting it too low can cause a large number of false detections. Frequent manual parameter adjustments are required in practice, resulting in a low level of automation and unsuitable for the continuous, high-speed operation required by modern production lines. Summary of the Invention

[0007] In response to the low accuracy of carton printing defect detection in the existing technology, this application provides a carton printing defect detection method and system based on visual recognition, which improves the detection accuracy of ink dots and broken line defects through edge-preserving fusion strategy and adaptive feature fusion.

[0008] One aspect of the present application provides a carton printing defect detection method based on visual recognition, comprising: acquiring a set of printed carton images; constructing an edge enhancement detection model based on YOLov, the detection model comprising a backbone network, a neck network and a detection network; wherein the neck network comprises a feature pyramid module and an edge perception module; taking the image set as input, and extracting multiple feature maps of different resolutions using the backbone network; inputting the multiple feature maps of different resolutions into the neck network, adaptively aggregating them through the feature pyramid module to obtain an aggregated feature map, and then obtaining the direction and edge response weights through the edge perception module, performing edge-preserving fusion, and obtaining edge-enhanced features; inputting the edge-enhanced features into the detection network, and obtaining the defect detection position and type through multi-task training of position-aware loss and direction-enhanced classification loss; wherein defects include ink dots and broken lines.

[0009] Furthermore, a detection model based on YOLov is constructed, including: using the YOLov network as the backbone network, and the backbone network outputs multiple feature maps of different resolutions; using feature maps of different resolutions as input to construct a neck network; the neck network includes a feature pyramid module and an edge perception module; the feature pyramid module adaptively aggregates feature maps of different resolutions through learnable weight parameters to obtain aggregated feature maps of multiple scales; the edge perception module is used to extract the directional features and edge features of the aggregated feature maps, generate position-sensitive attention weights and edge response weights, and perform edge-preserving fusion on the aggregated feature maps to obtain edge-enhanced features; using the edge-enhanced features as input to construct a detection network, the detection network includes a classification head and a regression head; the classification head outputs the category probability distribution of each candidate area through the convolution layer, and the regression head predicts the position of the defect bounding box through the convolution layer; setting a multi-task loss function of the detection model, the multi-task loss function includes position-aware loss and direction-enhanced classification loss, and training the detection model through back propagation.

[0010] Furthermore, obtaining aggregated feature maps at multiple scales includes: constructing a first path from high-level features to low-level features, and a second path from low-level features to high-level features; wherein high-level features represent features whose resolution is less than a threshold T1 output by the backbone network, and low-level features represent features whose resolution is greater than a threshold T2;

[0011] In the first path, for each layer of features: get the current layer input feature P i and the resized adjacent upper-level features P i+1 , respectively set P i and P i+1 The weights of the current layer are combined and weighted to obtain the intermediate feature P of the current layer. i,td ;

[0012] In the second path, for each layer of features: get the current layer feature P i , corresponding to the intermediate layer feature P i,td , and the resized adjacent lower-level features P i-1,out ; Set the feature P respectively i 、P i,td and P i-1,out The weights of the layers are combined and weighted to generate the output feature P of the current layer. i,out ; The output features of all layers on the second path {P i,out}, as the aggregated feature map generated by the feature pyramid module.

[0013] Furthermore, the aggregated feature maps are fused according to the attention weight to obtain edge enhancement features, including: setting an edge perception module after the feature pyramid module; i,out , extract the horizontal and vertical feature maps Z respectively h,i and Z w,i At the same time, P i,out Perform edge detection, calculate the gradient strength in the horizontal, vertical and diagonal directions, and fuse the gradient strength in each direction to form a gradient feature map G e,i ; The feature map Z h,i 、Z w,i and G e,i Combined along the channel dimension to form edge enhancement features [Z h,i ,Z w,i ,G e,i ]; Use shared convolutional layers to enhance edge features [Z h,i ,Z w,i ,G e,i ] to perform nonlinear transformation and obtain the position-sensitive attention weight g corresponding to each scale. h,i and g w,i , and the edge response weight g e,i ; Using weight gh,i 、g w,i and g e,i , for the corresponding aggregate feature map P i,out Perform edge-preserving fusion to obtain edge enhancement features {E i}.

[0014] Furthermore, the horizontal and vertical feature maps Z are extracted respectively h,i and Z w,i , including: aggregation feature map P i,out Apply sliding windows of different sizes to construct multi-scale pooling operations; the sizes of sliding windows include {k1×1,k2×1,......,k n ×1} is used to extract horizontal features, {1×k1,1×k2,......,1×k n} is used to extract vertical features; for horizontal feature extraction, different sizes of k×1 sliding windows are used to extract features in the horizontal direction of P i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a horizontal feature map Z h,i ; For vertical feature extraction, 1×k sliding windows of different sizes are used to extract the feature points along the vertical direction of P i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a vertical feature map Z w,i .

[0015] Furthermore, we get the edge enhancement feature {E i}, through the following formula:

[0016] E i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ), where α is the direction perception weight coefficient, β is the edge response weight coefficient, and λ is the edge preservation basic coefficient. Each coefficient is adaptively adjusted according to the characteristics of the defect type to enhance the detection capability of defects with different edge characteristics, such as ink dots and broken lines.

[0017] Furthermore, the multi-task loss function of the detection model is set, including: Loss = λ1×loc loss +λ2×cls loss , where loc loss is the position-aware loss, cls loss is the classification loss; λ1 and λ2 are weight coefficients.

[0018] Further, Where N is the number of samples in the batch; B iis the bounding box area containing the defect in the i-th sample; pred box,i and gt box,i They are the predicted defect bounding box and the real defect bounding box; IoU(pred box,i ,gt box,i ) is the intersection-over-union ratio of the predicted box and the true box;

[0019] g e,i (x, y) is the value of the edge response weight at position (x, y) obtained by edge detection calculation; w e It is the edge response weighting coefficient, which is used to adjust the contribution of edge features to position loss.

[0020] Further, Where N is the number of samples in the batch; C is the total number of defect categories; B i is the area containing defects in the i-th sample; y i,c is the true label (0 or 1) of the i-th sample for category c; p i,c is the predicted probability of the i-th sample for category c; g h,i (x,y) and g w,i (x, y) is the calculated horizontal and vertical attention weights at the position (x, y); w d is the directional weighting coefficient, which is used to adjust the contribution of directional features to classification loss.

[0021] Traditional detection systems usually use a general loss function, which is difficult to optimize for specific defect features. On the one hand, the traditional IoU loss treats all areas within the prediction box equally, ignoring the importance of edge areas. The position-aware loss function of this application introduces the edge response weight g e,i (x, y), so that the loss function pays special attention to the defect edge area, and the edge weight makes the model pay more attention to the precise positioning of the edge area during the training process, thereby improving the detection accuracy of the irregular edges of ink dots.

[0022] On the other hand, the traditional cross entropy loss gives the same weight to the classification errors of all regions, ignoring the importance of directional features for classification. The direction-enhanced classification loss of this application introduces the directional attention weight g h,i (x,y)×g w,i (x, y), the directional weight enables the model to pay special attention to areas with directional features during training, improving the classification accuracy of directional defects such as broken wires. The product term of the horizontal and vertical directional weights enables the system to capture complex combinations of directional features.

[0023] Another aspect of the present application also provides a carton printing defect detection system based on visual recognition, which is used to execute a carton printing defect detection method based on visual recognition of the present application.

[0024] Compared with the existing technology, the advantages of this application are:

[0025] (1) Ink dots and broken lines in carton printing exhibit unique morphological characteristics. Ink dots typically exhibit irregular shapes, complex edges, and varying sizes, while broken lines appear as elongated structures with distinct directionality and large contrast variations. Existing techniques typically employ standard convolution or global pooling for feature extraction, which does not distinguish between feature directions. This often results in loss of directional information, blurred edge details, and reduced feature differentiation. This results in poor detection performance, particularly when dealing with subtle broken lines and ink dots with complex edges.

[0026] Therefore, this application is based on a direction-aware edge extraction mechanism. Through directional sliding window pooling and multi-directional gradient calculation, the multi-scale pooling operation enables the system to adapt to broken line defects of different lengths. The multi-directional gradient intensity calculation enhances sensitivity to edge features and improves the detection accuracy of ink dot edges. The fusion of horizontal, vertical, and diagonal gradients enables the system to detect edges in any direction, enhancing the comprehensiveness of detection. This solves the problems of insufficient directional feature extraction and loss of edge information in traditional methods, improving the accuracy of ink dot edge positioning and the accuracy of broken line direction identification.

[0027] Ink dots and broken line defects often rely on the extraction of refined edge features. In existing technologies, simple additive fusion, feature concatenation, or feature averaging methods are generally used for feature fusion. However, these linear fusion methods cause edge information to be easily smoothed and lost, especially during deep network transmission.

[0028] Therefore, this application is approved by E i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ) to achieve edge-preserving fusion, direction-aware weight term α×g h,i ×g w,i Specifically enhance the directional feature expression of the broken line, the edge response weight term β×g e,i The focus is on preserving the edge information of the ink dots, and the basic coefficient λ ensures that the basic features are retained even in areas with smaller weights. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0030] Figure 1This is an exemplary flow chart of a method for detecting carton printing defects based on visual recognition according to some embodiments of the present application;

[0031] Figure 2 is a schematic diagram of printing defects according to some embodiments of the present application;

[0032] Figure 3 is a schematic diagram of an edge enhancement detection model according to some embodiments of the present application;

[0033] Figure 4 is a schematic diagram of a first path and a second path according to some embodiments of the present application;

[0034] Figure 5 is a schematic diagram of an edge perception module according to some embodiments of the present application;

[0035] Figure 6 This is a heat map of feature weights in different directions according to some embodiments of the present application;

[0036] Figure 7 This is a comparison diagram before and after feature fusion shown in some embodiments of the present application. DETAILED DESCRIPTION

[0037] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] like Figure 1 As shown, a set of printed carton images is obtained; an edge enhancement detection model based on YOLov is constructed, and the detection model includes a backbone network, a neck network and a detection network; wherein, the neck network includes a feature pyramid module and an edge perception module; the image set is used as input, and the backbone network is used to extract multiple feature maps of different resolutions; the multiple feature maps of different resolutions are input into the neck network, and adaptively aggregated by the feature pyramid module to obtain an aggregated feature map, and then the direction and edge response weights are obtained by the edge perception module, and edge-preserving fusion is performed to obtain edge enhancement features; the edge enhancement features are input into the detection network, and the defect detection position and type are obtained through multi-task training of position perception loss and direction enhancement classification loss; wherein, defects include ink dots and broken lines.

[0039] In this embodiment, the hardware platform is: NVIDIA RTX 3090 GPU (24GB), Intel i9-10900K CPU (10 cores and 20 threads), 64GB DDR4-3200 RAM; the software environment is: Ubuntu 20.04LTS, Python 3.8.10, PyTorch1.10.0, CUDA 11.3, and cuDNN 8.2.0.

[0040] like Figure 2 As shown, the data set constructed in this embodiment mainly includes two typical defect types, ink dots and broken lines. Ink dots: irregular spots formed by ink splashing during the printing process, with variable shapes and obvious edge features. Mainly caused by unstable ink jetting of the printing machine, uneven pressure of the ink roller or uneven ink absorption of the paper, usually appearing as local contamination of black or color, with sizes ranging from tiny dots (diameter <2mm) to medium areas (diameter >10mm), characterized by clear edge contours but highly irregular shapes. Broken lines: defects in which lines that should be continuous in the printed pattern are interrupted or missing. Mainly caused by plate wear, insufficient printing pressure or unstable ink supply, it manifests as sudden interruption of lines with obvious directional characteristics. The fracture is usually relatively neat or slightly blurred. The length of the broken line varies from tiny (<1mm) to obvious (>5mm), which has a direct impact on the integrity and aesthetics of the product information.

[0041] The physical components of the dataset include corrugated boxes, coated paper boxes, and pharmaceutical cartons. An industrial camera (resolution 3840×2160) was used to capture the original images and perform defect annotation. The dataset was divided into a training set and a validation set. min ,y min ,x max ,y max ,class id ] format, where ink dots (class id =0) and disconnection (class id =1).

[0042] like Figure 3 As shown, the edge enhancement detection model of the present application includes: using the YOLov network as the backbone network, and the backbone network outputs multiple feature maps of different resolutions; specifically, in this embodiment, the backbone network adopts CSPDarknet53, and the backbone network outputs feature maps: P1: 80×80×128 (low-level features); P2: 40×40×256 (middle-level features); P3: 20×20×512 (high-level features).

[0043] like Figure 4 As shown in the figure, the feature pyramid module is a key component of the neck network, designed to solve the problem of multi-scale feature fusion in carton printing defect detection. The feature pyramid module constructs a dual-path structure: a top-down path from high-level features to low-level features, and a bottom-up path from low-level features to high-level features.

[0044] In this embodiment, the feature level division standard is as follows: high-level features: feature maps with a resolution less than a threshold value T1: 40×40, such as P3: 20×20×512; low-level features: feature maps with a resolution greater than a threshold value T2: 80×80, such as P1: 80×80×128; intermediate-level features: feature maps with a resolution between T1 and T2, such as P2: 40×40×256.

[0045] The top-down pathway enhances the ability of lower-level features to understand the overall semantics of defects by passing high-level semantic information to lower layers. This is important for detecting large-scale defects or distinguishing defects in complex backgrounds. Specifically, the first pathway receives the multi-scale feature maps P3 (20×20×512), P2 (40×40×256), and P1 (80×80×128) output by the backbone network. A 1×1 convolution is applied to P3 to reduce the channel dimension (512→256) to reduce computational complexity. Bilinear interpolation is then used to upsample the feature map to twice its size (20×20→40×40).

[0046] Perform adaptive feature fusion and apply learnable weight parameters to fuse high-level and current-level features. The fusion formula is: P 2,td =w p ×P2+w up ×P3', where P3' represents the upsampled P3; the weight parameter is set to: w p =0.65, w up = 0.35; these weights can be automatically adjusted according to the gradient of the loss function during training. 2,td Repeat the same process: channel dimensionality reduction, upsampling (40×40→80×80), and fuse the processed features with P1 to obtain P 1,td , completing the cascade transmission of semantic information from high to low.

[0047] The bottom-up path transfers detailed information from the lower layers to the higher layers, ensuring that the high-level features gain powerful semantic understanding capabilities while not losing the fine edge information of the defects. This is especially important for detecting small-scale or weak-edge defects.

[0048] In this embodiment, for the second path, the fusion formula is: P 1,out =w curr ×P1+w td ×P 1,td , weight parameter: w curr =0.5, w td =0.35, at this stage, there is no need to consider the lower layer features (P1 is the lowest layer), and generate the low-level output features P 1,out :80×80×128. First, 1,out Perform downsampling (80×80→40×40) and channel adjustment (128→256); fusion formula: P2,out =w curr ×P2+w td ×P 2,td +w down ×P 1,out ', P 1,out ' represents the P after downsampling 1,out ; Weight parameter: w curr =0.5, w td =0.35, w down =0.15; generate middle-level output feature P 2,out :40×40×256. 2,out Perform downsampling (40×40→20×20) and channel adjustment (256→512); fusion formula:

[0049] P 3,out =w curr ×P3+w td ×P 3,td +w down ×P 2,out ', P 2,out ' represents the P after downsampling 2,out ; The weight parameters are the same as the middle layer fusion, generating high-level output features P 3,out :20×20×512. Collect output features of all levels

[0050] {P 1,out ,P 2,out ,P 3,out}, these features constitute a multi-scale feature set, which serves as the input of the edge perception module.

[0051] The dual-path architecture of this application forms a closed-loop information flow. High-level features are first passed downward to provide contextual information, helping the model understand the overall semantics of the defect. Low-level features then pass detailed information upward, preserving the fine structure of the defect's edges and texture. This bidirectional information flow not only improves the model's ability to perceive defects of varying scales, but also enables it to more effectively distinguish between normal printed textures on the carton surface and actual defects, reducing false positives and improving detection accuracy.

[0052] In paper printing, ink dots and broken lines are two common and critical printing defects. The characteristics of ink dot defects are: irregular shape, clear edges but complex contours, no obvious directional features, and high contrast between edges and background. The characteristics of broken line defects are: obvious linear structure, strong directionality, and may be distributed horizontally, vertically, or diagonally. The width is narrow but the length is variable, and the edge forms a linear contrast with the background. Traditional machine recognition-based defect detection algorithms often use a global pooling layer to extract features, but global pooling is essentially an averaging / maximization process for the entire row or column, which will weaken the strength of local edge features. In other words, global pooling will cause the precise position information of the edge to be lost. For defects such as ink dots and broken lines that rely on fine edge features, global pooling processing will weaken their feature expression and reduce recognition accuracy.

[0053] Therefore, if Figure 5 As shown, the present application fuses the aggregated feature maps according to the attention weight to obtain edge enhancement features, including: setting an edge perception module after the feature pyramid module; i,out , extract the horizontal and vertical feature maps Z respectively h,i and Z w,i , for the aggregated feature map P i,out Apply sliding windows of different sizes to construct multi-scale pooling operations; the sizes of sliding windows include {k1×1,k2×1,......,k n ×1} is used to extract horizontal features,

[0054] {1×k1,1×k2,......,1×k n} is used to extract vertical features.

[0055] For horizontal feature extraction, different sizes of k×1 sliding windows are used to move horizontally along P i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a horizontal feature map Z h,i ; For vertical feature extraction, 1×k sliding windows of different sizes are used to extract the feature points along the vertical direction of P i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a vertical feature map Z w,i ; The horizontal sliding window (k×1) is particularly suitable for capturing horizontal break features; the vertical sliding window (1×k) is particularly suitable for capturing vertical break features; the multi-scale combination can simultaneously process defects of different sizes and shapes.

[0056] At the same time, i,out Perform edge detection, calculate the gradient strength in the horizontal, vertical and diagonal directions, and fuse the gradient strength in each direction to form a gradient feature map G e,iSpecifically, on the one hand, the boundary between the ink dot and the background forms a closed high-response area in the gradient map, and multi-directional gradient fusion can fully capture the irregular edges of the ink dot. On the other hand, the edge of the broken line produces a linear strong response on the gradient of a specific direction, and the diagonal gradient can capture the oblique broken line.

[0057] The feature map Z h,i , Z w,i and G e,i Combined along the channel dimension to form edge enhancement features [Z h,i ,Z w,i ,G e,i ];

[0058] Using shared convolutional layers to enhance edge features[Z h,i ,Z w,i ,G e,i ] to perform nonlinear transformation and obtain the position-sensitive attention weight g corresponding to each scale. h,i and g w,i , and the edge response weight g e,i ; Direction perception weight g h,i ×g w,i Enhanced the detection capability of directional defects such as broken wires; edge response weight g e,i Improved sensitivity to irregular edge defects such as ink dots; adaptive coefficients α, β, λ automatically adjust the detection strategy according to the characteristics of different defect types.

[0059] Using weight g h,i 、g w,i and g e,i , for the corresponding aggregate feature map P i,out Perform edge-preserving fusion to obtain edge enhancement features {E i};E i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ), where α is the direction perception weight coefficient, β is the edge response weight coefficient, and λ is the edge preservation basic coefficient; each coefficient is adaptively adjusted according to the characteristics of the defect type to enhance the detection capability of defects with different edge characteristics such as ink dots and broken lines. Specifically, for ink dot defects: β×g e,i The term enhances the edge features of the ink dots; the λ basic coefficient ensures that the basic features are retained even in areas with unclear edges. For line break defects: α×g h,i ×g w,i The term enhances the directional characteristics of the broken line; the product of the directional weights g h,i ×g w,iRegions with both horizontal and vertical features can be highlighted, which is particularly effective for detecting broken lines at intersections or corners.

[0060] Specifically, in this embodiment, directional features are extracted and a multi-scale sliding window is designed: k = {3, 5, 7, 9}; i,out :80×80×128 Apply horizontal sliding window: window size: {3×1, 5×1, 7×1, 9×1}; step size: 1×1, pooling method: maximum pooling, result: 4 80×80×128 feature maps, channel dimension splicing, get Z h,1 :80×80×512; Similarly, apply the vertical sliding window {1×3, 1×5, 1×7, 1×9} to get Z w,1 :80×80×512.

[0061] Calculate the edge gradient and apply the improved Sobel operator to calculate the three-directional gradient: horizontal gradient G x : Size is 3×3, weight Vertical gradient G y : Size is 3×3, weight Diagonal gradient G d : Size is 3×3, weight Gradient feature fusion: Calculation result: 80×80×128 gradient feature map.

[0062] Z h,1 、Z w,1 and G e,1 Splicing along the channel dimension: [Z h,1 ,Z w,1 ,G e,1 ]:80×80×1152, apply shared convolutional network to generate weights: first layer: 3×3 convolution (1152→256)+LeakyReLU(0.1); second layer: 1×1 convolution (256→128)+Sigmoid; get weight: g h,1 、g w,1 and g e,1 (all 80×80×128); weight dynamic range: [0.15, 0.95]; average weight value of edge area: 0.82, average weight value of non-edge area: 0.23. In this embodiment, the original defect image is as follows Figure 6 As shown in a; the generated horizontal weight heat map is as follows Figure 6 As shown in b, the intensity of the horizontal feature is highlighted. The darker the blue, the higher the weight. The generated vertical weight heat map is as follows: Figure 6 As shown in c, the intensity of the vertical feature is highlighted. The darker the green, the higher the weight. The generated edge response weight heat map is as follows: Figure 6Figure d highlights the strength of edge features, with darker red indicating higher weights. It can be seen that the weight distributions generated by the edge perception module differ significantly for different defect types: broken wire defects are prominent in the directional weight map, while ink dot defects are more prominent in the edge response weight map.

[0063] When detecting ink dot defects: α=0.3, β=0.3, λ=0.2; when detecting broken wire defects: α=0.7, β=0.5, λ=0.2; for P 1,out :80×80×128 edge preservation fusion: ink dot edge information retention rate: increased by 43.5%; broken line direction feature enhancement rate: increased by 38.2%; 2,out and P 3,out The same process is performed to obtain E2 and E3. The feature comparison before and after edge preservation fusion is as follows Figure 7 As shown, the feature map heat map before fusion is shown in 7a, and the feature map heat map after edge-preserving fusion is shown in 7b. As shown in the figure, in the fused heat map, the edge contour of the ink dot is clearer, the edge response intensity is increased by about 43.5%, and the high edge response weight of β=0.9 enables the irregular edge of the ink dot to be accurately preserved. In the fused heat map, the broken line area is significantly enhanced along the broken line direction, and the directional feature is enhanced by about 38.2%. The high directional perception weight of α=0.7 enables the directional feature of the broken line to be effectively expressed. The fused heat map maintains the low response characteristics of the non-defective area, and the basic coefficient λ=0.2 ensures that the basic features of the low-feature area are still retained. The edge-preserving fusion strategy of this application uses adaptive weight coefficients to differentiate and enhance features for the characteristics of different types of defects, thereby significantly improving the detection accuracy of the two main types of defects, ink dots and broken lines.

[0064] S5, the edge enhancement feature {E i} is sent to the detection network; the classification head enhances the edge features {E i} is processed and the defect category probability distribution of each candidate area is output; the regression head uses the convolution layer to enhance the edge features {E i} is processed to predict the position coordinates of the defect bounding box.

[0065] The loss function in traditional machine recognition algorithms mostly focuses on single-task optimization and uniform feature weight processing. However, ink dots and broken lines have significant differences in edge complexity and directional characteristics. The traditional standard loss function has the defects of insufficient edge accuracy and loss of directional information for ink dot and broken line detection. Therefore, this application sets a multi-task loss function for the detection model, including: Loss = λ1×loc loss +λ2×cls loss , where loc lossis the position-aware loss, which is used to measure the deviation between the predicted defect bounding box and the true bounding box; cls loss is the classification loss, which is used to measure the difference between the predicted defect category and the true category; λ1 and λ2 are weight coefficients; Specifically, ink dots and broken lines are two typical printing defects with essential differences in morphological characteristics. Ink dots usually have irregular shapes and complex edge contours; while broken lines appear as linear structures with obvious directionality. It is difficult for a traditional single loss function to simultaneously optimize the detection effect of these two differentiating features. Therefore, this application loc loss By introducing the edge response weight g e,i (x, y), especially strengthening the precise positioning of complex edges of ink dots; cls loss By direction-aware weight g h,i (x,y)×g w,i (x,y), which significantly improves the classification accuracy of the broken line direction features.

[0066] Secondly, traditional object detection algorithms usually use the standard IoU loss function for bounding box optimization. However, for printing defects such as ink dots and broken lines, the standard IoU loss has technical defects such as insufficient edge sensitivity and uniform regional importance. Therefore, this application designs an edge-enhanced position-aware loss function:

[0067] Where N is the number of samples in the batch; B i is the bounding box area containing the defect in the i-th sample; pred box,i and gt box,i They are the predicted defect bounding box and the real defect bounding box; IoU(pred box,i ,gt box,i ) is the intersection-over-union ratio of the predicted box and the true box; g e,i (x, y) is the value of the edge response weight at position (x, y) obtained by edge detection calculation; w e It is the edge response weighting coefficient, which is used to adjust the contribution of edge features to position loss.

[0068] In detail, on the one hand, ink dots usually have irregular edge shapes, and traditional IoU calculation is difficult to accurately fit their contours, and the edge response weight g e,i (x,y) produces a high response value at the edge of the ink dot, and the expression [1+w e ×g e,i (x,y)] amplifies the loss value at the edge, forcing the model to learn the ink dot boundary more accurately. This mechanism is particularly suitable for processing ink dot defects of different sizes and shapes. On the other hand, although the broken line defect has a simple structure, it is extremely subtle and its boundary is easily ignored by the traditional IoU loss. e,i(x,y) weighting, the edge area of the broken line gets a higher loss weight, and the key positions such as the starting point and end point of the broken line are specially strengthened in the loss calculation. This design effectively solves the problem that slender defects are easily roughly located. Finally, the coefficient w e Allows the system to dynamically adjust the importance of edge response according to the defect characteristics. For defects with obvious edge features, w can be increased. e To strengthen edge constraints, for defects with blurred edges, ω_e can be reduced to avoid excessive focus on noise edges.

[0069] Finally, traditional object detection algorithms typically use a standard cross-entropy loss function in classification tasks, assigning uniform weights to image regions. However, for printing defects with significant directional differences, such as ink dots and broken lines, the standard cross-entropy loss has technical drawbacks such as insufficient utilization of directional information and limited feature differentiation capabilities. Therefore, this application designs a direction-aware classification loss function: Where N is the number of samples in the batch; C is the total number of defect categories; B i is the area containing defects in the i-th sample; y i,c is the true label (0 or 1) of the i-th sample for category c; p i,c is the predicted probability of the i-th sample for category c; g h,i (x,y) and g w,i (x, y) is the calculated horizontal and vertical attention weights at the position (x, y); w d is the direction weighting coefficient, which is used to adjust the contribution of directional features to classification loss;

[0070] In detail, by introducing the direction attention weight g h,i (x,y) and g w,i The product term of (x, y) focuses on regions with directional features. Traditional cross-entropy loss assigns equal weight to all pixels, while this loss function differentially weights them based on the significance of directional features, significantly enhancing the ability to classify directional defects.

[0071] On the one hand, the broken line defect usually appears as an obvious linear structure, and the directional feature is its key identification feature. h,i (x,y) produces a high response, and the vertical break is in g w,i (x,y) produces a high response, the product term g h,i (x,y)×g w,i (x,y) can capture the broken line features at various angles.

[0072] On the other hand, although the ink dots do not have obvious directionality, the response patterns of their edge areas in different directions form characteristic fingerprints. h,i (x,y) and gw,i The combination of (x,y) can capture the complex geometric characteristics of the ink dot edge. This multi-directional feature expression can effectively distinguish ink dots from other defects with similar appearance. The direction-aware weight enhances the model's adaptability to changes in ink dot size and shape.

[0073] In addition, the traditional cross entropy loss treats the entire region uniformly, ignoring the importance differences of different locations within the defect. h,i (x,y) and g w,i (x,y) is sensitive to spatial position, which makes the loss function focus on the area with the richest classification information in the defect. For broken lines, endpoints and center lines often contain the most critical classification information. For ink dots, edge contours often contain more classification information than internal areas.

[0074] The parameters of the detection model are optimized by the back propagation algorithm so that the detection network can enhance the edge features {E i} Accurately output the location and type of defects.

[0075] The above schematically describes the invention and its implementation methods. This description is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it and designs a structural method and embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A carton printing defect detection method based on visual recognition, characterized in that: include: Get a set of printed carton images; Build a YOLov-based edge enhancement detection model, which includes a backbone network, a neck network, and a detection network. The neck network includes a feature pyramid module and an edge perception module. Take the image set as input and use the backbone network to extract multiple feature maps of different resolutions; Multiple feature maps of different resolutions are input into the neck network, and adaptively aggregated through the feature pyramid module to obtain the aggregated feature map. The direction and edge response weights are then obtained through the edge perception module, and edge-preserving fusion is performed to obtain edge-enhanced features. The edge-enhanced features are input into the detection network, and the defect detection location and type are obtained through multi-task training of position-aware loss and direction-enhanced classification loss; defects include ink dots and broken lines.

2. The carton printing defect detection method based on visual recognition according to claim 1 is characterized in that: Build an edge enhancement detection model, including: The YOLov network is used as the backbone network, which outputs multiple feature maps of different resolutions; Taking feature maps of different resolutions as input, a neck network is constructed; the neck network includes a feature pyramid module and an edge perception module; The feature pyramid module adaptively aggregates feature maps of different resolutions through learnable weight parameters to obtain aggregated feature maps of multiple scales; The edge perception module is used to extract the directional features and edge features of the aggregated feature map, generate position-sensitive attention weights and edge response weights, and perform edge-preserving fusion on the aggregated feature map to obtain edge-enhanced features; The edge enhancement features are used as input to build a detection network, which includes a classification head and a regression head. The classification head outputs the category probability distribution of each candidate region through the convolution layer, and the regression head predicts the location of the defect bounding box through the convolution layer. Set the multi-task loss function of the detection model, which includes position perception loss and direction enhancement classification loss, and train the detection model through backpropagation.

3. The carton printing defect detection method based on visual recognition according to claim 2 is characterized in that: Get aggregated feature maps of multiple scales, including: Construct a first path from high-level features to low-level features, and a second path from low-level features to high-level features; wherein high-level features represent features whose resolution is less than a threshold T1 output by the backbone network, and low-level features represent features whose resolution is greater than a threshold T2; In the first path, for each layer of features: get the current layer input feature P i and the resized adjacent upper-level features P i+1 , respectively set P i and P i+1 The weights of the current layer are combined and weighted to obtain the intermediate feature P of the current layer. i,td ; In the second path, for each layer of features: get the current layer feature P i , corresponding to the intermediate layer feature P i,td , and the resized adjacent lower-level features P i-1,out ; Set the feature P respectively i 、P i,td and P i-1,out The weights of the layers are combined and weighted to generate the output feature P of the current layer. i,out ; The output features of all layers on the second path {P i,out }, as the aggregated feature map generated by the feature pyramid module.

4. The method for detecting carton printing defects based on visual recognition according to claim 3, characterized in that: Obtain edge enhancement features, including: An edge perception module is set after the feature pyramid module; For each scale aggregate feature map P i,out , extract the horizontal and vertical feature maps Z respectively h,i and Z w,i ; At the same time, i,out Perform edge detection, calculate the gradient strength in the horizontal, vertical and diagonal directions, and fuse the gradient strength in each direction to form a gradient feature map G e,i ; The feature map Z h,i , Z w,i and G e,i Combined along the channel dimension to form edge enhancement features [Z h,i ,Z w,i ,G e,i ]; Using shared convolutional layers to enhance edge features[Z h,i ,Z w,i ,G e,i ] to perform nonlinear transformation and obtain the position-sensitive attention weight g corresponding to each scale. h,i and g w,i , and the edge response weight g e,i ; Using weight g h,i 、g w,i and g e,i , for the corresponding aggregate feature map P i,out Perform edge-preserving fusion to obtain edge enhancement features {E i }.

5. The method for detecting carton printing defects based on visual recognition according to claim 4, characterized in that: For each scale aggregate feature map P i,out , extract the horizontal and vertical feature maps Z respectively h,i and Z w,i ,include: For the aggregated feature map P i,out Apply sliding windows of different sizes to construct multi-scale pooling operations; the sizes of sliding windows include {k1×1,k2×1,......,k n ×1} is used to extract horizontal features, {1×k1,1×k2,......,1×k n }Used to extract vertical features; For horizontal feature extraction, different sizes of k×1 sliding windows are used to move horizontally along P i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a horizontal feature map Z h,i ; For vertical feature extraction, 1×k sliding windows of different sizes are used to extract the vertical features in P. i,out Perform maximum pooling on the image and concatenate the pooling results of all scales along the channel dimension to form a vertical feature map Z w,i .

6. The method for detecting carton printing defects based on visual recognition according to claim 4, characterized in that: Get edge enhancement features {E i }, through the following formula: AND i =P i,out × (1 + α × g h,i × g w,i +β × g e,i +λ) Among them, α is the direction perception weight coefficient, β is the edge response weight coefficient, and λ is the edge preservation basic coefficient.

7. The method for detecting carton printing defects based on visual recognition according to any one of claims 2 to 6, characterized in that: Set the multi-task loss function of the detection model, including: Loss=λ1×loc loss +λ2×cls loss Among them, loc loss is the position-aware loss, cls loss is the classification loss; λ1 and λ2 are weight coefficients.

8. The method for detecting carton printing defects based on visual recognition according to claim 7, characterized in that: Where N is the number of samples in the batch; B i is the bounding box area containing the defect in the i-th sample; pred box,i and gt box,i They are the predicted defect bounding box and the real defect bounding box; IoU(pred box,i ,gt box,i ) is the intersection-over-union ratio of the predicted box and the true box; g e,i (x, y) is the value of the edge response weight at position (x, y) obtained by edge detection calculation; w e is the edge response weighting coefficient.

9. The method for detecting carton printing defects based on visual recognition according to claim 8, characterized in that: Where N is the number of samples in the batch; C is the total number of defect categories; B i is the area containing defects in the i-th sample; y i,c is the true label of the i-th sample for category c; p i,c is the predicted probability of the i-th sample for category c; g h,i (x,y) and g w,i (x, y) is the calculated horizontal and vertical attention weights at the position (x, y); w d is the directional weighting coefficient.

10. A carton printing defect detection system based on visual recognition, characterized in that: include: at least one processing unit; Used to execute instructions to implement the carton printing defect detection method based on visual recognition as described in any one of claims 1 to 9.

Images