A carton printing defect detection method and system based on visual recognition

Abstract

The application discloses a carton printing defect detection method and system based on visual identification, and relates to the field of defect detection, and comprises the following steps: acquiring a printed carton image set; constructing an edge enhancement detection model based on YOLov; taking the image set as input, and extracting a plurality of feature maps with different resolutions by using a backbone network; inputting the plurality of feature maps with different resolutions into a neck network, adaptively aggregating the feature maps by a feature pyramid module to obtain an aggregated feature map, and then acquiring a direction and an edge response weight by an edge perception module, performing edge reservation fusion, and obtaining edge enhancement features; inputting the edge enhancement features into a detection network, and obtaining a defect detection position and type by multi-task training of a position perception loss and a direction enhancement classification loss. In view of the low carton printing defect detection precision, the edge reservation fusion strategy and adaptive feature fusion are used, so that the detection precision of the ink dot and broken line defects is improved.

Description

Technical Field

[0001] This application relates to the field of defect detection, and in particular to a method and system for detecting defects in cardboard box printing based on visual recognition. Background Technology

[0002] Common defects in cardboard box printing mainly fall into two categories: ink spots and broken lines. Ink spots typically appear as irregular patches with complex edges and varying sizes; broken lines, on the other hand, are interruptions in a linear structure, exhibiting a clear directional characteristic, narrow in width but variable in length. Detecting these two types of defects presents the following technical challenges: ink spots have irregular shapes and complex edges; broken lines have variable lengths and directions, making them difficult to describe using simple geometric features. Defect sizes range from millimeters to centimeters, requiring detection algorithms with strong multi-scale adaptability.

[0003] In recent years, the YOLO (You Only Look Once) series of algorithms has been widely used in industrial defect detection due to its excellent balance between speed and accuracy. The evolution from YOLOv3 to YOLOv7 has significantly improved detection performance, especially in the detection of small targets. However, the YOLOv series still faces the following problems when applied to the detection of defects in carton printing: Although the standard YOLOv model adopts a feature pyramid structure, it lacks an adaptive aggregation mechanism for features at different resolutions, resulting in insufficient fusion of high and low layer features. The standard convolution operation on which YOLOv relies is not sensitive to directional information and cannot effectively capture the directional features of broken lines. During feature extraction and transmission, edge information is easily smoothed, affecting the accurate positioning of ink dot edges.

[0004] For example, relevant technical document CN119624971A discloses a method and system for detecting defects in cardboard box printing based on visual recognition. The method includes: acquiring cardboard box images through an image acquisition device; calculating grayscale feature values ​​of each pixel based on its grayscale value and the grayscale difference within its neighborhood, used to reflect the grayscale changes in local areas; calculating the degree of change of each pixel based on the grayscale feature values, where the degree of change characterizes the disorder of the pixel; constructing a co-occurrence matrix of the degree of change to describe the joint distribution characteristics between different degrees of change; calculating the defect degree of each pixel based on the co-occurrence matrix of the degree of change; and identifying pixels with a defect degree greater than a set threshold as defective pixels. However, this method has the following drawbacks:

[0005] First, this scheme adopts a traditional image processing approach, relying solely on grayscale features and statistical distribution characteristics for defect detection, which has limited feature representation capabilities. Faced with the complex background and varied defect morphologies of cardboard printing, especially the irregular edges of ink dots and the directional characteristics of broken lines, these simple features are insufficient to provide adequate discrimination, thus limiting detection accuracy.

[0006] Furthermore, this pixel-level detection method based on a fixed threshold lacks adaptability and is difficult to adapt to different batches of cartons, different printing processes, and changing ambient lighting conditions. Setting the threshold too high will lead to missed detections, while setting it too low will cause a large number of false detections. In practical applications, frequent manual adjustment of parameters is required, resulting in low automation and making it unsuitable for the continuous high-speed operation requirements of modern production lines. Summary of the Invention

[0007] To address the low accuracy of existing cardboard box printing defect detection technologies, this application provides a visual recognition-based cardboard box printing defect detection method and system. Through edge-preserving fusion strategies and adaptive feature fusion, the detection accuracy of ink dots and broken lines is improved.

[0008] One aspect of this application provides a visual recognition-based method for detecting defects in printed cardboard boxes, comprising: acquiring a set of printed cardboard box images; constructing a YOLov-based edge enhancement detection model, the detection model including a backbone network, a neck network, and a detection network; wherein the neck network includes a feature pyramid module and an edge perception module; taking the image set as input, extracting multiple feature maps of different resolutions using the backbone network; inputting the multiple feature maps of different resolutions into the neck network, adaptively aggregating them through the feature pyramid module to obtain an aggregated feature map, then obtaining orientation and edge response weights through the edge perception module, performing edge-preserving fusion to obtain edge enhancement features; inputting the edge enhancement features into the detection network, and obtaining the defect detection location and type through multi-task training using position-aware loss and orientation-enhanced classification loss; wherein the defects include ink dots and broken lines.

[0009] Furthermore, a YOLov-based detection model is constructed, including: using a YOLov network as the backbone network, which outputs multiple feature maps at different resolutions; using these feature maps as input to construct a neck network; the neck network includes a feature pyramid module and an edge-aware module; the feature pyramid module adaptively aggregates feature maps at different resolutions using learnable weight parameters to obtain aggregated feature maps at multiple scales; the edge-aware module extracts directional and edge features from the aggregated feature maps, generates position-sensitive attention weights and edge response weights, and performs edge-preserving fusion on the aggregated feature maps to obtain edge-enhanced features; using the edge-enhanced features as input to construct a detection network, which includes a classification head and a regression head; the classification head outputs the class probability distribution of each candidate region through convolutional layers, and the regression head predicts the location of the defect bounding box through convolutional layers; a multi-task loss function for the detection model is set, which includes position-aware loss and orientation-enhanced classification loss, and the detection model is trained through backpropagation.

[0010] Furthermore, aggregated feature maps of multiple scales are obtained, including: constructing a first path from high-level features to low-level features, and a second path from low-level features to high-level features; wherein, high-level features represent features whose resolution is less than the threshold T1 output by the backbone network, and low-level features represent features whose resolution is greater than the threshold T2.

[0011] In the first path, for each layer of features: obtain the input feature P of the current layer. i and the adjacent upper-layer feature P whose size has been adjusted i+1 Set P respectively i and P i+1 The weights are then weighted and fused to obtain the intermediate features P of the current layer. i,td ;

[0012] In the second path, for each layer of features: obtain the current layer feature P. i Corresponding intermediate layer feature P i,td and the adjacent lower-layer feature P whose size has been adjusted. i-1,out ; Set feature P respectively i P i,td and P i-1,out The weights are then weighted and fused to generate the output feature P of the current layer. i,out ; The output features of all layers on the second path {P i,out}, which is the aggregated feature map generated by the feature pyramid module.

[0013] Furthermore, the aggregated feature maps are fused according to attention weights to obtain edge enhancement features, including: setting an edge-aware module after the feature pyramid module; and fusing the aggregated feature maps P at each scale. i,out Extract the horizontal and vertical feature maps Z respectively. h,i and Z w,i Meanwhile, for P i,out Edge detection is performed, the gradient intensities in the horizontal, vertical, and diagonal directions are calculated, and the gradient intensities in each direction are fused to form a gradient feature map G. e,i ; the feature map Z h,i Z w,i and G e,i Edge enhancement features are formed by combining along the channel dimension [Z] h,i Z w,i G e,i [Z]; Utilizing shared convolutional layers to enhance edge features h,i Z w,i G e,i A nonlinear transformation is performed to obtain the position-sensitive attention weights g for each scale. h,i and g w,i and edge response weight g e,i Using weight gh,i g w,i and g e,i For the corresponding aggregated feature map P i,out Edge-preserving fusion is performed to obtain edge-enhanced features {E}. i}

[0014] Furthermore, feature maps Z in the horizontal and vertical directions are extracted separately. h,i and Z w,i This includes: the aggregated feature map P i,out Multi-scale pooling operations are constructed using sliding windows of different sizes; the sizes of the sliding windows include {k1×1,k2×1,......,k n ×1} is used to extract horizontal features, {1×k1,1×k2,......,1×k} n} is used to extract vertical features; for horizontal feature extraction, k×1 sliding windows of different sizes are used along the horizontal direction at P i,out Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a horizontal feature map Z. h,i For vertical feature extraction, 1×k sliding windows of different sizes are used along the vertical direction at P. i,out Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a vertical feature map Z. w,i .

[0015] Furthermore, the edge enhancement features {E} are obtained. i}, through the following formula:

[0016] E i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ), where α is the direction perception weight coefficient, β is the edge response weight coefficient, and λ is the edge preservation base coefficient; each coefficient is adaptively adjusted according to the defect type characteristics to enhance the detection capability of defects with different edge characteristics such as ink spots and broken lines.

[0017] Furthermore, the multi-task loss function of the detection model is set, including: Loss = λ1 × loc loss +λ2×cls loss , where loc loss For position-aware loss, cls loss λ1 and λ2 are the classification loss; λ1 and λ2 are the weighting coefficients.

[0018] Furthermore, Where N is the number of samples in the batch; B iThe bounding box region containing the defect in the i-th sample; pred box,i and gt box,i These are the predicted defect bounding box and the actual defect bounding box, respectively; IoU(pred box,i ,gt box,i The intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box is denoted as .

[0019] g e,i (x, y) represents the edge response weight calculated through edge detection at position (x, y); w e These are edge response weighting coefficients used to adjust the contribution of edge features to the location loss.

[0020] Furthermore, Where N is the number of samples in the batch; C is the total number of defect categories; B i Let y be the region containing the defect in the i-th sample; i,c p represents the true label (0 or 1) of the i-th sample pair for class c; i,c g represents the predicted probability of the i-th sample for class c; h,i (x,y) and g w,i (x, y) represents the calculated attention weights in the horizontal and vertical directions at position (x, y); w d is the direction weighting coefficient, used to adjust the contribution of directional features to the classification loss.

[0021] Traditional detection systems typically use generic loss functions, which are difficult to optimize for specific defect features. On one hand, traditional IoU loss treats all regions within the predicted bounding box equally, neglecting the importance of edge regions. This application's position-aware loss function introduces edge response weights g... e,i (x,y) makes the loss function pay special attention to the defect edge region, and the edge weight makes the model pay more attention to the accurate localization of the edge region during training, thus improving the detection accuracy of irregular edges of ink dots.

[0022] On the other hand, traditional cross-entropy loss assigns the same weight to the classification error across all regions, neglecting the importance of directional features for classification. This application's orientation-enhanced classification loss introduces directional attention weights g. h,i (x,y)×g w,i (x,y) directional weights enable the model to pay special attention to regions with directional features during training, improving the classification accuracy of directional defects such as broken lines. The product of horizontal and vertical directional weights allows the system to capture complex combinations of directional features.

[0023] Another aspect of this application provides a visual recognition-based cardboard box printing defect detection system for performing a visual recognition-based cardboard box printing defect detection method of this application.

[0024] Compared to existing technologies, the advantages of this application are:

[0025] (1) Ink spots and broken lines in carton printing exhibit unique morphological characteristics. Ink spots are usually irregular in shape, with complex edges and varying sizes, while broken lines are characterized by slender structures, obvious directionality, and large contrast variations. Existing technologies generally use standard convolution or global pooling for feature extraction, which does not distinguish the directionality of features, often resulting in loss of directional information, blurred edge details, and reduced feature discrimination. The detection effect is particularly poor when dealing with fine broken lines and ink spots with complex edges.

[0026] Therefore, this application, based on a direction-aware edge extraction mechanism, utilizes directional sliding window pooling and multi-directional gradient calculation. Multi-scale pooling operations enable the system to adapt to broken line defects of varying lengths. Multi-directional gradient intensity calculation enhances sensitivity to edge features and improves the detection accuracy of ink dot edges. The fusion of horizontal, vertical, and diagonal gradients allows the system to detect edges in any direction, enhancing the comprehensiveness of detection. This solves the problems of insufficient directional feature extraction and edge information loss in traditional methods, improving the accuracy of ink dot edge localization and the accuracy of broken line direction recognition.

[0027] Ink spots and broken lines often rely on the extraction of refined edge features. In existing technologies, simple additive fusion, feature connection or feature averaging methods are generally used for feature fusion. However, these linear fusion methods can easily cause edge information to be smoothed and lost, especially in the process of deep network transmission.

[0028] Therefore, this application is approved by E. i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ) achieves edge-preserving fusion, with orientation-aware weight term α×g h,i ×g w,i The directional feature representation of the broken line was specifically enhanced, and the edge response weight term β×g was used. e,i The edge information of the ink dots is preserved, and the basic coefficient λ ensures that basic features are preserved even in areas with smaller weights. Attached Figure Description

[0029] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0030] Figure 1This is an exemplary flowchart of a visual recognition-based method for detecting defects in carton printing, according to some embodiments of this application.

[0031] Figure 2 These are schematic diagrams illustrating printing defects according to some embodiments of this application;

[0032] Figure 3 This is a schematic diagram of an edge enhancement detection model according to some embodiments of this application;

[0033] Figure 4 This is a schematic diagram of a first path and a second path according to some embodiments of this application;

[0034] Figure 5 This is a schematic diagram of an edge sensing module according to some embodiments of this application;

[0035] Figure 6 These are feature weight heatmaps of different directions shown in some embodiments of this application;

[0036] Figure 7 These are comparison images of features before and after fusion, based on some embodiments of this application. Detailed Implementation

[0037] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] like Figure 1 As shown, an image set of printed cardboard boxes is obtained; a YOLov-based edge enhancement detection model is constructed, which includes a backbone network, a neck network, and a detection network; the neck network includes a feature pyramid module and an edge perception module; the image set is used as input, and the backbone network extracts multiple feature maps of different resolutions; the multiple feature maps of different resolutions are input into the neck network, and the feature pyramid module performs adaptive aggregation to obtain an aggregated feature map, which is then obtained through the edge perception module to obtain the orientation and edge response weights, and edge preservation fusion is performed to obtain edge enhancement features; the edge enhancement features are input into the detection network, and multi-task training is performed using position-aware loss and orientation-enhanced classification loss to obtain the defect detection location and type; the defects include ink spots and broken lines.

[0039] In this embodiment, the hardware platform is: NVIDIA RTX 3090 GPU (24GB), Intel i9-10900K CPU (10 cores and 20 threads), 64GB DDR4-3200 RAM; the software environment is: Ubuntu 20.04LTS, Python 3.8.10, PyTorch 1.10.0, CUDA 11.3, cuDNN 8.2.0.

[0040] like Figure 2 As shown, the dataset constructed in this embodiment mainly includes two typical defect types: ink spots and broken lines. Ink spots are irregular spots formed by ink splashing during the printing process. They are varied in shape and have distinct edge features. They are mainly caused by unstable ink jetting on the printing press, uneven ink roller pressure, or uneven ink absorption by the paper. They typically appear as localized black or colored contamination, ranging in size from tiny dots (diameter <2mm) to medium-sized areas (diameter >10mm). Their characteristic is a clear edge outline but a highly irregular shape. Broken lines are defects where lines that should be continuous in the printed pattern are interrupted or missing. They are mainly caused by plate wear, insufficient printing pressure, or unstable ink supply. They appear as sudden breaks in the lines, with obvious directional characteristics. The break is usually relatively neat or slightly blurred. The length of the broken line ranges from tiny (<1mm) to obvious (>5mm), directly affecting the integrity and aesthetics of the product information.

[0041] The dataset consists of corrugated cardboard boxes, coated paper boxes, and pharmaceutical packaging. Original images were acquired using an industrial camera (3840×2160 resolution) and defect annotations were performed. The dataset was divided into training and validation sets. [x] min ,y min ,x max ,y max ,class id The format is ], where the ink dot (class) id =0) and broken line (class) id =1).

[0042] like Figure 3 As shown, the edge enhancement detection model of this application includes: using the YOLov network as the backbone network, and the backbone network outputs multiple feature maps of different resolutions; specifically, in this embodiment, the backbone network adopts CSPDarknet53, and the backbone network outputs the following feature maps: P1: 80×80×128 (low-level features); P2: 40×40×256 (mid-level features); P3: 20×20×512 (high-level features).

[0043] like Figure 4 As shown, the feature pyramid module is a key component of the neck network, designed to address the problem of multi-scale feature fusion in cardboard printing defect detection. The feature pyramid module constructs a dual-path structure: a top-down path from high-level features to low-level features, and a bottom-up path from low-level features to high-level features.

[0044] In this embodiment, the feature hierarchy is defined as follows: High-level features: feature maps with a resolution less than the threshold T1: 40×40, such as P3: 20×20×512; Low-level features: feature maps with a resolution greater than the threshold T2: 80×80, such as P1: 80×80×128; Intermediate-level features: feature maps with a resolution between T1 and T2, such as P2: 40×40×256.

[0045] The top-down path enhances the ability of lower-level features to understand the overall semantics of defects by passing high-level semantic information to lower levels. This is important for detecting large-scale defects or identifying defects in complex backgrounds. Specifically, the first path receives multi-scale feature maps P3 (20×20×512), P2 (40×40×256), and P1 (80×80×128) output from the backbone network. A 1×1 convolution is applied to P3 for channel dimensionality reduction (512→256) to reduce computational complexity; bilinear interpolation is used to upsample the feature map to twice its size (20×20→40×40).

[0046] Adaptive feature fusion is performed by applying learnable weight parameters to fuse features from higher layers and the current layer. The fusion formula is: P 2,td =w p ×P2+w up ×P3', where P3' represents the upsampled P3; the weight parameter is set to: w p =0.65, w up =0.35; these weights can be automatically adjusted during training based on the gradient of the loss function. For P 2,td Repeat the same process: channel dimensionality reduction, upsampling (40×40→80×80), and fuse the processed features with P1 to obtain P. 1,td This completes the cascading transmission of semantic information from high to low.

[0047] The bottom-up path transmits detailed information from lower levels to higher levels, ensuring that high-level features gain strong semantic understanding capabilities without losing fine edge information of defects. This is especially important for detecting small-scale or weak-edge defects.

[0048] In this embodiment, for the second path, the fusion formula is: P 1,out =w curr ×P1+w td ×P 1,td Weighting parameter: w curr =0.5, w td =0.35, at this stage there is no need to consider the lower-level features (P1 is the lowest level), and the lower-level output features P are generated. 1,out :80×80×128. First, consider P. 1,out Perform downsampling (80×80→40×40) and channel adjustment (128→256); fusion formula: P2,out =w curr ×P2+w td ×P 2,td +w down ×P 1,out ', P 1,out ' indicates the downsampled P 1,out Weighting parameter: w curr =0.5, w td =0.35, w down =0.15; Generate intermediate layer output feature P 2,out :40×40×256. For P 2,out Perform downsampling (40×40→20×20) and channel adjustment (256→512); fusion formula:

[0049] P 3,out =w curr ×P3+w td ×P 3,td +w down ×P 2,out ', P 2,out ' indicates the downsampled P 2,out The weight parameters are the same as those used in the middle layer fusion, generating the high-level output feature P. 3,out 20×20×512. Collect output features from all levels.

[0050] {P 1,out ,P 2,out ,P 3,out These features constitute a multi-scale feature set, which serves as the input to the edge-aware module.

[0051] The dual-path structure of this application forms a closed-loop information flow. High-level features first pass down to provide contextual information, helping the model understand the overall semantics of the defect; low-level features then pass up to provide detailed information, preserving the edge and fine texture structure of the defect. This bidirectional information flow not only improves the model's ability to perceive defects at different scales, but also enables the model to more effectively distinguish between normal printing textures on the surface of the carton and actual defects, reducing the false detection rate and improving detection accuracy.

[0052] In paper printing, ink spots and broken lines are two common and critical printing defects. Ink spot defects are characterized by irregular shapes, clear edges but complex outlines, no obvious directional features, and high contrast between the edges and the background. Broken line defects are characterized by obvious linear structures, strong directionality, and may be distributed horizontally, vertically, or diagonally. They are narrow in width but variable in length, and the edges form a linear contrast with the background. Traditional machine-based defect detection algorithms often use global pooling layers to extract features. However, global pooling essentially averages / maximizes the entire row or column, which weakens the strength of local edge features. In other words, global pooling leads to the loss of precise edge location information. For defects such as ink spots and broken lines that rely on fine edge features, global pooling weakens their feature representation and reduces recognition accuracy.

[0053] Therefore, as Figure 5 As shown, this application fuses the aggregated feature maps according to attention weights to obtain edge-enhanced features, including: setting an edge-aware module after the feature pyramid module; and fusing the aggregated feature maps P at each scale. i,out Extract the horizontal and vertical feature maps Z respectively. h,i and Z w,i For the aggregated feature map P i,out Multi-scale pooling operations are constructed using sliding windows of different sizes; the sizes of the sliding windows include {k1×1,k2×1,......,k n ×1} is used to extract horizontal features.

[0054] {1×k1,1×k2,......,1×k n} is used to extract vertical features.

[0055] For horizontal feature extraction, k×1 sliding windows of different sizes are used along the horizontal direction at point P. i,out Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a horizontal feature map Z. h,i For vertical feature extraction, 1×k sliding windows of different sizes are used along the vertical direction at P. i,out Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a vertical feature map Z. w,i The horizontal sliding window (k×1) is particularly suitable for capturing horizontal break features; the vertical sliding window (1×k) is particularly suitable for capturing vertical break features; multi-scale combinations can handle defects of different sizes and shapes simultaneously.

[0056] At the same time, for P i,out Edge detection is performed, the gradient intensities in the horizontal, vertical, and diagonal directions are calculated, and the gradient intensities in each direction are fused to form a gradient feature map G. e,iSpecifically, on the one hand, the boundary between the ink dot and the background forms a closed high-response region in the gradient map, and multi-directional gradient fusion can fully capture the irregular edges of the ink dot. On the other hand, the edges of broken lines produce a strong linear response on a specific directional gradient, and diagonal gradients can capture oblique broken lines.

[0057] Feature map Z h,i Z w,i and G e,i Edge enhancement features are formed by combining along the channel dimension [Z] h,i Z w,i G e,i ];

[0058] Utilizing shared convolutional layers to enhance edge features [Z] h,i Z w,i G e,i A nonlinear transformation is performed to obtain the position-sensitive attention weights g for each scale. h,i and g w,i and edge response weight g e,i Direction-aware weights g h,i ×g w,i Enhanced the detection capability for directional defects such as broken wires; edge response weight g e,i It improves the sensitivity to irregular edge defects such as ink spots; the adaptive coefficients α, β, λ automatically adjust the detection strategy according to the characteristics of different defect types.

[0059] Using weight g h,i g w,i and g e,i For the corresponding aggregated feature map P i,out Edge-preserving fusion is performed to obtain edge-enhanced features {E}. i};E i =P i,out ×(1+α×g h,i ×g w,i +β×g e,i +λ), where α is the direction-aware weighting coefficient, β is the edge response weighting coefficient, and λ is the edge preservation baseline coefficient; each coefficient is adaptively adjusted according to the defect type characteristics to enhance the detection capability of defects with different edge characteristics such as ink spots and broken lines. Specifically, for ink spot defects: β×g e,i The term enhances the edge features of ink dots; the λ fundamental coefficient ensures that basic features are preserved even in areas with indistinct edges. For broken lines: α×g h,i ×g w,i The term enhances the directional characteristics of the broken line; the product of the directional weights, g h,i ×g w,iIt can highlight areas that have both horizontal and vertical features, which is particularly effective for detecting broken lines at intersections or corners.

[0060] Specifically, in this embodiment, directional features are extracted using a multi-scale sliding window design: k = {3, 5, 7, 9}; for P i,out An 80×80×128 feature map is applied using a horizontal sliding window: window size: {3×1, 5×1, 7×1, 9×1}; stride: 1×1; pooling method: max pooling; result: four 80×80×128 feature maps, concatenated along the channel dimensions to obtain Z. h,1 :80×80×512; Similarly, applying the vertical sliding window {1×3, 1×5, 1×7, 1×9}, we obtain Z. w,1 :80×80×512.

[0061] To calculate the marginal gradient, the improved Sobel operator is applied to calculate the three-directional gradients: the horizontal gradient G. x Size is 3×3, weight Vertical gradient G y Size is 3×3, weight Diagonal gradient G d Size is 3×3, weight Gradient feature fusion: Calculation result: 80×80×128 gradient feature map.

[0062] Z h,1 Z w,1 and G e,1 Stitching along the channel dimension: [Z h,1 Z w,1 G e,1 [80×80×1152, weights are generated using a shared convolutional network: First layer: 3×3 convolution (1152→256) + LeakyReLU (0.1); Second layer: 1×1 convolution (256→128) + Sigmoid; resulting in weights: g] h,1 g w,1 and g e,1 (All are 80×80×128); Weight dynamic range: [0.15, 0.95]; Average weight value of edge regions: 0.82, average weight value of non-edge regions: 0.23. The original defect image in this embodiment is as follows: Figure 6 As shown in a; the generated horizontal weighted heatmap is as follows. Figure 6 As shown in b, the intensity of horizontal features is highlighted, with darker blue indicating higher weights; the generated vertical weight heatmap is shown below. Figure 6 As shown in c, the intensity of vertical features is highlighted, with darker green indicating higher weights; the generated edge response weight heatmap is shown below. Figure 6As shown in d, the intensity of edge features is highlighted, with darker red indicating higher weights. It can be seen that the weight distribution generated by the edge perception module differs significantly for different types of defects: broken line defects are prominent in the direction weight map, while ink dot defects are more significant in the edge response weight map.

[0063] For ink dot defect detection: α = 0.3, β = 0.3, λ = 0.2; for broken line defect detection: α = 0.7, β = 0.5, λ = 0.2; for P 1,out 80×80×128 Application Edge Preservation Fusion: Ink dot edge information preservation rate: improved by 43.5%; Broken line direction feature enhancement rate: improved by 38.2%; For P 2,out and P 3,out Performing the same processing yields E2 and E3. Edge preservation and feature comparison before and after fusion are shown below. Figure 7 As shown in Figure 7a, the feature map heatmap before fusion is shown in Figure 7b, and the feature map heatmap after edge-preserving fusion is shown in Figure 7b. In the fused heatmap, the ink dot edge contours are clearer, and the edge response intensity is increased by approximately 43.5%. The high edge response weight β = 0.9 ensures accurate preservation of the irregular edges of the ink dots. In the fused heatmap, the broken line region is significantly enhanced along the broken line direction, with directional features enhanced by approximately 38.2%. The high directional perception weight α = 0.7 effectively expresses the directional features of the broken line. The fused heatmap maintains the low response characteristics of non-defect areas, and the basic coefficient λ = 0.2 ensures that the basic features of low-feature areas are still preserved. The edge-preserving fusion strategy of this application, through adaptive weight coefficients, enhances features differently for different types of defects, significantly improving the detection accuracy of the two main types of defects: ink dots and broken lines.

[0064] S5, edge enhancement features {E i The data is fed into the detection network; the classification head enhances edge features through convolutional layers {E}. i The process is performed to output the probability distribution of defect categories for each candidate region; the regression head enhances the edge features {E} through convolutional layers. i The process is performed to predict the location coordinates of the defect bounding box.

[0065] Traditional machine recognition algorithms often focus on single-task optimization and uniform feature weighting in their loss functions. However, ink dots and broken lines exhibit significant differences in edge complexity and directional features. Traditional standard loss functions suffer from insufficient edge accuracy and loss of directional information in ink dot and broken line detection. Therefore, this application proposes a multi-task loss function for the detection model, including: Loss = λ1 × loc loss +λ2×cls loss , where loc lossThe location-aware loss is used to measure the deviation between the predicted defect bounding box and the true bounding box; cls loss The classification loss is used to measure the difference between the predicted defect category and the true category; λ1 and λ2 are weighting coefficients; specifically, ink dots and broken lines are two typical printing defects, which differ fundamentally in their morphological features. Ink dots usually exhibit irregular shapes and complex edge contours; while broken lines show a linear structure with obvious directionality. Traditional single loss functions are difficult to simultaneously optimize the detection performance of these two differentiated features. Therefore, this application loc loss By introducing edge response weight g e,i (x,y), which particularly enhances the precise positioning of complex edges of ink dots; cls loss Weight g based on direction perception h,i (x,y)×g w,i (x,y) significantly improves the classification accuracy of broken line direction features.

[0066] Secondly, traditional object detection algorithms typically use the standard IoU loss function for bounding box optimization. However, for printing defects such as ink spots and broken lines, the standard IoU loss function suffers from insufficient edge sensitivity and uniformity of region importance. Therefore, this application designs an edge-enhanced position-aware loss function:

[0067] Where N is the number of samples in the batch; B i The bounding box region containing the defect in the i-th sample; pred box,i and gt box,i These are the predicted defect bounding box and the actual defect bounding box, respectively; IoU(pred box,i ,gt box,i ) represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box; g e,i (x, y) represents the edge response weight calculated through edge detection at position (x, y); w e These are edge response weighting coefficients used to adjust the contribution of edge features to the location loss.

[0068] In detail, on the one hand, ink dots typically have irregular edge shapes, making it difficult for traditional IoU calculations to accurately fit their contours; the edge response weights g e,i (x,y) produces a high response value at the edge of the ink dot, as shown in the expression [1+w]. e ×g e,i The loss value [x,y] is amplified at the edges, forcing the model to learn the ink dot boundaries more accurately. This mechanism is particularly suitable for handling ink dot defects of varying sizes and shapes. On the other hand, although broken lines have a simple structure, they are extremely subtle, and their boundaries are easily ignored by traditional IoU loss. Through g... e,i(x,y) weighting gives higher loss weight to the edge regions of the broken line, and special emphasis is placed on key locations such as the start and end points of the broken line in the loss calculation. This design effectively solves the problem that slender defects are easily coarsely located. Finally, the coefficient w e The system can dynamically adjust the importance of edge responses based on defect characteristics; for defects with distinct edge features, the importance of w can be increased. e To strengthen edge constraints, ω_e can be reduced to avoid over-focusing on noisy edges, which can address the issue of blurred edges.

[0069] Finally, traditional object detection algorithms typically employ the standard cross-entropy loss function in classification tasks, assigning uniform weights to image regions. However, for printing defects with significant directional differences, such as ink spots and broken lines, the standard cross-entropy loss suffers from insufficient utilization of directional information and limited feature discrimination capabilities. Therefore, this application designs a direction-aware classification loss function: Where N is the number of samples in the batch; C is the total number of defect categories; B i Let y be the region containing the defect in the i-th sample; i,c p represents the true label (0 or 1) of the i-th sample pair for class c; i,c g represents the predicted probability of the i-th sample for class c; h,i (x,y) and g w,i (x, y) represents the calculated attention weights in the horizontal and vertical directions at position (x, y); w d These are the orientation weighting coefficients, used to adjust the contribution of orientation features to the classification loss;

[0070] In detail, by introducing directional attention weights g h,i (x,y) and g w,i The product term (x, y) allows for focused attention on regions with directional features. While traditional cross-entropy loss assigns equal weights to all pixels, this loss function differentiates weights based on the salience of directional features, significantly enhancing its ability to classify directional defects.

[0071] On the one hand, broken lines typically exhibit a clear linear structure, with directional characteristics being the key identifying feature. Horizontal broken lines, in g... h,i A high response is generated in (x,y), and the vertical break line is in g. w,i A high response is generated in (x,y), the product term g h,i (x,y)×g w,i (x,y) can capture line break features from various angles.

[0072] On the other hand, although ink dots do not have obvious directionality, the response patterns of their edge regions in different directions form a characteristic fingerprint, which can be obtained through g. h,i (x,y) and gw,i The combination of (x,y) can capture the complex geometric characteristics of the ink spot edge. This multi-directional feature representation can effectively distinguish ink spots from other defects with similar appearances. The direction-aware weights enhance the model's adaptability to changes in ink spot size and shape.

[0073] Furthermore, traditional cross-entropy loss treats the entire region uniformly, ignoring the differences in importance at different locations within the defect. In this scheme, g h,i (x,y) and g w,i (x,y) has spatial location sensitivity, which makes the loss function focus on the region with the richest classification information in the defect. For broken lines, the endpoints and center lines often contain the most critical classification information, and for ink spots, the edge contours often contain more classification information than the interior regions.

[0074] The parameters of the detection model are optimized using the backpropagation algorithm, enabling the detection network to utilize edge enhancement features {E}. i Accurately output the location and type of the defect.

[0075] The invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the invention, such design should fall within the scope of protection of this patent. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for detecting printing defects in cardboard boxes based on visual recognition, characterized in that, include: Obtain a set of printed cardboard box images; An edge enhancement detection model based on YOLov is constructed. The detection model includes a backbone network, a neck network, and a detection network. The neck network includes a feature pyramid module and an edge sensing module. The image set is used as input, and the backbone network is used to extract multiple feature maps of different resolutions. Multiple feature maps of different resolutions are input into the neck network, and the feature pyramid module performs adaptive aggregation to obtain the aggregated feature map. Then, the edge perception module obtains the orientation perception weight and edge response weight, and performs edge preservation fusion to obtain the edge enhancement feature. Edge enhancement features are input into the detection network, and multi-task training is performed using position-aware loss and orientation-enhanced classification loss to obtain the location and type of defects; among which, defects include ink dots and broken lines; Edge enhancement features are obtained, including: Set an edge-aware module after the feature pyramid module; Aggregated feature maps for each scale Extract feature maps in the horizontal and vertical directions respectively. and ; At the same time, Edge detection is performed by calculating the gradient intensity in the horizontal, vertical, and diagonal directions, and then fusing the gradient intensities in each direction to form a gradient feature map. ; feature map , and Edge enhancement features are formed by combining along the channel dimension. ; Enhance edge features using shared convolutional layers A nonlinear transformation is performed to obtain the directional attention weights in the horizontal and vertical directions at each scale location. and and edge response weights Among them, the horizontal and vertical directional attention weights serve as directional perception weights; Using weights , and For the corresponding aggregated feature map Edge-preserving fusion is performed to obtain edge-enhanced features. ; Aggregated feature maps for each scale Extract feature maps in the horizontal and vertical directions respectively. and ,include: aggregated feature maps Multi-scale pooling operations are constructed by applying sliding windows of different sizes; the sizes of the sliding windows include... Used to extract horizontal features Used to extract vertical features; For horizontal feature extraction, k×1 sliding windows of different sizes are used along the horizontal direction. Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a horizontal feature map. ; For vertical feature extraction, 1×k sliding windows of different sizes are used along the vertical direction. Max pooling is performed on the top layer, and the pooling results at all scales are concatenated along the channel dimension to form a vertical feature map. .

2. The method for detecting printing defects in cardboard boxes based on visual recognition according to claim 1, characterized in that: Constructing an edge enhancement detection model includes: The backbone network outputs multiple feature maps at different resolutions; The neck network is constructed by taking feature maps of different resolutions as input; the neck network includes a feature pyramid module and an edge sensing module. The feature pyramid module adaptively aggregates feature maps of different resolutions using learnable weight parameters to obtain aggregated feature maps of multiple scales. The edge awareness module is used to extract the directional and edge features of the aggregated feature map, generate position-sensitive attention weights and edge response weights, and perform edge-preserving fusion on the aggregated feature map to obtain edge-enhanced features; Using edge enhancement features as input, a detection network is constructed, which includes a classification head and a regression head. The classification head outputs the class probability distribution of each candidate region through convolutional layers, and the regression head predicts the location of the defect bounding box through convolutional layers. The detection model is configured with a multi-task loss function, which includes position-aware loss and orientation-enhanced classification loss. The detection model is then trained using backpropagation.

3. The method for detecting printing defects in cardboard boxes based on visual recognition according to claim 2, characterized in that: The resulting aggregated feature maps have multiple scales, including: Construct a first path from high-level features to low-level features, and a second path from low-level features to high-level features; where high-level features represent the resolution of the backbone network output being less than a threshold. The features of the lower-level features represent resolutions greater than the threshold. Features; In the first path, for each layer of features: obtain the input features of the current layer. and adjacent upper-layer features with adjusted dimensions Set up separately and The weights are then calculated and weighted to obtain the intermediate features of the current layer. ; In the second path, for each layer of features: obtain the features of the current layer. Corresponding intermediate layer features and adjacent lower-layer features that have been resized. Set features separately , and The weights are then weighted and fused to generate the output features of the current layer. ; The output features of all layers on the second path This serves as the aggregated feature map generated by the feature pyramid module.

4. The method for detecting printing defects in cardboard boxes based on visual recognition according to claim 1, characterized in that: The edge enhancement features are obtained using the following formula. : ； Where α is the direction-aware weight coefficient, β is the edge response weight coefficient, and λ is the edge preservation baseline coefficient.

5. The method for detecting printing defects in cardboard boxes based on visual recognition according to any one of claims 2 to 4, characterized in that: Configure the multi-task loss function for the detection model, including: ； in, For location perception loss, For classification loss; and These are the weighting coefficients.

6. The method for detecting printing defects in cardboard boxes based on visual recognition according to claim 5, characterized in that: ； Where N is the number of samples in the batch; The bounding box region containing the defect in the i-th sample; and These are the predicted defect bounding box and the actual defect bounding box, respectively. This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This represents the edge response weight calculated through edge detection at position (x, y). These are the edge response weighting coefficients.

7. The method for detecting printing defects in cardboard boxes based on visual recognition according to claim 6, characterized in that: ； Where N is the number of samples in the batch; C is the total number of defect categories; Let be the true label of the i-th sample pair for class c; Let be the predicted probability of the i-th sample for class c; and The horizontal and vertical attention weights at the calculated position (x, y); These are the direction-weighted coefficients.

Images