Corrugated carton defect detection method and system based on adaptive spatial feature fusion and structure re-parameterization

Through the detection methods of adaptive spatial feature fusion and structural reparameterization, the problems of low accuracy and poor robustness in corrugated carton defect detection are solved, and high-precision corrugated carton defect detection is realized in high-speed flow operation, with fast speed and accuracy High advantage.

CN118918077AInactive Publication Date: 2024-11-08GUANGDONG UNIV OF TECH

Patent Information

Application Number
CN202410965021.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing corrugated carton defect detection methods have problems with low detection accuracy and poor robustness, especially in high-speed flow operation, which is difficult to maintain high-precision defect detection.

Method used

Using detection methods based on adaptive spatial feature fusion and structural reparameterization, the backbone feature extraction network, neck feature extraction network and adaptive spatial feature fusion detection network are used, combined with data set training, real-time corrugated carton image defect detection, using The structural reparameterization module introduces multi-scale and multi-receptive field features during model training, and simplifies the model structure during inference, performs feature fusion and weighting.

Benefits of technology

It improves the accuracy and robustness of corrugated carton defect detection, maintains high-speed detection without affecting the detection speed, and has a wide range of application prospects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918077A_ABST
    Figure CN118918077A_ABST
Patent Text Reader

Abstract

The invention provides a corrugated carton defect detection method and system based on adaptive spatial feature fusion and structure re-parameterization. The method comprises the following steps: acquiring image data of a corrugated carton to be detected; inputting the image data of the corrugated carton to be detected into a defect detection model, and performing real-time defect detection on the detected corrugated carton; wherein the defect detection model comprises a backbone feature extraction network, a check feature extraction network and an adaptive spatial feature fusion detection network, and is obtained based on data set training; the data set comprises corrugated carton image data containing defects and corresponding defect types. The system is simple in structure, can obtain the image of the detected corrugated carton in real time, is combined with a defect detection algorithm based on adaptive spatial feature fusion and structural re-parameterization, has high detection precision and speed, and is expected to be widely applied to the field of corrugated carton defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of corrugated paper box defect detection, and in particular relates to a corrugated paper box defect detection method and system based on adaptive spatial feature fusion and structural re-parameterization. Background Art

[0002] There are two main types of corrugated box defect detection methods: manual detection and optical detection. There are many disadvantages to manual detection methods. The detection process is inefficient, and the human eye may not be able to accurately identify some tiny, weak, and imperceptible defects, causing the defects to be ignored. Due to the above disadvantages, corrugated box defect detection has gradually turned to automated and semi-automated detection methods, such as machine vision, sensor technology, etc., to improve detection efficiency and accuracy.

[0003] Automatic optical inspection technology is a detection method based on optical principles and machine vision technology. The core is to use optical imaging technology to capture the image of the object, and then use a computer to process, analyze and identify these images to achieve automatic detection. Due to the limitations of traditional imaging systems, detection algorithms and computing power, current automatic optical inspection methods often encounter limitations such as low accuracy and slow detection speed for small defects. Therefore, it is of great significance to develop a target detection algorithm with fast detection speed and high accuracy for corrugated box defect detection. Summary of the invention

[0004] In view of the problems of low detection accuracy and poor robustness of existing corrugated cardboard defect detection algorithms, the present invention proposes a corrugated cardboard defect detection method and system based on adaptive spatial feature fusion and structural re-parameterization. The system has a simple structure and can obtain the image of the corrugated cardboard being detected in real time. Combined with the defect detection algorithm based on adaptive spatial feature fusion and structural re-parameterization, it has high detection accuracy and speed, and is expected to be widely used in the field of corrugated cardboard defect detection.

[0005] To achieve the above object, the present invention provides a corrugated box defect detection method based on adaptive spatial feature fusion and structural re-parameterization, comprising:

[0006] Collect image data of the corrugated box to be inspected;

[0007] The image data of the corrugated cardboard box to be detected is input into the defect detection model, and real-time defect detection is performed on the inspected corrugated cardboard box; wherein the defect detection model includes: a backbone feature extraction network, a neck feature extraction network and an adaptive spatial feature fusion detection network, and is obtained based on data set training; the data set includes: defective corrugated cardboard box image data and corresponding defect types.

[0008] Optionally, inputting the corrugated box image data to be detected into the defect detection model comprises:

[0009] The corrugated box image data to be detected is input into the backbone feature extraction network to obtain three feature maps of different scales. b1 , I b2 and I b3 ;

[0010] The feature map I b3 Input the neck feature extraction network to obtain three feature maps of different scales I n1 , I n2 and I n3 ;

[0011] The feature map I n1 , I n2 and I n3 Input into the adaptive spatial feature fusion detection network to obtain a prediction box; wherein the prediction box includes: defect type, prediction box center coordinates, normalized prediction box length and width, and confidence score.

[0012] Optionally, the backbone feature extraction network includes: a downsampling module, a feature extraction module and an SPPF module;

[0013] The corrugated box image data to be detected is input into the downsampling module, feature extraction module and SPPF module respectively, and three feature maps of different scales are output. b1 , I b2 and I b3 .

[0014] Optionally, the down-sampling module includes: a first down-sampling module and a second down-sampling module;

[0015] The first downsampling module is used to downsample the initial input corrugated box image with a resolution of H×W by 4 times, integrate channel information, and finally output a feature map with a resolution of H / 4×W / 4;

[0016] The second downsampling module is used to downsample the input feature map with a resolution of h×w by 2 times, integrate channel information, and finally output a feature map with a resolution of h / 2×w / 2.

[0017] Optionally, the first downsampling module implements 4-fold downsampling through two groups of convolutions with a step size of 2, and finally uses a convolution with a convolution kernel of 1 and a residual branch to integrate the channel information;

[0018] The first downsampling module uses a first dual-branch structure during the model training phase; wherein the first dual-branch structure includes: a convolution-normalization branch with a step size of 2 and a convolution kernel of 3 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 1 are connected in parallel, the outputs of the two branches are added and then pass through an activation function, and then pass through the same parallel structure, the outputs of the two branches are added and then pass through an activation function, and finally, a convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel, the outputs of the two branches are added and then pass through an activation function to obtain the output of the first downsampling module;

[0019] In the model inference stage, the first downsampling module fuses the weights through a structural reparameterization method, fuses the normalization layer into the convolution operation, and converts the first dual-branch structure into a first single-branch structure; wherein the first single-branch structure includes: a convolution-activation function with a step size of 2 and a convolution kernel of 3, a convolution-activation function with a step size of 2 and a convolution kernel of 3, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one;

[0020] The fusion process of the normalization layer is:

[0021] y conv =ω·x+b

[0022]

[0023] Among them, y conv is the output of the convolution operation, ω is the convolution weight, x is the corrugated box image input, b is the bias constant of the convolution, BN γ,β (·) is the output of the normalization function, γ and β are the trained scaling and translation parameters, μ B and is the mean and variance of all training data, ε is a constant used to prevent division by zero errors, and is the weight and bias constant of the fused convolution kernel;

[0024] The process of converting a double-branch structure to a single-branch structure is as follows:

[0025] f(x)=W 1×1 ·x+b 1×1 +W 3×3 ·x+b 3×3

[0026] =W′ 3×3 ·x+b′ 3×3

[0027] W′ 3×3 =pad0(W 1×1 )+W 3×3

[0028] b′ 3×3 =b 1×1 +b 3×3

[0029] Among them, f(·) represents the dual-branch convolution operation function, W 1×1 , W 3×3 and W′ 3×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution weight with a fused convolution kernel of 3, respectively. 1×1 、b 3×3 and b′ 3×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution bias constants of the fused convolution kernel of 3, respectively. pad0(·) represents a function that fills with 0.

[0030] Optionally, the second downsampling module uses a convolution double-branch structure with a convolution kernel of 7 and a convolution kernel of 3, both with a step size of 2, to perform 2-fold downsampling, and finally uses a convolution with a convolution kernel of 1 and a residual branch to integrate the channel information;

[0031] The second down-sampling module uses a second dual-branch structure during the model training phase; wherein the second dual-branch structure is connected in parallel by a convolution-normalization branch with a step size of 2 and a convolution kernel of 7 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 3, the outputs of the two branches are added and then passed through an activation function, and finally a convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel, the outputs of the two branches are added and then passed through an activation function to obtain the output of the second down-sampling module;

[0032] During the model inference stage, the second downsampling module fuses the weights through a structural reparameterization method, integrates the normalization layer into the convolution operation, and converts the second dual-branch structure into a second single-branch structure; wherein the second single-branch structure is composed of a convolution-activation function with a step size of 2 and a convolution kernel of 7, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one.

[0033] Optionally, the feature extraction module captures feature information at multiple scales and multiple receptive fields in space through residual connection and a three-branch feature extractor, and then integrates channel information through residual and a series of convolution, normalization, and activation operations;

[0034] The process of feature extraction in the feature extraction module is as follows:

[0035] F hd =F+BN(F)+BN(Conv 3×3 (F))+BN(Avg 3×3 (F)

[0036] F out =F hd +f seq (F hd )

[0037] Among them, F out ,F hd ∈R C×H×W represents the feature extraction module and the output feature map of the hidden layer in the module, BN(·) represents the normalization function, Conv 3×3 (·) represents the convolution branch function with a convolution kernel of 3, Avg 3×3 (·) represents the average pooling branch function with pooling kernel 3, f seq (·) represents the concatenation function of convolution with kernel 7, normalization, convolution with kernel 1, activation, and convolution with kernel 1.

[0038] Optionally, in the adaptive spatial feature fusion detection network, for any multiple upsampling, a convolution with a convolution kernel of 1 is used to adjust the number of channels to n0, and then an interpolation function is used to adjust the resolution to h0×w0; for 2x downsampling, a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the resolution and the number of channels to n0×h0×w0; for 4x downsampling, a pooling operation with a step size of 2 and a pooling kernel of 3 is first used for 2x downsampling, and then a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the number of channels and the resolution to n0×h0×w0; after adjusting the scale of the feature map, the outputs of the three layers are fused using adaptive weighted processing;

[0039] Adaptive weighted processing is used to fuse the three-layer outputs:

[0040] Y i =α i I 1→i +β i I 2→i +γ i I 3→i

[0041] α i +β i +γ i =1, and α i , β i , γ i ∈[0,1]

[0042] Among them, Y i It represents the output feature map of the i-th layer after adaptive spatial feature fusion, I n→i It represents the intermediate feature map of the nth layer after the channel number and resolution adjustment operation. When n=i, no adjustment operation is required. i , βi , γ i is the learnable weight coefficient, i∈{1, 2, 3}.

[0043] Optionally, real-time defect detection of the inspected corrugated box includes:

[0044] The inspected corrugated cardboard box image is input into the defect detection model in real time, the output prediction frames of the three detection heads are obtained, the output prediction frames are decoded and the confidence is calculated, the high confidence anchor frames are screened by non-maximum suppression, and then they are sorted from high to low according to the confidence, and then the IoU of the current prediction frame and other prediction frames is calculated, the prediction frame with the IoU overlap of the preset value is eliminated, the current frame is retained, and the selection is iterated multiple times, and finally the remaining detection frames are output; the output detection frame is drawn in the test corrugated cardboard box image; if the input inspected corrugated cardboard box image outputs the detection frame, it means that the inspected corrugated cardboard box image contains defects, otherwise, it means that the corrugated cardboard box image has no defects.

[0045] The present invention also proposes a corrugated box defect detection system based on adaptive spatial feature fusion and structural re-parameterization, the system comprising: a data processing unit, an imaging unit, an illumination device, a supporting device, and a conveying device;

[0046] The imaging unit and the lighting device are installed on the supporting device; the conveying device is located directly below the imaging unit; the corrugated paper box to be inspected is placed on the conveying device, and the image data of the corrugated paper box is collected by the imaging unit;

[0047] The data processing unit is preset with a defect detection model for performing real-time defect detection on corrugated cardboard image data; wherein the defect detection model includes: a backbone feature extraction network, a neck feature extraction network and an adaptive spatial feature fusion detection network, and the defect detection model is obtained based on data set training; the data set includes: corrugated cardboard image data containing defects and corresponding defect types.

[0048] The present invention has the following beneficial effects:

[0049] The present invention provides a method and system for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural reparameterization. In traditional corrugated paper box defect detection, it is difficult to maintain high-precision defect detection in high-speed assembly line work. The present invention proposes a re-parameterizable module that introduces rich multi-scale and multi-receptive field features during model training, increases diversified links and nonlinearity during training, effectively links contextual feature information and spatial local differences, and achieves equivalent simplification of model structure during reasoning. Adaptive spatial feature fusion is used to selectively perform feature fusion and selection, and filter conflicting spatial information between different layers. Not only is the accuracy and robustness of detection improved, but the detection speed is not affected. The method has the advantages of high speed and high precision, and has broad application prospects in the field of corrugated paper box defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0051] Figure 1 A schematic diagram of the structure of a corrugated box defect detection system based on adaptive spatial feature fusion and structural re-parameterization according to an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the overall architecture of a detection network according to an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the architecture of the downsampling module Stem according to an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the architecture of the downsampling module PatchEmb according to an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the feature extraction module architecture of an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the architecture of a feature fusion detection head module according to an embodiment of the present invention;

[0057] Among them, 101 is a computer; 102 is an imaging unit; 103 is an illumination device; 104 is a vertical support rod; 105 is a conveyor belt; 106 is an object to be inspected. DETAILED DESCRIPTION

[0058] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0059] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] like Figure 1 The embodiment shown in the figure proposes a corrugated box defect detection system based on adaptive spatial feature fusion and structural re-parameterization, including: a data processing unit, an imaging unit, a lighting device, a supporting device, and a conveying device;

[0061] The imaging unit and the lighting device are installed on the supporting device; the conveying device is located directly below the imaging unit; the corrugated paper box to be inspected is placed on the conveying device, and the image data of the corrugated paper box is collected by the imaging unit;

[0062] The data processing unit is preset with a defect detection model for real-time defect detection of corrugated cardboard image data; wherein the defect detection model includes: a backbone feature extraction network, a neck feature extraction network and an adaptive spatial feature fusion detection network, and the defect detection model is obtained based on data set training; the data set includes: defective corrugated cardboard image data and corresponding defect types.

[0063] Specifically, in this embodiment, the detection system includes: a computer 101, an imaging unit 102, an illumination device 103, a vertical support rod 104, a conveyor belt 105, and an object to be inspected 106; the computer 101 is connected to the imaging unit 102, and an image of the inspected corrugated cardboard box is obtained and defect detection is performed through a corrugated cardboard box defect detection algorithm based on adaptive spatial feature fusion and structural reparameterization. The imaging unit 102 is a high-resolution and high-frame-rate industrial camera. The imaging unit 102 and the illumination device 103 are mounted on the vertical support rod 104 to form an imaging system. The conveyor belt 105 is installed directly below the imaging unit 102. The inspected corrugated cardboard box is placed on the conveyor belt 105. As the conveyor belt 105 moves, the imaging system can capture images of the surface of the corrugated cardboard box.

[0064] The imaging system is used to capture images of corrugated paper boxes. The captured images are input into the computer 101 in real time, and the inspected corrugated paper boxes are subjected to real-time defect detection by using a network model constructed based on the yolov8 target detector, adaptive spatial feature fusion and structural reparameterization method.

[0065] This embodiment also proposes a corrugated box defect detection method based on adaptive spatial feature fusion and structural re-parameterization, including:

[0066] Collect the image data of the corrugated box to be inspected;

[0067] The image data of the corrugated cardboard box to be inspected is input into the defect detection model, and real-time defect detection is performed on the inspected corrugated cardboard box; the defect detection model includes: backbone feature extraction network, neck feature extraction network and adaptive spatial feature fusion detection network, and is obtained based on data set training; the data set includes: image data of corrugated cardboard boxes containing defects and corresponding defect types.

[0068] Specifically, in this embodiment, the RGB corrugated box image of size 3×640×640 (3 is the number of channels, 640 is the image height, and 640 is the image width) acquired by the imaging system is input into the backbone of the network model. The backbone consists of Stem and PatchEmb responsible for downsampling, feature extraction module and SPPF module. The downsampling, feature extraction module and SPPF module are used to output three feature maps of different scales. b1 , I b2 and I b3 , the scales are 96×80×80, 192×40×40 and 256×20×20 respectively. Figure 2 As shown in the figure, the number of feature extraction modules is set to 2×, 3×, 4×, and 2×. When deploying, the Stem module, PatchEmb module, and feature extraction module use the structural reparameterization method to simplify the module structure without losing accuracy to improve the model running speed.

[0069] In the neck of the network model, the feature map I obtained in the backbone network is b3 After a series of processing such as upsampling, concatenation and C2f layer, we get three feature maps of different scales I n1 , I n2 and I n3 The scales are 64×80×80, 128×40×40 and 256×20×20 respectively. At the same time, an adaptive spatial feature fusion detection head is designed to perform spatial information fusion on the feature maps of the three scales output by the neck, realize information interaction and fusion across feature maps, and thus improve the target detection performance.

[0070] The downsampling module Stem is as follows Figure 3 As shown in the figure, it is used to downsample the initial input corrugated box image with a resolution of H×W by 4 times, integrate the channel information, and finally output a feature map with a resolution of H / 4×W / 4; the downsampling module Stem achieves 4 times downsampling through 2 groups of convolutions with a step size of 2, and finally uses a convolution with a convolution kernel of 1 and a residual branch to integrate the channel information. In the model training stage, use Figure 3The dual-branch structure on the left can enrich the gradient flow information, extract features under different receptive fields, and obtain more types of feature information. In the model inference stage, the weights are fused through the structural reparameterization method, the normalization layer is fused into the convolution operation, and finally the dual-branch structure is converted to the single-branch structure on the right, which can shorten the inference time without affecting the performance of the model.

[0071] In the model training stage, the dual-branch structure on the left is: a convolution-normalization branch with a step size of 2 and a convolution kernel of 3 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 1 are connected in parallel. The outputs of the two branches are added and then pass through an activation function. Then, they pass through the same parallel structure again. The outputs of the two branches are added and then pass through an activation function. Finally, a convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel. The outputs of the two branches are added and then pass through an activation function to obtain the output of the first downsampling module.

[0072] In the model inference stage, the weights are fused through the structural reparameterization method, the normalization layer is integrated into the convolution operation, and the double-branch structure on the left is converted to the single-branch structure on the right; the single-branch structure on the right is composed of a convolution-activation function with a step size of 2 and a convolution kernel of 3, a convolution-activation function with a step size of 2 and a convolution kernel of 3, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one;

[0073] The normalization layer fusion process can be described as:

[0074] y conv =ω·x+b

[0075]

[0076]

[0077] In the above formula, y conv is the output of the convolution operation, ω is the convolution weight, x is the corrugated box image input, b is the bias constant of the convolution, BN γ,β (·) is the output of the normalization function, γ and β are the trained scaling and translation parameters, μ B and is the mean and variance of all training data, and ε is a small constant used to prevent division by zero errors. and is the weight and bias constant of the fused convolution kernel.

[0078] The process of converting a double-branch structure to a single-branch structure can be described as:

[0079] f(x)=W 1×1 ·x+b 1×1 +W3×3 ·x+b 3×3

[0080] =W′ 3×3 ·x+b′ 3×3

[0081] W′ 3×3 =pad0(W 1×1 )+W 3×3

[0082] b′ 3×3 =b 1×1 +b 3×3

[0083] In the above formula, f(·) represents the dual-branch convolution operation function, W 1×1 , W 3×3 and W′ 3×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution weight with a fused convolution kernel of 3, respectively. 1×1 , b 3×3 and b′ 3×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution bias constants of the fused convolution kernel of 3, respectively. pad0(·) represents a function that fills with 0.

[0084] The downsampling module PatchEmb is as follows Figure 4 As shown in the figure, a convolution with a convolution kernel of 7 and a convolution kernel of 3 with a stride of 2 is used for 2x downsampling, and finally a convolution with a convolution kernel of 1 and a residual branch are used to integrate the channel information. The relationship between blocks can be enhanced by using a large convolution kernel, while another smaller convolution kernel focuses on local differences. Similar to step 2.2, the structural reparameterization method is used to achieve the conversion from the left training structure to the right reasoning structure.

[0085] The downsampling module PatchEmb uses the second dual-branch structure during the model training phase; the second dual-branch structure is composed of a convolution-normalization branch with a step size of 2 and a convolution kernel of 7 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 3 in parallel. The outputs of the two branches are added and then passed through an activation function. Finally, the convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel. The outputs of the two branches are added and then passed through an activation function to obtain the output of the second downsampling module.

[0086] In the model inference stage, the downsampling module PatchEmb fuses the weights through the structural reparameterization method, integrates the normalization layer into the convolution operation, and converts the second dual-branch structure into the second single-branch structure; wherein, the second single-branch structure is composed of a convolution-activation function with a step size of 2 and a convolution kernel of 7, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one.

[0087] Feature extraction module such as Figure 5 As shown, assuming that the input feature map F∈R C×H×W , through residual connection and three-branch feature extractor, it captures feature information of multiple scales and multiple receptive fields in space, increases diversified links and nonlinearity during training, and can effectively connect the feature information of context and local differences in space. Then, it integrates channel information through residual and a series of convolution, normalization, and activation operations. The process can be described as:

[0088] F hd =F+BN(F)+BN(Conv 3×3 (F))+BN(Avg 3×3 (F)

[0089] F out =F hd +f seq (F hd )

[0090] In the above formula, F out ,F hd ∈R C×H×W represents the feature extraction module and the output feature map of the hidden layer in the module, BN(·) represents the normalization function, Conv 3×3 (·) represents the convolution branch function with a convolution kernel of 3, Avg 3×3 (·) represents the average pooling branch function with pooling kernel 3, f seq (·) represents the concatenation function of convolution with kernel 7, normalization, convolution with kernel 1, activation, and convolution with kernel 1. Similar to step 2.2, the structure reparameterization method is used to realize the conversion from the training structure on the left to the inference structure on the right.

[0091] Feature extraction module, the left structure is first the residual connection and the three-branch parallel structure output addition to obtain the middle hidden layer output feature map, where the three-branch parallel structure is composed of a normalization branch, a convolution with a convolution kernel of 3-normalization branch and an average pooling-normalization branch with a pooling kernel of 3, and then the middle hidden layer output feature map is input to the convolution kernel of 7-normalization-convolution with a convolution kernel of 1-activation function-convolution with a convolution kernel of 1 series structure and residual connection, and the results are added to obtain the feature extraction module output feature map; the right structure is first the residual connection and the convolution with a convolution kernel of 3 are added in parallel to obtain the middle hidden layer output feature map, and then the middle hidden layer output feature map is input to the convolution kernel of 7-normalization-convolution with a convolution kernel of 1-activation function-convolution with a convolution kernel of 1 series structure and residual connection, and the results are added to obtain the feature extraction module output feature map.

[0092] Adaptive spatial feature fusion detection head module such as Figure 6 As shown in the figure, assuming that the scale of the neck output feature map obtained by the current layer is n0×h0×w0, the number of channels and resolution of the remaining two neck output feature maps will be adjusted to n0×h0×w0. For any multiple upsampling, a convolution with a convolution kernel of 1 is used to adjust the number of channels to n0, and then the interpolation function is used to adjust the resolution to h0×w0; for 2x downsampling, a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the resolution and number of channels to n0×h0×w0; for 4x downsampling, a pooling operation with a step size of 2 and a pooling kernel of 3 is first used for 2x downsampling, and then a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the number of channels and resolution to n0×h0×w0. After adjusting the scale of the feature map, the outputs of the three layers are fused using adaptive weighting processing, the best fusion method of feature maps of different layers is learned, and feature fusion and selection are selectively performed to filter out conflicting spatial information between different layers. The process can be described as:

[0093] Y i =α i I 1→i +β i I 2→i +γ i I 3→i

[0094] α i +β i +γ i =1, and α i , β i , γ i ∈[0,1]

[0095] In the above formula, Y i It represents the output feature map of the i-th layer after adaptive spatial feature fusion, I n→iIt represents the intermediate feature map of the nth layer after the channel number and resolution adjustment operation. When n=i, no adjustment operation is required. i , β i , γ i is the learnable weight coefficient, i∈{1, 2, 3}.

[0096] The training is performed by gradually decreasing the learning rate, and the final learning rate is reduced to 0.00001. The loss function is consistent with the loss function in the original yolov8. The training parameters are set as follows: the initial learning rate is 0.001, the training batch size is set to 8, the training set and validation set split ratio is 100:264, and then the training set is expanded to 920 using data enhancement methods such as mosaic, the optimizer is set to AdamW, and the training cycle is set to 300.

[0097] The trained network is used for prediction, and the image of the corrugated box to be inspected taken by the imaging system is input in real time, and the inspected corrugated box image and the predicted target defect frame are output. t The image size is 3×640×640 when input into the network. After being processed by the network, the output prediction frames of the three detection heads are obtained. The prediction frames are decoded and the confidence is calculated. The high confidence anchor frames are selected by non-maximum suppression, and then sorted from high to low by confidence. Then the IoU between the current prediction frame and other prediction frames is calculated, and other prediction frames with high overlap (IoU) are removed. The current frame is retained, and multiple iterations are selected to finally output the remaining detection frames (class, x, y, w, h, conf). x, y, w, h represent the normalized center coordinates (x, y) of the prediction frame and the normalized length and width (w, h) of the prediction frame. Class is the defect type and conf is the confidence score. Draw it in the test corrugated box image. If the detection frame is output for the input corrugated box image to be inspected, it means that the inspected corrugated box image contains defects, and the defect type corresponds to the class. Otherwise, it means that the corrugated box image has no defects.

[0098] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A corrugated box defect detection method based on adaptive spatial feature fusion and structural re-parameterization, characterized in that: include: Collect the image data of the corrugated box to be inspected; Inputting the image data of the corrugated paper box to be inspected into the defect detection model, and performing real-time defect detection on the inspected corrugated paper box; The defect detection model includes: a backbone feature extraction network, a neck feature extraction network and an adaptive spatial feature fusion detection network, and is obtained based on data set training; the data set includes: defective corrugated box image data and corresponding defect types.

2. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 1 is characterized in that: Inputting the corrugated box image data to be detected into the defect detection model includes: The corrugated box image data to be detected is input into the backbone feature extraction network to obtain three feature maps of different scales. b1 , I b2 and I b3 ; The feature map I b3 Input the neck feature extraction network to obtain three feature maps of different scales I n1 , I n2 and I n3 ; The feature map I n1 , I n2 and I n3 Input into the adaptive spatial feature fusion detection network to obtain a prediction box; wherein the prediction box includes: defect type, prediction box center coordinates, normalized prediction box length and width, and confidence score.

3. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 2 is characterized in that: The backbone feature extraction network includes: a downsampling module, a feature extraction module and an SPPF module; The corrugated box image data to be detected is input into the downsampling module, feature extraction module and SPPF module respectively, and three feature maps of different scales are output. b1 , I b2 and I b3 .

4. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 3 is characterized in that: The down sampling module comprises: a first down sampling module and a second down sampling module; The first downsampling module is used to downsample the initial input corrugated box image with a resolution of H×W by 4 times, integrate channel information, and finally output a feature map with a resolution of H / 4×W / 4; The second downsampling module is used to downsample the input feature map with a resolution of h×w by 2 times, integrate channel information, and finally output a feature map with a resolution of h / 2×w / 2.

5. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 4 is characterized in that: The first downsampling module implements 4x downsampling through two groups of convolutions with a step size of 2, and finally integrates the channel information using a convolution with a convolution kernel of 1 and a residual branch; The first downsampling module uses a first dual-branch structure during the model training phase; wherein the first dual-branch structure includes: a convolution-normalization branch with a step size of 2 and a convolution kernel of 3 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 1 are connected in parallel, the outputs of the two branches are added and then pass through an activation function, and then pass through the same parallel structure, the outputs of the two branches are added and then pass through an activation function, and finally, a convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel, the outputs of the two branches are added and then pass through an activation function to obtain the output of the first downsampling module; In the model inference stage, the first downsampling module fuses the weights through a structural reparameterization method, fuses the normalization layer into the convolution operation, and converts the first dual-branch structure into a first single-branch structure; wherein the first single-branch structure includes: a convolution-activation function with a step size of 2 and a convolution kernel of 3, a convolution-activation function with a step size of 2 and a convolution kernel of 3, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one; The fusion process of the normalization layer is: y conv =ω·x+b Among them, y conv is the output of the convolution operation, ω is the convolution weight, x is the corrugated box image input, b is the bias constant of the convolution, BN γ,β (·) is the output of the normalization function, γ and β are the trained scaling and translation parameters, μ B and is the mean and variance of all training data, ε is a constant used to prevent division by zero errors, and is the weight and bias constant of the fused convolution kernel; The process of converting a double-branch structure to a single-branch structure is as follows: f(x)=W 1×1 ·x+b 1×1 +W 3×3 ·x+b 3×3 =W′ 3×3 ·x+b′ 3×3 IN' 3×3 =pad0(W 1×1 )+W 3×3 b3′ ×3 =b 1×1 +b 3×3 Among them, f(·) represents the dual-branch convolution operation function, W 1×1 , W 3×3 and W3′ ×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution weight with a fused convolution kernel of 3, respectively. 1×1 、b 3×3 and b3′ ×3 They represent the convolution with a convolution kernel of 1, the convolution with a convolution kernel of 3, and the convolution bias constants of the fused convolution kernel of 3, respectively. pad0(·) represents a function that fills with 0.

6. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 4 is characterized in that: The second downsampling module uses a convolution with a convolution kernel of 7 and a convolution kernel of 3 with a step size of 2 to perform 2 times downsampling, and finally uses a convolution with a convolution kernel of 1 and a residual branch to integrate the channel information; The second down-sampling module uses a second dual-branch structure during the model training phase; wherein the second dual-branch structure is connected in parallel by a convolution-normalization branch with a step size of 2 and a convolution kernel of 7 and a convolution-normalization branch with a step size of 2 and a convolution kernel of 3, the outputs of the two branches are added and then passed through an activation function, and finally a convolution-normalization branch with a step size of 1 and a convolution kernel of 1 and a separate normalization branch are connected in parallel, the outputs of the two branches are added and then passed through an activation function to obtain the output of the second down-sampling module; During the model inference stage, the second downsampling module fuses the weights through a structural reparameterization method, integrates the normalization layer into the convolution operation, and converts the second dual-branch structure into a second single-branch structure; wherein the second single-branch structure is composed of a convolution-activation function with a step size of 2 and a convolution kernel of 7, and a convolution-activation function with a step size of 1 and a convolution kernel of 1, which are connected in series one by one.

7. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 3 is characterized in that: The feature extraction module captures feature information at multiple scales and multiple receptive fields in space through residual connection and three-branch feature extractor, and then integrates channel information through residual and a series of convolution, normalization and activation operations; The process of feature extraction in the feature extraction module is as follows: F hd =F+BN(F)+BN(Conv 3×3 (F))+BN(Avg 3×3 (F)) F out =F hd +f seq (F hd ) Among them, F out ,F hd ∈R C×H×W represents the feature extraction module and the output feature map of the hidden layer in the module, BN(·) represents the normalization function, Conv 3×3 (·) represents the convolution branch function with a convolution kernel of 3, Avg 3×3 (·) represents the average pooling branch function with pooling kernel 3, f seq (·) represents the concatenation function of convolution with kernel 7, normalization, convolution with kernel 1, activation, and convolution with kernel 1.

8. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 1 is characterized in that: In the adaptive spatial feature fusion detection network, for any multiple upsampling, a convolution with a convolution kernel of 1 is used to adjust the number of channels to n0, and then an interpolation function is used to adjust the resolution to h0×w0; for 2x downsampling, a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the resolution and the number of channels to n0×h0×w0; for 4x downsampling, a pooling operation with a step size of 2 and a pooling kernel of 3 is first used for 2x downsampling, and then a convolution with a step size of 2 and a convolution kernel of 3 is used to adjust the number of channels and the resolution to n0×h0×w0; after adjusting the scale of the feature map, the outputs of the three layers are fused using adaptive weighted processing; Adaptive weighted processing is used to fuse the three-layer outputs: Y i =a i I 1→i +b i I 2→i +g i I 3→i α i + β i + γ i = 1, and α i , β i , γ i ∈ [0, 1] Among them, Y i It represents the output feature map of the i-th layer after adaptive spatial feature fusion, I n→i It represents the intermediate feature map of the nth layer after the channel number and resolution adjustment operation. When n=i, no adjustment operation is required. i , β i , γ i is the learnable weight coefficient, i∈{1, 2, 3}.

9. The method for detecting defects in corrugated paper boxes based on adaptive spatial feature fusion and structural re-parameterization according to claim 1, characterized in that: Real-time defect detection of corrugated boxes includes: The inspected corrugated cardboard box image is input into the defect detection model in real time, the output prediction frames of the three detection heads are obtained, the output prediction frames are decoded and the confidence is calculated, the high confidence anchor frames are screened by non-maximum suppression, and then they are sorted from high to low according to the confidence, and then the IoU of the current prediction frame and other prediction frames is calculated, the prediction frame with the IoU overlap of the preset value is eliminated, the current frame is retained, and the selection is iterated multiple times, and finally the remaining detection frames are output; the output detection frame is drawn in the test corrugated cardboard box image; if the input inspected corrugated cardboard box image outputs the detection frame, it means that the inspected corrugated cardboard box image contains defects, otherwise, it means that the corrugated cardboard box image has no defects.

10. A corrugated box defect detection system based on adaptive spatial feature fusion and structural re-parameterization, characterized in that: Used to implement the corrugated box defect detection method based on adaptive spatial feature fusion and structural re-parameterization as described in any one of claims 1 to 9, the system comprises: a data processing unit, an imaging unit, a lighting device, a supporting device, and a conveying device; The imaging unit and the lighting device are installed on the supporting device; the conveying device is located directly below the imaging unit; the corrugated paper box to be inspected is placed on the conveying device, and the image data of the corrugated paper box is collected by the imaging unit; The data processing unit is preset with a defect detection model for performing real-time defect detection on corrugated cardboard image data; wherein the defect detection model includes: a backbone feature extraction network, a neck feature extraction network and an adaptive spatial feature fusion detection network, and the defect detection model is obtained based on data set training; the data set includes: corrugated cardboard image data containing defects and corresponding defect types.

Citation Information

Patent Citations

  • Electrical equipment defect detection method based on structure re-parameterization

    CN116664490A

  • Strip steel surface defect detection method based on improved YOLO model

    CN117611571A

  • Lightweight steel surface defect detection method based on improved YOLOv8n

    CN118196529A

Cited By

  • Corrugated paper defect detection method based on transfer learning and scene adaptive segmentation

    CN120339162A