Corrugated box defect detection method based on enhanced feature fusion

Abstract

The present invention discloses a method for detecting defects of corrugated paper boxes based on enhanced feature fusion, comprising: constructing a defect detection network; extracting defects from an input image using a multi-layer feature extraction module to obtain a first feature, a second feature and a third feature; performing pyramid pooling on the third feature using a jump feature fusion module to obtain a fusion feature; performing convolution processing on the first feature, the second feature and the fusion feature using an image feature pyramid module to obtain a plurality of multi-scale feature vectors; extracting information from the multi-scale feature vectors using a defect detection head respectively, and merging and outputting information of the same type; the information includes a detection frame position and a detection classification; training a defect detection network using a loss function, and testing using the trained defect detection network. The present invention improves the accuracy of corrugated paper box area recognition and positioning when there are large differences in defect types and size scales through the extraction and fusion of multi-scale features.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial defect detection, and in particular relates to a corrugated box defect detection method based on enhanced feature fusion. Background Art

[0002] With the rapid development of modern industry, corrugated boxes tend to be produced at high speed, continuously and in large quantities. However, due to factors such as low precision of box production equipment, mismatch between equipment and image parameters or material failure, box defects may occur, which will have a negative impact on product quality. Traditional corrugated box quality inspection mainly relies on manual visual inspection or simple mechanical inspection equipment, which has problems such as low efficiency, large errors and inability to meet the needs of high-speed production lines. With the development of computer vision and image processing technology, automatic defect detection of corrugated boxes based on artificial intelligence has become an important means to improve the efficiency and accuracy of quality inspection.

[0003] Researchers have conducted extensive research on automated defect detection of corrugated cartons based on artificial intelligence, and great progress has been made in the online detection technology of carton defects. However, the existing defects of corrugated cartons, such as dirty spots, flying ink, folds, scratches, etc., have large scale differences and a large number of defect types, which are rarely considered in the algorithm, which easily leads to missed detection or misdetection of such defects. Therefore, how to improve the detection speed and detection accuracy of carton defect detection is one of the technical problems that technicians in this field need to solve urgently. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a corrugated cardboard defect detection method based on enhanced feature fusion, which improves the accuracy of corrugated cardboard area recognition and positioning when there are large differences in defect types and sizes through the extraction and fusion of multi-scale features.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a corrugated box defect detection method based on enhanced feature fusion, comprising the following steps:

[0007] Constructing a defect detection network, wherein the defect detection network includes a multi-layer feature extraction module, a jump feature fusion module, an image feature pyramid module and a defect detection head module;

[0008] A multi-layer feature extraction module is used to extract defects from an input image to obtain a first feature, a second feature, and a third feature; a jump feature fusion module is used to perform pyramid pooling on the third feature to obtain a fusion feature; an image feature pyramid module is used to perform convolution processing on the first feature, the second feature, and the fusion feature to obtain multiple multi-scale feature vectors; a defect detection head is used to extract information from the multi-scale feature vectors respectively, and information of the same type is merged and output; the information includes a detection frame position and a detection classification;

[0009] The defect detection network is trained using the loss function until the parameters converge and the training is completed; the trained defect detection network is used to output the final detection result.

[0010] As a preferred technical solution, the multi-layer feature extraction module includes a preprocessing module and a hierarchical feature extraction module, the preprocessing module includes multiple convolution kernels and pooling layers, and the hierarchical feature extraction module includes multiple residual connection blocks, each residual connection block includes multiple convolution kernels, activation functions and residual nodes; the output of the preprocessing module is connected to the input of the first residual connection block, and the other residual connection blocks are connected in series with the first residual connection block in sequence.

[0011] As a preferred technical solution, the method of extracting defects from an input image using a multi-layer feature extraction module includes the following two stages:

[0012] The preprocessing module is used to extract features from the image information to obtain the rough feature I0;

[0013] The hierarchical feature extraction module extracts features from the rough feature I0 step by step in sequence, and each residual connection block outputs a hierarchical feature, wherein the first residual connection block outputs the first feature, and the remaining residual connection blocks output the second feature and the third feature respectively.

[0014] As a preferred technical solution, the jump feature fusion module includes a feature fusion module and a jump connection module; the feature fusion module includes and n1×n1 convolution kernel, the jump connection module includes multiple diffusion pyramid pooling blocks of the same structure and a first element-by-element summation operation, each of the diffusion pyramid pooling blocks includes an n2×n2 convolution kernel and a second element-by-element summation operation, and the expansion rate of each n2×n2 convolution kernel is D and is different; each of the diffusion pyramid pooling blocks is connected through the second element-by-element summation operation, and the outputs of all diffusion pyramid pooling blocks are connected to the input of the n1×n1 convolution kernel through the first element-by-element summation operation.

[0015] As a preferred technical solution, the method of performing pyramid pooling on the third feature using the jump feature fusion module includes the following steps:

[0016] The third feature is input into the first diffusion pyramid pooling block to obtain the intermediate feature S1, and the intermediate feature S1 is input into the i-th diffusion pyramid pooling block step by step through the second element-by-element summation operation to obtain the intermediate feature S i , all the intermediate features are input into the n1×n1 convolution kernel through the first element-by-element sum operation, and the output of the n1×n1 convolution kernel is XOR-operated with the third feature to obtain the fused feature.

[0017] As a preferred technical solution, the image feature pyramid module includes several multi-scale feature extraction modules, the first multi-scale feature extraction module includes an n1×n1 convolution kernel, an upsampling operation, an XOR operation and an n2×n2 convolution kernel, the second multi-scale feature extraction module has the same structure as the first multi-scale feature extraction module, the third multi-scale feature extraction module includes an n1×n1 convolution kernel, an XOR operation and an n2×n2 convolution kernel, the fourth multi-scale feature extraction module includes an n2×n2 convolution kernel, and the fifth multi-scale feature extraction module includes an n2×n2 convolution kernel and an activation function; the first multi-scale feature extraction module is connected to the n2×n2 convolution kernel of the second multi-scale feature extraction module through the n1×n1 convolution kernel, the second multi-scale feature extraction module is connected to the n2×n2 convolution kernel of the third multi-scale feature extraction module through the n1×n1 convolution kernel, and the third multi-scale feature extraction module is connected to the activation function of the fourth multi-scale feature extraction module and the fifth multi-scale feature extraction module.

[0018] As a preferred technical solution, the convolution processing of the first feature, the second feature and the fusion feature using the image feature pyramid module includes the following steps:

[0019] Input the first feature into the first multi-scale feature extraction module through an XOR operation, input the second feature into the second multi-scale feature extraction module through an XOR operation, and input the fused feature into the third multi-scale feature extraction module and the fourth multi-scale feature extraction module respectively;

[0020] The third multi-scale feature extraction module obtains a third multi-scale feature vector through an n2×n2 convolution kernel, and transmits the third multi-scale feature vector to the second multi-scale feature extraction module;

[0021] The second multi-scale feature extraction module obtains a second multi-scale feature vector through an n2×n2 convolution kernel, and transmits the second multi-scale feature vector to the first multi-scale feature extraction module;

[0022] The first multi-scale feature extraction module obtains a first multi-scale feature vector through an n2×n2 convolution kernel;

[0023] The fourth multi-scale feature extraction module obtains a fourth multi-scale feature vector, and transmits the fourth multi-scale feature vector to the fifth multi-scale feature extraction module;

[0024] The fifth multi-scale feature extraction module obtains a fifth multi-scale feature vector.

[0025] As a preferred technical solution, the defect detection head module includes a defect detection target frame subnetwork and a defect classification subnetwork. The defect detection target frame subnetwork includes several defect frame classification feature extraction modules and defect target frame positioning modules. Each defect frame classification feature extraction module has the same weight. The defect classification subnetwork includes several defect target frame feature extraction modules and defect target classification modules.

[0026] As a preferred technical solution, the defect detection head is used to extract information from multi-scale feature vectors respectively, and the same type of information is combined and output, including:

[0027] The multi-scale feature vectors are input into the defect detection target frame sub-network and the defect classification sub-network respectively, and the defect target frame positioning module is used to obtain the defect target classification information. Use the defect target classification module to obtain defect target frame information The defect target classification information obtained and defect target frame information Merge them separately as follows:

[0028] I box_i =F box (P i ) 1≤i≤5

[0029] I class_i =F class (P i ) 1≤i≤5

[0030] I box =[I box_1 ,I box_2 ,I box_3 ,I box_4 ,I box_5 ]

[0031] I class =[I class_1 ,I class_2 ,I class_3 ,I class_4 ,I class_5 ]

[0032] Among them, F box represents the defect detection target box subnetwork, F class represents the defect classification subnetwork. box Indicates the merged defect target frame information, I class Indicates the defect target classification information after merging.

[0033] As a preferred technical solution, in the process of training the defect detection network, the Adam function is used to optimize the loss function L, as shown in the following formula:

[0034] L=L box ·W box +L class ·W class

[0035]

[0036] L class =-(1-p t ) γ log(p t )

[0037]

[0038] Among them, L box is the defect detection box loss function, W box is the defect detection box loss weight, L class is the defect detection category loss function, W class is the defect detection category loss weight, L box is the defect detection box loss function, W box is the defect detection box loss weight, L class is the defect detection category loss function, W class is the defect detection category loss weight, B is the predicted defect detection box position, G is the true defect box position, C is the overlap area between the predicted defect detection box and the true defect box, γ is the adjustment factor, y∈{-1,1} represents the foreground and background, p t Indicates the probability that the defect detection box belongs to the foreground.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] (1) By fusing multi-scale feature galaxies, the present invention can more accurately identify and locate the corrugated box area, especially when the defect types and size scales vary greatly.

[0041] (2) The present invention uses a jump feature fusion module to perform pyramid pooling on the third feature, which can extract and fuse image features in a refined manner, provide more detailed information for defect detection, and help the defect detection network quickly capture defects in corrugated boxes while reducing missed detections or false detections, thereby improving detection speed and accuracy.

[0042] (3) The present invention improves the stability and reliability of corrugated box defect detection through the image feature pyramid module and the defect detection head module, which helps to improve the quality and consistency of the final product. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 It is a flow chart of a corrugated box defect detection method based on enhanced feature fusion according to an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the structure of a multi-layer feature extraction module according to an embodiment of the present invention;

[0046] Figure 3 It is a structural schematic diagram of a jump feature fusion module according to an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the structure of an image feature pyramid module according to an embodiment of the present invention;

[0048] Figure 5 It is a structural schematic diagram of a defect detection head module according to an embodiment of the present invention;

[0049] Figure 6 This is a defect detection effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0051] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0052] Example 1

[0053] See also Figure 1 This embodiment provides a corrugated box defect detection method based on enhanced feature fusion, comprising the following steps:

[0054] S1. Construct a defect detection network, which includes a multi-layer feature extraction module 101, a jump feature fusion module 102, an image feature pyramid module 103 and a defect detection head module 104. The multi-layer feature extraction module 101 is used to extract the underlying details and high-level semantic features of the image respectively; the jump feature fusion module 102 enhances the detection capability of small defects by fusing features of different levels; the image feature pyramid module 103 uses a multi-scale feature structure to process defects of different sizes; the defect detection head module 104 realizes accurate positioning and classification of defects, and outputs the location and category of the defects.

[0055] like Figure 1 As shown, this embodiment uses the above four modules in the defect detection network to quickly and accurately detect the defects of the corrugated box. For details, please refer to step S2, which is the training step of the defect detection network:

[0056] S2, multi-layer feature extraction module 101 inputs image information I0 into network F multi , obtain image features I1, I2, I3, and input the obtained feature I3 into the jump feature fusion module 102 to obtain the multi-scale feature I4. The image feature pyramid module 103 inputs the above features I1, I2, I4 into the network F fpn Obtain image feature maps P1, P2, P3, P4, P5; the defect detection head module 104 obtains the multi-scale feature maps P1, P2, P3, P4, P5 and passes them to the defect detection target frame sub-network F box 401, defect classification subnetwork F class 402, finally generating defect target frame information I of different sizes at each position of the feature map box and defect target classification information I class .

[0057] The module structures and processing steps in this embodiment are as follows:

[0058] like Figure 2 As shown, in this embodiment, the multi-layer feature extraction module 101 includes a preprocessing module 201 and a hierarchical feature extraction module 202, the preprocessing module 201 includes multiple convolution kernels and pooling layers, and the hierarchical feature extraction module 202 includes multiple residual connection blocks, each residual connection block includes multiple convolution kernels, activation functions and residual nodes; the output of the preprocessing module 201 is connected to the input of the first residual connection block, and the other residual connection blocks are connected in series with the first residual connection block in sequence.

[0059] The multi-layer feature extraction module 101 is divided into two stages in feature processing. The calculation process of the first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2, denoted as F pre , transform the image of dimension C Processed as a tensor with a dimension of 32 Where C = 32, H, W are the image height and length, and the mathematical model is as follows:

[0060] I0=F pre (I)

[0061] The second stage consists of three residual connection blocks, where the calculation process of each residual connection block is as follows: Figure 2 As shown, denoted as F res , each residual connection block extracts the feature tensor Its mathematical model is as follows:

[0062] I i =F res (I i-1 ),i=1,2,3

[0063] Among them, feature tensors I1 and I2 are directly input into the image feature pyramid module, and I3 is input into the jump feature fusion module for the next step of calculation.

[0064] The jump feature fusion module 102 is as follows Figure 3 As shown, the jump feature fusion module 102 includes a feature fusion module 301 and a jump connection module 302; the feature fusion module 301 includes and a 1×1 convolution kernel, the jump connection module 302 includes 5 diffusion pyramid pooling blocks of the same structure and an element-by-element summation operation (referred to as the “first element-by-element summation operation”), each of the diffusion pyramid pooling blocks has a 3×3 convolution kernel and an element-by-element summation operation (referred to as the “second element-by-element summation operation”), the expansion rate of each 3×3 convolution kernel is D and they are all different, starting from the first input diffusion pyramid pooling block, the expansion rate of the 3×3 convolution kernel D increases successively, which are 3, 6, 12, 18, and 24 respectively; each of the diffusion pyramid pooling blocks is connected through the second element-by-element summation operation, and the outputs of all the diffusion pyramid pooling blocks are connected to the input of the 1×1 convolution kernel through the first element-by-element summation operation.

[0065] During feature processing in the jump feature fusion module 102, the obtained feature tensor I3 is subjected to multi-scale fusion feature extraction, and its mathematical model is as follows:

[0066] I scale =F skip (I3)

[0067] In the above formula, F skip Represents the skip feature fusion module.

[0068] Specifically, the jump feature fusion module 102 inputs the feature tensor I3 into the first diffusion pyramid pooling block to obtain the first intermediate feature, and then inputs the first intermediate feature into the next diffusion pyramid pooling block step by step through the second element-by-element summation operation to obtain the next intermediate feature. Before the last diffusion pyramid pooling block is processed, the intermediate features output by the previous diffusion pyramid pooling block are passed to the next diffusion pyramid pooling block for further processing. After being processed by all diffusion pyramid pooling blocks, each diffusion pyramid pooling block can obtain an intermediate feature. In this embodiment, these intermediate features are recorded as: S1, S2, S3, S4, S5.

[0069] Next, all the intermediate features are input into the 1×1 convolution kernel through the first element-by-element summation operation, and the output of the 1×1 convolution kernel is XORed with the feature tensor I3 to obtain the fused feature tensor

[0070] Intermediate features and fused feature tensors The calculation formula is as follows:

[0071]

[0072] S assp =F add (Conv1(F cat (S1,S2,S3,S4,S5)),I3)

[0073] In the above formula, Conv D Conv1 represents a convolution operation with a convolution kernel size of 1×1 and a dilation rate of D. cat Indicates the connection operation, F add Represents the element-by-element sum operation. The fused feature tensor S add Input the image feature pyramid module for the next step of calculation.

[0074] See also Figure 4, the image feature pyramid module includes five multi-scale feature extraction modules, wherein the first multi-scale feature extraction module 401 includes a 1×1 convolution kernel, an upsampling operation, an XOR operation and a 3×3 convolution kernel, the second multi-scale feature extraction module 402 has the same structure as the first multi-scale feature extraction module 401, the third multi-scale feature extraction module 403 includes a 1×1 convolution kernel, an XOR operation and a 3×3 convolution kernel, the fourth multi-scale feature extraction module 404 includes a 3×3 convolution kernel, and the fifth multi-scale feature extraction module 405 includes a 3×3 convolution kernel and a Relu activation function; the first multi-scale feature extraction module 401 is connected to the 3×3 convolution kernel of the second multi-scale feature extraction module 402 through the 1×1 convolution kernel, the second multi-scale feature extraction module 402 is connected to the 3×3 convolution kernel of the third multi-scale feature extraction module 403 through the 1×1 convolution kernel, and the third multi-scale feature extraction module 403 is connected to the Relu activation function of the fifth multi-scale feature extraction module 405 through the fourth multi-scale feature extraction module 404.

[0075] When the feature tensor obtained by the jump feature fusion module 102 and the feature tensor obtained in the multi-layer feature extraction module 103 The image feature pyramid module 104 is passed to process to obtain a multi-scale feature vector

[0076] It is worth explaining that Figure 1 The image feature pyramid module 104 part and Figure 4 The internal processing arrows shown in the figure show that the extraction of multi-scale feature vectors is divided into two directions and five dimensions. Each multi-scale feature vector has its own characteristics and details. For example, for multi-scale feature vectors In terms of The features of the feature tensor will also be saved and details.

[0077] In order to more clearly illustrate the process of obtaining the multi-scale feature vector, this embodiment lists the following mathematical model:

[0078]

[0079] In the above formula, Conv1 and Conv3 represent convolution operations with kernel sizes of 1×1 and 3×3 respectively, and F up represents an upsampling operation with an amplitude of 2, F relu Represents the activation function.

[0080] In the above feature tensor as well as When the image feature pyramid module 104 is input, the processing in two directions is as follows:

[0081] First direction: the third multi-scale feature extraction module 403 receives the feature tensor Then the feature vector is output through a 3×3 convolution kernel And the feature vector The second multi-scale feature extraction module 402 processes the feature tensor At the same time, receiving multi-scale feature vectors Perform convolution and upsampling operations and transform the feature tensor Perform an XOR operation on the multi-scale feature vector P3 after the above operation to output the multi-scale feature vector The first multi-scale feature extraction module 401 processes the feature tensor in the same way as the second multi-scale feature extraction module 402 and multi-scale feature vectors Output multi-scale feature vector

[0082] Second direction: transform the feature tensor Input the fourth multi-scale feature extraction module, which uses a 3×3 convolution kernel to transform the feature tensor Process and obtain multi-scale feature vectors At the same time, the multi-scale feature vector Passed to the fifth multi-scale feature extraction module; the fifth multi-scale feature extraction module inputs the multi-scale feature vector After that, it is processed by Relu activation function and 3×3 convolution kernel to output multi-scale feature vector

[0083] like Figure 5 As shown, the defect detection head module 104 includes a defect detection target frame sub-network and a defect classification sub-network, wherein the defect detection target frame sub-network includes 4 defect frame classification feature extraction modules 501 and 1 defect target frame positioning module 503, each defect frame classification feature extraction module 501 has the same weight, and the defect classification sub-network also includes 4 defect target frame feature extraction modules 502 and 1 defect target classification module 504. The difference between the two sub-networks is that a sigmoid activation function is added after the convolution kernel in the defect target frame positioning module 503.

[0084] Specifically, in a defect detection head module with 8 defect types and a scene in which the number of defects is 6, the defect frame classification feature extraction module 501 includes a 3×3 convolution kernel, a Relu activation function and an XOR operation. When the feature vector is input, it is passed to the 3×3 convolution kernel and the XOR operation respectively, and the structure of each defect target frame feature extraction module 502 is consistent with the defect frame classification feature extraction module 501; as for the defect target frame positioning module 503, it includes a 3×3 convolution kernel, a sigmod activation function and a Reshape function. When the defect frame classification feature extraction module 501 completes feature processing, the vector features are sequentially passed through the 3×3 convolution kernel, the sigmod activation function and the Reshape function to obtain the defect target classification information. The defect target classification module 504 includes a 3×3 convolution kernel and a Reshape function. After the feature processing of the defect target classification module 504, the defect target frame information is obtained.

[0085] The calculation formula of the defect detection head module is as follows:

[0086] I box_i =F box (P i ) 1≤i≤5

[0087] I class_i =F class (P i ) 1≤i≤5

[0088] I box =[I box_1 ,I box_2 ,I box_3 ,I box_4 ,I box_5 ]

[0089] I class =[I class_1 ,I class_2 ,I class_3 ,I class_4 ,I class_5 ]

[0090] In the above formula, F box represents the defect detection target box subnetwork, F class represents the defect classification subnetwork. box , I class The defects of each scale are merged and output.

[0091] S3. Use the loss function to train the defect detection network until the training is completed after the parameters converge; use the trained defect detection network to output the final detection result.

[0092] During the neural network training process, the Adam function is used to optimize the loss function L. The loss function is expressed as follows:

[0093] L=L box *W box +L class +W class

[0094] In the above formula, L box is the defect detection box loss function, W box is the defect detection box loss weight, L class is the defect detection category loss function, W class is the defect detection category loss weight. box and L class It is expressed as follows:

[0095]

[0096] L class =-(1-p t ) γ log(p t )

[0097] In the above formula, B is the predicted defect detection frame position, G is the real defect frame position, C is the overlapping area between the predicted defect detection frame and the real defect frame; γ is the adjustment factor, p t It is expressed as:

[0098]

[0099] In the above formula, y∈{-1,1} represents the foreground and background, p t Indicates the probability that the defect detection box belongs to the foreground.

[0100] Example 2

[0101] A more specific example is given in Example 2. When training and verifying the defect detection network in Example 2, the data set is prepared as follows:

[0102] The acquisition device is an industrial area scanning camera, which obtains a high-definition image set I with a resolution of 2048×2048. HQ ,The dataset mainly contains printing color difference defects. However, a single corrugated box is too large, so we HQ The images are cropped to a size of 1024 × 1024, and then the defect category and defect location annotations are completed using Labelimg software. During the training process, 70% of all images are used as training data, 20% as validation data, and 10% as test data.

[0103] The remaining steps are similar to those in Example 1. The number of residual connection blocks, diffuse pyramid pooling blocks, defect frame classification feature extraction modules, and defect target frame feature extraction modules can be adjusted according to actual conditions.

[0104] like Figure 6 As shown in the figure, the defect detection effect of corrugated boxes is good. The defects such as creases and scratches in the input image are accurately detected and marked, the target frame is accurately positioned, and the classification result is correct.

[0105] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0106] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A corrugated box defect detection method based on enhanced feature fusion, characterized in that: The steps include: Constructing a defect detection network, wherein the defect detection network includes a multi-layer feature extraction module, a jump feature fusion module, an image feature pyramid module and a defect detection head module; A multi-layer feature extraction module is used to extract defects from an input image to obtain a first feature, a second feature, and a third feature; a jump feature fusion module is used to perform pyramid pooling on the third feature to obtain a fusion feature; an image feature pyramid module is used to perform convolution processing on the first feature, the second feature, and the fusion feature to obtain multiple multi-scale feature vectors; a defect detection head is used to extract information from the multi-scale feature vectors respectively, and information of the same type is merged and output; the information includes a detection frame position and a detection classification; The defect detection network is trained using the loss function until the parameters converge and the training is completed; the trained defect detection network is used to output the final detection result.

2. The corrugated box defect detection method based on enhanced feature fusion according to claim 1 is characterized in that: The multi-layer feature extraction module includes a preprocessing module and a hierarchical feature extraction module, the preprocessing module includes multiple convolution kernels and pooling layers, and the hierarchical feature extraction module includes multiple residual connection blocks, each residual connection block includes multiple convolution kernels, activation functions and residual nodes; the output of the preprocessing module is connected to the input of the first residual connection block, and the other residual connection blocks are connected in series with the first residual connection block in sequence.

3. The corrugated box defect detection method based on enhanced feature fusion according to claim 2 is characterized in that: The method of extracting defects from an input image using a multi-layer feature extraction module includes the following two stages: The preprocessing module is used to extract features from the image information to obtain the rough feature I0; The hierarchical feature extraction module extracts features from the rough feature I0 step by step in sequence, and each residual connection block outputs a hierarchical feature, wherein the first residual connection block outputs the first feature, and the remaining residual connection blocks output the second feature and the third feature respectively.

4. According to claim 1, the method for detecting defects in corrugated paper boxes based on enhanced feature fusion is characterized in that: The jump feature fusion module includes a feature fusion module and a jump connection module; the feature fusion module includes and n1×n1 convolution kernel, the jump connection module includes a plurality of diffusion pyramid pooling blocks of the same structure and a first element-by-element summation operation, each of the diffusion pyramid pooling blocks includes an n2×n2 convolution kernel and a second element-by-element summation operation, and the expansion rate of each n2×n2 convolution kernel is D and is different; each of the diffusion pyramid pooling blocks is connected through the second element-by-element summation operation, and the outputs of all the diffusion pyramid pooling blocks are connected to the input of the n1×n1 convolution kernel through the first element-by-element summation operation.

5. The corrugated box defect detection method based on enhanced feature fusion according to claim 4 is characterized in that: The step of performing pyramid pooling on the third feature by using the jump feature fusion module comprises the following steps: The third feature is input into the first diffusion pyramid pooling block to obtain the intermediate feature S1, and the intermediate feature S1 is input into the i-th diffusion pyramid pooling block step by step through the second element-by-element summation operation to obtain the intermediate feature S i , all the intermediate features are input into the n1×n1 convolution kernel through the first element-by-element sum operation, and the output of the n1×n1 convolution kernel is XOR-operated with the third feature to obtain the fused feature.

6. The corrugated box defect detection method based on enhanced feature fusion according to claim 1 is characterized in that: The image feature pyramid module includes several multi-scale feature extraction modules, wherein the first multi-scale feature extraction module includes an n1×n1 convolution kernel, an upsampling operation, an XOR operation and an n2×n2 convolution kernel, the second multi-scale feature extraction module has the same structure as the first multi-scale feature extraction module, the third multi-scale feature extraction module includes an n1×n1 convolution kernel, an XOR operation and an n2×n2 convolution kernel, the fourth multi-scale feature extraction module includes an n2×n2 convolution kernel, and the fifth multi-scale feature extraction module includes an n2×n2 convolution kernel and an activation function; the first multi-scale feature extraction module is connected to the n2×n2 convolution kernel of the second multi-scale feature extraction module through the n1×n1 convolution kernel, the second multi-scale feature extraction module is connected to the n2×n2 convolution kernel of the third multi-scale feature extraction module through the n1×n1 convolution kernel, and the third multi-scale feature extraction module is connected to the activation function of the fourth multi-scale feature extraction module and the fifth multi-scale feature extraction module.

7. The corrugated box defect detection method based on enhanced feature fusion according to claim 6 is characterized in that: The method of using the image feature pyramid module to perform convolution processing on the first feature, the second feature and the fusion feature includes the following steps: Input the first feature into the first multi-scale feature extraction module through an XOR operation, input the second feature into the second multi-scale feature extraction module through an XOR operation, and input the fused feature into the third multi-scale feature extraction module and the fourth multi-scale feature extraction module respectively; The third multi-scale feature extraction module obtains a third multi-scale feature vector through an n2×n2 convolution kernel, and transmits the third multi-scale feature vector to the second multi-scale feature extraction module; The second multi-scale feature extraction module obtains a second multi-scale feature vector through an n2×n2 convolution kernel, and transmits the second multi-scale feature vector to the first multi-scale feature extraction module; The first multi-scale feature extraction module obtains a first multi-scale feature vector through an n2×n2 convolution kernel; The fourth multi-scale feature extraction module obtains a fourth multi-scale feature vector, and transmits the fourth multi-scale feature vector to the fifth multi-scale feature extraction module; The fifth multi-scale feature extraction module obtains a fifth multi-scale feature vector.

8. According to claim 1, the method for detecting defects in corrugated paper boxes based on enhanced feature fusion is characterized in that: The defect detection head module includes a defect detection target frame subnetwork and a defect classification subnetwork. The defect detection target frame subnetwork includes several defect frame classification feature extraction modules and defect target frame positioning modules. Each defect frame classification feature extraction module has the same weight. The defect classification subnetwork includes several defect target frame feature extraction modules and defect target classification modules.

9. The corrugated box defect detection method based on enhanced feature fusion according to claim 8, characterized in that: The defect detection head is used to extract information from multi-scale feature vectors respectively, and information of the same type is combined and output, including: The multi-scale feature vectors are input into the defect detection target frame sub-network and the defect classification sub-network respectively, and the defect target frame positioning module is used to obtain the defect target classification information. Use the defect target classification module to obtain defect target frame information The defect target classification information obtained and defect target frame information Merge them separately as follows: I box_i =F box (P i )1≤i≤5 I class_i =F class (P i )1≤i≤5 I box =[I box_1 ,I box_2 ,I box_3 ,I box_4 ,I box_5 ] I class =[I class_1 ,I class_2 ,I class_3 ,I class_4 ,I class_5 ] Among them, F box represents the defect detection target box subnetwork, F class represents the defect classification subnetwork, I box Indicates the merged defect target frame information, I class Indicates the defect target classification information after merging.

10. The corrugated box defect detection method based on enhanced feature fusion according to claim 1, characterized in that: In the process of training the defect detection network, the Adam function is used to optimize the loss function L, as shown below: L=L box ·W box +L class ·W class L class =-(1-p t ) γ log(p t ) Among them, L box is the defect detection box loss function, W box is the defect detection box loss weight, L class is the defect detection category loss function, W class is the defect detection category loss weight, L box is the defect detection box loss function, W box is the defect detection box loss weight, L class is the defect detection category loss function, W class is the defect detection category loss weight, B is the predicted defect detection box position, G is the true defect box position, C is the overlap area between the predicted defect detection box and the true defect box, γ is the adjustment factor, y∈{-1,1} represents the foreground and background, p t Indicates the probability that the defect detection box belongs to the foreground.

Images