Corrugated carton chromatic aberration area detection method based on edge guiding and differential attention mechanism

Abstract

The invention discloses a corrugated carton chromatic aberration area detection method based on an edge guiding and differential attention mechanism. The method comprises the following steps: acquiring a corrugated carton image data set; constructing a color difference region detection model, wherein the color difference region detection model comprises a multilayer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a depth supervision module; training the chromatic aberration area detection model to obtain a trained chromatic aberration area detection model; in the training process, an Adam function is adopted to optimize the loss function L, after network training is completed, a corrugated carton picture with chromatic aberration is input, and a trained network model is utilized to calculate and output a regional mask image with chromatic aberration defects. According to the method, the color difference area detection performance under the complex background and different illumination conditions can be effectively improved, and the method has important significance on improving the production quality of corrugated cartons.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial defect detection, and in particular relates to a method for detecting color difference areas of corrugated boxes based on edge guidance and differential attention mechanisms. Background Art

[0002] During the production and printing of corrugated boxes, ensuring the consistency of the color of the box surface is a key factor in ensuring product quality and aesthetics. As a packaging material, corrugated boxes play an important role in visual display and brand image building, so it is particularly important to accurately detect the color difference areas on their surface. Color difference areas, that is, the parts of the box surface with inconsistent colors, are usually caused by uneven printing, inconsistent materials or other problems in the production process. If these color difference areas are not identified and corrected in time, they may cause the visual effect of the finished product to be impaired, affecting consumer perception and market acceptance of the product. Traditional color difference area detection methods are common in the corrugated box industry, but they mainly rely on manual visual inspection, which is inefficient and the accuracy is limited by personnel judgment and external environmental factors such as light conditions. This method is particularly inappropriate on high-speed production lines and cannot meet the high efficiency and high quality standards of modern corrugated box production. Summary of the invention

[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a method for detecting color difference areas of corrugated cardboard boxes based on edge-guided and differential attention mechanisms. The edge-guided technology is used to enhance the model's ability to recognize the edges and texture features of the corrugated cardboard box surface. The differential attention mechanism is used to distinguish between color difference areas and non-color difference areas, so that the model can more accurately locate and identify color differences, thereby effectively improving the color difference area detection performance under complex backgrounds and different lighting conditions. This is of great significance for improving the production quality of corrugated cardboard boxes and is expected to be widely used in the printing industry.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms, comprising the following steps:

[0006] Get the corrugated box image dataset;

[0007] Constructing a color difference region detection model, the color difference region detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0008] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0009] The color difference area detection model is trained to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a color difference corrugated cardboard box picture is input, and the trained network model is used to calculate and output a mask image of the area with color difference defects.

[0010] The corrugated box image to be detected is input into the trained color difference area detection model, and the mask image of the area with color difference defects is calculated and output.

[0011] As a preferred technical solution, the multi-layer feature extraction module includes six stages:

[0012] The first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2; the second stage consists of two convolution layers with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. After the first and second stages, the first low-level feature I1 and the second low-level feature I2 of the input image are obtained;

[0013] The network structures of the third, fourth and second stages are almost the same, except that there is no maximum pooling layer in the third stage. After the processing of the third and fourth stages, the first intermediate feature I3 and the second intermediate feature I4 are obtained respectively;

[0014] The fifth stage consists of a convolution layer with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. The sixth stage changes the maximum pooling layer to an average pooling layer based on the fifth stage. After processing in the fifth and sixth stages, the first high-level feature I5 ​​and the second high-level feature I6 are obtained respectively.

[0015] 3. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1 is characterized in that the edge prediction module is composed of three identical one-dimensional convolutional layers and an upsampling layer; the first low-level feature I1 and the second low-level feature I2 are preferentially convolved with the one-dimensional convolutional layer to perform inter-channel information fusion to generate multi-channel fusion features I1′ and I′2, I′2 is upsampled and summed with I1′ pixels to generate fused edge information, and finally, it is compressed into a single-channel edge prediction map through a 1×1 convolution to obtain the predicted edge feature I edge .

[0016] As a preferred technical solution, the multi-scale feature fusion module includes 4 convolutional layers, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 are four features extracted from the multi-layer feature extraction module, where I6 is the highest-level feature and I3 is the lowest-level feature; these features are multi-scale feature fusion network F mfusion It is divided into two categories: main features, i.e., the highest level feature I6, and auxiliary features, i.e., the remaining features I3, I4, and I5. Then, a convolution operation is used to ensure that the size of the auxiliary features is consistent with the main features. Finally, features I3, I4, I5, and I6 are connected together in the channel to obtain the multi-scale fusion feature I mfusion .

[0017] As a preferred technical solution, the edge fusion module includes a spatial attention layer and two convolutional layers; the edge fusion module transforms the predicted edge features I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, apply spatial attention mechanism to dynamically extract feature maps from different channel feature maps, and fuse them through self-learning, F efusion It is divided into 4 edge fusion modules, respectively denoted as EFM 1 ,EFM 2 ,EFM 3 ,EFM 4 , when edge fusion, F efusion First, the predicted edge feature I output by the edge prediction module edge The multi-scale fusion feature I output by the multi-scale feature fusion module mfusion Then, the convolution operation is used to generate the dual-channel feature maps a1 and a2; next, a1 and a1 are mapped to the attention maps A1 and A2 respectively through the Softmax function; finally, the feature map I is obtained by weighted summation. cat , implement the convolution operation to get I efusion .

[0018] As a preferred technical solution, the differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, guide the entire network to find the target area, and obtain the feature map I of the color difference area dam ; The differential attention network F dam It is divided into 4 differential attention modules, denoted as DAM 1 ,DAM 2 ,DAM 3 ,EFM 4 ;EFM 4 The output is DAM 3 The current prediction features are used to eliminate background noise by differentiating the current side output features pixel by pixel, and are upsampled and multiplied by EFM 3 Output I efusion As the next independent DAM 2 The input of , and so on; where the current side output features are input into the spatial and channel attention.

[0019] As a preferred technical solution, the loss function of the deep supervision module is defined as follows:

[0020] BCEloss(x i ,y i )=-wi[y i logx i+(1-y i )log(1-x i )]

[0021] where x i and i Respectively represent the probability and label of the i-th sample being correctly predicted, and wi represents the weight of the target. Finally, according to the contribution of each part to the model, a model with different weight coefficients is designed to calculate the color difference area mask image I fn , for the mask image I fn The color difference of the area containing color difference represented is calculated, and the color difference area mask image and the maximum color difference value ΔE are finally output. max and the average color difference ΔE avg .

[0022] As a preferred technical solution, the loss function L is expressed as follows:

[0023] L=0.1loss1+0.45loss2+0.2loss3+0.15loss4+0.1loss5

[0024] loss1=Gradient(Label)-I edge

[0025] loss5=Label-DAM1,DAM1=DAM(EFM1),EFM1=EFM(I edge ,I mfusion ,I6)

[0026] loss4=Label-DAM2,DAM2=DAM(DAM1×EFM2),EFM2=EFM(I edge ,I mfusion ,I5)

[0027] loss3=Label-DAM3,DAM3=DAM(DAM2×EFM3),EFM3=EFM(I edge ,I mfusion ,I4)

[0028] loss2=Label-DAM4,DAM4=DAM(DAM3×EFM4),EFM4=EFM(I edge ,I mfusion ,I3)

[0029] Among them, loss1-loss5 represent the difference between the label value and the predicted value at each stage, and finally combine into the final loss function, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value.

[0030] In a second aspect, the present invention provides a color difference region detection system for a corrugated paper box based on edge guidance and differential attention mechanism, which is applied to the color difference region detection method for a corrugated paper box based on edge guidance and differential attention mechanism, and includes a data acquisition module, a model construction module, a model training module, and a color difference defect detection module;

[0031] The data acquisition module is used to acquire a corrugated box image dataset;

[0032] The model building module is used to build a color difference area detection model, and the color difference area detection model includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0033] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module If Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0034] The model training module is used to train the color difference area detection model to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a color difference corrugated cardboard box image is input, and the trained network model is used to calculate and output a regional mask image with color difference defects.

[0035] The color difference defect detection module is used to input the corrugated box image to be detected into the trained color difference area detection model, and calculate and output the area mask image with color difference defects.

[0036] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0037] at least one processor; and,

[0038] a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the corrugated box color difference area detection method based on edge guidance and differential attention mechanism.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] 1) Improve the accuracy of color difference area recognition: Through edge-guided technology, the present invention can enhance the model's sensitivity to the edge and texture features of the corrugated box surface, especially in complex backgrounds or uneven lighting conditions, and can more accurately identify color difference areas. Edge-guided technology effectively suppresses the interference of background noise, making the positioning of color difference areas more accurate.

[0042] 2) The introduction of the differential attention mechanism enhances the ability to distinguish color differences: The differential attention mechanism can better distinguish color difference areas from normal areas by focusing on the feature differences in color difference areas, thereby reducing false detections and missed detections. This mechanism not only improves the recognition ability of the model, but also enhances its adaptability in different production batches and environments.

[0043] 3) Improved stability and reliability: In traditional color difference detection methods, changes in illumination and background often lead to unstable detection results. The technical solution adopted by the present invention significantly improves the model's adaptability to illumination changes and background complexity by integrating edge guidance and differential attention mechanisms, thereby improving the stability and reliability of color difference detection and ensuring high consistency in the production process.

[0044] 4) Broad application prospects: This invention is not only of great significance in the production quality control of corrugated boxes, but can also be extended to other printing industries. By improving the accuracy and stability of color difference detection, it can effectively improve the quality of the final product, reduce the defective product rate in production, and reduce the quality control cost, which has significant economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 The overall scheme diagram of the edge-guided and differential attention mechanisms in the implementation method.

[0047] Figure 2 Schematic diagram of a multi-layer feature extraction module in an implementation method.

[0048] Figure 3 Schematic diagram of an edge prediction module in an implementation manner.

[0049] Figure 4 Schematic diagram of a multi-scale feature fusion module in an implementation method.

[0050] Figure 5 FIG. 4 is a schematic diagram of an edge fusion module in an implementation manner. FIG.

[0051] Figure 6 Schematic diagram of the differential attention module in an implementation method.

[0052] Figure 7 Graph showing the experimental results of the overall scheme in the implementation scheme.

[0053] Figure 8 The present invention is a block diagram of a system for detecting color difference regions of corrugated boxes based on edge guidance and differential attention mechanisms according to an embodiment of the present invention.

[0054] Fig. 9 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0056] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0057] like Figure 1 As shown, this embodiment provides a method for detecting color difference regions of corrugated boxes based on edge guidance and differential attention mechanism, comprising the following steps:

[0058] S1. Obtain a corrugated box image dataset.

[0059] Furthermore, when acquiring the corrugated box image dataset, the acquisition device is an industrial area scanning camera, and a high-definition image set I with a resolution of 1390×1390 is obtained. HQ , the data set mainly contains printing color difference defects. However, a single corrugated paper box is too large, so this embodiment uses the original image I HQ The images are cropped to 256x256 size and then the groundtruth annotation is completed using Labelme software. During the training process, 70% of all images are used as training data, 20% as validation data, and 10% as test data.

[0060] S2. Construct a color difference area detection model, which includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module.

[0061] Furthermore, the multi-layer feature extraction module is used for the multi-layer feature extraction module to input the image information I0 with an input size of 512*512*3 into the network F multi , extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network Fedge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn .

[0062] In a specific embodiment, Figure 2 As shown, the multi-layer feature extraction module consists of six stages, denoted as F multi , where the first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2; the second stage consists of two convolution layers with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. After these two stages, the input image I0∈R 512×512×3 The low-level features I1∈R 256 ×256×64 and I2∈R 128×128×64 The network structures of the third, fourth and second stages are almost the same, except that there is no maximum pooling layer in the third stage. After processing in the third and fourth stages, the intermediate features I3∈R 128×128×256and I4∈R 64×64×512 The fifth stage consists of a convolution layer with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. The sixth stage changes the maximum pooling layer to an average pooling layer based on the fifth stage. After the fifth and sixth stages, the high-level features I5∈R 32×32×1024 and I6∈R 16×16×2048 The mathematical model is shown below:

[0063] I1,I2,I3,I4,I5,I6=F multi (I0)

[0064] Through multi-layer feature extraction, the edge information and color difference information in the image are separated and enhanced, and the corresponding features can be selected for more effective processing when performing multi-scale feature fusion and edge prediction.

[0065] In a specific embodiment, Figure 3 As shown in the figure, the low-level features I1 and I2 obtained by the multi-layer feature extraction module are passed to the edge prediction module for processing. The edge prediction module consists of three identical one-dimensional convolutional layers and an upsampling layer. The 64-channel low-level features I1 and I2 are first convolved with the one-dimensional convolutional layer to fuse the information between channels and generate 64-channel fused features, which are I1′ and I′2. Next, I′2 is upsampled and summed with I1′ pixels to generate fused edge information. Finally, it is compressed into a single-channel edge prediction image I through a 1×1 convolution. edge I edge The output is represented as follows:

[0066] I edge =Conv 1×1 [Conv 1×1 (I1)+UP2(Conv 1×1 (I2))]

[0067] Among them I edge To predict edge features, Con1×1 represents a convolution operation with a convolution kernel size of 1, and UP2 represents 2 upsampling.

[0068] In a specific embodiment, Figure 4 As shown in Figure 1, the multi-scale feature fusion module mainly has 4 convolutional layers. I3-I6 are four features extracted from the multi-layer feature extraction module, where I6 is the highest-level feature and I3 is the lowest-level feature. These features are combined by the multi-scale feature fusion network F mfusionThey are divided into two categories: main features, i.e., the highest level features (I6); auxiliary features, i.e., the remaining features (I3, I4, I5). Then, a convolution operation is used to ensure that their size is consistent with the main features. Finally, these features are connected together in the channel to obtain the multi-scale fusion feature I mfusion The mathematical model is shown below:

[0069] I mfusion =Cat{Conv1(I6),Cat[Conv2(I5),Conv3(I4),Conv4(I3)]}

[0070] Among them, Conv1, Conv2, Conv3, and Conv4 represent convolutional layers with different convolution kernel sizes, and Cat represents connection in the channel.

[0071] In a specific example, Figure 5 As shown in Figure 2, the edge fusion module mainly includes a spatial attention layer and two convolutional layers. The edge fusion module combines the predicted edge features I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, apply spatial attention mechanism to dynamically extract feature maps from different channel feature maps, and fuse them through self-learning, F efusion It is divided into 4 edge fusion modules, respectively denoted as EFM 1 ,EFM 2 ,EFM 3 ,EFM 4 During the implementation, F efusion First, the output feature I of the edge prediction module edge The output feature I of the multi-scale feature fusion module mfusion Then, the convolution operation is used to generate the dual-channel feature maps a1 and a2. Next, a1 and a1 are mapped to the attention maps A1 and A2 respectively through the Softmax function. Where A1, A2∈R H×W (H: height of feature map, W: width of feature map). The specific calculation process is as follows:

[0072]

[0073] Finally, the feature map I is obtained by weighted summation cat , implement the convolution operation to get I efusion The specific calculation process is as follows:

[0074] I efusion =Conv{Cat[(I edge ×A1),(I mfusion ×A2)]}

[0075] Among them I edge and I mfusion They represent the output of the edge fusion module and the output of the multi-scale feature fusion module, respectively. efusion Represents the final fusion result.

[0076] In a specific embodiment, Figure 6 As shown in Figure 2, the differential attention module mainly consists of two identical convolutional layers, a spatial attention layer and a channel attention layer. The differential attention module combines the edge fusion features I efusion Submitted to the differential attention network F dam Processing, guide the entire network to find the target area, and obtain the feature map I of the color difference area dam . F dam It is divided into 4 differential attention modules, denoted as DAM 1 ,DAM 2 ,DAM 3 , specifically EFM 4 For example, EFM 4 The output is DAM 3 It achieves background noise removal by differentiating the current side output features pixel by pixel, and is upsampled and multiplied by EFM 3 Output I efusion As the next independent DAM 2 The current side output features are input into the spatial and channel attention. It is worth noting that in order to strengthen the expansion of the target area, DAM 3 The output of is also constrained by the true value Gt. This top-down difference method can even eventually eliminate the deviation of the background area.

[0077] In a specific embodiment, the multi-level design of the model significantly increases the parameter loading. In order to speed up and simplify network optimization, deep supervision is applied at each output stage. The loss function is defined as follows:

[0078] BCEloss(x i ,y i )=-wi[y i logx i +(1-y i )log(1-x i )]

[0079] where x i and i Respectively represent the probability and label of the i-th sample being correctly predicted, and wi represents the weight of the target. Finally, according to the contribution of each part to the model, a model with different weight coefficients is designed to calculate the color difference area mask image I fn, for the mask image I fn The color difference of the area containing color difference represented is calculated, and the color difference area mask image and the maximum color difference value ΔE are finally output. max and the average color difference ΔE avg .

[0080] S3. Training the color difference area detection model to obtain a trained color difference area detection model.

[0081] During the neural network training process, the Adam function is used to optimize the loss function L. The loss function is expressed as follows:

[0082] L=0.1loss1+0.45loss2+0.2loss3+0.15loss4+0.1loss5

[0083] loss1=Gradient(Label)-I edge

[0084] loss5=Label-DAM1,DAM1=DAM(EFM1),EFM1=EFM(I edge ,I mfusion ,I6)

[0085] loss4=Label-DAM2,DAM2=DAM(DAM1×EFM2),EFM2=EFM(I edge ,I mfusion ,I5)

[0086] loss3=Label-DAM3,DAM3=DAM(DAM2×EFM3),EFM3=EFM(I edge ,I mfusion ,I4)

[0087] loss2=Label-DAM4,DAM4=DAM(DAM3×EFM4),EFM4=EFM(I edge ,I mfusion ,I3)

[0088] In the above formula, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value. After the network training is completed, the color-difference corrugated box picture is input, and the trained network model is used to calculate and output the mask image of the area with color-difference defects.

[0089] S4. Input the corrugated box image to be detected into the trained color difference area detection model, and calculate and output the mask image of the area with color difference defects.

[0090] See also Figure 7In an experimental process based on the technical solution of this patent, original images and input images of corrugated boxes under different conditions (corrugated box surfaces with slight or obvious color difference) were selected. The images include complex backgrounds and uneven lighting conditions to test the detection ability of the present invention. The model uses edge-guided technology to enhance the surface edge and texture features, combines the differential attention mechanism to accurately distinguish between color difference areas and non-color difference areas, and finally outputs the detection results. The experimental results show that the present invention can accurately locate the color difference area and give the color difference detection results under different conditions, and the quantitative index ΔE max and ΔE avg The values ​​of the coefficients of detection are between 1.6 and 3.1, respectively, which shows high-precision detection performance. In addition, the detection results still show good stability and robustness under complex lighting and background interference, which proves the practicality and reliability of this method in the actual production environment, and can effectively guide the quality control of corrugated box production process, with significant application value.

[0091] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0092] Based on the same idea as the method for detecting color difference regions of corrugated cardboard boxes based on edge-guided and differential attention mechanisms in the above-mentioned embodiment, the present invention also provides a system for detecting color difference regions of corrugated cardboard boxes based on edge-guided and differential attention mechanisms, which can be used to execute the above-mentioned method for detecting color difference regions of corrugated cardboard boxes based on edge-guided and differential attention mechanisms. For ease of explanation, the structural schematic diagram of the embodiment of the system for detecting color difference regions of corrugated cardboard boxes based on edge-guided and differential attention mechanisms only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0093] See also Figure 8 In another embodiment of the present application, a color difference region detection system 100 for a corrugated box based on edge-guided and differential attention mechanism is provided, the system comprising a data acquisition module 101, a model building module 102, a model training module 103 and a color difference defect detection module 104;

[0094] The data acquisition module 101 is used to acquire a corrugated box image dataset;

[0095] The model building module 102 is used to build a color difference area detection model, and the color difference area detection model includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0096] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0097] The model training module 103 is used to train the color difference area detection model to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a color difference corrugated cardboard box image is input, and the trained network model is used to calculate and output a mask image of the area with color difference defects.

[0098] The color difference defect detection module 104 is used to input the corrugated box image to be detected into the trained color difference area detection model, and calculate and output the area mask image with color difference defects.

[0099] It should be noted that the corrugated cardboard box color difference area detection system based on edge-guided and differential attention mechanism of the present invention corresponds one-to-one to the corrugated cardboard box color difference area detection method based on edge-guided and differential attention mechanism of the present invention. The technical features and beneficial effects described in the embodiment of the corrugated cardboard box color difference area detection method based on edge-guided and differential attention mechanism are applicable to the embodiment of the corrugated cardboard box color difference area detection based on edge-guided and differential attention mechanism. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. Hereby declared.

[0100] In addition, in the implementation of the corrugated cardboard box color difference area detection system based on edge-guided and differential attention mechanism in the above-mentioned embodiment, the logical division of each program module is only an example. In actual application, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the corrugated cardboard box color difference area detection system based on edge-guided and differential attention mechanism is divided into different program modules to complete all or part of the functions described above.

[0101] See also Fig. 9 In one embodiment, an electronic device for implementing a method for detecting color difference areas of a corrugated cardboard box based on edge-guided and differential attention mechanisms is provided. The electronic device 200 may include a first processor 201, a first memory 202, and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a program 203 for detecting color difference areas of a corrugated cardboard box based on edge-guided and differential attention mechanisms.

[0102] Among them, the first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the corrugated box color difference area detection program 203 based on edge guidance and differential attention mechanism, but also can be used to temporarily store data that has been output or is to be output.

[0103] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0104] Fig. 9 Only an electronic device with components is shown, and those skilled in the art will understand that Fig. 9 The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0105] The corrugated box color difference area detection program 203 based on edge guidance and differential attention mechanism stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0106] Get the corrugated box image dataset;

[0107] Constructing a color difference region detection model, the color difference region detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0108] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0109] The color difference area detection model is trained to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a color difference corrugated cardboard box picture is input, and the trained network model is used to calculate and output a mask image of the area with color difference defects.

[0110] The corrugated box image to be detected is input into the trained color difference area detection model, and the mask image of the area with color difference defects is calculated and output.

[0111] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0113] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms, characterized in that: The steps include: Get the corrugated box image dataset; Constructing a color difference region detection model, the color difference region detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module; The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ; Training the color difference area detection model to obtain a trained color difference area detection model; During the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a corrugated cardboard picture with color difference is input, and the trained network model is used to calculate and output the mask image of the area with color difference defects. The corrugated box image to be detected is input into the trained color difference area detection model, and the mask image of the area with color difference defects is calculated and output.

2. According to claim 1, the method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism is characterized in that: The multi-layer feature extraction module consists of six stages: The first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2; the second stage consists of two convolution layers with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. After the first and second stages, the first low-level feature I1 and the second low-level feature I2 of the input image are obtained; The network structures of the third, fourth and second stages are almost the same, except that there is no maximum pooling layer in the third stage. After the processing of the third and fourth stages, the first intermediate feature I3 and the second intermediate feature I4 are obtained respectively; The fifth stage consists of a convolution layer with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. The sixth stage changes the maximum pooling layer to an average pooling layer based on the fifth stage. After processing in the fifth and sixth stages, the first high-level feature I5 ​​and the second high-level feature I6 are obtained respectively.

3. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The edge prediction module consists of three identical one-dimensional convolutional layers and an upsampling layer; the first low-level feature I1 and the second low-level feature I2 are first convolved with the one-dimensional convolutional layer to fuse the information between channels, generate multi-channel fusion features I1′ and I′2, upsample I′2 and sum it with I1′ pixels to generate fused edge information, and finally compress it into a single-channel edge prediction map through a 1×1 convolution to obtain the predicted edge feature I edge .

4. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The multi-scale feature fusion module includes 4 convolutional layers, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 are four features extracted from the multi-layer feature extraction module, where I6 is the highest-level feature and I3 is the lowest-level feature; These features are combined by the multi-scale feature fusion network F mfusion It is divided into two categories: main features, i.e., the highest level feature I6, and auxiliary features, i.e., the remaining features I3, I4, and I5. Then, a convolution operation is used to ensure that the size of the auxiliary features is consistent with the main features. Finally, features I3, I4, I5, and I6 are connected together in the channel to obtain the multi-scale fusion feature I mfusion .

5. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The edge fusion module includes a spatial attention layer and two convolutional layers; the edge fusion module combines the predicted edge features I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, apply spatial attention mechanism to dynamically extract feature maps from different channel feature maps, and fuse them through self-learning, F efusion It is divided into 4 edge fusion modules, respectively denoted as EFM 1 ,EFM 2 ,EFM 3 ,EFM 4 , when edge fusion, F efusion First, the predicted edge feature I output by the edge prediction module edge The multi-scale fusion feature I output by the multi-scale feature fusion module mfusion Then, the convolution operation is used to generate the dual-channel feature maps a1 and a2; next, a1 and a1 are mapped to the attention maps A1 and A2 respectively through the Softmax function; finally, the feature map I is obtained by weighted summation. cat , implement the convolution operation to get I efusion .

6. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The differential attention module combines the edge fusion features I efusion Submitted to the differential attention network F dam Processing, guide the entire network to find the target area, and obtain the feature map I of the color difference area dam ; The differential attention network F dam It is divided into 4 differential attention modules, denoted as DAM 1 ,DAM 2 ,DAM 3 ,EFM 4 ;EFM 4 The output is DAM 3 The current prediction features are used to eliminate background noise by differentiating the current side output features pixel by pixel, and are upsampled and multiplied by EFM 3 Output I efusion As the next independent DAM 2 The input of , and so on; where the current side output features are input into the spatial and channel attention.

7. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The loss function of the deep supervision module is defined as follows: BCEloss(x i ,y i )=-wi[y i logx i +(1-y i )log(1-x i )] where x i and i Respectively represent the probability and label of the i-th sample being correctly predicted, and wi represents the weight of the target. Finally, according to the contribution of each part to the model, a model with different weight coefficients is designed to calculate the color difference area mask image I fn , for the mask image I fn The color difference of the area containing color difference represented is calculated, and the color difference area mask image and the maximum color difference value ΔE are finally output. max and the average color difference ΔE avg .

8. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanism according to claim 1, characterized in that: The loss function L is expressed as follows: L=0.1loss1+0.45loss2+0.2loss3+0.15loss4+0.1loss5 loss1=Gradient(Label)-I edge loss5=Label-DAM1,DAM1=DAM(EFM1),EFM1=EFM(I edge ,I mfusion ,I6) loss4=Label-DAM2,DAM2=DAM(DAM1×EFM2),EFM2=EFM(I edge ,I mfusion ,I5) <h2 style=";text-align:left;direction:ltr">loss3 = Label-DAM3, DAM3 = DAM(DAM2×EFM3), EFM3 = EFM(I<h2 style=";text-align:left;direction:ltr"> edge <h2 style=";text-align:left;direction:ltr"> ,I<h2 style=";text-align:left;direction:ltr"> mfusion <h2 style=";text-align:left;direction:ltr"> ,I4) loss2=Label-DAM4,DAM4=DAM(DAM3×EFM4),EFM4=EFM(I edge ,I mfusion ,I3) Among them, loss1-loss5 represent the difference between the label value and the predicted value at each stage, and finally combine into the final loss function, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value.

9. Corrugated box color difference area detection system based on edge-guided and differential attention mechanism, characterized by: A method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms as described in any one of claims 1 to 8, comprising a data acquisition module, a model building module, a model training module, and a color difference defect detection module; The data acquisition module is used to acquire a corrugated box image dataset; The model building module is used to build a color difference area detection model, and the color difference area detection model includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module; The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module sends the extracted first low-level feature I1 and second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5 ​​and the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge And multi-scale fusion feature I mfusion Edge Fusion Network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain the edge fusion feature I efusion ; The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then assign all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ; The model training module is used to train the color difference area detection model to obtain a trained color difference area detection model; During the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a corrugated cardboard picture with color difference is input, and the trained network model is used to calculate and output the mask image of the area with color difference defects. The color difference defect detection module is used to input the corrugated box image to be detected into the trained color difference area detection model, and calculate and output the area mask image with color difference defects.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the corrugated box color difference area detection method based on edge-guided and differential attention mechanism as described in any one of claims 1-7.

Images