Corrugated box color difference region detection method based on edge guidance and differential attention mechanism

Abstract

The application discloses a corrugated box color difference area detection method based on edge guidance and differential attention mechanism, comprising: acquiring a corrugated box image data set; constructing a color difference area detection model, the color difference area detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module; training the color difference area detection model to obtain a trained color difference area detection model; in the training process, an Adam function is used to optimize a loss function L, after network training is completed, a corrugated box picture with color difference is input, and a trained network model is used to calculate an output region mask image with color difference defects. The application can effectively improve the color difference area detection performance under complex background and different illumination conditions, and has important significance for improving the production quality of corrugated boxes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial defect detection, and particularly relates to a corrugated box color difference region detection method based on edge guidance and differential attention mechanism. BACKGROUND

[0002] In the production and printing process of corrugated boxes, ensuring the consistency of the surface color of the corrugated box is a key factor to ensure product quality and aesthetics. As a packaging material, corrugated boxes play an important role in visual display and brand image shaping, so it is particularly important to accurately detect the color difference regions on the surface of the corrugated box. Color difference regions, i.e. parts of the surface of the corrugated box where the color is inconsistent, are usually caused by uneven printing, inconsistent materials or other problems in the production process. If these color difference regions are not identified and corrected in time, the visual effect of the finished product may be damaged, affecting the perception of consumers and the market acceptance of the product. Traditional color difference region detection methods are common in the corrugated box industry, but mainly rely on manual visual inspection, which is low in efficiency and accuracy is limited by personnel's judgment and external environmental factors such as light conditions. This method is particularly unsuitable for high-speed production lines and cannot meet the high efficiency and high quality standards of modern corrugated box production. SUMMARY

[0003] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a corrugated box color difference region detection method based on edge guidance and differential attention mechanism. The edge guidance technology enhances the model's ability to recognize the edge and texture features of the corrugated box surface, and the differential attention mechanism distinguishes between color difference regions and non-color difference regions, enabling the model to more accurately locate and identify color differences, thereby effectively improving the color difference region detection performance in complex backgrounds and different lighting conditions. This has important significance for improving the production quality of corrugated boxes and is expected to be widely used in the printing industry.

[0004] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0005] In a first aspect, the present application provides a corrugated box color difference region detection method based on edge guidance and differential attention mechanism, comprising the following steps:

[0006] Obtain the corrugated box image data set;

[0007] Construct a color difference region detection model, the color difference region detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0008] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0009] The color difference area detection model is trained to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a corrugated cardboard box image with color difference is input, and the trained network model is used to calculate and output a mask image of the area with color difference defects.

[0010] The corrugated cardboard image to be detected is input into the trained color difference area detection model, and the mask image of the area with color difference defects is calculated and output.

[0011] As a preferred technical solution, the multi-layer feature extraction module comprises six stages:

[0012] The first stage is composed of a convolution layer with a kernel size of 7, a convolution layer with a kernel size of 3, and a maximum value pooling layer with a kernel size of 2; the second stage is composed of two convolution layers with a kernel size of 3 and a maximum value pooling layer with a kernel size of 2, and after the first and second stages, the first low-level feature I1 and the second low-level feature I2 of the input image are obtained;

[0013] The network structures of the third stage, the fourth stage and the second stage are almost the same, except that the third stage does not have a maximum value pooling layer, and after the processing of the third and fourth stages, the first intermediate feature I3 and the second intermediate feature I4 are obtained, respectively;

[0014] The fifth stage is composed of a convolution layer with a kernel size of 3 and a maximum value pooling layer with a kernel size of 2, and the sixth stage changes the maximum value pooling layer to an average pooling layer based on the fifth stage, and after the processing of the fifth and sixth stages, the first high-level feature I5 and the second high-level feature I6 are obtained, respectively.

[0015] 3. The corrugated box color difference region detection method based on edge orientation and differential attention mechanism according to claim 1, wherein the edge prediction module is composed of three same one-dimensional convolution layers and an up-sampling layer; the first low-level feature I1 and the second low-level feature I2 are preferentially convolved with the one-dimensional convolution layer, the inter-channel information is fused, the multi-channel fusion features I1' and I'2 are generated, I'2 is up-sampled and summed with I1' pixels to produce fused edge information, and finally, the edge prediction feature I is obtained by compressing it into a single-channel edge prediction map through 1x1 convolution. edge .

[0016] As a preferred technical solution, the multi-scale feature fusion module comprises four convolution layers, and the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5 and the second high-level feature I6 are four features extracted from the multi-layer feature extraction module, wherein I6 is the highest layer feature and I3 is the lowest layer feature; these features are divided into two categories: main features, i.e. the highest level feature I6, and auxiliary features, i.e. the remaining features I3, I4 and I5; then, convolution operation is used to ensure that the size of the auxiliary features is consistent with the main feature; finally, the features I3, I4, I5 and I6 are connected together in the channel to obtain the multi-scale fusion feature I mfusion .

[0016] As a preferred technical solution, the multi-scale feature fusion module comprises four convolution layers, and the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5 and the second high-level feature I6 are four features extracted from the multi-layer feature extraction module, wherein I6 is the highest layer feature and I3 is the lowest layer feature; these features are divided into two categories: main features, i.e. the highest level feature I6, and auxiliary features, i.e. the remaining features I3, I4 and I5; then, convolution operation is used to ensure that the size of the auxiliary features is consistent with the main feature; finally, the features I3, I4, I5 and I6 are connected together in the channel to obtain the multi-scale fusion feature I mfusion .

[0017] As a preferred technical solution, the edge fusion module comprises a spatial attention layer and two convolution layers; the edge fusion module combines the predicted edge feature I edge and the multi-scale fusion feature I mfusion delivered to the edge fusion network F efusion for processing, dynamically extracts feature maps from different channel feature maps by applying a spatial attention mechanism, and performs fusion through self-learning, F efusion is divided into four edge fusion modules, respectively denoted as EFM 1 , EFM 2 , EFM 3 , and EFM 4 . During edge fusion, F efusion first concatenates the predicted edge feature I edge output by the edge prediction module with the multi-scale fusion feature I mfusion output by the multi-scale feature fusion module; then, convolution operation is performed to generate double-channel feature maps a1 and a2; next, a1 and a2 are respectively mapped to attention maps A1 and A2 through Softmax functions; finally, feature map I cat is obtained through weighted summation, and I efusion is obtained through convolution operation.

[0018] As a preferred technical solution, the differential attention module processes the edge fusion feature I efusion through a differential attention network F dam to guide the entire network to find the target region and obtain the feature map I dam of the color difference region; the differential attention network F dam is divided into four differential attention modules, respectively denoted as DAM 1 , DAM 2 , DAM 3 , and EFM 4 ; the output of EFM 4 is the current prediction feature of DAM 3 , which realizes background noise elimination by distinguishing the current side output feature pixel by pixel, and is up-sampled and multiplied by the output I 3 of EFM efusion as the input of the next independent DAM 2 , and so on; wherein, the current side output feature is input to the spatial and channel attention.

[0019] As a preferred technical solution, the loss function of the deep supervision module is defined as follows:

[0020] BCEloss(x i ,y i )=-wi[y i logx i+(1-y i )log(1-x i )]

[0021] where x i and y i represent the probability of the i-th sample being correctly predicted and the label respectively, and wi represents the weight of the target. Finally, according to the contribution of each part to the model, a model with different weight coefficients is designed, and the color difference region mask image I fn is calculated. The color difference calculation is performed on the region containing color difference represented by the mask image I fn , and finally the color difference region mask image, the maximum color difference value ΔE max and the average color difference value ΔE avg are output.

[0022] As a preferred technical solution, the loss function L is represented as follows:

[0023] L = 0.1loss1 + 0.45loss2 + 0.2loss3 + 0.15loss4 + 0.1loss5

[0024] loss1 = Gradient(Label) - I edge

[0025] loss5 = Label - DAM1, DAM1 = DAM(EFM1), EFM1 = EFM(I edge , I mfusion , I6)

[0026] loss4 = Label - DAM2, DAM2 = DAM(DAM1 x EFM2), EFM2 = EFM(I edge , I mfusion , I5)

[0027] loss3 = Label - DAM3, DAM3 = DAM(DAM2 x EFM3), EFM3 = EFM(I edge , I mfusion , I4)

[0028] loss2 = Label - DAM4, DAM4 = DAM(DAM3 x EFM4), EFM4 = EFM(I edge , I mfusion , I3)

[0029] Wherein, loss1-loss5 represent the difference between the label value and the predicted value at each stage, and finally combined into the final loss function, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value.

[0030] In a second aspect, the present application provides a corrugated carton color difference region detection system based on edge guidance and differential attention mechanism, which is applied to the corrugated carton color difference region detection method based on edge guidance and differential attention mechanism, and includes a data acquisition module, a model construction module, a model training module, and a color difference defect detection module.

[0031] The data acquisition module is configured to acquire a corrugated carton image dataset.

[0032] The model construction module is configured to construct a color difference region detection model, which includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module, and a deep supervision module.

[0033] The multi-layer feature extraction module is configured to extract first low-level features I1, second low-level features I2, first middle-level features I3, second middle-level features I4, first high-level features I5, and second high-level features I6 of an input image. edge The edge prediction module is configured to process the first low-level features I1 and the second low-level features I2 by an edge prediction network F edge , and obtain predicted edge features I edge by performing pixel summation on the first low-level features I1 and the second low-level features I2 through convolution and up-sampling operations. mfusion The multi-scale feature fusion module is configured to process the first middle-level features I3 and the second middle-level features I4 and the first high-level features I5 and the second high-level features I6 by a neural network F mfusion , and obtain multi-scale fusion features I mfusion by fusing the features I3, I4, I5, and I6 through convolution. edge The edge fusion module is configured to process the predicted edge features I mfusion and the multi-scale fusion features I efusion by an edge fusion network F efusion , and obtain edge fusion features I efusion by dynamically extracting feature maps from different channel feature maps through a spatial attention mechanism and fusing the feature maps through self-learning. efusion The differential attention module is configured to process the edge fusion features I dam by a differential attention network F dam , and obtain a feature map I f representing color difference defects by guiding the entire network to find a target region starting from the feature map with high confidence generated at the topmost layer. f Finally, the feature map I ds is input to a deep supervision network F ds , and the output I of the differential attention module is obtained.f The most appropriate weight is calculated, and all I f The weighted sum is obtained to obtain a single-channel mask image I fn

[0034] The model training module is configured to train the color difference area detection model to obtain a trained color difference area detection model; during the training process, the Adam function is used to optimize the loss function L, and after the network training is completed, the corrugated box picture with color difference is input, and the trained network model is used to calculate and output the region mask image with color difference defects.

[0035] The color difference defect detection module is configured to input the corrugated box image to be detected into the trained color difference area detection model to calculate and output the region mask image with color difference defects.

[0036] In a third aspect, the present application provides an electronic device, which comprises:

[0037] at least one processor; and

[0038] a memory connected in communication with the at least one processor; wherein

[0039] The memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the corrugated box color difference area detection method based on the edge orientation and differential attention mechanism.

[0040] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0041] 1) Improve the color difference area recognition accuracy: through the edge orientation technology, the present application can enhance the sensitivity of the model to the edge and texture features of the corrugated box surface, especially in the case of complex background or uneven illumination conditions, the color difference area can be more accurately identified. The edge orientation technology effectively suppresses the interference of background noise, making the positioning of the color difference area more accurate.

[0042] 2) The introduction of differential attention mechanism enhances the color difference distinguishing ability: the differential attention mechanism can better distinguish the color difference area from the normal area by focusing on the feature differences of the color difference area, thereby reducing the false detection and missed detection. This mechanism not only improves the recognition ability of the model, but also enhances the adaptability in different production batches and environments.

[0043] ​3) Improve stability and reliability: in the traditional color difference detection method, the change of light and background often leads to the instability of the detection result. While the technical scheme adopted by the present application, through the fusion of edge guidance and differential attention mechanism, significantly improves the adaptability of the model to the light change and background complexity, thereby improving the stability and reliability of the color difference detection, and ensuring the high consistency in the production process.

[0044] 4) Wide application prospect: the present application not only has important significance in the production quality control of corrugated carton, but also can be popularized to other printing industry fields. Through improving the precision and stability of color difference detection, the quality of final product can be effectively improved, the defective rate in production can be reduced, and the quality control cost can be reduced, which has significant economic benefits. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Fig. 1 is a general scheme diagram of edge guidance and differential attention mechanism in the embodiment.

[0047] Figure 2 Fig. 4 is a schematic diagram of multi-layer feature extraction module in the embodiment.

[0048] Figure 3 Fig. 5 is a schematic diagram of edge prediction module in the embodiment.

[0049] Figure 4 Fig. 6 is a schematic diagram of multi-scale feature fusion module in the embodiment.

[0050] Figure 5 Fig. 7 is a schematic diagram of edge fusion module in the embodiment.

[0051] Figure 6 Fig. 8 is a schematic diagram of differential attention module in the embodiment.

[0052] Figure 7 Fig. 9 is an experimental result diagram of the general scheme in the embodiment.

[0053] Figure 8 Fig. 10 is a block diagram of corrugated carton color difference area detection system based on edge guidance and differential attention mechanism in the embodiment of the present application.

[0054] Figure 9 Fig. 11 is a structural diagram of electronic equipment in the embodiment of the present application. DETAILED DESCRIPTION

[0055] In the interest of a better understanding of the present application, reference will be made to the accompanying drawings comprising examples of embodiments of the application. It is to be understood that the examples described are only some of the embodiments of the application and that no limitation of the scope of the application is implied by describing the examples. Any other embodiments not explicitly described or shown are within the scope of the application.

[0056] Reference to "an embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to one or more group of alternative embodiments.

[0057] As shown in the Figure 1 , the embodiment provides a corrugated box color difference area detection method based on edge guidance and differential attention mechanism, comprising the following steps:

[0058] S1, obtaining a corrugated box image data set.

[0059] Further, when obtaining the corrugated box image data set, the acquisition device is an industrial area scanning camera, and a high-definition image set I HQ with a resolution of 1390x1390 is obtained. The data set is mainly a printing color difference defect. However, a single corrugated box is too large, so the embodiment crops the original image I HQ to a size of 256x256, and then uses Labelme software to complete groundtruth annotation. In the training process, 70% of all images are used as training data, 20% are used as verification data, and 10% are used as test data.

[0060] S2, constructing a color difference area detection model, the color difference area detection model comprising a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module, and a deep supervision module.

[0061] Further, the multi-layer feature extraction module is configured to input image information I0 with an input size of 512*512*3 into a network F multi to extract first low-level features I1, second low-level features I2, first intermediate features I3, second intermediate features I4, first high-level features I5, and second high-level features I6 of the input image. The edge prediction module is configured to process the extracted first low-level features I1 and second low-level features I2 by an edge prediction network F edge , and Fedge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn .

[0062] In a specific embodiment, Figure 2 As shown, the multi-layer feature extraction module consists of six stages, denoted as F multi , where the first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2; the second stage consists of two convolution layers with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. After these two stages, the input image I0∈R 512×512×3 The low-level features I1∈R 256 ×256×64 and I2∈R 128×128×64 The network structures of the third, fourth and second stages are almost the same, except that there is no maximum pooling layer in the third stage. After the third and fourth stages, the intermediate features I3∈R 128×128×256 and I4∈R 64×64×512The fifth stage consists of a convolutional layer with a kernel size of 3 and a max-pooling layer with a kernel size of 2, and the sixth stage changes the max-pooling layer to an average-pooling layer based on the fifth stage. After the fifth and sixth stages, high-level features I5∈R 32×32×1024 and I6∈R 16×16×2048 are obtained, respectively. The mathematical model is shown as follows:

[0063] I1, I2, I3, I4, I5, I6 = F multi (I0)

[0064] Through multi-layer feature extraction, the edge information and color difference information in the image are separated and strengthened, and corresponding features can be selected for more effective processing during multi-scale feature fusion and edge prediction.

[0065] In a specific embodiment, as shown in Figure 3 , the low-level features I1 and I2 obtained by the multi-layer feature extraction module are transmitted to the edge prediction module for processing. The edge prediction module consists of three one-dimensional convolutional layers and an up-sampling layer. The 64-channel low-level features I1 and I2 are first convolved with the one-dimensional convolutional layer, the inter-channel information is fused, and 64-channel fusion features I1' and I'2 are generated. Next, I'2 is up-sampled and summed with I1' pixels to produce fused edge information. Finally, it is compressed into a single-channel edge prediction map I edge by 1x1 convolution. The output of I edge is shown as follows:

[0066] I edge = Conv 1×1 [Conv 1×1 (I1)+UP2(Conv 1×1 (I2))]

[0067] where I edge is the predicted edge feature, Con1x1 represents convolution operation with a kernel size of 1, and UP2 is 2x up-sampling.

[0068] In a specific embodiment, as shown in Figure 4 , the multi-scale feature fusion module mainly has four convolutional layers. I3-I6 are four features extracted from the multi-layer feature extraction module, among which I6 is the highest-level feature and I3 is the lowest-level feature. These features are divided into two categories by the multi-scale feature fusion network F mfusion : main features, i.e., the highest-level feature (I6); and auxiliary features, i.e., the remaining features (I3, I4, I5). Then, convolution operation is used to ensure that their sizes are consistent with the main feature. Finally, these features are connected together in the channel to obtain multi-scale fusion features I mfusionThe mathematical model is shown below:

[0069] I mfusion =Cat{Conv1(I6),Cat[Conv2(I5),Conv3(I4),Conv4(I3)]}

[0070] Among them, Conv1, Conv2, Conv3, and Conv4 represent convolutional layers with different convolution kernel sizes, and Cat represents connection in the channel.

[0071] In a specific example, Figure 5 As shown in Figure 2, the edge fusion module mainly includes a spatial attention layer and two convolutional layers. The edge fusion module combines the predicted edge features I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, apply spatial attention mechanism to dynamically extract feature maps from different channel feature maps, and fuse them through self-learning, F efusion It is divided into 4 edge fusion modules, respectively denoted as EFM 1 ,EFM 2 ,EFM 3 ,EFM 4 During the implementation process, F efusion First, the output feature I of the edge prediction module is edge The output feature I of the multi-scale feature fusion module mfusion Then, convolution operation is used to generate dual-channel feature maps a1 and a2. Next, a1 and a1 are mapped to attention maps A1 and A2 respectively through the Softmax function. Where A1, A2∈R H×W (H: height of feature map, W: width of feature map). The specific calculation process is as follows:

[0072]

[0073] Finally, the feature map I is obtained by weighted summation cat , implement convolution operation to get I efusion The specific calculation process is as follows:

[0074] I efusion =Conv{Cat[(I edge ×A1),(I mfusion ×A2)]}

[0075] Among them I edge and I mfusion Represent the output of the edge fusion module and the output of the multi-scale feature fusion module, I efusion Represents the final fusion result.

[0076] In one specific embodiment, as shown in FIG. 2, the differential attention module mainly includes two identical convolution layers, a spatial attention layer and a channel attention layer. The differential attention module fuses the edge feature I efusion to the differential attention network F dam for processing, guiding the entire network to find the target region, and obtaining the feature map I dam of the color difference region. F dam is divided into four differential attention modules, respectively denoted as DAM 1 , DAM 2 , DAM 3 , and DAM 4 . Take EFM 4 as an example. The output of EFM 3 is the current prediction feature of DAM 3 . It realizes background noise elimination by distinguishing the current side output feature pixel by pixel, and is up-sampled and multiplied by the output I efusion of EFM 2 as the input of the next independent DAM 3 . Among them, the current side output feature is input to the spatial and channel attention. It is worth noting that in order to strengthen the expansion of the target region, the output of DAM i is also constrained by the ground truth Gt. This top-down differential method can even eliminate the bias of the background region at the end.

[0077] In one specific embodiment, the multi-level design of the model significantly increases the parameter loading. In order to accelerate and simplify the network optimization, deep supervision is applied at each output stage. The loss function is defined as follows:

[0078] BCEloss(x i ,y i )=-wi[y i logx i +(1-y i )log(1-x i )]

[0079] where x i and y fn represent the probability and label of the i-th sample being correctly predicted, and wi represents the weight of the target. Finally, according to the contribution of each part to the model, a model with different weight coefficients is designed, and the color difference region mask image I fn is calculated. The color difference of the region represented by the mask image I max is calculated, and finally the color difference region mask image, the maximum color difference value ΔE avg and the average color difference value ΔE edge are output.

[0080] S3, training the color difference region detection model to obtain a trained color difference region detection model.

[0081] In the neural network training process, the Adam function is used to optimize the loss function L, and the loss function is expressed as follows:

[0082] L = 0.1loss1 + 0.45loss2 + 0.2loss3 + 0.15loss4 + 0.1loss5

[0083] loss1 = Gradient(Label) - I edge

[0084] loss5 = Label - DAM1, DAM1 = DAM(EFM1), EFM1 = EFM(I edge ,I mfusion ,I6)

[0085] loss4 = Label - DAM2, DAM2 = DAM(DAM1 x EFM2), EFM2 = EFM(I edge ,I mfusion ,I5)

[0086] loss3 = Label - DAM3, DAM3 = DAM(DAM2 x EFM3), EFM3 = EFM(I edge ,I mfusion ,I4)

[0087] loss2 = Label - DAM4, DAM4 = DAM(DAM3 x EFM4), EFM4 = EFM(I edge ,I mfusion ,I3)

[0088] In the above formula, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value. After the network training is completed, the corrugated box picture with color difference is input, and the trained network model is used to calculate the output region mask image with color difference defects.

[0089] S4, input the corrugated box image to be detected into the trained color difference region detection model, and calculate the output region mask image with color difference defects.

[0090] Please refer to Figure 7In an experimental process based on the patent technical solution, the original images and input images of corrugated boxes under different conditions (corrugated box surfaces with slight or obvious color difference) are selected. The images include complex background and uneven lighting conditions to test the detection ability of the application. The model enhances the surface edge and texture features through edge-oriented technology, accurately distinguishes the color difference area and non-color difference area by combining the differential attention mechanism, and finally outputs the detection result. The experimental results show that the application can accurately locate the color difference area and give the color difference detection result under different conditions, and the quantitative indicators ΔE max and ΔE avg are between 1.6 and 3.1, respectively, which embodies the high-precision detection performance. In addition, the detection result still shows good stability and robustness under complex lighting and background interference, proving the practicality and reliability of the method in actual production environment, which can effectively guide the quality control in the production process of corrugated boxes, and has significant application value.

[0091] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all described as a combination of a series of actions, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, certain steps can be performed in other order or simultaneously.

[0092] Based on the same idea as the corrugated box color difference area detection method based on edge orientation and differential attention mechanism in the above embodiment, the application also provides a corrugated box color difference area detection system based on edge orientation and differential attention mechanism, which can be used to execute the corrugated box color difference area detection method based on edge orientation and differential attention mechanism. For the sake of convenience, in the structural schematic diagram of the embodiment of the corrugated box color difference area detection system based on edge orientation and differential attention mechanism, only the part related to the embodiment of the application is shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, which can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0093] Please refer to Figure 8 In another embodiment of the present application, a corrugated box color difference area detection system 100 based on edge orientation and differential attention mechanism is provided, which includes a data acquisition module 101, a model construction module 102, a model training module 103, and a color difference defect detection module 104.

[0094] The data acquisition module 101 is used to acquire corrugated box image data set.

[0095] The model construction module 102 is used to construct a color difference area detection model, which includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0096] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0097] The model training module 103 is configured to train the color difference region detection model to obtain a trained color difference region detection model; during the training process, the Adam function is used to optimize the loss function L, and after the network training is completed, the corrugated box picture with color difference is input, and the trained network model is used to calculate the region mask image with color difference defects.

[0098] The color difference defect detection module 104 is configured to input the corrugated box image to be detected into the trained color difference region detection model to calculate the region mask image with color difference defects.

[0099] It should be noted that the corrugated box color difference region detection system based on the edge guidance and differential attention mechanism of the present application corresponds to the corrugated box color difference region detection method based on the edge guidance and differential attention mechanism of the present application. The technical features and advantages described in the embodiment of the corrugated box color difference region detection method based on the edge guidance and differential attention mechanism are applicable to the embodiment of the corrugated box color difference region detection based on the edge guidance and differential attention mechanism. For specific content, please refer to the description in the method embodiment of the present application. Here, no further description is given, and this is hereby declared.

[0100] In addition, in the embodiment of the corrugated box color difference region detection system based on the edge guidance and differential attention mechanism of the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be completed by different program modules according to the needs, for example, considering the configuration requirements of the corresponding hardware or the convenience of software implementation, that is, the internal structure of the corrugated box color difference region detection system based on the edge guidance and differential attention mechanism is divided into different program modules to complete all or part of the functions described above.

[0101] Please refer to Figure 9 In one embodiment, an electronic device implementing the corrugated box color difference region detection method based on the edge guidance and differential attention mechanism is provided. The electronic device 200 can include a first processor 201, a first memory 202 and a bus, and can further include a computer program stored in the first memory 202 and executable on the first processor 201, such as a corrugated box color difference region detection program 203 based on the edge guidance and differential attention mechanism.

[0102] The first memory 202 includes at least one type of readable storage medium, such as flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the first memory 202 can include both an internal storage unit and an external storage device of the electronic device 200. The first memory 202 can be used to store application software and various data installed in the electronic device 200, such as the code of the corrugated box color difference region detection program 203 based on edge orientation and differential attention mechanism, and can also be used to temporarily store data that has been output or will be output.

[0103] The first processor 201 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202 and calling data stored in the first memory 202.

[0104] Figure 9 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 9 The structure shown does not constitute a limitation on the electronic device 200, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0105] The corrugated box color difference region detection program 203 based on edge orientation and differential attention mechanism stored in the first memory 202 of the electronic device 200 is a combination of a plurality of instructions, which, when running in the first processor 201, can achieve:

[0106] Obtain a corrugated box image data set;

[0107] Constructing a color difference region detection model, the color difference region detection model includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module;

[0108] The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ;

[0109] The color difference region detection model is trained to obtain a trained color difference region detection model; during the training process, the Adam function is used to optimize the loss function L, and after the network training is completed, the corrugated box image with color difference is input, and the trained network model is used to calculate the output region mask image with color difference defects.

[0110] The corrugated box image to be detected is input into the trained color difference region detection model, and a region mask image with color difference defects is calculated and output.

[0111] Further, the modules / units of the electronic device 200, if implemented in the form of software function units and sold or used as independent products, can be stored in a non-volatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. In the embodiments provided in the present application, any reference to memory, storage, database or other medium can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0113] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0114] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, etc. made without departing from the spirit and principles of the present application should be equivalent replacement manners and should be included in the protection scope of the present application.

Claims

1. A method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms, characterized by: The steps include: Get a dataset of corrugated cardboard images; Constructing a color difference region detection model, the color difference region detection model includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module; The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ; Training the color difference area detection model to obtain a trained color difference area detection model; During the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a corrugated cardboard picture with color difference is input, and the trained network model is used to calculate and output the mask image of the area with color difference defects; The corrugated cardboard image to be detected is input into the trained color difference area detection model, and the mask image of the area with color difference defects is calculated and output.

2. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 1, characterized in that: The multi-layer feature extraction module consists of six stages: The first stage consists of a convolution layer with a convolution kernel size of 7, a convolution layer with a convolution kernel size of 3, and a maximum pooling layer with a kernel size of 2; the second stage consists of two convolution layers with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. After these first and second stages, the first low-level features I1 and the second low-level features I2 of the input image are obtained; The network structures of the third and fourth stages are the same as those of the second stage except that there is no maximum pooling layer. After the processing of the third and fourth stages, the first intermediate feature I3 and the second intermediate feature I4 are obtained respectively; The fifth stage consists of a convolution layer with a convolution kernel size of 3 and a maximum pooling layer with a kernel size of 2. The sixth stage changes the maximum pooling layer to an average pooling layer based on the fifth stage. After processing in the fifth and sixth stages, the first high-level feature I5 ​​and the second high-level feature I6 are obtained respectively.

3. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 1, characterized in that: The edge prediction module consists of three identical one-dimensional convolutional layers and an upsampling layer; the first low-level feature I1 and the second low-level feature I2 are first convolved with the one-dimensional convolutional layer to perform inter-channel information fusion to generate multi-channel fusion features I1' and I'2, I'2 is upsampled and summed with I1' pixels to generate fused edge information, and finally, it is compressed into a single-channel edge prediction map through a 1×1 convolution to obtain the predicted edge feature I edge .

4. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 1, characterized in that: The multi-scale feature fusion module includes 4 convolutional layers, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 are four features extracted from the multi-layer feature extraction module, where I6 is the highest-level feature and I3 is the lowest-level feature; These features are combined with the multi-scale feature fusion network F mfusion It is divided into two categories: main features, i.e. the highest level feature I6, and auxiliary features, i.e. the remaining features I3, I4, and I5. Then, a convolution operation is used to ensure that the size of the auxiliary features is consistent with the main features. Finally, features I3, I4, I5, and I6 are connected together in the channel to obtain the multi-scale fusion feature I mfusion .

5. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 1, characterized in that: The edge fusion module includes a spatial attention layer and two convolutional layers; the edge fusion module combines the predicted edge features I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, apply spatial attention mechanism to dynamically extract feature maps from different channel feature maps, and fuse them through self-learning, F efusion It is divided into 4 edge fusion modules, respectively denoted as EFM 1 ,EFM 2 ,EFM 3 ,EFM 4 , when edge fusion, F efusion First, the predicted edge features I output by the edge prediction module edge The multi-scale fusion feature I output by the multi-scale feature fusion module mfusion Then, the convolution operation is used to generate the dual-channel feature maps a1 and a2; next, a1 and a1 are mapped to the attention maps A1 and A2 respectively through the Softmax function; finally, the feature map I is obtained by weighted summation. cat , implement convolution operation to get I efusion .

6. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 5, characterized in that: The differential attention module combines the edge fusion feature I efusion Submitted to the differential attention network F dam Processing, guide the entire network to find the target area, and obtain the feature map I of the color difference area dam ; The differential attention network F dam It is divided into 4 differential attention modules, respectively denoted as DAM 1 ,DAM 2 ,DAM 3 ,DAM 4 ;DAM 4 The output is DAM 3 The current prediction feature of the current side is used to eliminate background noise by differentiating the current side output feature pixel by pixel, and is upsampled and multiplied by EFM 3 Output I efusion As the next independent DAM 2 The input of , and so on; where the current side output features are input into the spatial and channel attention.

7. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 1, characterized in that: The loss function of the deep supervision module is defined as follows: BCEloss(x i ,y i )=-wi[y i logx i +(1-y i )log(1-x i )] where x i and y i Represent the probability and label of the i-th sample being correctly predicted, and wi represents the weight of the target; finally, according to the contribution of each part to the model, a model with different weight coefficients is designed to calculate the color difference area mask image I fn , for the mask image I fn The color difference of the area represented by the color difference is calculated, and the color difference area mask image and the maximum color difference value ΔE are finally output. max and the average color difference ΔE avg .

8. The method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms according to claim 6, characterized in that: The loss function L is expressed as follows: L=0.1loss1+0.45loss2+0.2loss3+0.15loss4+0.1loss5 loss1=Gradient(Label)-I edge loss5=Label-DAM 1 ,DAM 1 =DAM(EFM 1 ),EFM 1 =EFM(I edge ,I mfusion ,I6) loss4=Label-DAM 2 ,DAM 2 =DAM(DAM 1 ×EFM 2 ),EFM 2 =EFM(I edge ,I mfusion ,I5) loss3=Label-DAM 3 ,DAM 3 =DAM(DAM 2 ×EFM 3 ),EFM 3 =EFM(I edge ,I mfusion ,I4) loss2=Label-DAM 4 ,DAM 4 =DAM(DAM 3 ×EFM 4 ),EFM 4 =EFM(I edge ,I mfusion ,I3) Among them, loss1-loss5 respectively represent the difference between the label value and the predicted value at each stage, and are finally combined into the final loss function, DAM represents the differential attention network, EFM represents the edge fusion network, and Label is the label value.

9. A corrugated cardboard color difference region detection system based on edge-guided and differential attention mechanism, characterized by: A method for detecting color difference regions of corrugated boxes based on edge-guided and differential attention mechanisms, as applied to any one of claims 1 to 8, comprising a data acquisition module, a model building module, a model training module, and a color difference defect detection module; The data acquisition module is used to acquire a corrugated box image dataset; The model construction module is used to construct a color difference area detection model, which includes a multi-layer feature extraction module, an edge prediction module, a multi-scale feature fusion module, an edge fusion module, a differential attention module and a deep supervision module; The multi-layer feature extraction module is used to extract the first low-level feature I1, the second low-level feature I2, the first intermediate feature I3, the second intermediate feature I4, the first high-level feature I5, and the second high-level feature I6 of the input image; the edge prediction module passes the extracted first low-level feature I1 and the second low-level feature I2 to the edge prediction network F edge Processing, F edge Through convolution and upsampling operations, the first low-level feature I1 and the second low-level feature I2 are summed to obtain the predicted edge feature I edge The multi-scale feature fusion module passes the first intermediate feature I3, the second intermediate feature I4 and the first high-level feature I5, the second high-level feature I6 to the neural network F mfusion Processing, F mfusion The features I3, I4, I5 and I6 are fused by convolution to obtain the multi-scale fusion feature I mfusion ; The edge fusion module predicts the edge feature I edge and multi-scale fusion features I mfusion Handed over to the edge fusion network F efusion Processing, F efusion The spatial attention mechanism is used to dynamically extract feature maps from different channel feature maps and fuse them through self-learning to obtain edge fusion feature I efusion ; The differential attention module fuses the edge features I efusion Submitted to the differential attention network F dam Processing, F dam Starting from the top-level feature map with high confidence, the entire network is guided to find the target area, and the feature map I representing the color difference defect is obtained. f Finally, the feature graph I f Input to the deep supervision network F ds , F ds is the output of the differential attention module I f Calculate the most appropriate weights and then put all I f Weighted summation is performed to obtain the final single-channel mask image I representing the area with color difference fn ; The model training module is used to train the color difference area detection model to obtain a trained color difference area detection model; During the training process, the Adam function is used to optimize the loss function L. After the network training is completed, a corrugated cardboard picture with color difference is input, and the trained network model is used to calculate and output the mask image of the area with color difference defects; The color difference defect detection module is used to input the corrugated cardboard box image to be detected into a trained color difference area detection model, and calculate and output a mask image of the area with color difference defects.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the corrugated cardboard color difference area detection method based on edge-guided and differential attention mechanism as described in any one of claims 1-7.

Images