A method for image registration and detection of pet food packaging bags

Abstract

This invention provides a method for image registration and detection of pet food packaging bags, belonging to the field of image analysis technology based on computer vision. The method includes: acquiring RGB images and thermal images of pet food packaging bags using an RGB camera and a thermal imaging camera, respectively; performing standardized preprocessing on both images and interpolating the thermal imaging image to the same resolution as the RGB image; inputting the preprocessed images into a trained image registration model, and outputting the printing quality detection result of the pet food packaging bag; the image registration model includes a newly designed image registration module, image segmentation module, and defect detection module. This invention combines multimodal image registration with deep learning technology, improving detection accuracy in complex backgrounds and the ability to perceive latent defects. Through comprehensive evaluation of color consistency, blur, offset, and missing values, it achieves all-round detection of packaging bag printing quality.

Description

Technical Field

[0001] This invention belongs to the field of image analysis technology based on computer vision, and particularly relates to a method for image registration and detection of pet food packaging bags. Background Technology

[0002] Pet food packaging bags, as an important form of food packaging, not only play a crucial role in protecting food safety and extending shelf life, but also attract consumers through exquisite printing designs. However, with the advancement of packaging technology and the increase in market demand, the printing quality of packaging bags directly affects brand image and product competitiveness. High-quality packaging bags typically feature complex patterns, multiple colors, and precise typography; therefore, the inspection of printing quality has become a vital part of the production process.

[0003] Traditional print quality inspection methods mostly rely on manual visual inspection or simple machine vision systems. While manual inspection is flexible, it is inefficient and easily affected by subjective factors; while traditional machine vision inspection methods are mostly based on single-mode image processing techniques, such as RGB image analysis. These methods show significant limitations when dealing with complex printed patterns, especially when faced with print designs with similar colors, complex textures, or fine details, resulting in low accuracy and robustness.

[0004] In recent years, the rise of multimodal image processing technology has provided new solutions for print quality inspection. By combining RGB images and thermal imaging images, multidimensional information about the surface of packaging bags, including color, texture, and temperature distribution, can be obtained. This multimodal fusion technology can effectively compensate for the shortcomings of single modes, especially showing significant advantages in detecting material defects, printing misalignments, and color unevenness. Furthermore, the rapid development of deep learning technology has provided powerful algorithmic support for image registration and defect detection. Based on convolutional neural networks (CNNs) and object detection models (such as YOLO and Faster R-CNN), automated and high-precision inspection of packaging bag print quality can be achieved.

[0005] Although multimodal fusion technology and deep learning algorithms have been widely applied in industrial inspection, the inspection of printing quality on pet food packaging bags still faces many challenges. This is mainly because pet food packaging bags are usually made of flexible materials, and their surfaces are prone to wrinkles, reflections, or temperature changes, further increasing the complexity of inspection. Existing technologies have the following shortcomings:

[0006] 1. Limitations of Single-Modal Image Processing: Traditional printing quality inspection systems are mostly based on RGB image analysis, which lacks the ability to distinguish between regions with similar colors. For patterns or text with similar colors, RGB images struggle to differentiate, easily leading to missed or false detections. Furthermore, they are highly dependent on lighting conditions; changes in ambient light significantly affect detection results, especially in high-reflectivity or low-contrast scenarios. They also lack the ability to detect material defects; RGB images cannot provide information about the physical properties of materials (such as temperature distribution), making it difficult to detect latent defects on material surfaces.

[0007] 2. Insufficient Application of Multimodal Fusion Technology: Although multimodal technology has made significant progress in fields such as medical imaging and remote sensing image processing, its application in packaging and printing quality inspection is still immature. Multimodal images (such as RGB and thermal imaging) have significant differences in viewpoint and resolution, making it difficult for traditional geometric feature-based registration methods to meet high-precision requirements. Existing methods mostly employ simple pixel-level overlay, lacking deep fusion of multimodal features, resulting in low information utilization.

[0008] 3. Adaptability Issues of Deep Learning Technology: While deep learning excels in image detection, its application in print quality inspection faces the following challenges: The lack of publicly available, standardized multimodal datasets in the field of packaging bag print quality inspection limits the training and optimization of deep learning models. Deep learning models are sensitive to wrinkles, reflections, and deformations on the packaging bag surface, easily leading to unstable detection results. Industrial inspection demands high precision and real-time performance, but deep learning models are typically computationally complex, making direct deployment to edge devices difficult.

[0009] 4. Low level of automation in existing systems: Traditional printing quality inspection systems often require manual assistance and have a low level of automation. Significant manual intervention is needed in areas such as image annotation, feature extraction, and defect classification, resulting in low efficiency. Inspection results are difficult to feed back to the production line in real time, making automatic defect removal or repair impossible.

[0010] Therefore, designing an efficient and accurate printing quality inspection method based on multimodal image registration is of great practical significance. Summary of the Invention

[0011] To address the above problems, this invention provides a method for image registration and detection of pet food packaging bags, characterized by the following steps:

[0012] S1, which acquires RGB images and thermal images of pet food packaging bags using an RGB camera and a thermal imaging camera, respectively;

[0013] S2, performs normalization preprocessing on the RGB image and thermal imaging image of the pet food packaging bag, and interpolates the thermal imaging image to the same resolution as the RGB image;

[0014] S3: Input the pre-processed RGB image and thermal imaging image of the pet food packaging bag into the trained image registration model, and output the printing quality detection result of the pet food packaging bag;

[0015] The image registration model includes an image registration module, an image segmentation module, and a defect detection module;

[0016] The image registration module performs feature extraction, feature point matching, matching point pair optimization and fusion on the acquired RGB image and thermal imaging image to generate a pixel-level fused image.

[0017] The image segmentation module uses UNet as the segmentation model to segment the packaging bag area and the printed pattern area from the input pixel-level fused image. The UNet consists of two parts: an encoder and a decoder. The encoder is responsible for extracting features and gradually reducing the resolution, while the decoder is responsible for restoring the spatial resolution of the image.

[0018] The defect detection module performs detection based on the segmented defect region image and outputs the location information and classification result of each printing quality defect.

[0019] Preferably, the process of constructing the dataset used to train the image registration model is as follows:

[0020] Collect samples of pet food packaging bags from different batches, including normal samples and defective samples; the types of defects include missing print, misaligned print, and blurry print;

[0021] The images acquired by the RGB camera and the thermal imaging camera have the same viewing angle and uniform brightness. The acquisition frequency and acquisition time for each sample are designed to ensure complete coverage of the packaging bag.

[0022] Printing defects in RGB and thermal images are annotated. Defects in the printing section of pet food packaging in the image are annotated with rectangular bounding boxes, with the bounding box being B = {x}. min y min x max y max},(x min y min ) and (x max y max The coordinates of the top left and bottom right corners of the bounding box are represented by 0, 1 and 2 respectively. Defects are divided into three types: printing blur, printing offset and printing missing, which are represented by codes 0, 1 and 2 respectively.

[0023] Preferably, the feature extraction in the image registration module involves multi-layer feature extraction, where each layer uses a pre-trained convolutional neural network to extract high-dimensional features from the RGB and thermal imaging images.

[0024] F RGB =f CNN (I RGB )

[0025] F T =f CNN (I T1 )

[0026] Among them, f CNN F represents a convolutional neural network. RGB and F T These are the feature maps for the RGB image and the thermal image, respectively; the dimension of the output feature map is F∈R. H′×W ' ×C Where H′ and W′ represent the height and width of the feature map, which are downsampled from the input image resolution; C represents the number of feature channels and the extracted feature dimension.

[0027] The pre-trained convolutional neural network includes convolutional layers, pooling layers, and normalization layers;

[0028] The convolutional layer is used to extract local features:

[0029]

[0030] Among them, F l (x, y, c) represents the value of channel c at position (x, y) in the output feature map of layer l, i.e., the output feature map of layer l; W l (i, j, c) represent the convolution kernel weights, with a kernel size of (2k+1)×(2k+1); b l Here, k is the bias term, and k is the kernel radius.

[0031] The pooling layer uses max pooling to downsample, reducing the spatial resolution of the feature map.

[0032]

[0033] Where R represents the pooling window range, and x′ and y′ represent the coordinates after pooling;

[0034] Finally, batch normalization and the non-linear activation function ReLU are used to enhance the feature representation capability, resulting in a multi-layer feature map F. RGB and F T .

[0035] Preferably, the feature point matching in the image registration module achieves cross-modal key point alignment through feature point detection and matching, establishing the correspondence between multimodal images:

[0036] First, keypoint extraction is performed; keypoint P is detected in each feature map:

[0037] P = {p i |i = 1, 2, ..., N}

[0038] Each keypoint p includes two-dimensional coordinates (x, y) and a feature descriptor d, and N is the number of keypoints detected in each feature map;

[0039] Next, a feature descriptor d is generated for each keypoint, and extracted from its local feature map:

[0040] d(p) = desc(F, p)

[0041] Where desc(F, p) represents the feature map F∈{F RGB F T Extract local feature descriptors centered at p;

[0042] Secondly, matching point pairs are generated; using a similarity metric between feature descriptors, key points in different feature maps are matched:

[0043]

[0044] Where sim(d1, d2) represents the similarity metric function, which is calculated using Euclidean distance in this invention, and τ is the similarity threshold. Let i be the i-th pixel in the RGB image. Let be the j-th pixel in the thermal imaging image.

[0045] Preferably, in the image registration module, the matching point pair optimization and fusion first employs the Random Sample Consensus Algorithm (RANSAC) to remove erroneous matching point pairs, retaining the inlier set, M f The optimized set of matching point pairs:

[0046] M f =RANSAC(M)

[0047] Then, the homography transformation matrix is ​​estimated based on the optimized matching point pairs M. f Estimate the homography transformation matrix H from thermal imaging to RGB image:

[0048]

[0049] Among them, h ** These represent the effects of transformations in the x and y directions and perspective transformations, respectively; the homography transformation relationship is:

[0050]

[0051] Where (x, y) and (x′, y′) are the coordinates of the matching point in the thermal image and the RGB image, respectively;

[0052] Finally, pixel-level image fusion is performed. During the fusion process, the pixel values ​​of multiple images are merged into a single final image. Let the weighting coefficient α be used, and the weighted average method is applied:

[0053] I R =αI RGB +(1-α)I T

[0054] Ultimately, I R This is the registered image obtained after image registration processing.

[0055] Preferably, the image segmentation module adopts the UNet model, wherein the encoder consists of a series of convolutional layers and pooling layers;

[0056] In each layer, the input registered image undergoes a convolution operation, and a nonlinearity is introduced through the ReLU activation function. The input registered image I... R As input to layer 0, it undergoes a convolution operation:

[0057]

[0058] This represents the output feature map after the convolution operation; These are the parameters of the convolution kernel;

[0059] Next, pooling is performed for downsampling, introducing dynamic context modeling, and adaptive pooling generates multi-scale contextual features:

[0060]

[0061] The decoder aims to restore the low-resolution feature map to the size of the original image and combine the high-resolution features from the encoder part through skip connections.

[0062] Multi-scale contextual features are mapped back to the original resolution, and then upsampled through weighted fusion and deconvolution operations:

[0063]

[0064] The feature map is upsampled, with its size progressively restored, where l represents the l-th layer. In the decoder, skip connections concatenate the high-resolution features from the encoder with the upsampled results from the decoder to preserve details.

[0065]

[0066] in, This is the spliced ​​feature map. These are learnable weight parameters used to dynamically adjust the contribution of contexts at different scales;

[0067] The decoder performs convolution on the concatenated feature maps and then performs a non-linear transformation using ReLU activation.

[0068]

[0069] Finally, the decoder output F fin The number of channels is reduced to 1 by using a 1×1 convolutional layer, and the class probability of each pixel is mapped to the interval [0, 1] by the Sigmoid activation function, so that each pixel belongs to the packaging bag region R. p and the printed pattern area R s probability

[0070]

[0071] in, The output probability map takes values ​​of [0, 1], representing the probability that each pixel belongs to its respective category, σ is the activation function, and k represents the k-th image;

[0072] Next, by using probability graphs Binarization yields the packaging bag area. and printed pattern area Using threshold when Then, the image (x, y) belongs to the packaging bag area, and vice versa, it belongs to the printing area, thus obtaining the segmented image I. seg .

[0073] Preferably, the UNet model is trained using a cross-entropy loss function, with the goal of minimizing the difference between the predicted results and the true labels. The formula for the cross-entropy loss function is as follows:

[0074]

[0075] Among them, Y k (x, y) represents the ground truth label, indicating whether pixel (x, y) belongs to the target region. H and W are the height and width of the segmentation map, respectively. Through backpropagation, the model parameters are updated, thereby minimizing the loss function.

[0076] Preferably, the defect detection module includes two parts: a modality-guided dynamic feature enhancement unit and a YOLOv5 target detection unit.

[0077] The dynamic feature enhancement unit performs channel compression and global modeling of thermal imaging features to generate guiding weights.

[0078] w T =σ(FC(Gap(F) T )))

[0079] Where Gap(·) represents global average pooling, used to extract global features, FC(·) represents a fully connected layer, which maps global features to the weight space, and σ represents the Sigmoid activation function.

[0080] Dynamic feature enhancement is achieved by adjusting the channel importance of RGB features using thermal imaging weights.

[0081]

[0082] The enhanced RGB features are fused with the original thermal imaging features, and convolution operations are used to further extract local information from the fused features:

[0083]

[0084] Using YOLOv5 target detection unit and enhanced feature F en To enable defect detection in pet food packaging bags.

[0085] Preferably, the YOLOv5 target detection unit and the enhanced feature F are used. en The specific process for detecting defects in pet food packaging bags is as follows:

[0086] The segmented image I seg Mesh generation and bounding box prediction are performed on the input YOLOv5 object detection unit; for the k-th input image... First, the network is divided into s×s grids; then, bounding box prediction is performed for each grid cell. For the i-th network cell gi, the YOLOv5 object detection unit outputs the predicted bounding box B of that network cell. i =(x i y i w i h i ), (x i y i w i h i The coordinates of the predicted bounding box are shown in the diagram. For the i-th bounding box, YOLOv5 outputs a class probability C. i This indicates the probability that the bounding box belongs to a certain type of defect;

[0087] After object detection is completed, a set of bounding boxes and class probabilities are generated. Based on the detection results, non-maximum suppression (NMS) is applied. For multiple overlapping bounding boxes, NMS will apply a confidence threshold δ. NMS Select the best bounding boxes and remove boxes with high overlap and low confidence.

[0088] For bounding box B i =(x i y i w i h i ) and B j Calculate the intersection-union ratio of the two bounding boxes:

[0089]

[0090] If the IoU between the two bounding boxes is greater than the threshold δ NMS If the confidence level is higher, then the bounding boxes with higher confidence levels are retained, and the bounding boxes with lower confidence levels are removed.

[0091] Using F en The confidence level for printing defects in the mesh cells is calculated to correct the detection results of YOLOv5; for the i-th mesh cell g i Confidence level with defects The Feature Pyramid Network (FPN) is used to fuse features at different resolutions, and the F... en Input into the multi-scale feature layers of the pyramid network:

[0092]

[0093] in, The original confidence scores predicted by the model. The normalized confidence score obtained after the Sigmoid activation function has a value in the range [0, 1].

[0094] The defect class probability output by YOLOv5 and the confidence level output by the multi-scale feature layer of the pyramid network are combined and then input into the softmax activation function to obtain the printing defect detection class; for the i-th network unit g i The specific calculation formula is as follows:

[0095]

[0096] Defects are classified into three categories: printing blur (0), printing misalignment (1), and printing missing (2). If class... i =0 indicates that the region selected by the predicted bounding box in the i-th grid cell has a printing blur problem. If class i=1 indicates that the region selected by the predicted bounding box in the i-th grid cell has a printing offset problem. If class i =2 indicates that there is a printing defect in the area selected by the predicted bounding box in the i-th grid cell.

[0097] Compared with the prior art, the present invention has the following beneficial effects:

[0098] 1. Combination of Multimodal Image Registration and Deep Learning: This invention proposes a detection method combining multimodal image registration and deep learning algorithms for detecting the printing quality of pet food packaging bags. By jointly using RGB images and thermal imaging images, multidimensional information of the packaging bag is obtained, and high-precision alignment is achieved through deep learning-assisted registration. RGB images provide color and texture information of the packaging bag, while thermal imaging images capture the temperature distribution of the material for detecting latent defects. High-dimensional features of the multimodal images are extracted using a convolutional neural network (CNN), and high-precision alignment of the multimodal images is achieved by combining feature point matching and homography transformation matrix estimation. The dataset is expanded by methods such as rotation, scaling, and noise addition, and random sampling consistency is used to optimize the feature point matching results, improving the robustness and accuracy of registration.

[0099] 2. Dynamic Context-Aware Module Improves Segmentation Accuracy: Addressing the issue of insufficient segmentation accuracy in complex backgrounds by traditional UNet models, this invention innovatively introduces a dynamic context-aware module. Adaptive pooling generates multi-scale contextual features, capturing global information about the packaging bag background. Learnable weights dynamically adjust the contribution of contextual features at different scales, improving adaptability to complex backgrounds. Combining high-resolution features from the encoder and low-resolution features from the decoder, skip connections preserve edge details, improving edge blurring.

[0100] 3. Modality-Guided Dynamic Feature Enhancement Module: This invention designs a modality-guided dynamic feature enhancement module, which, combined with the YOLOv5 target detection framework, enhances the detection capability for printing defects. Guided weights are generated through thermal imaging features to dynamically adjust the channel importance of RGB features, enhancing the model's perception of defect areas. The enhanced RGB features and the original thermal imaging features are fused to generate a more robust multimodal feature map. The modality-guided dynamic feature module is seamlessly integrated into the YOLOv5 framework, significantly improving the detection accuracy for complex defects (such as color unevenness, printing misalignment, and blurring) while maintaining real-time detection capabilities.

[0101] 4. Text and Pattern Integrity Assessment on Packaging Bags: This invention further designs a text and pattern integrity assessment module based on defect detection results, and innovatively proposes the following detection method: By comparing the color distribution of the pattern area on the packaging bag with that of a standard template, the color difference is calculated to detect defects of inconsistent colors. The defect areas marked by the defect detection model are used in conjunction with template information to determine blurred areas and the degree of text offset. By analyzing the areas marked as missing, missing text or patterns on the packaging bag are accurately identified. Attached Figure Description

[0102] Figure 1 This is an overall flowchart of the image registration and detection method of the present invention.

[0103] Figure 2 The data acquisition and processing flowchart of this invention.

[0104] Figure 3 The image registration module data processing flowchart of this invention.

[0105] Figure 4 The data processing flowchart of the image segmentation module and defect detection module of this invention.

[0106] Figure 5 This is a comparison chart of the performance of image registration and printing quality testing for pet food packaging bags according to the present invention. Detailed Implementation

[0107] The overall logic of the present invention is as follows: Figure 1 As shown, the invention will be further described below with reference to specific embodiments.

[0108] I. Data Collection

[0109] This phase aims to design and implement a system architecture based on multimodal imaging, data acquisition and annotation, and camera calibration and image preprocessing workflow, providing the hardware foundation for the printing quality inspection of pet food packaging bags, such as... Figure 2 As shown, the combined use of an RGB camera and a thermal imaging camera acquired multidimensional information about the packaging bags, improving the comprehensiveness and accuracy of the detection. During data acquisition, standardized imaging settings and annotation procedures were employed to ensure sample diversity and the accuracy of ground truth. The multimodal fusion image generated through image preprocessing provided high-quality data input for subsequent deep learning detection, ensuring the overall detection performance and robustness of the system.

[0110] 1. System Architecture Design

[0111] Preparation stage of multimodal imaging equipment. This invention employs an RGB camera, a thermal imaging camera, and a light source system. The RGB camera acquires the color and texture information of the packaging bag, generating an RGB image. RGBThermal imaging cameras are used to capture the temperature distribution on the surface of packaging bags, detect hidden defects, and generate thermal images. T The light source system employs a uniform, adjustable light source to ensure image quality and reduce glare interference. These components are mounted on a platform, fixing the camera and light source to ensure a consistent imaging angle, with the packaging bag placed in a fixed position. Secondly, it is equipped with a GPU computing unit to support the training and inference of deep learning models.

[0112] 2. Data Collection and Labeling

[0113] Collect samples of pet food packaging bags from different batches, including normal sample S. n and defect sample S a Defects include printing defects, printing misalignment, and printing blur. The RGB camera and thermal imaging camera are fixedly mounted to ensure consistent viewing angles in the acquired images. The imaging distance D and the light source intensity of the light source system are set to I. light This ensures uniform image brightness. RGB image I RGB and thermal imaging image I T Synchronous acquisition ensures that each pair of images corresponds one-to-one.

[0114] In this invention, the camera acquires 5 frames per second, with each sample requiring 2 seconds of acquisition. This ensures that images are captured under sufficient angle and lighting conditions to cover the entire area of ​​the packaging bag. A total of 3400 images were obtained, including 210 normal samples and 130 defective samples.

[0115] Furthermore, the data annotation tool labellmg was used to annotate printing defects in the RGB and thermal images. Specifically, rectangular bounding boxes were used to annotate defects in the printing section of pet food packaging in the images, with the bounding box defined as B = {x}. min y min x max y max},(x min y min ) and (x max y max The coordinates of the top left and bottom right corners of the bounding box are represented by 0, 1, and 2, respectively. Defects are classified into three types: printing blur, printing offset, and printing missing, which are represented by codes 0, 1, and 2, respectively.

[0116] Furthermore, data augmentation is performed on the acquired images to increase data diversity. The input for data augmentation is the original acquired multimodal image I. RGB and I T Together they constitute dataset I o .

[0117] First, the image is rotated. The image is rotated by an angle θ ∈ [-θ].max θ max ], θ max Maximum rotation angle:

[0118] I1(x,y)=I o (xcosθ-ysinθ, xsinθ+ycosθ)

[0119] Where (x, y) represents the coordinates of a pixel.

[0120] Secondly, the image is scaled, with a scaling factor s∈[s min s max ], scaled according to the following formula:

[0121] I2(x, y) = I o (s*x, s*y)

[0122] Next, noise is added. Gaussian noise N(0, σ) is added to the original image. 2 ), where σ is the standard deviation of the noise:

[0123] I3(x,y)=I o (x, y) + δ, δ ~ N(0, σ) 2 )

[0124] The images I1, I2, and I3, obtained after rotation, scaling, and noise addition, are merged with the original image dataset to form the final image dataset I. f New images generated during image data enhancement are categorized as follows: RGB images are merged into the RGB subset, and thermal images are merged into the thermal subset.

[0125] 3. Data interpolation:

[0126] Dataset I f It consists of the original image and the data-enhanced image:

[0127] I f ={I RGB I T}

[0128] The thermal image is interpolated to the same resolution H×W as the RGB image, where H is the image height and W is the image width.

[0129] I T1 =r(I T (H, W)

[0130] Where r is the interpolation function.

[0131] II. Image Registration Module Design

[0132] The purpose of image registration is to register the acquired RGB images I RGB and thermal imaging image I T Precise alignment to generate a multimodal fused image T RGB-T This invention employs deep learning-based assisted registration technology to perform image registration on pet food packaging bags. Based on the obtained augmented multimodal dataset I... f Feature extraction, feature point matching, transformation matrix estimation, and optimization are performed to achieve RGB image I RGB and thermal imaging image I T High-precision alignment, the overall process is as follows Figure 3 As shown.

[0133] 1. Feature Extraction

[0134] Feature extraction is the core step in image registration, and its purpose is to extract features from dataset I. f High-dimensional features are extracted from RGB and thermal images to facilitate subsequent keypoint matching and image alignment. In the application scenario of pet food packaging bags, feature extraction needs to consider the complex patterns, text layout, diverse colors, and potential thermal imaging characteristics of the packaging bags.

[0135] First, from dataset I f Select a pair of RGB and thermal images (I RGB I T1 Feature extraction is performed on each pair of images separately.

[0136] High-dimensional features are extracted from RGB and thermal imaging images using a pre-trained convolutional neural network. Feature extraction process:

[0137] F RGB =f CNN (I RGB )

[0138] F T =f CNN (I T1 )

[0139] Among them, f CNN This represents a convolutional neural network. F RGB and F T These are the feature maps for the RGB image and the thermal image, respectively. The dimension of the output feature map is F∈R. H′×W′×C Where H′ and W′ represent the height and width of the feature map, respectively, and are downsampled from the input image resolution. C represents the number of feature channels, and represents the extracted feature dimension, which is set to 256 in this invention.

[0140] Secondly, the design of a pre-trained convolutional neural network includes the following: designing convolutional layers. Convolutional operations are used to extract local features:

[0141]

[0142] Among them, F l (x, y, c) represents the value of channel c at position (x, y) in the output feature map of layer l, i.e., the output feature map of layer l. W l (i, j, c) represent the convolution kernel weights, with a kernel size of (2k+1)×(2k+1). b l is the bias term, and k is the kernel radius.

[0143] Next, design the pooling layer. Use max pooling to downsample and reduce the spatial resolution of the feature map:

[0144]

[0145] Where R represents the pooling window range, and x′ and y′ represent the coordinates after pooling.

[0146] Finally, normalization is performed. Batch normalization and the non-linear activation function ReLU are used to enhance feature representation. The features of each channel are normalized as follows:

[0147]

[0148] F ReLU =max(0, F BN )

[0149] Where μ and σ are the mean and standard deviation of the feature map, respectively.

[0150] Furthermore, multi-layer feature extraction is performed. The feature extraction network consists of multiple convolutional layers (conv), pooling layers (pool), normalized batch normalization (BN), and ReLU activation functions stacked together. For the input F of the l-th layer... l-1 and output F l The recursive relation is:

[0151] F l =ReLU(BN(Pool(conv(F)) l-1 ))))

[0152] 2. Feature point matching unit

[0153] In the feature extraction stage, after processing by a convolutional neural network, multi-layer feature maps F were extracted from the RGB image and the thermal imaging image, respectively. RGB F T The total feature map F∈{F RGB F TThese feature maps contain high-dimensional feature information from multimodal images. The next step is to achieve cross-modal keypoint alignment through feature point detection and matching, and to establish the correspondence between multimodal images.

[0154] First, keypoint extraction is performed. Keypoint P is detected in each feature map:

[0155] P = {p i |i = 1, 2, ..., N}

[0156] Each keypoint p includes two-dimensional coordinates (x, y) and a feature descriptor d, and N is the number of keypoints detected in each feature map.

[0157] Next, a feature descriptor d is generated for each keypoint, and extracted from its local feature map:

[0158] d(p) = desc(F, p)

[0159] Where desc(F, p) represents the feature map F∈{F RGB F T Local feature descriptors are extracted with p as the center.

[0160] Next, matching point pairs are generated. Using a similarity metric between feature descriptors, key points in different feature maps are matched:

[0161]

[0162] Where sim(d1, d2) represents the similarity metric function, which is calculated using Euclidean distance in this invention, and τ is the similarity threshold. Let i be the i-th pixel in the RGB image. Let be the j-th pixel in the thermal imaging image.

[0163] 3. Matching point pair optimization

[0164] After generating matching points, the matching point pairs are optimized, and erroneous matches are removed. This invention uses the Random Sample Consensus Algorithm (RANSAC) to remove erroneous matching point pairs. The set of interior points, M, is retained. f The optimized set of matching point pairs:

[0165] M f =RANSAC(M)

[0166] Furthermore, the homography transformation matrix is ​​estimated. Based on the optimized matching point pairs M f Estimate the homography transformation matrix H from thermal imaging to RGB image:

[0167]

[0168] Among them, h ** These represent the effects of transformations in the x and y directions and perspective transformations, respectively. The homography transformation relationship is:

[0169]

[0170] Where (x, y) and (x′, y′) are the coordinates of the matching point in the thermal image and the RGB image, respectively.

[0171] Furthermore, pixel-level image fusion is performed. During the fusion process, the pixel values ​​of multiple images are merged into a single final image. Let the weighting coefficient α be used, and according to the weighted average method:

[0172] I R =αI RGB +(1-α)I T

[0173] Ultimately, I R This is an image of a pet food packaging bag obtained after image registration processing.

[0174] III. Image Segmentation Module Design

[0175] To segment images of pet food packaging bags, this invention employs UNet as the segmentation model. It is used to extract regions of the pet food packaging bag and the printed pattern regions within it. Traditional UNet models suffer from blurred edges or inaccurate segmentation when handling complex packaging bags due to insufficient local features. This invention introduces a dynamic context-aware module, combining global context information at different scales, which helps improve the segmentation model's adaptability to complex backgrounds. The overall process is as follows: Figure 4 As shown.

[0176] Based on the obtained image registration image dataset I R Using UNet as the segmentation model, from the input image dataset I R Middle-divided packaging bag area R p and the printed pattern area R s Specifically, UNet consists of two main parts: an encoder and a decoder. The encoder is responsible for extracting features and gradually reducing the resolution, while the decoder is responsible for restoring the spatial resolution of the image.

[0177] The encoder consists of a series of convolutional and pooling layers, with the main purpose of progressively extracting high-level image features while reducing the spatial resolution of the feature maps.

[0178] In each layer, the input image undergoes a convolution operation with a kernel size of 3×3, and non-linearity is introduced through the ReLU activation function. Input image IR As input to layer 0, it undergoes a convolution operation:

[0179]

[0180] This represents the output feature map after the convolution operation. These are the parameters of the convolution kernel.

[0181] Next, pooling is performed for downsampling. Dynamic context modeling is introduced to improve the segmentation model's adaptability to complex backgrounds. Adaptive pooling generates multi-scale contextual features:

[0182]

[0183] In the decoder, the goal of the decoder part is to restore the low-resolution feature map to the size of the original image and combine the high-resolution features from the encoder part through skip connections.

[0184] Multi-scale contextual features are mapped back to the original resolution and then fused using weighted fusion. The feature maps are then upsampled using a deconvolution operation.

[0185]

[0186] The image represents the upsampled feature map, with its size progressively restored, where l denotes the l-th layer. In the decoder, skip connections concatenate the high-resolution features from the encoder with the upsampled result from the decoder to preserve details.

[0187]

[0188] in, This is the spliced ​​feature map. These are learnable weight parameters used to dynamically adjust the contribution of contexts at different scales.

[0189] Furthermore, the decoder performs convolution on the concatenated feature maps and performs a non-linear transformation using ReLU activation:

[0190]

[0191] Finally, the decoder output F fin The number of channels is reduced to 1 by using a 1×1 convolutional layer, and the class probability of each pixel is mapped to the interval [0, 1] by the Sigmoid activation function, so that each pixel belongs to the packaging bag region R. p and the printed pattern area R s probability

[0192]

[0193] in, The output probability map takes values ​​in the range [0, 1], representing the probability that each pixel belongs to its respective category. σ is the activation function, and k represents the k-th image.

[0194] Next, by using probability graphs Binarization yields the packaging bag area. and printed pattern area Using threshold when Then (x, y) belongs to the packaging bag area, and vice versa, it belongs to the printing area.

[0195] Furthermore, UNet uses the cross-entropy loss function to train the model, aiming to minimize the difference between the predicted results and the true labels. The formula for the cross-entropy loss function is as follows:

[0196]

[0197] Among them, Y k (x, y) represents the ground truth label, indicating whether pixel (x, y) belongs to the target region. H and W are the height and width of the segmentation map, respectively. Through backpropagation, the model's parameters are updated, thereby minimizing the loss function.

[0198] Based on the above, the segmented image I is finally obtained. seg This serves as input to a defect detection model for pet food packaging.

[0199]

[0200] IV. Defect Detection Module Design

[0201] In pet food packaging bag inspection, the goal is to identify blurry printing, misaligned printing, and missing printing on the packaging. These defective areas need to be accurately identified for subsequent quality control.

[0202] Based on the defect region image obtained by the segmentation module, this invention uses YOLOv5 as the target detection unit, with I... seg The image is used as input to the model to identify defect regions in the image. Image I is input to YOLOv5. seg Only the packaging bag and printing areas are included, so YOLOv5 can focus on defect detection in these two areas. Since the feature extraction part of the YOLOv5 object detection network mainly relies on features from single-modal inputs, it is difficult to fully utilize the complementary information of multimodal data. Therefore, this invention designs a feature mapping F = {F...} RGB F TAs a guide, a modality-guided dynamic feature enhancement unit is introduced, which guides the RGB features through thermal imaging features and dynamically adjusts the feature weights of different regions, thereby improving the detection network's ability to perceive defects.

[0203] The goal of the YOLOv5 object detection unit is to detect defective regions in an image, including printing blur, printing misalignment, and printing defects. YOLOv5 detects defects by analyzing the input image I... seg The process is performed, outputting a set of bounding boxes and class probabilities, representing the location information and classification result of each defect in the image. The overall process is as follows: Figure 4 As shown.

[0204] The design incorporates a modally guided dynamic feature enhancement unit. Channel compression and global modeling are performed on thermal imaging features to generate guiding weights.

[0205] w T =σ(FC(Gap(F) T )))

[0206] Where Gap(·) represents global average pooling, used to extract global features, FC(·) represents a fully connected layer, which maps global features to the weight space, and σ represents the Sigmoid activation function.

[0207] Dynamic feature enhancement is achieved by adjusting the channel importance of RGB features using thermal imaging weights.

[0208]

[0209] Furthermore, the enhanced RGB features are fused with the original thermal imaging features, and convolution operations are used to further extract local information from the fused features:

[0210]

[0211] Furthermore, the YOLOv5 target detection unit and the enhanced feature F are utilized. en The specific steps for detecting defects in pet food packaging bags are as follows:

[0212] Will I seg The dataset is input into YOLOv5 for grid generation and bounding box prediction; for the k-th input image First, it is divided into s×s grids; then, bounding box prediction is performed for each grid cell. For the i-th network cell g i YOLOv5 outputs the predicted bounding box B of this network unit. i =(x i y i w i h i ), (xi y i w i h i The coordinates of the predicted bounding box are shown in the diagram. For the i-th bounding box, YOLOv5 outputs a class probability C. i This represents the probability that the bounding box belongs to a certain type of defect. Further, after object detection is complete, the YOLOv5 model generates a set of bounding boxes and class probabilities, and sets non-maximum suppression (NMS) based on the detection results. That is, for multiple overlapping bounding boxes, NMS is applied based on the confidence threshold δ. NMS Select the best bounding boxes and remove boxes with high overlap and low confidence.

[0213] Specifically, for bounding box B i =(x i y i w i h i ) and B j Calculate the intersection-union ratio of the two bounding boxes:

[0214]

[0215] If the IoU between the two bounding boxes is greater than the threshold δ NMS If the confidence level is higher, then the bounding boxes with higher confidence levels are retained, and the bounding boxes with lower confidence levels are removed.

[0216] Using F en The confidence level of printing defects in the mesh cells is used to correct the detection results of the YOLOv5 target detection unit; for the i-th mesh cell g i Confidence level with defects The Feature Pyramid Network (FPN) is used to fuse features at different resolutions, thereby integrating the F... en Input into the multi-scale feature layers of the pyramid network:

[0217]

[0218] in, The original confidence scores predicted by the model. The normalized confidence score obtained after the Sigmoid activation function has a value in the range [0, 1].

[0219] The defect class probability output by YOLOv5 and the confidence level output by the multi-scale feature layer of the pyramid network are combined and then input into the softmax activation function to obtain the printing defect detection class; for the i-th network unit g i The specific calculation formula is as follows:

[0220]

[0221] The defects in this invention are classified into three categories: printing blur (0), printing misalignment (1), and printing defects (2). i =0 indicates that the region selected by the predicted bounding box in the i-th grid cell has a printing blur problem. If class i =1 indicates that the region selected by the predicted bounding box in the i-th grid cell has a printing offset problem. If class i =2 indicates that there is a printing defect in the area selected by the predicted bounding box in the i-th grid cell.

[0222] V. Model Training

[0223] The image registration module, image segmentation module, and defect detection module are trained using image data of pet food packaging bags obtained through camera calibration and data acquisition. This process includes:

[0224] 1) Using the information about the RGB image I obtained in S1 RGB and thermal imaging image I T Together they constitute dataset I o For the dataset I that together constitute o Data augmentation is performed by adding augmented datasets through image rotation, scaling, and noise, and then merging these augmented datasets with the original image data to obtain the total image dataset I. f .

[0225] 2) Using the total dataset I f The image registration module is trained; firstly, feature extraction is performed, using a pre-trained convolutional neural network to extract high-dimensional features of the image to obtain F. RGB F T Secondly, regarding F RGB F T Feature point matching is performed to find key points P, and matching point pairs M are generated. The matching point pairs M are then optimized by removing incorrect matching point pairs, resulting in M0. F The homography transformation matrix is ​​used to obtain the coordinates (x, y) and (x′, y′) of the matching points in the RGB and thermal imaging images. The set of corresponding images is I. R Complete image registration.

[0226] 3) Image dataset I obtained through image registration training R Train the image segmentation module. Use the image dataset I... R The input is used for training a UNet-based segmentation model, which consists of two parts: an encoder and a decoder. In the decoder, non-linearity is introduced through convolutional operations and activation functions to obtain the output feature map. Next, pooling is performed for downsampling. To improve the segmentation model's adaptability to complex backgrounds, dynamic context modeling is introduced, and adaptive pooling generates multi-scale contextual features. right Upsampling is performed through weighted fusion deconvolution to concatenate the results of the decoder part, and the decoder output F fin By mapping the class of each pixel to the interval [0, 1] using an activation function, the class probability of that pixel can be obtained. according to It was identified as the packaging bag area R. p Or printing area R s The image dataset after image registration is composed of I R Represented as I seg .

[0227] 4) Image dataset I obtained using the image segmentation module seg Training the defect detection module. A modality-guided dynamic feature enhancement module is introduced to enhance the feature mapping F of the RGB image obtained from the image registration module. T Perform channel compression and global modeling to generate guiding weights w T Using guiding weights w T Dynamic feature enhancement of RGB image feature mapping F RGB ,get The two enhanced feature maps are fused, and convolution operations are used to extract local information from the fused features to obtain F. en This is used as the detection head input for the YOLOv5 object detection model for model training. F en The input is fed into the multi-scale feature layer of the pyramid network to obtain the raw confidence values ​​predicted by the model. After training with activation functions, the confidence value of the target's existence is obtained. and category scores

[0228] VI. Model Deployment and Application

[0229] The trained image registration module, image segmentation module, and defect detection module are deployed on the server backend to achieve real-time detection of the printing quality of pet food packaging bags.

[0230] After the RGB image and thermal image of the pet food packaging bag are acquired by the RGB camera and thermal camera respectively, the following steps are performed to complete the detection and verification of the printing quality of the pet food packaging bag and to grade its level:

[0231] The RGB and thermal images of pet food packaging bags are standardized and their sizes are adjusted to ensure that the images meet the resolution and size requirements when input into the deployed model.

[0232] The pre-processed RGB image and thermal image of the pet food packaging bag are input into the trained image registration module to obtain the registered image.

[0233] The registered image is input into the trained image segmentation module to perform image segmentation and obtain images of the pet food packaging bag area and the printing area.

[0234] Input the images of the pet food packaging bag area and the printing area into the defect detection module to output the pet food packaging bag printing quality inspection results.

[0235] Experimental Evaluation and Comparison: The algorithm of this invention is compared with traditional YOLO series object detection methods in experiments, such as... Figure 5 As shown, the algorithm proposed in this invention performs best in pet food packaging bag inspection, achieving an accuracy of 93%, a recall of 91%, and an overall F1 score of 92%, significantly outperforming YOLOv3 (F1 score 82.5%) and YOLOv5 (F1 score 89%). Compared to the traditional YOLOv5, the multi-scale improvement enhances the detection capability for small targets and complex backgrounds, effectively improving recall and accuracy, making it particularly suitable for detecting printing defects and detail anomalies. This demonstrates that the method designed in this invention is an effective solution for pet food packaging bag image registration and printing quality inspection.

[0236] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0237] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for image registration and detection of pet food packaging bags, characterized in that, Includes the following steps: S1, which acquires RGB images and thermal images of pet food packaging bags using an RGB camera and a thermal imaging camera, respectively; S2, performs normalization preprocessing on the RGB image and thermal imaging image of the pet food packaging bag, and interpolates the thermal imaging image to the same resolution as the RGB image; S3: Input the pre-processed RGB image and thermal imaging image of the pet food packaging bag into the trained image registration model, and output the printing quality detection result of the pet food packaging bag; The image registration model includes an image registration module, an image segmentation module, and a defect detection module; The image registration module performs feature extraction, feature point matching, matching point pair optimization and fusion on the acquired RGB image and thermal imaging image to generate a pixel-level fused image. The feature extraction involves multi-layer feature extraction, where each layer uses a pre-trained convolutional neural network to extract high-dimensional features from RGB and thermal imaging images. ； ； in, Represents a convolutional neural network. and These are the feature maps for the RGB image and the thermal image, respectively; the dimension of the output feature map is... ,in, and Represents the height and width of the feature map, which are downsampled from the input image resolution; This represents the number of feature channels and the dimension of the extracted features. The pre-trained convolutional neural network includes convolutional layers, pooling layers, and normalization layers; The convolutional layer is used to extract local features: ； in, Indicates the first Position in the layer output feature map passage The value of, i.e., the first Layer output feature mapping; This represents the convolution kernel weights, with a kernel size of . ; Here, k is the bias term, and k is the kernel radius. The pooling layer uses max pooling to downsample, reducing the spatial resolution of the feature map. ； Where R represents the pooling window range, Represents the coordinates after pooling; Finally, batch normalization and the ReLU nonlinear activation function are used to enhance feature representation capabilities, resulting in multi-layer feature maps. and ; The feature point matching is achieved by aligning key points across modalities through feature point detection and matching, thus establishing correspondences between multimodal images. First, keypoint extraction is performed; keypoint P is detected in each feature map: ； Among them, each key point Including two-dimensional coordinates And feature descriptor d, N is the number of keypoints detected in each feature map; Next, a feature descriptor d is generated for each keypoint, and extracted from its local feature map: ； in, Represented as in feature mapping above Extract local feature descriptors centered on the target area; Secondly, matching point pairs are generated; using a similarity metric between feature descriptors, key points in different feature maps are matched: ； in, To represent the similarity measurement function, this invention uses Euclidean distance for calculation. For similarity threshold, For the RGB image, the first 1 pixel The first in the thermal imaging image 1 pixel; The optimization and fusion of matching point pairs first employs the Random Sample Consensus Algorithm (RANSAC) to remove erroneous matching point pairs, retaining the set of interior points. The optimized set of matching point pairs: ； Then, the homography transformation matrix is ​​estimated based on the optimized matching point pairs. Estimate the homography transformation matrix H from thermal imaging to RGB image: ； in, These represent the effects of transformations in the x and y directions and perspective transformations, respectively; the homography transformation relationship is: ； in, and These are the coordinates of the matching point in the thermal image and the RGB image, respectively; Finally, pixel-level image fusion is performed. During the fusion process, the pixel values ​​of multiple images are merged into a single final image, with weighting coefficients set. According to the weighted average method: ； final, The registered image is obtained after image registration processing; The image segmentation module uses UNet as the segmentation model to segment the packaging bag area and the printed pattern area from the input pixel-level fused image. The UNet consists of two parts: an encoder and a decoder. The encoder is responsible for extracting features and gradually reducing the resolution, while the decoder is responsible for restoring the spatial resolution of the image. The defect detection module performs detection based on the segmented defect region image and outputs the location information and classification result of each printing quality defect.

2. The method for image registration and detection of pet food packaging bags as described in claim 1, characterized in that, The process of constructing the dataset used to train the image registration model is as follows: Collect samples of pet food packaging bags from different batches, including normal samples and defective samples; the types of defects include missing print, misaligned print, and blurry print; The images acquired by the RGB camera and the thermal imaging camera have the same viewing angle and uniform brightness. The acquisition frequency and acquisition time for each sample are designed to ensure complete coverage of the packaging bag. Printing defects in RGB and thermal images are annotated. Defects in the printing of pet food packaging in the images are annotated with rectangular boxes. , and These represent the coordinates of the lower left and upper right corners of the bounding box, respectively; defects are divided into three types: printing blur, printing offset, and printing missing, which are represented by codes 0, 1, and 2, respectively.

3. The image registration and detection method for pet food packaging bags as described in claim 1, characterized in that: The image segmentation module adopts the UNet model, in which the encoder consists of a series of convolutional layers and pooling layers; In each layer, the input registered image undergoes a convolution operation, and a nonlinearity is introduced through the ReLU activation function, resulting in a registered image. As input to layer 0, it undergoes a convolution operation: ； This represents the output feature map after the convolution operation; These are the parameters of the convolution kernel; Next, pooling is performed for downsampling, introducing dynamic context modeling, and adaptive pooling generates multi-scale contextual features: ； The decoder aims to restore the low-resolution feature map to the size of the original image and combine the high-resolution features from the encoder part through skip connections. Multi-scale contextual features are mapped back to the original resolution, and then upsampled through weighted fusion and deconvolution operations: ； This is the upsampled feature map, with its size gradually restored. l Indicates the first l Layers; in the decoder, skip connections concatenate the high-resolution features of the encoder with the upsampled results of the decoder to preserve details: ； in, This is the spliced ​​feature map. These are learnable weight parameters used to dynamically adjust the contribution of contexts at different scales; The decoder performs convolution on the concatenated feature maps and then performs a non-linear transformation using ReLU activation. ； Finally, the decoder output Through a The convolutional layer reduces the number of channels to 1 and uses the sigmoid activation function to map the class probability of each pixel to the [0,1] interval, thus determining whether each pixel belongs to the packaging bag region. and printed pattern area probability : ； in, The output probability map takes values ​​in the range [0,1], representing the probability that each pixel belongs to its respective category. Let k be the activation function, and k represent the k-th image. Next, by using probability graphs Binarization yields the packaging bag area. and printed pattern area Using threshold ,when , then like The image is segmented as follows: if it belongs to the packaging area, it belongs to the printing area. .

4. The image registration and detection method for pet food packaging bags as described in claim 3, characterized in that: The UNet model is trained using the cross-entropy loss function, with the goal of minimizing the difference between the predicted results and the true labels; the formula for the cross-entropy loss function is as follows: ； in, The true label represents whether pixel (x, y) belongs to the target region. H and W are the height and width of the segmentation map, respectively. Through backpropagation, the model parameters are updated, thereby minimizing the loss function. .

5. The image registration and detection method for pet food packaging bags as described in claim 1, characterized in that: The defect detection module includes two parts: a dynamic feature enhancement unit with modality guidance and a YOLOv5 target detection unit. The dynamic feature enhancement unit performs channel compression and global modeling of thermal imaging features to generate guiding weights. ； in, This indicates global average pooling, used to extract global features. This represents a fully connected layer that maps global features to a weight space. This represents the Sigmoid activation function; Dynamic feature enhancement is achieved by adjusting the channel importance of RGB features using thermal imaging weights. ； The enhanced RGB features are fused with the original thermal imaging features, and convolution operations are used to further extract local information from the fused features: ； Using YOLOv5 object detection unit and enhanced features To enable defect detection in pet food packaging bags.

6. The image registration and detection method for pet food packaging bags as described in claim 5, characterized in that: Using YOLOv5 object detection unit and enhanced features The specific process for detecting defects in pet food packaging bags is as follows: The segmented image Input the YOLOv5 object detection unit for mesh generation and bounding box prediction; for the first Input image First, divide it into The first grid cell is then used for bounding box prediction; for the first grid cell... Network unit The YOLOv5 object detection unit outputs the predicted bounding box of this network unit. , To predict the position coordinates of the bounding box; for the th Given a bounding box, YOLOv5 will output a class probability. This indicates the probability that the bounding box belongs to a certain type of defect; After object detection is completed, a set of bounding boxes and class probabilities are generated. Based on the detection results, Non-maximum Suppression (NMS) is applied. For multiple overlapping bounding boxes, NMS will apply a confidence threshold. Select the best bounding boxes and remove boxes with high overlap and low confidence. For bounding box and Calculate the intersection-union ratio of the two bounding boxes: ； If the IoU between the two bounding boxes is greater than the threshold If the confidence level is higher, then the bounding boxes with higher confidence levels are retained, and the bounding boxes with lower confidence levels are removed. use The confidence level for printing defects in the calculated mesh cells is corrected for the detection results of YOLOv5; for the first... Network unit Confidence level with defects ; The Feature Pyramid Network (FPN) is used to fuse features at different resolutions. Input into the multi-scale feature layers of the pyramid network: ； ； in, The original confidence scores predicted by the model. The normalized confidence score obtained after the Sigmoid activation function has a value in the range [0,1]. The defect category probabilities output by YOLOv5 and the confidence scores output by the multi-scale feature layers of the pyramid network are combined and then input into the softmax activation function to obtain the printing defect detection category. For the first Network unit The specific calculation formula is as follows: ； Defects are classified into three categories: blurry printing (0), misaligned printing (1), and missing printing (2). Then it represents the first The region selected by the predicted bounding box in each grid cell has a printing blur problem. Then it represents the first There is a printing offset issue in the region selected by the predicted bounding box within each grid cell. Then it represents the first There is a printing defect in the area selected by the predicted bounding box in each grid cell.

Images