A packaging box printing quality detection method based on image processing

Abstract

The application discloses a kind of based on image processing's packaging box printing quality detection method, and the present application relates to packaging box quality detection technical field.The steps of the method include: collecting depth variation information and natural texture image and carrying out geometric registration, obtain initial spatial information field, construct printing surface three-dimensional model based on initial spatial information field;Three-dimensional topographic anomaly detection model is constructed, feature extraction is carried out to printing surface three-dimensional model, and actual topographic feature is obtained, and is input into three-dimensional topographic anomaly detection model, and topographic anomaly area is output, and defect candidate set is constructed based on topographic anomaly area;Two-dimensional texture consistency verification model is constructed, feature extraction is carried out based on defect candidate set, and actual texture feature is obtained, and is input into two-dimensional texture consistency verification model, and real printing defect is output;Based on real printing defect, packaging box printing quality detection result is obtained.The application generates packaging box printing quality detection result by image processing, and realizes the detection of printing quality.

Description

Technical Field

[0001] This invention relates to the field of packaging box quality inspection technology, specifically to a packaging box printing quality inspection method based on image processing. Background Technology

[0002] In modern industrial production systems, packaging boxes serve as the core carriers for product storage, transportation, and display. Their printing quality directly impacts market acceptance and brand value. Various defects generated during the printing process can severely affect the appearance integrity and performance of packaging boxes. Therefore, efficient and accurate automated inspection of packaging box printing quality is a crucial step in ensuring production efficiency and reducing defect rates, and has become an important requirement for the intelligent upgrading of the manufacturing industry.

[0003] In the texture detection stage of packaging box printing quality inspection, the traditional method often uses template matching. The core process is to extract features from the natural texture image of a defect-free packaging box and construct a standard texture template library; obtain the natural texture image of the packaging box to be inspected and segment it into several texture image blocks; extract the actual shape features of the texture image blocks and compare them one by one with the corresponding reference shape feature set in the standard texture template library to calculate the similarity index; and comprehensively determine whether there are any abnormalities in the texture of the packaging box to be inspected based on the preset similarity threshold.

[0004] However, template matching relies on standard templates and cannot adapt to the diversity of textures by learning the inherent rules of textures. It is difficult to accurately distinguish between normal texture changes and texture anomalies, which ultimately leads to a large number of misjudgments during the detection process and cannot meet the stringent requirements for detection accuracy in high-end packaging box production. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a packaging box printing quality inspection method based on image processing, thereby resolving the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a packaging box printing quality inspection method based on image processing, comprising the following steps: Step S1: Collect depth variation information and natural texture images of the packaging box surface, perform geometric registration on the depth variation information and natural texture images to obtain an initial spatial information field, and construct a three-dimensional model of the printed surface based on the initial spatial information field; Step S2: Extract the morphological features of the three-dimensional model of the printed surface to obtain the actual morphological features, and input them into the three-dimensional morphological anomaly detection model based on the autoencoder to output the morphological anomaly region. Construct a defect candidate set based on the morphological anomaly region. Step S3: Based on the defect candidate set, extract texture features from the three-dimensional model of the printed surface to obtain the actual texture features, and input them into the two-dimensional texture consistency verification model built based on the self-supervised learning framework to output the real printing defects. Step S4: Analyze and locate the actual printing defects to obtain the packaging box printing quality inspection results.

[0007] Preferably, depth variation information and natural texture images of the packaging box surface are acquired, and geometric registration is performed on the depth variation information and natural texture images to obtain an initial spatial information field, including: Collect information on depth variations and natural texture images of the packaging box surface; Unified geometric registration is performed between depth variation information and natural texture images to form an initial spatial information field. ,in Identify a data point, These are the target 3D coordinates of data point d. It is the texture color vector of data point d. It is the absolute phase value.

[0008] Preferably, constructing a three-dimensional model of the printed surface based on the initial spatial information field includes: For the initial spatial information field The target three-dimensional coordinate set Denoising and fusion are performed to remove outliers; a surface reconstruction algorithm based on Poisson reconstruction or rolling guided filtering is used to generate a continuous watertight triangular mesh surface from the target's three-dimensional coordinates, i.e., the three-dimensional model of the printed surface. ; For the three-dimensional model of the printed surface Each vertex on Based on the vertex's position and normal, select the optimal viewing camera—that is, the camera with the smallest angle between the line of sight and the normal—and position the vertex... Substitute the three-dimensional coordinates into the projection equation of the optimal viewing angle camera to calculate its projected coordinates on the texture image. : ; in, These are the three-dimensional coordinates of the vertex; and These are the intrinsic and extrinsic parameter matrices of the camera with the best viewing angle; These are the intermediate homogeneous coordinates; These are texture coordinates; Projected coordinates That is, the vertex The sampling location on the corresponding natural texture image; for the three-dimensional model of the entire printed surface. After calculating the texture coordinates of all vertices and internal pixels of each triangular facet, the acquired natural texture image can be fitted onto the 3D model of the printed surface. On top, a textured three-dimensional model of the printed surface is formed. .

[0009] Preferably, the actual morphological features are obtained by extracting morphological features from the three-dimensional model of the printed surface, including: 3D model of the printed surface Sampling is performed within the effective printing area to obtain a set of sampling points; for each sampling point... Analyze the geometric information in its local neighborhood and calculate a set of multidimensional feature vectors. Geometric features include curvature features, obtained by fitting sampling points. The local neighborhood of the point is a quadratic surface; calculate the principal curvature at that point. and Gaussian curvature is derived from principal curvature. and mean curvature Projecting points within the local neighborhood onto On the tangent plane of a point, we obtain the set of height values ​​of each point along the normal vector. Based on this set of height values Calculate the arithmetic mean height Root mean square height skewness and kurtosis ; Each sampling point Represented as actual morphological features For all Perform this operation on each sampling point to obtain a three-dimensional model representing the entire printed surface. Actual morphological feature set ; Using a reference morphological feature set Calculate the mean of each feature dimension. and standard deviation Then, the actual morphological feature set Each actual morphological feature Z-score standardization is performed to obtain the normalized actual morphological features. .

[0010] Preferably, the input is a 3D shape anomaly detection model constructed based on an autoencoder, and the output is a shape anomaly region. A defect candidate set is constructed based on the shape anomaly region, including: A 3D anomaly detection model is built and trained based on an autoencoder. The autoencoder consists of an encoder. and decoder It consists of two parts, the encoder Actual morphological features Mapping to latent encoded features The decoder extracts this latent encoded feature. The reconstructed feature vector is obtained by reconstructing the feature vector. ; During the training phase, a reference topography feature set was extracted using a large number of defect-free normal printed surface 3D models. As training data, for a single sample, its reconstruction error The calculation formula is: ; in, Representing vectors Norm; and These are the actual morphological features and reconstructing feature vectors The l-th component; l is the component index; After the 3D anomaly detection model is trained, the actual anomaly feature set to be tested is... The data is input into a pre-trained 3D anomaly detection model, and forward propagation yields each actual anomaly feature. Reconstructed feature vectors and the corresponding reconstruction error ; Based on reconstruction error Anomalies were identified; all these anomalies were represented in a 3D model of the printed surface. The corresponding set of spatial locations constitutes the region of morphological anomaly. ; areas with abnormal shapes Clustering and merging are performed to ultimately form a defect candidate set. .

[0011] Preferably, based on the defect candidate set, texture features are extracted from the three-dimensional model of the printed surface to obtain the actual texture features, including: A texture feature extraction network based on self-supervised contrastive learning is constructed, and an improved cut-and-paste data augmentation method is used to construct training samples for self-supervised learning, which are two-dimensional texture image patches randomly sampled from a normal sample library. The improved cut-and-paste data augmentation method generates textured image pairs. ; This texture feature extraction network is trained using cross-entropy loss to extract texture images. Input Feature Extraction Network To obtain the corresponding actual texture features and simulated texture features Using comparative loss Optimize: ; in These are actual texture features; It simulates texture features; Cosine similarity It is a temperature hyperparameter; is the negative sample feature; N is the training batch size; a is the actual texture feature index; b is the negative sample feature index; After training, for each region with abnormal morphology Extract its two-dimensional texture image patches And through feature extraction network Obtain its actual texture features .

[0012] Preferably, the input is a two-dimensional texture consistency verification model built based on a self-supervised learning framework, and the output includes real printing defects such as: Based on the trained feature extraction network A complete dictionary was learned through an improved dictionary learning algorithm. ; For each region with abnormal morphology Extract its two-dimensional texture image patches And through feature extraction network Obtain its actual texture features In a complete dictionary Solve its sparse coding And calculate the actual texture features. Reconstruction error : ; in, It is a reconstruction error; It is the optimal sparse coding vector; D is the overcomplete dictionary; Set defect threshold , judge when At that time, areas with abnormal morphology Verified as a genuine printing defect Otherwise, it is determined to be a normal process texture and excluded from the defect candidate set; ultimately, it is based on the actual printing defects. Construct a set of real printing defects .

[0013] Preferably, the overcomplete dictionary includes: Use a trained feature extraction network Extract standard texture features and construct a standard texture feature set. An overcomplete dictionary is learned through an improved dictionary learning algorithm. The optimization objective is: ; in, It is the nth standard texture feature; It is a complete dictionary that needs to be studied; It corresponds to the first A sparse coding coefficient vector of standard texture features; for Norm; for Norm; Regularization parameter

[0014] Preferably, the analysis and location of the actual printing defects include: Extract each real printing defect fused feature vector ,in, It is the projected area; It is the average depth deviation; It is a shape type identifier; It is a texture anomaly type identifier; Based on the preset classification rules, the fused feature vectors will be... Mapped to specific defect categories ; Meanwhile, based on the fused feature vector For each real printing defect Calculate a comprehensive severity score ; Next, based on the actual set of printing defects Calculate the overall quality score Based on overall quality score To obtain the final quality judgment ; 3D model of the printed surface Back-projecting the defect coordinates onto the surface to represent actual printing defects. Each vertex within the region Utilizing the best perspective camera parameters and Project it back to the projection coordinates of the natural texture image. Simultaneously based on vertices Calculate actual printing defects The positioning coordinates of the center of the 3D bounding box in the world coordinate system .

[0015] Preferably, the packaging box printing quality inspection results include: Generate and output a structured packaging box printing quality inspection result, the core content of which includes: comprehensive judgment result. Overall Defect Overview; Each Real Printing Defect A detailed list covering its defect categories Severity score Projected coordinates on natural texture images and the positioning coordinates in the world coordinate system. A natural texture image with visual annotations of all defect locations.

[0016] Beneficial effects This invention provides a packaging box printing quality inspection method based on image processing, involving image processing and deep learning technologies, which has the following beneficial effects: (1) By constructing a three-dimensional anomaly detection model based on an autoencoder, only three-dimensional morphological samples of normal packaging boxes are needed for training, perfectly addressing the real-world pain point of scarce defect samples in industrial scenarios. By extracting multi-dimensional geometric features such as principal curvature, Gaussian curvature, and height distribution statistics, the three-dimensional anomaly detection model can accurately capture microscopic undulation anomalies on the printed surface, such as bulges caused by ink accumulation and ripples formed by lamination wrinkling. By using reconstruction error to determine abnormal areas, interference from normal structures such as folds and pressure lines is effectively eliminated. The constructed defect candidate set is accurately located, providing a clear target for subsequent texture consistency verification, significantly reducing computational redundancy in subsequent processes, and improving overall detection efficiency and coarse screening accuracy.

[0017] (2) A two-dimensional texture consistency verification model based on a self-supervised learning framework is constructed. By using an improved cut-and-paste data augmentation method, the feature differences between minute texture defects and normal process textures are simulated. Combined with contrastive learning to train the feature extraction network, the sensitivity to irregular anomalies such as texture breaks and blurring is significantly improved. This two-dimensional texture consistency verification model focuses on local texture analysis of the defect candidate set, rather than global detection, which greatly improves the verification efficiency, effectively makes up for the shortcomings of traditional methods in detecting minute defects, and further reduces the false negative rate and false positive rate.

[0018] (3) By employing an improved dictionary learning method in the two-dimensional texture consistency verification model, and by reducing the low-rank constraint of dictionary learning, the constructed overcomplete dictionary retains richer semantic information of normal textures, enabling it to accurately depict the periodic and regular characteristics of embossing, relief, and other process textures. Based on the reconstruction error judgment logic of this dictionary, it can effectively distinguish between normal process areas with abnormal shapes but regular textures and real defects with abnormal shapes and textures, thus avoiding misjudging compliant process textures as printing defects from the root. This design significantly improves the reliability of the detection results, adapts to the complex decorative process requirements of high-end packaging boxes, ensures the accuracy of quality judgment, and meets the stringent requirements of the manufacturing industry for high-precision quality inspection. Attached Figure Description

[0019] Figure 1 This is a flowchart of a packaging box printing quality inspection method based on image processing proposed in this invention.

[0020] Figure 2 This invention provides a hierarchical diagram of a defect candidate set obtained from a packaging box printing quality inspection method based on image processing.

[0021] Figure 3 This invention presents a hierarchical diagram of the packaging box printing quality inspection results obtained through an image processing-based packaging box printing quality inspection method. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figures 1-3 This invention provides a technical solution: a method for inspecting the printing quality of packaging boxes based on image processing. Specifically, the method for inspecting the printing quality of packaging boxes based on image processing is provided below. Figure 1 The method includes the following steps: Step S1: Collect depth variation information and natural texture images of the packaging box surface, perform geometric registration on the depth variation information and natural texture images to obtain an initial spatial information field, and construct a three-dimensional model of the printed surface based on the initial spatial information field.

[0024] This step aims to construct a high-precision 3D model of the printed surface of the packaging box using multi-angle structured light scanning, multi-source information registration, and 3D reconstruction technology. This 3D model of the printed surface will serve as the geometric and textural basis for subsequent quality analysis.

[0025] First, depth variation information and natural texture images of the packaging box surface are acquired. The packaging box to be inspected is fixed at the inspection station, and a multi-view imaging unit arranged in a ring performs synchronous scanning. This multi-view imaging unit includes a structured light projection device and at least two sets of cameras from different directions. The structured light projection device projects a series of bright and dark stripes with a sinusoidal distribution onto the surface of the packaging box. When there are micro-undulations on the surface of the packaging box caused by processes or defects, the projected regular stripes will undergo geometric deformation. By capturing this deformation, the depth variation information of the packaging box can be obtained. At the same time, each camera simultaneously acquires natural texture images (i.e., color and pattern) of the packaging box surface under uniform illumination. Through the combination of active structured light and passive texture acquisition, the multi-view imaging unit simultaneously obtains a phase map reflecting the surface geometry and a texture map reflecting the visual appearance.

[0026] Next, the depth variation information acquired from multiple perspectives is geometrically registered with the natural texture images to form an initial spatial information field. The core purpose of registration is to unify data from different sensors (deformed stripe patterns, texture maps) into the same world coordinate system, forming a fused dataset that can be subsequently computed, i.e., the initial spatial information field. Mathematically, this initial spatial information field can be represented as a set consisting of discrete three-dimensional points and their associated attributes: ,in Identify a data point, These are the target 3D coordinates of data point d. It is the texture color vector (RGB) of data point d. It is the calculated absolute phase value. The process of obtaining this field depends on calibration parameters and triangulation.

[0027] Specifically, the intrinsic parameter matrix of each camera has been obtained during the initial calibration. (Including focal length) and the main point ) and extrinsic parameter matrix (Rotation and translation), and the equivalent intrinsic parameters of the structured light projector. and external references For a given camera pixel... Its relationship with the target's three-dimensional coordinates Associated via camera projection model: ; in, It is a non-zero scaling factor (i.e., the reciprocal of the depth value); It is an intrinsic parameter matrix; It is an extrinsic parameter matrix; These are projected coordinates. This formula defines the coordinates from the pixel point. A three-dimensional ray is projected outwards, but the point cannot be determined. The specific location on this ray. To solve for the target's three-dimensional coordinates. Constraints from the structured light projector need to be introduced. An absolute phase value is calculated for each pixel by performing phase calculations on multiple acquired phase-shifted fringe patterns. This phase value corresponds to a virtual projected coordinate in the structured light projector coordinate system, which is the encoded coordinate of the structured light projector. ( (The phase-pixel scaling factor obtained from calibration). Therefore, the target's three-dimensional coordinates The projection relationship under the projector model must also be satisfied: ; in, It is the scaling factor of the projection equation of the projector; These are the coded coordinates of the structured light projector; and These are the intrinsic and extrinsic parameter matrices of the structured light projector. Solving the two projection equations simultaneously allows us to eliminate the unknowns. and Construct three-dimensional coordinates of the target A system of overdetermined equations with three components. Solving this system of overdetermined equations using the least squares method yields the target's 3D coordinates corresponding to the pixel. Perform the above calculations on each set of synchronously acquired multi-view images, and then convert all the obtained target 3D coordinates... and the texture color vector obtained from the corresponding camera pixels. By summarizing, we obtain the initial spatial information field after registration. This is a discrete point dataset that already possesses precise geometric relationships and texture attachments.

[0028] Finally, based on the initial spatial information field Constructing a high-precision 3D model of the printing surface Due to the initial spatial information field It is a discrete set of points and needs to be reconstructed into a continuous surface model for analysis. The reconstruction process first involves reconstructing the target's three-dimensional coordinate set. Denoising and fusion are performed to remove outliers. Then, a surface reconstruction algorithm based on Poisson reconstruction or rolling guided filtering is used to generate a continuous watertight triangular mesh surface from the point cloud. This results in the final 3D model of the printed surface. It can be expressed in parameterized form as Where V is the set of model vertices, Each vertex That is, from the target three-dimensional coordinate set Three-dimensional coordinates obtained through reconstruction algorithms or interpolation; It is a set of edges, which connect the vertices to form the boundary of the triangular facet; It is a collection of triangular facets.

[0029] Obtaining the three-dimensional model of the printing surface Next, the texture needs to be mapped onto its surface. For a 3D model of a printed surface... Each vertex on The three-dimensional coordinates of the vertex are known. Based on the position and normal of this vertex, select the optimal viewing camera, i.e., the camera with the smallest angle between the line of sight and the normal. Then, select this vertex... Substitute the three-dimensional coordinates into the projection equation of the camera with the optimal viewing angle (i.e., using the camera's...). and ), calculate its projected coordinates on the texture image. : ; in, These are the three-dimensional coordinates of the vertex; and These are the intrinsic and extrinsic parameter matrices of the camera with the best viewing angle; These are the intermediate homogeneous coordinates; These are texture coordinates.

[0030] Texture coordinates calculated using the above formula That is, the vertex Sampling locations on the corresponding natural texture image. A 3D model of the entire printed surface. After calculating the texture coordinates of all vertices and internal pixels of each triangular facet, the acquired natural texture image can be fitted onto the 3D model of the printed surface. On top, a textured three-dimensional model of the printed surface is formed. This 3D model of the printed surface accurately characterizes the microscopic geometry and macroscopic visual texture of the packaging box surface, providing a complete digital twin for subsequent anomaly detection and texture analysis.

[0031] Step S2: Extract the morphological features of the three-dimensional model of the printed surface to obtain the actual morphological features, and input them into the three-dimensional morphological anomaly detection model constructed based on the autoencoder to output the morphological anomaly region. Construct a defect candidate set based on the morphological anomaly region.

[0032] The three-dimensional model of the printing surface constructed in step S1 Building upon the previous steps, this step aims to identify potential local geometric anomalies in the 3D model of the printed surface caused by printing process defects. To achieve this goal, this step constructs a 3D anomaly detection model based on an autoencoder. The core idea is to allow the 3D anomaly detection model to learn the morphological feature distribution of a "normal" printed surface, and any area deviating from this distribution is judged as "abnormal".

[0033] First, the three-dimensional model of the printing surface. Shape feature extraction is performed to obtain high-dimensional actual shape features representing its local geometric properties. Shape feature extraction is applied to the 3D model of the printed surface. The process is performed on the surface of the triangular mesh. First, a 3D model of the printed surface is created. Dense and uniform sampling is performed within the effective printing area to obtain a set of sampling points. For each sampling point Analyze its local neighborhood (usually defined as ) Centered on, with radius Using the geometric information of all vertices within a spherical region (or within a k-ring neighborhood defined by mesh topology), calculate a set of multidimensional feature vectors. The extracted features encompass multiple dimensions to comprehensively characterize the local topography. Core geometric features include curvature features, obtained by fitting sampling points. The local neighborhood of the point is a quadratic surface; calculate the principal curvature at that point. and (That is, the maximum and minimum values ​​of the normal curvature). The principal curvature directly reflects the degree of curvature of the surface in the most curved and flattest directions at that point. From the principal curvature, two intrinsic geometric quantities can be further derived: Gaussian curvature. This is used to describe the local shape type of a surface (such as an elliptic point, hyperbolic point, or parabolic point); and the mean curvature. It is used to characterize the average curvature of a surface and is particularly sensitive to identifying uniform bulges or depressions.

[0034] Based on curvature analysis, the height distribution characteristics, which characterize the statistical properties of surface micro-undulations, are further calculated. This process projects points within the local neighborhood onto... On the tangent plane of a point, we obtain the set of height values ​​of each point along the normal vector. Based on this set of height values Calculate a series of statistical moments, arithmetic mean height and root mean square height Used to quantify the overall undulation intensity of the region; skewness Used to determine the asymmetry of height distribution (e.g., whether the surface is dominated by peaks or valleys); kurtosis Used to measure the sharpness of a distribution curve, it can distinguish between flat surfaces and surfaces with sharp peaks and valleys.

[0035] Based on the above calculations, each sampling point Represented as a Actual morphological features of the dimension For all Perform this operation on each sampling point to obtain a three-dimensional model representing the entire printed surface. Actual morphological feature set .

[0036] In the actual morphological feature set Before inputting the 3D anomaly detection model, standardization preprocessing is required to eliminate the influence of dimensions. Specifically, a reference morphology feature set based on a large number of normal samples is used. Calculate the mean of each feature dimension (8 dimensions in total). and standard deviation Then, the actual morphological feature set Each actual morphological feature Z-score standardization is performed to obtain the normalized actual morphological features. ,in .

[0037] Next, a 3D shape anomaly detection model is built and trained based on an autoencoder. An autoencoder is an unsupervised learning model designed to learn efficient representations of data. It consists of two parts: the encoder and the encoder. and decoder ,in and These represent the learnable parameters of the encoder and decoder, respectively. Encoder Actual morphological features Mapped to a low-dimensional, dense latent encoded feature The decoder then extracts this latent encoding feature. The original input features are reconstructed, i.e., the feature vector is reconstructed. Its learning objective is to minimize the difference between the input and the reconstructed output. For a given actual morphological feature... The forward propagation process of an autoencoder can be described as follows: ; ; in, It is to reconstruct the feature vector; These are latent coding features. Dimension Much smaller than the input dimension This forces the three-dimensional anomaly detection model to learn the most representative information in the data; and These are the weight matrices and bias vectors of the encoder and decoder, respectively, which are the learnable parameters that need to be optimized through training. and ; This refers to nonlinear activation functions, such as ReLU, which introduce nonlinear transformation capabilities into 3D shape anomaly detection models, enabling them to learn more complex feature patterns. Latent encoded features It is the three-dimensional shape anomaly detection model that detects the actual shape features of the input. An abstract and compressed representation of a surface should ideally contain all the key information needed to reconstruct normal surface morphology features.

[0038] During the training phase, a reference topography feature set was extracted using a large number of defect-free normal printed surface 3D models. As training data, the learnable parameters are optimized using the backpropagation algorithm. and The goal is to minimize the reconstruction error across all training samples. The reconstruction error is typically calculated using mean squared error, which measures the autoencoder's ability to reconstruct the input. For a single sample, the reconstruction error is... The calculation formula is: ; in, Representing vectors Norm (Euclidean distance); and These are the actual morphological features and reconstructing feature vectors The I-th component; l is the component index. This reconstruction error... This directly reflects the familiarity of the 3D topography anomaly detection model with the current input features and its reconstruction accuracy.

[0039] Once the 3D anomaly detection model is trained, it can be used to detect anomalies in new, untested 3D printed surface models. This involves setting the actual morphological feature set of the 3D printed surface model to be tested. The data is input into a pre-trained 3D anomaly detection model, and forward propagation yields each actual anomaly feature. Reconstructed feature vectors and the corresponding reconstruction error Because the autoencoder is trained only on normal samples, it learns how to effectively compress and reconstruct normal surface topography features. When inputting region features with normal topography, the 3D topography anomaly detection model can reconstruct them well, with a reconstruction error of [missing information]. The actual shape features of a defective region (such as abnormal protrusions caused by ink buildup or ripples caused by coating wrinkles) are lower. Conversely, when the actual shape features are input, they deviate from the normal feature distribution learned by the 3D shape anomaly detection model in the latent space, causing the decoder to fail to reconstruct accurately, resulting in a reconstruction error that is significantly higher than normal.

[0040] The difference between normal and abnormal can be distinguished by setting a dynamic threshold or performing statistical tests. For example, the reconstruction error of all sampling points can be used. The data is sorted, and points with errors greater than a preset percentile (e.g., 99%), or points with errors exceeding several times the standard deviation of the mean of the normal sample error distribution, are initially marked as outliers. All these outliers are then mapped onto the 3D model of the printed surface. The corresponding set of spatial locations constitutes the region of morphological anomaly. These spatially adjacent morphologically abnormal regions are clustered and merged to form a candidate set of defects. Each of them This represents an abnormal area of ​​shape that is suspected of being a defect.

[0041] It should be noted that the clustering algorithm initially labeled all morphologically abnormal regions in this step. As input, the core criterion is spatial adjacency, that is, calculating the three-dimensional model of each abnormal shape region on the printed surface. On the surface, if the shortest Euclidean distance between the boundaries of two morphological anomaly regions is less than a preset neighborhood threshold d, they are considered adjacent. By traversing all morphological anomaly regions, adjacent morphological anomaly regions are analyzed for connectivity and merged to form a spatially continuous and larger morphological anomaly region. Finally, the merged defect candidate set C is output. All regions that deviate from the normal standard in terms of geometric shape were accurately located, providing clear targets for analysis in the subsequent step S3 for texture consistency verification.

[0042] Step S3: Based on the defect candidate set, extract texture features from the three-dimensional model of the printed surface to obtain the actual texture features, and input them into the two-dimensional texture consistency verification model built based on the self-supervised learning framework to output the real printing defects.

[0043] The abnormal morphology regions (i.e., the defect candidate set) obtained in step S2 Building upon the previous step, the core task of this step is to construct and apply a two-dimensional texture consistency verification model based on a self-supervised learning framework. The purpose of this two-dimensional texture consistency verification model is to determine the authenticity of textures in abnormal regions within the defect candidate set, strictly distinguishing between texture anomalies caused by genuine printing defects and regions caused by normal decorative processes such as embossing and relief, which are only morphologically suspicious but have regular textures. This step first uses the three-dimensional model of the printed surface obtained in step S1... Extract the defect candidate set from the texture map. Middle morphological anomaly region Corresponding two-dimensional texture image block Each 2D texture image block The corresponding actual texture features are the objects of analysis in this step.

[0044] First, a texture feature extraction network based on self-supervised contrastive learning is constructed as the front-end of a two-dimensional texture consistency verification model. This model aims to learn a robust feature representation unsupervised, making it insensitive to regular variations in normal textures (such as repeated embossing) but highly sensitive to irregular variations caused by defects, such as texture breaks and blurring. An improved cut-and-paste data augmentation method is used to construct training samples for self-supervised learning. Two-dimensional texture image patches are randomly sampled from a normal sample library. The improved cut-and-paste data augmentation method will randomly crop out a long, narrow rectangle or a square region with extremely small side lengths. (Its side length is set to the original image width) (one-twentieth), then the square area Randomly paste back 2D texture image blocks Generate a simulated texture image patch containing simulated minute irregularities at any location. .thus, This forms a texture image pair. Through this construction, the feature extraction network... The model is trained to recognize localized, subtle disruptions to the regularity of the texture background, thereby improving the ability of the 2D texture consistency verification model to perceive real defects. The feature extraction network uses shallow convolutional layers of AlexNet pre-trained on ImageNet, as it can effectively capture mid-to-low-level texture features.

[0045] Next, the texture feature extraction network is trained using cross-entropy loss. The texture image is then compared with... Input Feature Extraction Network To obtain the corresponding actual texture features and simulated texture features In a containing In the training batch of image pairs, contrastive loss is used. Optimize: ; in, These are actual texture features; It simulates texture features; It is cosine similarity. ; It is a temperature hyperparameter used to adjust the sharpness of the distribution; These are negative sample features; the summation term in the denominator iterates through the batch, excluding... All other than itself The training batch size is N; a is the actual texture feature index; b is the negative sample feature index. This contrastive loss... Drive positive sample pairs In the feature space, they are close to each other while being far away from all other samples, thus enabling the feature extraction network to... The network learns to encode feature representations that discriminate between local texture consistency and inconsistency. After training, the feature extraction network... It can map any input 2D texture image patch to a discriminative feature space in which normal texture features cluster while abnormal texture features deviate.

[0046] Then, based on the trained feature extraction network An anomaly verification module based on improved dictionary learning was constructed to complete the final construction of the two-dimensional texture consistency verification model. This anomaly verification module utilizes the reconstruction principle of dictionary learning for consistency judgment. First, a pre-trained feature extraction network is used... Extract standard texture features from a large number of normal training samples to construct a standard texture feature set. Then, an overcomplete dictionary is learned using an improved dictionary learning algorithm. Where d is the dimension of the texture feature y (i.e., the feature extraction network) The output dimension is m, where m is the number of dictionaries and m>d, to ensure its overcompleteness.

[0047] The improved dictionary learning algorithm removes the low-rank constraint term, i.e., the nuclear norm, imposed on the overcomplete dictionary D in the traditional algorithm from the dictionary learning optimization objective function. This prevents the overcomplete dictionary D from being forced to compress into a lower-dimensional subspace during training, allowing it to learn and preserve higher-dimensional, more complex normal texture patterns. The improved optimization objective is: ; in, It is the nth standard texture feature; It is a complete dictionary that needs to be studied; It corresponds to the first A sparse coding coefficient vector of standard texture features; for Norm, a measure of reconstruction error; for Norm, forcing coefficient sparsity; For regularization parameters, The improvement in this step lies in reducing the reliance on overcomplete dictionaries. The low-rank constraint allows it to retain richer semantic information of normal textures, thus enabling it to more accurately represent the complex regularity of normal process textures such as periodic embossing, and avoid misjudging them as anomalies.

[0048] Finally, the constructed two-dimensional texture consistency verification model is applied to verify the defect candidate set. For each morphologically abnormal region... Extract its two-dimensional texture image patches And through feature extraction network Obtain its actual texture features Next, after going through a complete dictionary... Solve its sparse coding : ; in, It is the optimal sparse coding vector to be found; It is a complete dictionary that has already been learned; It is an area with abnormal morphology. The actual texture features.

[0049] And calculate the actual texture features. Reconstruction error : ; in, It is a reconstruction error; is the optimal sparse coding vector; D is the overcomplete dictionary.

[0050] Due to the completeness of the dictionary If only normal features are learned, then areas with abnormal morphology will be affected. The texture is normal (e.g., it belongs to regular embossing), and its actual texture characteristics are... Can be passed through a complete dictionary Reconstruction was performed well, and the reconstruction error was minimal. The error is relatively small; if it is a real defect, its texture pattern deviates from the normal distribution, and the reconstruction error is relatively small. This will increase significantly. By setting a defect threshold... , judge when At that time, areas with abnormal morphology Verified as a genuine printing defect Otherwise, it is judged as a normal process texture and excluded from the defect candidate set. Ultimately, the defect is determined based on actual printing defects. Construct a set of real printing defects .

[0051] It should be noted that, regarding the specific criteria for determining the defect threshold T, an adaptive strategy based on the statistical distribution of normal samples is adopted. During the model training phase, the texture features of all normal training samples are used to reconstruct the model using the learned overcomplete dictionary D, and the reconstruction error set { is calculated}. Subsequently, statistical analysis was performed on the error set to calculate its mean reconstruction error. and reconstruction error standard deviation Ultimately, the defect threshold T ,in This coefficient is preset based on the stringency of actual process requirements, with a value range of 3 to 6. The principle behind this setting is that the reconstruction error of normal textures follows a stable distribution, while real defects act as significant outliers, exceeding this distribution range. This strategy ensures that the determination of the defect threshold T has a clear statistical basis and can automatically adapt to the inherent fluctuations in the textures of different products, thereby achieving a high detection rate while controlling the false alarm rate.

[0052] Thus, the two-dimensional texture consistency verification model constructed in this step has enabled accurate verification of the initial morphology screening results in step S2, ensuring high reliability and low false alarm rate in the detection.

[0053] Step S4: Analyze and locate the actual printing defects to obtain the packaging box printing quality inspection results.

[0054] The set of actual printing defects output in step S3, verified for texture consistency. Based on this, each actual printing defect Having already linked the morphological anomaly information (such as region, protrusion / depression type) provided in step S2 and the texture anomaly discrimination results provided in step S3, this step aims to fuse and analyze this multi-source information to complete the identification of each real printing defect. Quantitative assessment and classification are used to make a comprehensive judgment on the overall quality of the packaging box and to realize the true printing defects. Precise positioning within the original image and physical space ultimately generates structured packaging box printing quality inspection results. This step connects all front-end sensing and analysis results, forming the final decision-making and output terminal for quality control instructions.

[0055] First, based on real printing defects The associated morphological and texture attributes are quantified and automatically classified. Each real printing defect is extracted. fused feature vector .in, It is the projected area, obtained by calculating the defect area in step S1 to create a three-dimensional model of the printed surface. The total area of ​​the triangular facets covering the surface is obtained; The average depth deviation is the average difference in height between all points in the region and the reference height of the surrounding normal region, provided by the topography analysis in step S2. It is a topography type identifier, a discrete parameter based on the average depth deviation. The positive and negative values ​​and curvature characteristics are taken from the predefined set {convex, concave, wavy, flat}; This is a texture anomaly type identifier, also a discrete parameter, given by the verification result of step S3, such as {missing, broken, blurry, color difference, normal}. Based on the preset classification rules, the fused feature vectors are... Mapped to specific defect categories For example, it can be defined that when feature vectors are fused... Morphology type identifier A marker for raised and textured abnormal types. When there is a color difference, (Ink buildup); when It is concave and When it breaks, (Imprinting anomaly); when For ripples and When it is normal, (Wrinkled coating category). This classification logic substantially integrates the judgment results of steps S2 and S3.

[0056] At the same time, for each real printing defect Calculate a comprehensive severity score This severity score Quantified the actual printing defects The negative impact is calculated using the following formula: ; in, It is a genuine printing defect. The projected area; This represents the total area of ​​the printed area on the packaging box, used to normalize the defect area. It is a genuine printing defect. The average depth deviation; It is the characteristic thickness of the printed layer of a qualified product (from the process standard), used to normalize the depth deviation; It is a weighting coefficient function set according to the severity of texture anomaly type. Its mapping value is determined based on prior process knowledge of the impact of various defects (e.g., assigning a higher coefficient to missing defects and a lower coefficient to color difference), which transforms the qualitative discrimination in step S3 into quantitative contribution. The preset weighting coefficients satisfy... This is used to balance the relative importance of area, depth, and texture type in severity assessment.

[0057] It should be noted that the weighting coefficient function It is a mapping from a discrete set of defect types to positive real-valued weight coefficients. The specific rules are implemented through a predefined weight mapping table, which is built upon prior knowledge of the specific product's process requirements. For example, a typical mapping rule is: severe defects such as missing or illegible images are assigned a higher weight, e.g., I(missing) = 1.0; moderate defects such as uneven ink density or misregistration are assigned a medium weight, e.g., I(uneven) = 0.6; and minor defects such as slight color differences or blemishes are assigned a lower weight, e.g., I(color difference) = 0.3. The specific values ​​of this mapping relationship are pre-set and fixed after normalization, based on the analysis of the impact of various defects on the product pass rate in historical quality data, or according to the deduction weight ratios for different defects in industry standards.

[0058] Next, based on the actual set of printing defects Based on all information, assess the overall quality of the packaging box. Define the overall quality score. Its calculation formula comprehensively considers actual printing defects. The total volume, average severity, and number are calculated using the following formulas: ; in, Approximately the total volume of all defects; This is the reference tolerance for the total permissible volume of defects; It is the reference severity used for normalization; It is the sum of all defect severity scores; It is a genuine printing defect. The total quantity; This is the maximum number of defects allowed. The weighting coefficients and Final quality assessment For a binary decision function: ; in, It is the overall quality score threshold; These are all real printing defects. The highest severity score; This is the upper limit threshold for the severity score of a single genuine printing defect. This judgment rule means that the packaging box is only judged as qualified when the overall quality is good, there are no extremely serious defects, and the total number of defects does not exceed the limit.

[0059] Then, the defect location information is mapped to a coordinate system that the production line can execute. To achieve automatic rejection of defective products or annotation on natural texture images, a 3D model of the printed surface needs to be created. The defect coordinates are back-projected. For actual printing defects... Each vertex within the region Using the optimal viewpoint camera parameters calibrated in step S1 for texture mapping and Project it back to the projection coordinates of the natural texture image. These projection coordinates are used to locate and annotate defects on the natural texture image, generating an inspection report. The projection process is as follows: ; in, It is a genuine printing defect. Three-dimensional vertices within the region; and These are the intrinsic and extrinsic parameter matrices of the camera with the best viewing angle; These are the intermediate homogeneous coordinates after projection transformation.

[0060] Meanwhile, based on the fixed transformation relationship between the inspection station and the world coordinate system Calculate actual printing defects The positioning coordinates of the center of the 3D bounding box in the world coordinate system The calculation formula is: ; here, It is a genuine printing defect. Position coordinates in the world coordinate system; It is a genuine printing defect. The set of vertices contained therein; It is the number of vertices; It is a genuine printing defect. Average coordinates of all vertices. It provides the physical location of the actual printing defects; This is a transformation matrix from the 3D model coordinate system of the printed surface to a fixed world coordinate system. These positioning coordinates can be directly sent to the robotic arm controller to achieve automated operation.

[0061] Finally, a structured packaging box printing quality inspection result is generated and output, the core content of which includes: comprehensive judgment result. (Pass / Fail); Overall Defects Overview (Total Quantity) Overall quality score ); each real printing defect A detailed list covering its defect categories Severity score Projected coordinates on natural texture images and the positioning coordinates in the world coordinate system. A natural texture image with visual annotations of all defect locations.

[0062] This step integrates morphology and texture information for comprehensive judgment and precise positioning, transforming the front-end perception and recognition results into clear quality control instructions and traceable data records. This closes the loop of the complete automated detection process from 3D imaging to quality decision-making, demonstrating the method's comprehensive capability to handle complex surface detection in high-end packaging based on the fusion of 3D and 2D information.

[0063] This technical solution acquires depth and texture information of the packaging box surface through multi-angle structured light scanning. A 3D model of the printed surface is constructed through geometric registration. Then, an autoencoder is used to build a 3D anomaly detection model to coarsely screen the defect candidate set. Subsequently, a 2D texture consistency verification model is constructed based on a self-supervised learning framework combined with improved dictionary learning to accurately distinguish between real printing defects and normal process textures. Finally, multi-dimensional information is integrated to complete defect classification, location, and quality judgment. This solution effectively addresses pain points in industrial scenarios such as scarce defect samples, misjudgment of process textures, and difficulty in detecting minute defects, significantly improving detection accuracy and efficiency. It avoids misjudgment of normal processes such as embossing and relief, adapts to the complex printing quality inspection needs of high-end packaging boxes, and provides reliable automated quality inspection support for production lines.

[0064] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the phrase "comprising an element defined as..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their likenesses.

Claims

1. A method for inspecting the printing quality of packaging boxes based on image processing, characterized in that, Includes the following steps: Step S1: Collect depth variation information and natural texture images of the packaging box surface, perform geometric registration on the depth variation information and natural texture images to obtain an initial spatial information field, and construct a three-dimensional model of the printed surface based on the initial spatial information field; Step S2: Extract the morphological features of the three-dimensional model of the printed surface to obtain the actual morphological features, and input them into the three-dimensional morphological anomaly detection model based on the autoencoder to output the morphological anomaly region. Construct a defect candidate set based on the morphological anomaly region. Step S3: Based on the defect candidate set, extract texture features from the three-dimensional model of the printed surface to obtain the actual texture features, and input them into the two-dimensional texture consistency verification model built based on the self-supervised learning framework to output the real printing defects. Step S4: Analyze and locate the actual printing defects to obtain the packaging box printing quality inspection results.

2. The packaging box printing quality inspection method based on image processing according to claim 1, characterized in that, Acquire depth variation information and natural texture images of the packaging box surface, perform geometric registration on the depth variation information and natural texture images to obtain an initial spatial information field, including: Collect information on depth variations and natural texture images of the packaging box surface; Unified geometric registration is performed between depth variation information and natural texture images to form an initial spatial information field. ,in Identify a data point, These are the target 3D coordinates of data point d. It is the texture color vector of data point d. It is the absolute phase value.

3. The packaging box printing quality inspection method based on image processing according to claim 2, characterized in that, Constructing a three-dimensional model of the printed surface based on the initial spatial information field includes: For the initial spatial information field The target three-dimensional coordinate set Denoising and fusion are performed to remove outliers; a surface reconstruction algorithm based on Poisson reconstruction or rolling guided filtering is used to generate a continuous watertight triangular mesh surface from the target's three-dimensional coordinates, i.e., the three-dimensional model of the printed surface. ; For the three-dimensional model of the printed surface Each vertex on Based on the vertex's position and normal, select the optimal viewing camera—that is, the camera with the smallest angle between the line of sight and the normal—and position the vertex accordingly. Substitute the three-dimensional coordinates into the projection equation of the optimal viewing angle camera to calculate its projected coordinates on the texture image. : ； in, These are the three-dimensional coordinates of the vertex; and These are the intrinsic and extrinsic parameter matrices of the camera with the best viewing angle; These are the intermediate homogeneous coordinates; These are texture coordinates; Projected coordinates That is, the vertex The sampling location on the corresponding natural texture image; for the three-dimensional model of the entire printed surface. After calculating the texture coordinates of all vertices and internal pixels of each triangular facet, the acquired natural texture image can be fitted onto the 3D model of the printed surface. On top, a textured three-dimensional model of the printed surface is formed. .

4. The packaging box printing quality inspection method based on image processing according to claim 3, characterized in that, The morphological features of the printed surface three-dimensional model are extracted to obtain the actual morphological features, including: 3D model of the printed surface Sampling is performed within the effective printing area to obtain a set of sampling points; for each sampling point... Analyze the geometric information in its local neighborhood and calculate a set of multidimensional feature vectors. Geometric features include curvature features, obtained by fitting sampling points. The local neighborhood of the point is a quadratic surface; calculate the principal curvature at that point. and Gaussian curvature is derived from principal curvature. and mean curvature Projecting points within the local neighborhood onto... On the tangent plane of a point, we obtain the set of height values ​​of each point along the normal vector. Based on this set of height values Calculate the arithmetic mean height Root mean square height skewness and kurtosis ; Each sampling point Represented as actual morphological features For all Perform this operation on each sampling point to obtain a three-dimensional model representing the entire printed surface. Actual morphological feature set ; Using a reference morphological feature set Calculate the mean of each feature dimension. and standard deviation Then, the actual morphological feature set Each actual morphological feature Z-score standardization is performed to obtain the normalized actual morphological features. .

5. The packaging box printing quality inspection method based on image processing according to claim 4, characterized in that, The input is a 3D anomaly detection model built based on an autoencoder, and the output is anomaly regions. A defect candidate set is then constructed based on these anomaly regions, including: A 3D anomaly detection model is built and trained based on an autoencoder. The autoencoder consists of an encoder. and decoder It consists of two parts, the encoder Actual morphological features Mapping to latent encoded features The decoder extracts this latent encoded feature. The reconstructed feature vector is obtained by reconstructing the feature vector. ; During the training phase, a reference topography feature set was extracted using a large number of defect-free normal printed surface 3D models. As training data, for a single sample, its reconstruction error The calculation formula is: ； in, Representing vectors Norm; and These are the actual morphological features and reconstructing feature vectors The One component; l is the component index; After the 3D anomaly detection model is trained, the actual anomaly feature set to be tested is... The data is input into a pre-trained 3D anomaly detection model, and forward propagation yields each actual anomaly feature. Reconstructed feature vectors and the corresponding reconstruction error ; Based on reconstruction error Anomalies were identified; all these anomalies were represented in a 3D model of the printed surface. The corresponding set of spatial locations constitutes the region of morphological anomaly. ; areas with abnormal shapes Clustering and merging are performed to ultimately form a defect candidate set. .

6. The packaging box printing quality inspection method based on image processing according to claim 5, characterized in that, Based on the defect candidate set, texture features are extracted from the three-dimensional model of the printed surface to obtain the actual texture features, including: A texture feature extraction network based on self-supervised contrastive learning is constructed, and an improved cut-and-paste data augmentation method is used to construct training samples for self-supervised learning, which are two-dimensional texture image patches randomly sampled from a normal sample library. The improved cut-and-paste data augmentation method generates textured image pairs. ; This texture feature extraction network is trained using cross-entropy loss to extract texture images. Input Feature Extraction Network To obtain the corresponding actual texture features and simulated texture features Using comparative loss Optimize: ； in These are actual texture features; It simulates texture features; It is cosine similarity; It is a temperature hyperparameter; is the negative sample feature; N is the training batch size; a is the actual texture feature index; b is the negative sample feature index; After training, for each region with abnormal morphology Extract its two-dimensional texture image patches And through feature extraction network Obtain its actual texture features .

7. The packaging box printing quality inspection method based on image processing according to claim 6, characterized in that, The input is a two-dimensional texture consistency verification model built on a self-supervised learning framework, and the output is the actual printing defects, including: Based on the trained feature extraction network A complete dictionary was learned through an improved dictionary learning algorithm. ; For each region with abnormal morphology Extract its two-dimensional texture image patches And through feature extraction network Obtain its actual texture features In a complete dictionary Solve its sparse coding And calculate the actual texture features. Reconstruction error : ； in, It is a reconstruction error; It is the optimal sparse coding vector; D is the overcomplete dictionary; Set defect threshold , judge when At that time, areas with abnormal morphology Verified as a genuine printing defect Otherwise, it is determined to be a normal process texture and excluded from the defect candidate set; ultimately, it is based on the actual printing defects. Construct a set of real printing defects .

8. The packaging box printing quality inspection method based on image processing according to claim 7, characterized in that, The overcomplete dictionary includes: Use a trained feature extraction network Extract standard texture features and construct a standard texture feature set. An overcomplete dictionary is learned through an improved dictionary learning algorithm. The optimization objective is: ； in, It is the nth standard texture feature; It is a complete dictionary that needs to be studied; It corresponds to the first A sparse coding coefficient vector of standard texture features; for Norm; for Norm; This is the regularization parameter.

9. The packaging box printing quality inspection method based on image processing according to claim 8, characterized in that, The analysis and location of the actual printing defects include: Extract each real printing defect fusion feature vector ,in, It is the projected area; It is the average depth deviation; It is a shape type identifier; It is a texture anomaly type identifier; Based on the preset classification rules, the fused feature vectors will be... Mapped to specific defect categories ; Meanwhile, based on the fused feature vector For each real printing defect Calculate a comprehensive severity score ; Next, based on the actual set of printing defects Calculate the overall quality score Based on overall quality score To obtain the final quality judgment ; 3D model of the printed surface Back-projecting the defect coordinates onto the surface to represent actual printing defects. Each vertex within the region Utilizing the best perspective camera parameters and Project it back to the projection coordinates of the natural texture image. Simultaneously based on vertices Calculate actual printing defects The positioning coordinates of the center of the 3D bounding box in the world coordinate system .

10. The packaging box printing quality inspection method based on image processing according to claim 9, characterized in that, The results of the packaging box printing quality inspection were obtained, including: Generate and output a structured packaging box printing quality inspection result, the core content of which includes: comprehensive judgment result. Overall Defect Overview; Each Real Printing Defect A detailed list covering its defect categories Severity score Projected coordinates on natural texture images and the positioning coordinates in the world coordinate system. A natural texture image with visual annotations of all defect locations.

Images