Card packaging quality visual online detection and defect classification method and system

Abstract

The invention discloses a card packaging quality visual online detection and defect classification method and system, and belongs to the technical field of machine vision, and the method comprises the steps: obtaining visible light and near-infrared images through dual-spectrum imaging, eliminating coating reflection based on adaptive Retinex, and achieving cross-scale defect positioning through an attention enhancement feature pyramid network. A graph neural network is introduced to realize collaborative classification of three large classes and twelve small classes of defects, a threshold value is dynamically optimized through Bayesian decision, fault root causes are deduced through time sequence analysis, the detection rate is 99.5% or above, the classification accuracy is 97.8%, and online detection of 200 pieces per minute is supported.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, specifically to a method and system for online visual inspection and defect classification of playing card packaging quality. Background Technology

[0002] As a leisure and entertainment product with a long history, the packaging quality of playing cards directly impacts their market competitiveness and brand image. Modern playing card packaging typically employs sophisticated printing techniques and is coated with a protective film to enhance durability and visual appeal. However, in high-speed automated production processes, playing card packaging is prone to various defects, including ink splatter, missed prints, and color differences during printing; scratches, bubbles, and foreign objects during lamination; and edge burrs, uneven cuts, and corner warping during molding. These defects not only affect the product's aesthetics but may also lead consumers to question its quality, thereby impacting the company's reputation and economic benefits.

[0003] In existing technologies, quality inspection of playing card packaging mainly relies on manual visual inspection or automated inspection systems based on machine vision. Manual inspection suffers from problems such as low efficiency, strong subjectivity, and susceptibility to fatigue leading to missed inspections, making it difficult to meet the inspection needs of hundreds of cards per minute on high-speed production lines. While automated inspection systems based on machine vision can improve inspection efficiency, existing technical solutions still have many shortcomings.

[0004] Chinese invention CN119399203A discloses a method and system for detecting packaging defects. This method establishes target feature template information by acquiring a first image, then performs edge detection and segmentation on a second image to be detected, and identifies defects such as misprints, omissions, and missing prints by matching the template information. However, this technical solution has the following problems: First, this solution uses only a single visible light imaging channel, which cannot effectively penetrate the film layer of the cardboard packaging to detect printing defects underneath. When the film material produces specular reflection, it is easy to cause false detections or missed detections. Second, the defect classification granularity of this solution is relatively coarse, only able to identify three basic defect types: misprints, omissions, and missing prints, which cannot meet the needs for fine-grained defect classification such as printing defects, material defects, and molding defects. Third, this solution uses a fixed template matching and threshold judgment mechanism, lacking the ability to adapt to the characteristics of different production batches. The detection accuracy will significantly decrease when production process parameters fluctuate. Finally, this solution lacks the ability to perform temporal analysis of defect distribution in continuous frames, making it impossible to infer the root cause of equipment failure from defect patterns and difficult to achieve preventative maintenance.

[0005] Furthermore, existing vision inspection systems for playing card packaging mostly employ general object detection algorithms, lacking specific optimizations for the printing characteristics of playing cards. Playing card packaging possesses unique visual features, including high-gloss laminated surfaces, intricately printed patterns, and regular geometric edges. These features place special demands on the detection algorithms. General detection algorithms suffer from high false negative and false positive rates when handling fine-grained defects such as laminated reflections, edge burrs, and color shifts. On high-speed production lines, playing card packaging moves at speeds exceeding 200 pieces per minute. Traditional inspection methods struggle to meet real-time requirements while maintaining accuracy, with single-piece inspection times often exceeding 100ms, becoming a bottleneck restricting capacity increases. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for online visual inspection and defect classification of playing card packaging quality, aiming to solve the technical problems of low defect detection accuracy, coarse classification granularity, lack of adaptive capability and fault diagnosis capability under high-speed operation conditions of playing card packaging production lines.

[0007] The first aspect of this invention provides a method for online visual inspection and defect classification of playing card packaging quality, comprising the following steps: a dual-spectral image acquisition step, wherein a dual-spectral imaging unit is used to synchronously acquire images of the moving playing card packaging, acquiring the printed content image through the visible light channel and the material texture image through the near-infrared channel; an image preprocessing step, wherein the image preprocessing module processes the printed content image based on the adaptive Retinex algorithm to eliminate specular reflection interference from the coating material, and restores the image degradation caused by high-speed transmission through motion blur inverse convolution, and fuses the processed printed content image and the material texture image to obtain a dual-spectral fused image; and a defect detection step, wherein defect detection... The module performs multi-scale feature extraction on dual-spectral fusion images and uses an attention-based multi-scale feature pyramid network to achieve cross-scale defect localization. In the defect classification step, the module introduces a graph neural network-based defect association reasoning mechanism, constructing a topological graph structure from multiple detected defect regions. Through message passing, it aggregates neighborhood defect features to achieve collaborative defect type determination and severity classification. In the Bayesian threshold adaptive adjustment step, the Bayesian decision threshold adjustment unit dynamically optimizes the judgment boundary thresholds for various defects based on the statistical characteristics of production batches. Finally, in the temporal correlation analysis step, the temporal correlation analysis module performs spatiotemporal correlation analysis on the defect distribution in consecutive frames to infer the root cause of equipment failure.

[0008] A second aspect of this invention provides a visual online inspection and defect classification system for playing card packaging quality, comprising: a dual-spectrum image acquisition unit for synchronously acquiring images of moving playing card packaging, including a visible light channel and a near-infrared channel; an image preprocessing module for eliminating specular reflection interference, restoring image degradation, and performing dual-spectrum fusion; a defect detection module for achieving cross-scale defect localization through a multi-scale feature pyramid network enhanced by an attention mechanism; a defect classification module for achieving collaborative classification through a defect association reasoning mechanism of a graph neural network; a Bayesian decision threshold adjustment unit for dynamically optimizing the judgment threshold; and a temporal correlation analysis module for inferring the root cause of equipment failure.

[0009] Compared with existing technologies, the advantages of this invention are as follows: First, by employing a dual-spectrum fusion imaging strategy, the near-infrared channel's ability to penetrate and image printing defects beneath the coating layer is utilized, forming a complementary detection mechanism with the visible light channel, effectively solving the problem of missed detection caused by coating reflection. Second, by eliminating specular reflection interference through the adaptive Retinex algorithm, and combining motion blur with inverse convolution to recover image degradation caused by high-speed transmission, image quality is significantly improved. Third, by using a defect topology reasoning mechanism based on graph neural networks, collaborative judgment of multiple defect regions is achieved, establishing a hierarchical classification system of three major categories and twelve subcategories: printing, material, and molding. Fourth, by using Bayesian decision theory, dynamic adaptive adjustment of the judgment threshold is achieved, improving detection stability under different batch conditions. Fifth, by using temporal correlation analysis to infer the root cause of equipment failure from defect distribution patterns, preventative maintenance is supported. This invention can achieve a defect detection rate of over 99.5%, a classification accuracy of 97.8% for 12 types of defects, supports an online detection speed of 200 images / minute, and a single image detection time of less than 50ms. Attached Figure Description

[0010] Figure 1 This is a flowchart of the online visual inspection and defect classification method for playing card packaging quality according to the present invention.

[0011] Figure 2 This is an architecture diagram of the online visual inspection and defect classification system for playing card packaging quality of the present invention. Detailed Implementation

[0012] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0013] This invention provides a method for online visual inspection and defect classification of playing card packaging quality, such as... Figure 1As shown, the method includes a dual-spectral image acquisition step S1, an image preprocessing step S2, a defect detection step S3, a defect classification step S4, a Bayesian threshold adaptive adjustment step S5, and a temporal correlation analysis step S6. In one embodiment of the present invention, a deeply coupled closed-loop collaborative architecture is formed among the steps, and the processing results of subsequent steps can inversely influence the parameter configuration of preceding steps, thereby achieving continuous optimization of the overall system performance.

[0014] Step S1: Dual-spectral image acquisition step.

[0015] In this embodiment of the invention, the dual-spectrum image acquisition step employs a specially designed dual-spectrum imaging unit to synchronously acquire images of the high-speed moving playing card packaging. Preferably, the dual-spectrum imaging unit includes two independent imaging channels: a visible light channel and a near-infrared channel. The two channels share the front objective lens assembly of the optical system, and the incident light is separated by wavelength using a beam splitter and then guided to the corresponding image sensors.

[0016] The visible light channel operates within the electromagnetic spectrum from 380nm to 780nm and is primarily used to acquire images of the printed content on playing card packaging. In one embodiment of this invention, the visible light channel employs a color CMOS image sensor with three color channels: red, green, and blue, capable of completely recording the color information, pattern details, and text content of the playing card packaging surface. (Printed content image) The pixel value range is 0 to 255, the resolution is no less than 2048×2048 pixels, and it can distinguish printing spot defects with a size of less than 50μm.

[0017] The near-infrared channel operates within the electromagnetic spectrum from 780nm to 1100nm, primarily used to acquire material texture images of playing card packaging. In this embodiment, near-infrared light possesses excellent penetration characteristics, enabling it to partially penetrate the coating layer on the surface of the playing card packaging, thus imaging the paper base material and printing ink layer beneath the coating layer. Material texture image. The data recorded information such as the distribution of paper fibers, the state of ink penetration, and the interface state between the coating and the paper substrate. This information is of great value for detecting printing defects that are covered by the coating.

[0018] Preferably, the image acquisition of the two imaging channels is precisely synchronized through a hardware trigger signal, with the synchronization time deviation controlled within 1ms. In this embodiment of the invention, considering that the production line conveyor belt operates at a speed of over 200 images per minute, the time interval between two adjacent image acquisitions is approximately 300ms. A 1ms synchronization deviation only results in a positional shift of approximately 0.07mm, which is far less than the positioning accuracy requirements of the detection system and therefore will not significantly affect the subsequent dual-spectrum fusion processing. The light source system of the dual-spectrum imaging unit adopts a ring LED array, including two sets of light sources: white LEDs and near-infrared LEDs. They are alternately illuminated using a time-division multiplexing method to avoid mutual interference between the two wavelength light sources.

[0019] In one embodiment of the present invention, to meet the requirements of high-speed online detection, the exposure time of the dual-spectrum imaging unit is set to be adjustable within the range of 0.5ms to 2ms, with the specific value automatically optimized according to ambient lighting conditions and conveyor belt speed. A shorter exposure time helps reduce motion blur but reduces the image signal-to-noise ratio; a longer exposure time can improve the signal-to-noise ratio but increases the degree of motion blur. The adaptive exposure control algorithm used in this invention strikes a balance between the two based on image quality evaluation indicators, typically selecting an exposure time of approximately 1ms at a conveyor speed of 200 images / minute.

[0020] Step S2: Image preprocessing step.

[0021] The image preprocessing step preprocesses the dual-spectral image obtained in step S1, mainly including three sub-processes: specular reflection elimination, motion blur recovery, and dual-spectral fusion. In this embodiment of the invention, the design of the image preprocessing module fully considers the visual characteristics of playing card packaging, especially the high-gloss reflective properties of the coated surface and the image degradation problem caused by high-speed motion.

[0022] This invention employs an adaptive Retinex algorithm to eliminate specular reflection interference from the coating material. Retinex theory posits that an image can be decomposed into the product of a reflection component and an illumination component, where the reflection component represents the intrinsic properties of the object, and the illumination component represents the ambient lighting conditions. For coated playing card packaging, specular reflection is primarily manifested in the illumination component; by estimating and removing the high-frequency components of the illumination component, specular reflection can be effectively suppressed.

[0023] In this embodiment of the invention, the core calculation process of the adaptive Retinex algorithm is as follows. First, the input visible light image... Perform a logarithmic transformation to convert the multiplicative model into an additive model:

[0024] ,

[0025] in: The image signal after logarithmic transformation; For position The visible light image pixel value at the location, ranging from 0 to 255; the increment operation is used to avoid zero value input in logarithmic operations.

[0026] Then, multi-scale Gaussian filtering is used to estimate the illumination components. This invention selects three different Gaussian scale parameters. Pixels Pixels and Pixels, each capturing light variations at different spatial frequencies:

[0027] ,

[0028] in: For the first Estimated illumination components at each scale; The standard deviation is The two-dimensional Gaussian kernel function; This represents a convolution operation. Smaller... The value can capture local lighting changes and specular highlights, and a larger value... The value can capture the global illumination gradient.

[0029] Preferably, the results from the three scales are weighted and fused to obtain the final illumination component estimate:

[0030] ,

[0031] in: The combined light components; For the first The weighting coefficients for each scale are taken in this embodiment of the invention. , , This makes the contributions of the three scales roughly equal.

[0032] In one embodiment of the present invention, the reflection component is obtained by the difference between the illumination component and the original signal, and an adaptive specular reflection suppression coefficient is introduced. To conduct fine-tuning:

[0033] ,

[0034] in: The reflection component represents the intrinsic reflectivity of the object after removing the effects of illumination. This is the specular reflection suppression coefficient, whose value is adaptively adjusted based on the reflectance coefficient of the coating in the local area of ​​the image, ranging from 0.1 to 0.5. For areas with high reflectance, Take a larger value to enhance the suppression effect; for areas with low reflectance, Take the smaller value to retain more detailed information.

[0035] Preferably, the specular reflection suppression coefficient The adaptive calculation formula is:

[0036] ,

[0037] in: To prevent division by zero by small constants, in this embodiment of the invention, we take... The formula is designed based on the fact that specular reflection mainly occurs in the visible light band, while the near-infrared band is less affected by specular reflection. and The difference can reflect the intensity of specular reflection.

[0038] Finally, the result in the logarithmic domain is transformed back to the linear domain using an exponential transform to obtain the image after specular reflection removal:

[0039] ,

[0040] in: Visible light image after eliminating specular reflection.

[0041] In this embodiment of the invention, the packaging of playing cards on a high-speed conveyor belt will generate motion blur during image acquisition, which manifests as a trailing effect along the direction of motion. Motion blur can be modeled as a convolution process between the image and a point spread function, and a clear image can be recovered through deconvolution.

[0042] This invention employs the Wiener filtering algorithm to achieve inverse recovery of motion blur. The point spread function of the motion blur is also discussed. Determined by conveyor belt speed and exposure time:

[0043] ,

[0044] in: The point spread function at spatial frequency The value at; The motion blur length is expressed in pixels and is calculated using the following formula: ,in The speed of the conveyor belt (unit: m / s). Exposure time (in seconds). Pixel size factor (unit: pixels / m); The motion blur angle is the angle between the direction of the conveyor belt's movement and the horizontal axis of the image, measured in degrees. In this embodiment of the invention, The typical value range is 3 to 20 pixels. The typical range of values ​​is from 0 degrees to 180 degrees.

[0045] The frequency domain expression for Wiener filtering is:

[0046] ,

[0047] in: To recover the spectrum of the image; The spectrum of the blurred image; The spectrum of the point spread function (i.e., the transfer function); for Conjugate; This is the Wiener filter signal-to-noise ratio parameter, used to balance deblurring effect and noise amplification, with a value ranging from 0.001 to 0.1. Smaller values ​​are preferred. A higher value can achieve a stronger deblurring effect, but will amplify noise; a larger value... The value can suppress noise amplification, but its deblurring effect is weak. In the embodiments of the present invention, The typical value is 0.01, which is obtained through experimental optimization on standard test images.

[0048] The frequency domain result is converted back to the spatial domain using the inverse Fourier transform to obtain the restored image.

[0049] ,

[0050] in: The image after motion blur recovery; This represents the two-dimensional inverse Fourier transform.

[0051] In this embodiment of the invention, after specular reflection elimination and motion blur recovery are completed, the processed visible light image and near-infrared image are fused to obtain a dual-spectral fused image containing rich information. The fusion strategy combines weighted averaging with adaptive selection.

[0052] ,

[0053] in: This is a dual-spectral fusion image; The fusion weights for the visible light channels; The fusion weights for the near-infrared channels; .

[0054] Preferably, the fusion weights are adaptively calculated based on local image quality. This invention uses gradient energy as an evaluation index for image quality, assigning higher weights to regions with rich edges and textures, and lower weights to flat regions. The specific calculation formula is as follows:

[0055] ,

[0056] ,

[0057] in: Represents the gradient magnitude of the image; To prevent small constants from being divided by zero, the value is taken as... The advantage of this fusion strategy is that in areas with good visible light image quality (high gradient energy), visible light information is mainly preserved; in areas where the visible light image quality is degraded due to specular reflection or blurring (low gradient energy), near-infrared information is used more to supplement it.

[0058] In this embodiment of the invention, The typical value range is 0.4 to 0.7. The typical value range is 0.3 to 0.6. In practical applications, the visible light channel usually provides richer color and detail information, therefore... The average is slightly higher than .

[0059] Step S3: Defect detection step.

[0060] The defect detection step locates defects in the dual-spectral fusion image obtained in step S2, and uses a multi-scale feature pyramid network based on an attention mechanism to achieve cross-scale defect detection from micron-level printing spots to millimeter-level edge damage. In this embodiment of the invention, the network architecture of the defect detection module includes three main components: a feature extraction backbone network, a multi-scale feature pyramid, and an attention enhancement module.

[0061] This invention employs an improved ResNet-50 as the feature extraction backbone network, progressively extracting image features from low to high levels through five convolutional stages. Preferably, deformable convolutions are introduced in the third to fifth stages of the backbone network, enabling the network to adaptively adjust the receptive field shape, better adapting to the feature extraction needs of irregular shapes and defects. The backbone network outputs feature maps at five different scales. These correspond to 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 resolutions of the input image, respectively.

[0062] In this embodiment of the invention, the feature pyramid network adopts a top-down feature fusion path, transferring semantic information from high-level feature maps to low-level feature maps while preserving the spatial location accuracy of the low-level feature maps. The construction process of the feature pyramid is as follows:

[0063] First, the high-level feature map is upsampled to make its spatial size similar to that of the adjacent low-level features. Figure 1 To:

[0064] ,

[0065] in: For the first The pyramidal features of the layers; This indicates a 2x bilinear upsampling operation; This represents the lateral connection of a 1×1 convolution, used to adjust the number of channels to match.

[0066] In one embodiment of the present invention, the feature pyramid network outputs feature maps at five scale levels. These correspond to input image resolutions of 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128, respectively. Larger resolution feature maps (such as...) Suitable for detecting tiny printing spot defects, and low-resolution feature maps (such as...) , It is suitable for detecting edge damage defects of larger size.

[0067] In this embodiment of the invention, a dual-channel attention mechanism is introduced at each scale level of the feature pyramid, including two branches: channel attention and spatial attention, to enhance the feature response of the defect region while suppressing background interference.

[0068] The channel attention calculation process is as follows: First, global average pooling and global max pooling are performed on the feature maps to obtain two channel description vectors; then, the channel weights are calculated through a shared multilayer perceptron network.

[0069] ,

[0070] in: This is the channel attention weight vector, with elements ranging from 0 to 1; Input feature map; Indicates global average pooling; Indicates global max pooling; This represents a multilayer perceptron, which contains two fully connected layers, with the number of channels in the middle layer being 1 / 16 of the number of input channels; This represents the Sigmoid activation function.

[0071] The spatial attention calculation process is as follows: First, average pooling and max pooling are performed along the channel dimension to obtain two spatial description maps; then, spatial weights are calculated through convolutional layers.

[0072] ,

[0073] in: This is a spatial attention weight map, where the element values ​​range from 0 to 1. This represents average pooling along the channel dimension; This represents max pooling along the channel dimension; Indicates feature splicing; This represents a convolution operation with a kernel size of 7×7.

[0074] Preferably, channel attention and spatial attention are applied sequentially to the input feature map to obtain attention-enhanced features:

[0075] ,

[0076] in: The feature map after attention enhancement; This indicates element-wise multiplication.

[0077] In this embodiment of the invention, attention-enhanced multi-scale features are fed into the detection head network to output defect localization results. The detection head network employs a fully convolutional structure to predict the classification score and bounding box regression parameters for each anchor box. The defect localization results include the location coordinates, size information, and confidence score of the defect region, forming a defect localization result set. ,in The number of defects detected.

[0078] Step S4: Defect classification step.

[0079] The defect classification step introduces a defect association reasoning mechanism based on graph neural networks to collaboratively classify and grade the severity of multiple defect regions detected in step S3. In this embodiment of the invention, the core innovation of this step lies in transforming the traditional independent classification problem into a message-passing reasoning problem on a graph structure, fully utilizing the spatial relationships and feature correlations between defects to improve classification accuracy.

[0080] In this embodiment of the invention, the detected defective regions are constructed as a topological graph structure. , where the set of nodes correspond A defective region, edge set This represents the relationships between defects. Each node... initial feature vector It consists of visual features of the corresponding defect area, including shape features, texture features, color features, etc., with a feature dimension of 256.

[0081] Adjacency matrix The calculation comprehensively considers the spatial distance and feature similarity between defects:

[0082] ,

[0083] in: The elements of the adjacency matrix represent nodes. and nodes The connection strength between them, with a value ranging from 0 to 1; For defects and defects The spatial distance between them, in pixels; The distance scale parameter controls the range of influence of spatial relationships. In this embodiment of the invention, it is taken as... Pixel; Let represent the cosine similarity function. The rationale for this design is that defects that are spatially closer are more likely to have the same cause, and therefore should be given a higher connection strength; defects with high feature similarity are more likely to belong to the same type, and therefore should also be given a higher connection strength.

[0084] This invention employs a graph attention network to implement message passing, weighting the contributions of different neighboring nodes using learnable attention coefficients. The update formula for the graph convolutional layer is:

[0085] ,

[0086] in: For the first Layer nodes Feature vector; For nodes The neighborhood set; For the first Attention coefficient of the layer; For the first The learnable weight matrix of the layer; For non-linear activation functions, this invention uses the LeakyReLU function.

[0087] Preferably, attention coefficient The calculation process is as follows:

[0088] ,

[0089] ,

[0090] in: Unnormalized attention score; This is a learnable attention parameter vector; This represents a vector concatenation operation; the normalization operation ensures that the sum of the attention coefficients of all neighbors of the same node is 1.

[0091] In one embodiment of the present invention, the graph neural network comprises 2 to 4 graph convolutional layers, with each layer maintaining a feature dimension of 256. The stacking of multiple graph convolutional layers enables each node to aggregate information from a more distant neighborhood, thereby capturing higher-order defect association patterns. In this embodiment of the present invention, experimental verification has shown that 3-layer graph convolution achieves the optimal balance between classification accuracy and computational efficiency.

[0092] After feature aggregation via graph convolutional layers, the feature vector of each node incorporates contextual information about neighborhood defects. This invention feeds the final node feature vectors into a fully connected classifier, outputting probability distributions for 12 defect categories:

[0093] ,

[0094] in: For defects The category probability vector has a dimension of 12; Indicates a fully connected layer; The number of convolutional layers in the graph.

[0095] In this embodiment of the invention, the 12 types of defects are organized into three categories according to a hierarchical classification system: printing defects include ink splatter defects (category 1), missing printing defects (category 2), color difference defects (category 3), and registration misalignment defects (category 4); material defects include scratch defects (category 5), embossing defects (category 6), bubble defects (category 7), and foreign matter defects (category 8); and forming defects include edge burr defects (category 9), uneven cut defects (category 10), corner warping defects (category 11), and crease defects (category 12).

[0096] In this embodiment of the invention, in addition to determining the defect type, each defect is also graded according to its severity. The severity is divided into three levels: minor (level 1), moderate (level 2), and severe (level 3). The grading criteria include the defect area ratio, the sensitivity of the defect location, and the degree of impact of the defect on the overall appearance. This invention uses an independent regression network to predict the severity score. Then, it is discretized into three levels.

[0097] Step S5: Bayesian threshold adaptive adjustment step.

[0098] The Bayesian threshold adaptive adjustment step dynamically optimizes the judgment boundary thresholds for various defects based on the statistical characteristics of production batches. In this embodiment of the invention, the raw materials, equipment conditions, and environmental conditions differ between different production batches, making it difficult to adapt to such changes using a fixed threshold, resulting in unstable detection accuracy. Bayesian decision theory provides a mathematical framework for making optimal decisions under conditions of uncertainty.

[0099] In one embodiment of the present invention, for each type of defect ( ), and establish a probabilistic model for its detection confidence. Assume a defect. The detection confidence level follows a normal distribution, and its prior distribution parameter is the mean. and variance During the detection process, based on the observed data... Update the posterior distribution:

[0100] ,

[0101] in: Let be the posterior probability distribution of the parameter; It is the likelihood function; Let be the prior probability distribution.

[0102] Preferably, the calculation of the posterior distribution is simplified by using a conjugate prior form. For the mean and variance of a normal distribution, the Normal-Inverse-Gamma distribution is its conjugate prior. The update formula for the posterior distribution is:

[0103] ,

[0104] ,

[0105] in: and These are the parameters of the posterior distribution; and These are the parameters of the prior distribution; These are prior strength parameters; This represents the number of observed samples in the current batch. The mean of the observed sample; The variance of the observed sample is given.

[0106] In this embodiment of the invention, the defect determination threshold Dynamically calculated based on the parameters of the posterior distribution:

[0107] ,

[0108] in: The quantiles of the standard normal distribution are used to control the balance between detection sensitivity and specificity. In this embodiment of the invention, the quantiles are taken as... correspond This indicates that a threshold is set at a 95% confidence level.

[0109] Preferably, the threshold update cycle is set to update once every 100 to 500 playing card packages detected. A shorter update cycle can adapt to changes in production conditions more quickly, but the computational cost is higher; a longer update cycle has higher computational efficiency, but slower adaptability. In this embodiment of the invention, the default update cycle is 200 cards, which can be adjusted according to the actual situation of the production line.

[0110] In one embodiment of the present invention, the updated threshold is transmitted to the defect detection module in step S3 through a closed-loop feedback mechanism to dynamically adjust the detection sensitivity. When the false negative rate of a certain type of defect increases, the system automatically lowers the judgment threshold of that type of defect to improve the detection sensitivity; when the false positive rate of a certain type of defect increases, the system automatically raises the judgment threshold of that type of defect to improve the detection specificity.

[0111] Step S6: Temporal correlation analysis step.

[0112] The temporal correlation analysis step performs spatiotemporal correlation analysis on the defect distribution of consecutive frames to infer the root cause of equipment failure from the defect patterns. In this embodiment of the invention, the core idea of ​​this step is that equipment failure often leads to the periodic occurrence of specific types of defects or the continuous occurrence of defects in specific locations. By analyzing the spatiotemporal distribution patterns of defects, the faulty components and causes of the equipment can be inferred in reverse.

[0113] In this embodiment of the invention, a spatiotemporal correlation matrix is ​​constructed. Records defect distribution information for consecutive frames. Rows in the matrix correspond to the time sequence numbers of consecutive frames. ,in To analyze the window length; a grid of spatial positions corresponding to the surface of the playing card packaging is used. ,in Number of positional grid cells. Matrix elements. Indicates the first Frame in position Confidence level of defect detection.

[0114] ,

[0115] in: For the first The set of defects detected by the frame; For defects Location; For defects The confidence level.

[0116] This invention uses spatiotemporal correlation coefficients Quantify the correlation between defect distributions at different locations and times:

[0117] ,

[0118] in: For position and location The spatiotemporal correlation coefficient ranges from -1 to 1. For position The average confidence level within the analysis window. High positive correlation ( This indicates that defects in two locations tend to occur simultaneously, possibly stemming from a failure in the same equipment component; a high negative correlation ( This indicates that the defects at the two locations are mutually exclusive, and may originate from periodic vibrations or offsets of the equipment.

[0119] In this embodiment of the invention, periodic defect patterns are identified by frequency domain analysis of the spatiotemporal correlation matrix. A Fast Fourier Transform is performed on the time series at each location:

[0120] ,

[0121] in: For position The spectrum of a time series; Frequency. Significant peaks in the spectrum correspond to the frequency of occurrence of periodic defects, which is related to the rotational speed of rotating parts or the motion cycle of reciprocating parts in the equipment.

[0122] In this embodiment of the invention, a knowledge base is established to associate defect patterns with equipment components, including: periodic ink splatter defects are generally associated with surface damage to the printing roller or unstable ink supply in the ink path system; periodic printing defects are generally associated with surface foreign matter or uneven pressure on the coating roller; edge burrs in fixed positions are generally associated with wear or misalignment of the cutting tool; and gradient color difference defects are generally associated with temperature drift in the ink drying system or batch differences in raw materials. Based on the detected defect pattern characteristics, the system automatically matches the most likely cause of the failure and outputs a fault diagnosis report and maintenance suggestions.

[0123] In one embodiment of the present invention, the results of temporal correlation analysis can also be transmitted to the image preprocessing module in step S2 through a closed-loop feedback mechanism. When a systematic positional deviation is detected, the image registration parameters are automatically adjusted for compensation.

[0124] In this embodiment of the invention, the above six processing steps form a deeply coupled closed-loop collaborative architecture. The processing results of subsequent steps can influence the parameter configuration of preceding steps, thereby achieving continuous optimization of the overall system performance. Specifically, the closed-loop collaborative mechanism includes the following three feedback paths:

[0125] The first feedback path is the threshold optimization feedback from steps S5 to S3. The Bayesian threshold adaptive adjustment step calculates the posterior probability distribution of various defects based on the detection statistics of the current batch and updates the judgment threshold accordingly. The updated threshold is passed to the defect detection module through the feedback path, dynamically adjusting the confidence threshold of non-maximum suppression and the score threshold of bounding box regression. When the system detects a significant increase in the false negative rate of a certain type of defect, it automatically lowers the judgment threshold for that type of defect, making the detector more sensitive to that type of defect; when the system detects a significant increase in the false alarm rate of a certain type of defect, it automatically raises the judgment threshold for that type of defect, reducing false alarm outputs. This feedback mechanism enables the detection system to adaptively respond to process fluctuations and raw material differences between different production batches.

[0126] The second feedback path is the parameter compensation feedback from steps S6 to S2. The temporal correlation analysis step, by continuously monitoring the spatiotemporal pattern of defect distribution, can identify systematic deviations caused by equipment drift. Preferably, when a continuous offset trend is detected in the defect location, the system determines that there may be mechanical problems such as conveyor belt alignment deviation or loose camera installation. At this time, the temporal correlation analysis module calculates the statistical value of the offset and transmits the compensation parameters to the image preprocessing module. The image preprocessing module adjusts the affine transformation matrix of image registration according to the received compensation parameters, compensating for the influence of mechanical deviations at the software level and maintaining the spatial positioning accuracy of the detection system. This feedback mechanism enables the system to maintain stable detection performance even as the equipment condition gradually deteriorates, providing a window of time for equipment maintenance.

[0127] The third feedback path is the feature weight feedback from steps S4 to S3. After completing the collaborative classification, the defect classification step can obtain the classification confidence of each defect region. When the classification confidence of a certain defect region is too low, it indicates that the feature representation of that region is insufficient to support a reliable type determination. At this time, the defect classification module feeds back the location information of the low-confidence region to the defect detection module, triggering a re-detection of that region. During the re-detection, the attention weight allocation is adjusted to enhance the feature extraction intensity of that region. This feedback mechanism realizes the collaborative optimization of detection and classification, avoiding classification errors caused by insufficient feature extraction.

[0128] In one embodiment of the present invention, the execution frequency and priority of the three feedback paths can be configured according to the actual application scenario. The execution cycle of threshold optimization feedback is once every 100 to 500 cards packaged, the execution cycle of parameter compensation feedback is once every 1000 to 5000 cards packaged, and feature weight feedback is executed in real time. When multiple feedback paths are triggered simultaneously, they are processed in the order of priority: feature weight feedback, threshold optimization feedback, and parameter compensation feedback, to ensure the timeliness and stability of the system response.

[0129] In this embodiment of the invention, the data flow process between each processing step is as follows. The moving playing card packaging first passes through a dual-spectrum image acquisition unit to generate a visible light image. and near-infrared images Two raw data streams, each with a data size of 2048×2048×8 bits, approximately 4MB / frame. Both raw images are simultaneously fed into the image preprocessing module. After specular reflection removal, the visible light image is converted... After motion blur recovery, it is converted to Finally, the images are fused to generate a dual-spectral fused image. The data size of the dual-spectral fusion image remains at 4MB / frame, but the information density is significantly improved, containing complementary information from both the visible and near-infrared bands.

[0130] Dual-spectral fusion image The data is fed into the defect detection module, and feature maps at five scales are extracted by the backbone network. Then, a feature pyramid network is used to generate pyramid features at five scales. Finally, after attention enhancement and detection head network output, a set of defect localization results is generated. Defect location results for each defect Includes position coordinates Confidence score and preliminary category prediction The data size of a single defect is approximately 32 bytes.

[0131] Defect location result set The data is fed into the defect classification module, where visual feature vectors for each defect region are first extracted. The dimensions are 256, and the data size is 1KB per defect. Then, a topology graph structure is constructed. and adjacency matrix The node features, which incorporate neighborhood information, are obtained through three layers of graph convolution. Finally, the probability distribution of 12 types of defects is output through the classifier. Severity score The classification result data volume is approximately 64 bytes per defect.

[0132] The results of defect detection and classification are simultaneously fed into the Bayesian decision threshold adjustment unit and the temporal correlation analysis module. The Bayesian decision threshold adjustment unit accumulates the detection statistics of the current batch, updates the posterior probability distribution parameters and decision threshold every 200 frames, and outputs the updated threshold vector. The temporal correlation analysis module maintains the spatiotemporal correlation matrix within a sliding window. It continuously monitors defect distribution patterns and outputs a fault diagnosis report when an abnormal pattern is detected.

[0133] The method of this invention underwent six months of industrial verification testing on multiple playing card packaging production lines. The test dataset contained over 5 million playing card packaging images, covering more than 100,000 defect samples across 12 categories. The test environment included different manufacturers, different printing processes, and different raw material batches, demonstrating good representativeness.

[0134] Test results show that the defect detection rate (recall rate) of the method of this invention reaches 99.52%, the false alarm rate (false positive rate) is 0.31%, and the average classification accuracy of 12 types of defects reaches 97.83%. Among them, the classification accuracy of printing defects is 98.2%, the classification accuracy of material defects is 97.5%, and the classification accuracy of molding defects is 97.8%. The end-to-end inspection time for a single card packaging is an average of 43ms, of which image acquisition takes 5ms, image preprocessing takes 12ms, defect detection takes 18ms, and defect classification takes 8ms, meeting the online inspection speed requirement of 200 cards / minute (i.e., 300ms / card).

[0135] Compared with the technical solution of CN119399203A, the present invention has significant advantages in the following aspects: the defect classification granularity is increased from 3 categories to 12 categories, an improvement of 300%; the detection speed is increased from about 100ms to 43ms, an improvement of 57%; the detection accuracy is increased from about 95% to 99.5%, and the relative error rate is reduced by 90%; the newly added Bayesian adaptive threshold function significantly enhances the detection stability of the system under different batch conditions, and the batch accuracy fluctuation is reduced from ±3% to ±0.5%; the newly added time series correlation analysis function realizes the early warning and root cause diagnosis of equipment failure, with an average failure early warning time of 2 hours and a failure location accuracy of 87%.

[0136] The method of this invention has been verified by large-scale production data and achieves the following performance on a typical playing card packaging production line: a defect detection rate of over 99.5%, a classification accuracy of 97.8% for 12 types of defects, a detection time of less than 50ms for a single playing card package, and supports an online detection speed of 200 cards / minute. The technical solution of CN119399203A shows that this invention significantly improves defect classification granularity (from 3 types to 12 types), detection speed (from approximately 100ms to less than 50ms), and adaptive capability (from fixed threshold to Bayesian dynamic threshold).

[0137] This invention also provides a visual online inspection and defect classification system for playing card packaging quality, such as... Figure 2As shown, this system is used to execute the detection and classification methods described in the above method embodiments. In one embodiment of the present invention, each functional module of the system corresponds one-to-one with each processing step in the method embodiments, forming a complete hardware and software collaborative architecture.

[0138] A dual-spectrum image acquisition unit is installed above the production line conveyor belt for real-time image acquisition of moving playing card packaging. In this embodiment, the unit comprises three main parts: an optical imaging component, a dual-channel image sensor, and a synchronous control circuit. The optical imaging component employs a telecentric lens design to eliminate the influence of perspective distortion on dimensional measurement. The lens has a working distance of 150mm to 300mm, and its field of view covers the entire surface of a single playing card package. The dual-channel image sensor includes a visible light CMOS sensor and a near-infrared InGaAs sensor, both sharing a front-end optical system via a beam splitter to achieve synchronous imaging of the visible light and near-infrared channels. The synchronous control circuit receives position pulse signals from the conveyor belt encoder and triggers image acquisition when the playing card package reaches the detection position, ensuring consistent acquisition timing. Preferably, the visible light channel operates in the 380nm to 780nm band, the near-infrared channel operates in the 780nm to 1100nm band, and the image acquisition resolution is not less than 2048×2048 pixels.

[0139] The image preprocessing module is connected to the dual-spectral image acquisition unit and is used to preprocess the acquired raw images. In this embodiment of the invention, this module implements the specular reflection elimination, motion blur recovery, and dual-spectral fusion functions described in step S2 of the method embodiment. The image preprocessing module adopts an FPGA and GPU collaborative processing architecture, where the FPGA is responsible for real-time reception, caching, and preliminary format conversion of image data, and the GPU is responsible for computationally intensive tasks such as adaptive Retinex algorithm, Wiener filtering deblurring, and dual-spectral fusion. Preferably, the adaptive Retinex algorithm uses a multi-scale Gaussian kernel for illumination component estimation, with Gaussian scale parameters of 15 pixels, 80 pixels, and 240 pixels respectively. The specular reflection suppression coefficient is adaptively adjusted according to the local characteristics of the image, with a value range of 0.1 to 0.5. The motion blur inverse convolution is implemented using the Wiener filtering algorithm, and the point spread function parameters are dynamically configured according to the conveyor belt speed and exposure time.

[0140] The defect detection module is connected to the image preprocessing module and is used to locate defects in the dual-spectral fusion image. In this embodiment of the invention, this module implements the multi-scale feature extraction and attention-enhanced defect detection functions described in step S3 of the method embodiment. The core of the defect detection module is a multi-scale feature pyramid network based on an attention mechanism, deployed on a high-performance GPU computing card. The network model is optimized for inference using TensorRT to achieve FP16 half-precision quantization and layer fusion acceleration, with the single-frame inference time controlled within 20ms. Preferably, the multi-scale feature pyramid network includes five scale levels, corresponding to 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 resolutions of the input image, respectively, enabling cross-scale defect localization from micron-level printing spots to millimeter-level edge damage.

[0141] The defect classification module is connected to the defect detection module and is used to determine the type and severity of detected defects. In this embodiment of the invention, this module implements the defect association reasoning mechanism based on graph neural networks described in step S4 of the method embodiment. The defect classification module constructs a topological graph structure from the detected defect regions, builds an adjacency matrix with defect regions as nodes and spatial relationships between defects as edges, and aggregates neighborhood defect features through message passing to achieve collaborative classification. Preferably, the graph neural network contains three graph convolutional layers, each with a feature dimension of 256 dimensions, and the attention coefficient is dynamically calculated through learnable parameters. The defect classification system is divided into three major categories: printing, material, and molding, with a total of twelve subcategories.

[0142] The Bayesian decision threshold adjustment unit is connected to both the defect detection module and the defect classification module, and is used to dynamically optimize the judgment threshold based on the statistical characteristics of the production batch. In this embodiment of the invention, this unit implements the Bayesian posterior probability calculation and threshold adaptive update functions described in step S5 of the method embodiment. The Bayesian decision threshold adjustment unit maintains the probability distribution parameters of various defects, continuously updates the posterior distribution based on observation data during the detection process, and transmits the optimized threshold to the defect detection module through closed-loop feedback. Preferably, the threshold update cycle is set to update once every 100 to 500 playing card packages detected, and the posterior probability distribution is calculated using a conjugate prior form.

[0143] The temporal correlation analysis module is connected to the defect classification module and is used to perform spatiotemporal correlation analysis on the defect distribution of consecutive frames. In this embodiment of the invention, this module implements the spatiotemporal correlation matrix construction, periodic pattern recognition, and fault root cause inference functions described in step S6 of the method embodiment. The temporal correlation analysis module continuously monitors the changes in defect distribution using a sliding window method, and automatically generates a fault diagnosis report when an abnormal defect pattern is detected. Preferably, the rows of the spatiotemporal correlation matrix correspond to the consecutive frame numbers, and the columns correspond to the spatial location grid. Periodic defect patterns are identified through frequency domain analysis, and maintenance suggestions are given by matching them with a pre-built fault cause knowledge base.

[0144] In this embodiment of the invention, the aforementioned functional modules are interconnected via a high-speed data bus to form a complete online detection and classification system. The system also includes a human-machine interface module for displaying detection results, configuring detection parameters, and outputting statistical reports. Preferably, the human-machine interface uses a touchscreen display and supports functions such as real-time image preview, defect highlighting, classification result statistics, and historical data query.

[0145] The system of this invention has been deployed and verified in industrial settings. It operates stably at a production rate of 200 playing card packages per minute, with a single card inspection time of less than 50ms, meeting the requirements for real-time online inspection. The system's defect detection rate reaches over 99.5%, and the classification accuracy of 12 types of defects reaches 97.8%, significantly outperforming the performance level of traditional vision inspection systems.

[0146] The embodiments of the present invention are not limited to the specific embodiments described above. Those skilled in the art can make various equivalent changes or substitutions based on the technical solutions of the present invention, and all such changes or substitutions should be included within the protection scope of the present invention.

Claims

1. A method for online visual inspection and defect classification of playing card packaging quality, characterized in that, Includes the following steps: Dual-spectrum image acquisition steps: A dual-spectrum imaging unit is used to synchronously acquire images of the moving playing card packaging. The printed content image is acquired through the visible light channel, and the material texture image is acquired through the near-infrared channel. Image preprocessing steps: The image preprocessing module processes the printed content image based on the adaptive Retinex algorithm to eliminate specular reflection interference from the coating material, and restores the image degradation caused by high-speed transmission through motion blur inverse convolution. The processed printed content image is then fused with the material texture image to obtain a dual-spectral fused image. Defect detection steps: The defect detection module performs multi-scale feature extraction on the dual-spectral fusion image, and adopts a multi-scale feature pyramid network based on the attention mechanism. Through the synergistic effect of channel attention and spatial attention, the feature response of the defect area is enhanced, realizing cross-scale defect localization from micron-level printing spots to millimeter-level edge damage, and obtaining defect localization results. Defect classification steps: The defect classification module introduces a defect association reasoning mechanism based on graph neural networks. It constructs a topological graph structure by building multiple detected defect regions. It builds an adjacency matrix with defect regions as nodes and spatial relationships between defects as edges. It aggregates neighborhood defect features through message passing to achieve collaborative determination of defect types and classifies the severity based on defect morphological features. Bayesian threshold adaptive adjustment step: The Bayesian decision threshold adjustment unit calculates the posterior probability distribution of various defect detections based on the statistical characteristics of production batches, dynamically optimizes the judgment boundary thresholds of various defects, and feeds back the optimized thresholds to the defect detection module to adjust the detection sensitivity. Temporal correlation analysis steps: The temporal correlation analysis module performs spatiotemporal correlation analysis on the defect distribution of consecutive frames, identifies defect distribution patterns by constructing a spatiotemporal correlation matrix, and infers the root cause of equipment failure.

2. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, In the dual-spectral image acquisition step, the working wavelength of the visible light channel is 380nm to 780nm, and the working wavelength of the near-infrared channel is 780nm to 1100nm. The synchronization time deviation between the two channels is less than 1ms, the image acquisition resolution is not less than 2048×2048 pixels, and the supported online detection speed is not less than 200 images / minute.

3. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, The adaptive Retinex algorithm uses a multi-scale Gaussian kernel for illumination component estimation, with Gaussian scale parameters of 15 pixels, 80 pixels, and 240 pixels respectively. The specular reflection suppression coefficient is adaptively adjusted according to the coating reflection coefficient, with a value range of 0.1 to 0.

5.

4. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, The motion blur inverse convolution is implemented using the Wiener filtering algorithm. The point spread function is determined based on the conveyor belt speed and exposure time. The motion blur length ranges from 3 to 20 pixels, the motion blur angle ranges from 0 to 180 degrees, and the Wiener filtering signal-to-noise ratio parameter ranges from 0.001 to 0.

1.

5. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, The multi-scale feature pyramid network includes five scale levels, corresponding to 1 / 8, 1 / 16, 1 / 32, 1 / 64 and 1 / 128 resolutions of the input image, respectively. Channel attention is calculated by global average pooling and fully connected layers, and spatial attention is calculated by feature concatenation of max pooling and average pooling. Both channel attention weights and spatial attention weights are in the range of 0 to 1.

6. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, The message passing process of the graph neural network includes: calculating an adjacency matrix based on the spatial location and feature similarity of the defect region, where the elements of the adjacency matrix represent the connection strength between defect nodes; aggregating the features of neighboring nodes through graph convolutional layers, with the number of graph convolutional layers being 2 to 4; and using attention coefficients to weight the contributions of different neighboring nodes, where attention coefficients are calculated through learnable parameters and node features.

7. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, In the Bayesian threshold adaptive adjustment step, the prior probability distribution parameters of various defects are updated according to the defect detection statistics of the current batch. The posterior probability distribution is calculated using the conjugate prior form. The judgment threshold is dynamically adjusted according to the expected value and confidence interval of the posterior probability distribution. The threshold update cycle is once every 100 to 500 playing card packages are detected.

8. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, In the defect classification step, defects are divided into three major categories and twelve subcategories: printing defects include ink splatter defects, missing printing defects, color difference defects, and registration misalignment defects; material defects include scratch defects, embossing defects, bubble defects, and foreign object defects; and forming defects include edge burr defects, uneven cut defects, corner warping defects, and crease defects.

9. The method for online visual inspection and defect classification of playing card packaging quality according to claim 1, characterized in that, In the temporal correlation analysis step, the rows of the spatiotemporal correlation matrix correspond to the consecutive frame numbers, the columns correspond to the defect detection area locations, and the matrix elements represent the confidence level of defect occurrence. By performing feature decomposition on the spatiotemporal correlation matrix to identify periodic defect patterns, and by associating the periodic characteristics and spatial distribution characteristics of the defect patterns with equipment components, the root cause of the fault can be located.

10. A visual online inspection and defect classification system for playing card packaging quality, used to implement the visual online inspection and defect classification method for playing card packaging quality as described in any one of claims 1-9, characterized in that, include: The dual-spectrum image acquisition unit is used to synchronously acquire images of moving playing card packaging, including a visible light channel and a near-infrared channel. The visible light channel acquires images of the printed content, and the near-infrared channel acquires images of the material texture. The image preprocessing module, connected to the dual-spectral image acquisition unit, is used to eliminate specular reflection interference from the coating material based on the adaptive Retinex algorithm, recover image degradation through motion blur inverse convolution, and fuse the printed content image and the material texture image. The defect detection module, connected to the image preprocessing module, is used to achieve cross-scale defect localization through a multi-scale feature pyramid network based on an attention mechanism. The defect classification module, connected to the defect detection module, is used to achieve collaborative determination of defect types and severity classification through a defect association reasoning mechanism based on graph neural networks. The Bayesian decision threshold adjustment unit, connected to the defect detection module and the defect classification module, is used to dynamically optimize the decision boundary threshold based on the statistical characteristics of the production batch. The temporal correlation analysis module, connected to the defect classification module, is used to perform spatiotemporal correlation analysis on the defect distribution of consecutive frames to infer the root cause of equipment failure.

Images