Nameplate printing defect detection method and system based on image segmentation

Abstract

The invention provides a nameplate printing defect detection method and system based on image segmentation. The method comprises the following steps: acquiring a nameplate surface image and carrying out distortion correction and brightness homogenization preprocessing; generating a pixel-level segmentation mask by using the semantic segmentation network; template registration is carried out based on the segmentation result, and a deformation field vector and a registration difference chart are calculated; performing defect area extraction and type identification on the difference image; according to the method, deep learning semantic segmentation is adopted to realize pixel-level analysis of a nameplate printing area, deformation field registration is combined to accurately quantify overprinting offset, defect quantitative evaluation is realized through multi-dimensional scoring, the detection rate reaches 99% or above, and the detection time of a single piece is less than 0.5 second.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial inspection technology, specifically to a method and system for detecting defects in nameplate printing based on image segmentation. Background Technology

[0002] Plastic and metal nameplates are widely used in electronic products, home appliances, automotive parts, and industrial equipment. The patterns, text, and color blocks printed on their surfaces directly affect the product's appearance and brand image. During nameplate production, the printing process is prone to various defects, including omissions, overprinting, misregistration, ink splatter, color deviation, blurred patterns, pinholes, and air bubbles. These defects not only affect the product's aesthetics but can also, in severe cases, lead to unclear product function markings, impacting user experience and even creating safety hazards.

[0003] Currently, nameplate printing quality inspection mainly relies on manual visual inspection. Inspectors visually examine the nameplate surface to determine and classify printing defects. However, manual visual inspection has significant technical bottlenecks: First, manual inspection is inefficient and cannot meet the demands of high-volume, high-speed production, with single-piece inspection time typically ranging from several seconds to tens of seconds. Second, manual inspection is heavily influenced by the subjective factors of the inspectors, easily leading to missed or false detections, especially for minor defects such as misprints and misregistration, which are difficult for the human eye to accurately identify and quantify. Third, prolonged visual inspection can cause visual fatigue among inspectors, further reducing accuracy. In addition, manual quality inspection of large-volume products requires a large number of inspectors, resulting in high labor costs.

[0004] In the prior art, Chinese invention patent application CN119722674A discloses a high-precision printing quality inspection method and system. This technical solution uses an image sensor to acquire printed images, obtains grayscale printed images through boundary segmentation and grayscale processing, and then employs a two-stage registration method of pixel transition sampling and pixel-interval sampling to perform pixel-level registration and correction between the image to be inspected and a standard template. Finally, a defect matcher is used to match defect types and output printing quality data. This technical solution improves the automation level of printing quality inspection to a certain extent, but still has the following shortcomings: First, grayscale processing loses color information, limiting its ability to detect color deviation defects; second, pixel transition sampling and pixel-interval sampling are essentially still pixel-level registration methods, lacking an understanding of the semantic information of the printed area, making it difficult to distinguish the feature differences of different printed elements such as patterns, text, and color blocks; third, the two-stage registration method has high computational complexity, and the registration accuracy is greatly affected by the sampling step size; fourth, the defect matcher uses a rule-based matching method, limiting its adaptability to novel defects.

[0005] Therefore, there is an urgent need for a high-precision and high-efficiency intelligent detection technology for nameplate printing defects to solve the technical problems of insufficient detection accuracy, lack of semantic understanding, and poor adaptability in existing technologies. Summary of the Invention

[0006] To address the technical problems of insufficient accuracy, lack of semantic level understanding, and poor adaptability in existing printing defect detection technologies, this invention provides a method and system for detecting nameplate printing defects based on image segmentation.

[0007] In a first aspect, the present invention provides a method for detecting defects in nameplate printing based on image segmentation. The method includes: performing an image acquisition and preprocessing step, deploying an industrial camera at a production line station to acquire an image of the nameplate surface under constant light source conditions, and sequentially performing distortion correction and brightness homogenization processing on the nameplate surface image to obtain a preprocessed image; performing a semantic segmentation step, inputting the preprocessed image into a semantic segmentation network, extracting multi-scale features through an encoder to generate a multi-scale feature pyramid, performing cross-layer feature fusion and upsampling reconstruction through a decoder, and outputting a pixel-level segmentation mask; performing a template registration and comparison step, acquiring a standard template image, establishing a spatial correspondence based on the pixel-level segmentation mask, calculating a deformation field vector, and generating a registration difference map after spatial transformation and alignment based on the deformation field vector; performing a defect identification and classification step, extracting defect regions by threshold segmentation and connected component analysis on the registration difference map, and determining defect type labels; and performing a quantitative evaluation and judgment step, calculating the defect area, location coordinates, and severity score, and outputting detection result data and defective product rejection signals.

[0008] A second aspect of the present invention provides a nameplate printing defect detection system based on image segmentation. The system includes: an image acquisition module deployed at a production line station, equipped with an industrial camera and a constant light source, for acquiring images of the nameplate surface; an image preprocessing module connected to the image acquisition module, for performing distortion correction and brightness homogenization processing on the nameplate surface image, and outputting a preprocessed image; a semantic segmentation module connected to the image preprocessing module, equipped with a semantic segmentation network, for performing multi-scale feature extraction and cross-layer feature fusion on the preprocessed image, and outputting a pixel-level segmentation mask; a template registration module connected to the semantic segmentation module, for spatially aligning the preprocessed image with a standard template image based on the pixel-level segmentation mask, and calculating a deformation field vector and a registration difference map; a defect identification module connected to the template registration module, for extracting defect regions and identifying defect types from the registration difference map; and a quantitative evaluation module connected to the defect identification module, for calculating quantitative defect indicators and severity scores, and outputting detection result data and defect rejection signals.

[0009] The technical solution provided by this invention has at least the following technical effects or advantages: First, by using a semantic segmentation network to perform pixel-level analysis on the nameplate image, it can accurately distinguish different printing elements such as patterns, text, and color blocks, achieving semantic-level defect understanding, and has stronger feature expression capabilities compared to pixel-level registration methods; Second, through a multi-scale feature pyramid and cross-layer feature fusion mechanism, it can simultaneously capture detailed information and semantic information, improving the detection capability for both minor and complex defects; Third, the template registration method based on deformation field can accurately quantify positional deviation defects such as overprinting offset, with high registration accuracy and excellent computational efficiency; Fourth, the multi-dimensional defect severity scoring system realizes quantitative assessment of defects, providing an objective basis for quality judgment; Fifth, the system as a whole achieves full-process automation from image acquisition to defect judgment, with a defect detection rate of over 99% and a single-piece inspection time of less than 0.5 seconds, significantly improving detection efficiency and reducing labor costs. Attached Figure Description

[0010] Figure 1 This is a flowchart of the nameplate printing defect detection method based on image segmentation provided in the embodiments of the present invention;

[0011] Figure 2 This is an architecture diagram of a nameplate printing defect detection system based on image segmentation provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0013] like Figure 1 As shown, this invention provides a method for detecting defects in nameplate printing based on image segmentation. This method constructs a complete detection process from image acquisition to defect determination. It achieves pixel-level analysis of the nameplate printing area through a semantic segmentation network, and combines deformation field registration and multi-dimensional quantitative evaluation to achieve high-precision and high-efficiency defect detection. In one embodiment of this invention, the method includes the following five core steps, with deeply coupled data flow relationships between each step, and adaptive optimization of system parameters is achieved through a closed-loop feedback mechanism.

[0014] Step S1: Image Acquisition and Preprocessing. This step aims to acquire high-quality images of the nameplate surface and eliminate distortion and uneven illumination during the imaging process through preprocessing, providing standardized input for subsequent semantic segmentation.

[0015] In one embodiment of the invention, the image acquisition device employs a high-resolution industrial camera, preferably with a resolution of 2048 pixels by 1536 pixels or higher, to ensure the capture of minute printing features on the nameplate surface. The camera is mounted directly above the inspection station on the production line, with the lens optical axis perpendicular to the nameplate surface. The shooting distance is adjusted according to the nameplate size and the camera's field of view, typically set within the range of 20cm to 50cm. The light source uses a ring-shaped LED light source or a strip LED light source array to provide uniform diffused illumination, maintaining a constant light source brightness to minimize the impact of illumination variations on image quality. Preferably, the light source color temperature is selected as cool white light of 5500K to 6500K, providing good contrast with common printing ink colors.

[0016] When the nameplate moves to the inspection station along the production line conveyor belt, the position sensor triggers the camera to acquire an image, obtaining the original image of the nameplate surface, which is recorded as follows: The original image is an RGB three-channel color image, containing complete visual information about the surface of the nameplate.

[0017] Distortion correction is the first step in preprocessing, used to eliminate geometric distortions introduced by the camera's optical system. Industrial camera lenses inherently possess radial and tangential distortions; without correction, this will lead to pattern distortion in the nameplate edge area, affecting subsequent registration accuracy. In one embodiment of this invention, distortion correction employs the following mathematical model for pixel coordinate mapping:

[0018] ,

[0019] ,

[0020] in: and These are the corrected pixel coordinates, in pixels; and These are the pixel coordinates of the original distorted image, in pixels. The normalized distance from a pixel to the center of the image is dimensionless. and The radial distortion coefficient is... The typical value range is -0.3 to 0.3. The typical value range is from -0.1 to 0.1, and the specific value is obtained through camera calibration; and The tangential distortion coefficient, ranging from -0.1 to 0.1, is caused by the non-parallel mounting of the lens and sensor plane. The distortion coefficient is obtained through pre-calibration using a checkerboard calibration board. The calibration process involves taking at least 20 calibration images from multiple angles, and the camera intrinsic parameters and distortion coefficient are solved using Zhang Zhengyou's calibration method. This distortion correction model effectively eliminates barrel or pincushion distortion introduced by the lens, and the geometric accuracy error of the corrected image is controlled within 0.5 pixels, providing a geometrically accurate image foundation for subsequent registration.

[0021] Brightness uniformity processing is the second step in preprocessing, used to eliminate differences in image brightness distribution caused by uneven illumination. Although constant light source illumination is used, localized brightness unevenness may still exist in the acquired image due to differences in the curvature and reflectivity of the nameplate surface, as well as slight variations in the light source's angle. In one embodiment of this invention, brightness uniformity employs an adaptive local mean compensation algorithm:

[0022] ,

[0023] in: For pixels after brightness uniformity The grayscale value at that location ranges from 0 to 255. Pixels after distortion correction The grayscale value at that location; The global average brightness of the entire image is obtained by taking the arithmetic mean of the gray values ​​of all pixels. In pixels The average brightness within a local window centered on the target, with the window size preferably being 64 pixels by 64 pixels; To prevent small constants from being divided by zero, a value of 1.0 is used. The technical advantage of this algorithm is that it can effectively address low-brightness local areas. Less than The compensation coefficient is greater than 1 to improve brightness; for local areas with excessive brightness, the compensation coefficient is less than 1 to suppress brightness, ultimately making the brightness distribution of the entire image more uniform.

[0024] Preferably, the brightness uniformity processing further includes a contrast enhancement step. In one embodiment of the present invention, the contrast enhancement employs adaptive Gamma correction:

[0025] ,

[0026] in: These are the pixel values ​​after contrast enhancement; Here is the Gamma correction factor, when Enhance shadow detail, when To enhance highlight details, the preferred value range is 0.8 to 1.2, with the specific value adaptively determined based on the image histogram distribution. The adaptive selection strategy for gamma correction is as follows: calculate the peak position of the image histogram; if the peak is located in the low grayscale range (grayscale value less than 100), then select... To enhance dark areas, if the peak value is in the high grayscale range (grayscale value greater than 180), then select... Compress the dynamic range of highlights; otherwise, select [the appropriate option]. Maintain the original contrast.

[0027] After distortion correction, brightness homogenization, and contrast enhancement, the preprocessed image is obtained. The preprocessed image is then normalized, scaling it to the standard input size for the semantic segmentation network, preferably 512 pixels by 384 pixels. Size normalization uses bilinear interpolation to maintain the continuity of the image content. Finally, the image pixel values ​​are normalized, mapping the integer range of pixel values ​​from 0 to 255 to the floating-point range of 0 to 1 to suit the input requirements of the neural network.

[0028] Step S2: Semantic Segmentation. This step aims to use a deep learning semantic segmentation network to perform pixel-level classification on the preprocessed image, dividing the nameplate surface into different semantic categories such as pattern regions, text regions, color block regions, and background regions, providing region-level feature representation for subsequent accurate defect identification.

[0029] In one embodiment of the present invention, the semantic segmentation network adopts an encoder-decoder architecture. The encoder is responsible for hierarchical feature extraction from the input image, and the decoder is responsible for restoring the extracted features to the original image resolution and outputting the classification result. The encoder adopts a convolutional neural network structure, which includes a cascaded combination of multiple convolutional blocks and downsampling layers. Preferably, the encoder contains 5 convolutional blocks, each containing 2 to 3 convolutional layers. The convolutional layers use 3x3 convolutional kernels for feature extraction, and the number of convolutional kernels increases progressively from 64 in the first convolutional block to 64, 128, 256, 512, and 512. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. Batch normalization helps to accelerate network training convergence and improve the model's generalization ability. Adjacent convolutional blocks are downsampled using 2x2 max pooling layers, which halves the spatial size of the feature map, thereby expanding the receptive field and reducing computational complexity.

[0030] The multi-scale feature pyramid is a crucial output of the encoder. During the encoding process, it preserves the feature maps of each downsampling level, forming a feature pyramid structure from shallow to deep layers. Shallow feature maps (such as the outputs of the first and second convolutional blocks) have high spatial resolution, preserving detailed information such as image edges and textures, which is beneficial for accurately locating defect boundaries. Deep feature maps (such as the outputs of the fourth and fifth convolutional blocks) have a larger receptive field and a higher level of semantic abstraction, containing category discrimination information for printed areas. The technical advantage of the multi-scale feature pyramid lies in its ability to achieve accurate pixel-level classification by simultaneously utilizing detailed and semantic information through the fusion of features from different levels.

[0031] In one embodiment of the present invention, the decoder adopts a structure design of progressive upsampling and cross-layer feature fusion. The decoder contains an equal number of upsampling layers as the encoder downsampling layers. Each upsampling layer first enlarges the feature map size by a factor of 2 through transposed convolution or bilinear interpolation, and then fuses it with the encoder features of the corresponding layer in the multi-scale feature pyramid. Cross-layer feature fusion adopts a skip connection mechanism, fusing the feature map of a certain layer of the encoder with the upsampling features of the corresponding layer of the decoder through channel concatenation or element-wise addition. The technical effect of skip connections is that they compensate for the spatial information lost during downsampling, enabling the decoder to utilize the high-resolution detail features extracted by the encoder and improve the accuracy of segmentation boundaries.

[0032] In one embodiment of the present invention, the cross-layer feature fusion is specifically implemented using an attention weighting mechanism. For encoder features... and decoder upsampling features The fusion process is represented as:

[0033] ,

[0034] in: The fused feature map; The attention weight coefficient is obtained by performing channel compression on the concatenated features through a 1x1 convolutional layer and then applying the Sigmoid activation function. Its value ranges from 0 to 1. Adaptive learning enables the network to dynamically adjust the fusion weights based on the feature importance of different regions, giving higher weights to encoder features in edge regions that require detailed information and higher weights to decoder features in internal regions that require semantic information.

[0035] The last layer of the decoder is a classification layer, which uses a 1x1 convolution to map the number of feature channels to the number of segmentation categories. The probability distribution of each pixel belonging to each category is then output using the Softmax function. In one embodiment of the present invention, the segmentation categories include: background (category 0), pattern area (category 1), text area (category 2), color block area (category 3), and border area (category 4), for a total of 5 categories. Pixel-level segmentation mask. Each location stores the predicted class label for that pixel, with values ​​ranging from 0 to 4.

[0036] The semantic segmentation network is trained using supervised learning with a pre-annotated dataset of nameplate images. The training dataset contains normal nameplate images and images of nameplates with various defects, each image accompanied by manually annotated pixel-level segmentation labels. In one embodiment of this invention, the training dataset contains over 5000 images, with normal samples accounting for approximately 70% and samples containing various defects accounting for approximately 30%, ensuring the model can learn the differences between normal printing features and defect features.

[0037] The loss function uses a weighted combination of cross-entropy loss and Dice loss, and the total loss function is defined as:

[0038] ,

[0039] in: This represents the total loss value. Cross-entropy loss is used to optimize pixel classification accuracy. The Dice loss is used to optimize the overall integrity and boundary quality of region segmentation. and The optimal values ​​for the loss weights are 0.6 and 0.4. The cross-entropy loss penalizes the classification error of each pixel, while the Dice loss evaluates the segmentation quality from the perspective of region overlap. Combining the two can simultaneously optimize pixel-level accuracy and region-level integrity.

[0040] The optimizer used is the Adam optimizer, with an initial learning rate set to 0.001 and a momentum parameter. and The values ​​were set to 0.9 and 0.999 respectively. A cosine annealing strategy was used for learning rate scheduling, allowing the learning rate to smoothly decay from its initial value to its minimum. The decay period was set to half the total number of training epochs. The total number of training epochs was set to 100, and the batch size was set to 16. Data augmentation techniques were used during training to improve the model's robustness and generalization ability. Data augmentation included random rotation (rotation angle range of -15 degrees to +15 degrees), random horizontal flip (probability of 0.5), random vertical flip (probability of 0.3), random cropping and scaling (scaling ratio of 0.8 to 1.2), and color dithering (brightness, contrast, and saturation variations ranging from 0.8 to 1.2).

[0041] Preferably, during training, mixed-precision training techniques are employed to accelerate the training process and reduce memory usage, gradient accumulation techniques are used to support larger equivalent batch sizes, and early stopping strategies are employed to prevent overfitting. After model training is complete, segmentation performance is evaluated on an independent validation dataset containing 500 images. Evaluation metrics include pixel accuracy, average intersection-over-union (IoU), and IoU for each category. When pixel accuracy reaches 98% or higher and average IoU reaches 90% or higher, the model is considered successfully trained and can be used for actual detection.

[0042] Step S3: Template Registration and Comparison. This step aims to accurately align the currently detected nameplate image with the standard template image, and detect printing defects by calculating the differences between the two. Based on the semantic segmentation results obtained in Step S2, the registration process can adopt differentiated matching strategies for different types of printing areas, improving registration accuracy and the targeting of defect detection.

[0043] In one embodiment of the present invention, the standard template image is a pre-acquired and stored image of a qualified nameplate. For each model of nameplate product, the system stores one or more standard template images that have been manually verified to be defect-free. The standard template images also undergo the preprocessing process in step S1 and the semantic segmentation process in step S2 to obtain a preprocessed image of the standard template. Segmentation mask with standard template During the actual testing process, the corresponding standard template image is retrieved from the database based on the product model of the current nameplate.

[0044] Deformation field estimation is a core step in template registration, used to calculate the local displacement of the preprocessed image relative to the standard template image. Deformation field It is a two-dimensional vector field defined on image coordinates, representing pixels in the preprocessed image. The displacement offset relative to the corresponding position in the standard template image. In one embodiment of the present invention, the deformation field estimation employs an iterative optimization algorithm based on the gradient field:

[0045] ,

[0046] ,

[0047] in: The registration energy function is expressed in squared pixels. The deformation field vector, and These are the horizontal and vertical displacement components, respectively, in pixels; To preprocess the pixel values ​​at the coordinates after deformation of the image, bilinear interpolation is used for calculation; For the standard template image in coordinates Pixel value at; This is the regularization coefficient, used to control the smoothness of the deformation field. Its value ranges from 0.01 to 1.0, with a preferred value of 0.1. A larger value is also acceptable. It makes the deformation field smoother but reduces the registration accuracy, and the smaller It allows for finer local displacements but may introduce noise; The gradient of the deformation field. ; For the first The deformation field estimate for the next iteration; The iteration step size ranges from 0.01 to 0.5, with 0.1 being the preferred value. The iteration termination condition is that the energy function converges or the maximum number of iterations is reached. The convergence threshold is set to an energy change of less than 0.01, and the maximum number of iterations is set to 100.

[0048] The first term of the energy function is the data term, which measures the similarity between the preprocessed image after deformation and the standard template image. Minimizing this term makes the two images as aligned as possible. The second term is the regularization term, which constrains the spatial continuity of the deformation field to avoid unreasonable and severe local deformation. The iterative optimization process starts with zero deformation field initialization and gradually updates the deformation field to minimize the energy function.

[0049] Preferably, the deformation field estimation employs a multi-resolution coarse-fine combination strategy. First, a coarse estimate is performed on a low-resolution image to obtain initial values ​​for a large range of displacements. Then, the estimation is refined step by step to the original resolution to obtain precise local displacements. This strategy can accelerate convergence and avoid getting trapped in local optima.

[0050] Region-weighted registration based on semantic segmentation masks is an innovation of this invention. Different types of printed areas have different requirements for registration accuracy. For example, the edge alignment requirements for text areas are high, while the internal alignment requirements for large color block areas are relatively lenient. In one embodiment of this invention, a region weight map is introduced. Weighting the registration energy function:

[0051] ,

[0052] in: The region weight is determined based on the category label of the pixel-level segmentation mask, and its value ranges from 0.5 to 2.0; for edge pixels of text regions, To emphasize precise alignment; for patterned areas, For the area inside the color block, To reduce sensitivity to slight differences in brightness; for background areas, To ignore the influence of background noise, the weight map setting allows the registration process to automatically focus on key areas of the printed content, improving the effectiveness of detection.

[0053] Based on the estimated deformation field vector, the preprocessed image is spatially transformed and aligned to obtain the aligned preprocessed image. :

[0054] ,

[0055] Subsequently, the registration difference between the aligned preprocessed image and the standard template image was calculated. :

[0056] ,

[0057] in: For position The difference value ranges from 0 to 255. The difference map reflects the pixel-level differences between the preprocessed image and the standard template image; areas with large difference values ​​may indicate printing defects.

[0058] Furthermore, the deformation field vector itself also contains important defect information. For overprinting misalignment defects, the deformation field exhibits an overall translational trend within a specific printing area. In one embodiment of the present invention, overprinting misalignment is identified by analyzing the statistical characteristics of the deformation field:

[0059] ,

[0060] ,

[0061] in: For the first Each segmented region; This represents the total number of pixels within the region. and This represents the average value of the horizontal and vertical components of the deformation field within this region, expressed in pixels. The standard deviation of the deformation field is expressed in pixels; when the average displacement... Exceeding the set threshold (Preferred to be 3 pixels, corresponding to approximately 0.15mm) and standard deviation Less than the threshold When the value is 1 pixel (preferably), the area is determined to have an overprinting misalignment defect. The technical basis for this determination logic is: the deformation field of normal printing should be close to the zero vector. If the deformation field of a certain area shows an overall offset and a concentrated distribution (small standard deviation), it indicates that the printed content in that area has undergone a systematic displacement relative to the template, i.e., overprinting misalignment.

[0062] Step S4: Defect Identification and Classification. This step aims to extract defect regions from the registration difference map, identify and classify the type of each defect region, and determine the specific defect type label.

[0063] In one embodiment of the present invention, defect region extraction first involves threshold segmentation of the registration difference map, separating regions with differences exceeding a threshold. Pixels marked as potentially defective pixels. Threshold The selection adopts an adaptive strategy, dynamically determined based on the statistical characteristics of the difference plot:

[0064] ,

[0065] in: The global mean for registering the difference map; The standard deviation of the registration difference plot; This is a multiplier coefficient, ranging from 2 to 4, with a preferred value of 3. The technical advantage of this adaptive threshold is that when the overall image registration quality is good (…). and When the image has a smaller overall brightness difference, the threshold is lowered accordingly to detect minute defects; when the image has a smaller overall brightness difference, the threshold is lowered accordingly to detect minute defects. (If the threshold is large), the threshold should be increased accordingly to avoid false detections.

[0066] After thresholding, connected component analysis is performed on the binarized potential defect pixel map. The eight-neighbor connectivity criterion is used to aggregate adjacent defect pixels into independent defect regions. Each connected component corresponds to an independent defect region, and its geometric features, such as the coordinates of its bounding rectangle, area, and centroid position, are recorded. To eliminate noise interference, a minimum area threshold is set in one embodiment of this invention. (Preferably 5 pixels by 5 pixels, i.e., 25 square pixels), connected regions with an area smaller than this threshold are considered noise points and are removed.

[0067] Defect type identification is the core step in this process. In one embodiment of the present invention, a classification system comprising the following seven categories of defects is established for typical defect types in nameplate printing:

[0068] The first type is the missing printing defect, which manifests as the absence of areas where printed content should be present. In the registration difference map, this appears as bright spots where the standard template contains content but the inspection image does not. The criteria for determining the missing printing defect are: the segmentation mask label of the defective area is classified as a printed area in the standard template, but as a background in the inspection image.

[0069] The second type is overprinting defects, which manifest as additional printing marks appearing in areas where there should be no printed content. In the registration difference map, this appears as dark spots where the standard template is the background but the detected image contains content. The criteria for determining overprinting defects are: the defective area is classified as background in the segmentation mask of the standard template, but as printed area in the segmentation mask of the detected image.

[0070] The third type is registration misalignment defect, which manifests as an overall positional shift in the printed content. This has been preliminarily identified in the deformation field analysis in step S3. A characteristic of registration misalignment defect is that the deformation field of a specific printing area exhibits a systematic displacement trend.

[0071] The fourth type is ink splatter defects, which manifest as scattered ink spots around the printed area. In the registration difference map, these appear as small, discrete bright spots on the outer edge of the printed area. The criteria for determining ink splatter defects are: the defect area is small (less than 100 square pixels), the shape is close to circular (circularity greater than 0.7), and it is located within a certain distance (less than 20 pixels) outside the boundary of the printed area.

[0072] The fifth category is color deviation defects, which manifest as a significant difference between the printed color and the standard color. In one embodiment of the present invention, color deviation detection is performed in the HSV color space, calculating the differences between the detected image and the standard template in the three channels of hue, saturation, and lightness. The criterion for determining a color deviation defect is: hue difference. Degree, or saturation difference or differences in brightness .

[0073] The sixth category is pattern blurring defects, characterized by unclear edges and loss of detail in the printed pattern. Pattern blurring defects are detected by calculating the local gradient magnitude of the image. A blurring defect is determined when the gradient magnitude of the detected image is significantly lower than that of the corresponding area of ​​the standard template. The criterion for determining a blurring defect is: the ratio of local gradient magnitudes is less than 0.5, meaning the edge sharpness of the detected image is less than 50% of that of the standard template.

[0074] The seventh category is pinhole bubble defects, which manifest as tiny, circular or near-circular blank spots within the printed area. The criteria for identifying pinhole bubble defects are: the defect area is located within the printed area, has an area of ​​10 to 200 square pixels, and a circularity greater than 0.6.

[0075] In one embodiment of the present invention, a cascaded judgment strategy is adopted for defect type identification, which sequentially detects the above seven types of defects, and assigns a unique defect type label to each defect area. If a defect area meets the judgment conditions of multiple defect types at the same time, the labels are assigned according to the priority of defect severity, with the priority from high to low as follows: missing print, multiple print, overprinting, misregistration, color deviation, ink splatter, pattern blur, and pinholes / bubbles.

[0076] Preferably, morphological feature analysis is also used as an auxiliary criterion for defect type identification. For each defect region, its morphological feature parameters are calculated, including area, perimeter, aspect ratio of the circumscribed rectangle, roundness, and convex hull area ratio. The formulas for calculating the morphological feature parameters are as follows:

[0077] Circularity The calculation formula is: ,in: This represents the roundness, with a value ranging from 0 to 1. The closer the value is to 1, the closer the shape is to a circle. The area of ​​the defective region is expressed in square pixels. Perimeter of the defect area, in pixels. Circularity is used to distinguish between circular defects (such as pinholes, bubbles, and ink splatter) and non-circular defects (such as missing prints and duplicate prints).

[0078] Aspect Ratio of the Circumscribed Rectangle The aspect ratio is calculated as the ratio of the longer side to the shorter side of the circumscribed rectangle. The aspect ratio is used to distinguish between linear and blocky defects. It may be a linear defect, otherwise it may be a blocky or point defect.

[0079] Convex hull area ratio The calculation formula is: ,in: The area of ​​the convex hull of the defect region is expressed in square pixels. The value ranges from 0 to 1, with smaller values ​​indicating more irregular shapes. The convex hull area ratio is used to distinguish between regular defects (such as ghosting caused by misregistration) and irregular defects (such as ink splatter).

[0080] In one embodiment of the present invention, texture feature analysis is further performed on the defect region to assist in type determination. Texture feature extraction employs the gray-level co-occurrence matrix method to calculate statistical quantities such as contrast, correlation, energy, and entropy of the defect region. Texture features help distinguish pattern blur defects (where texture features exhibit low contrast and high correlation) from other defect types.

[0081] Step S5: Quantitative assessment and judgment. This step aims to quantitatively assess each identified defective area, calculate the severity score of the defect, and make a final judgment on whether it is qualified or not according to the product quality standards, outputting the test result data and automatic rejection signal.

[0082] In one embodiment of the present invention, the calculation of the quantitative defect index includes the defect area, defect location coordinates, and overprint offset.

[0083] Defect area The calculation involves counting the number of pixels within the defect area and converting it into the actual physical area based on the image resolution.

[0084] ,

[0085] in: The defect area is expressed in units of... ; This represents the number of pixels within the defective area. and The physical size of the pixels in the horizontal and vertical directions, in mm / pixel, is determined by camera calibration and shooting distance, with a typical value of 0.05 mm / pixel.

[0086] Defect location coordinates The centroid coordinates of the defect region are calculated using the following formula:

[0087] ,

[0088] in: This is a defective area; The pixel coordinates of the centroid can be further converted into physical coordinates relative to the origin at the top left corner of the nameplate.

[0089] For overprinting misalignment defects, the offset amount The calculation is as follows:

[0090] ,

[0091] in: This is the overprint offset, in mm; This represents the average pixel size, expressed in mm per pixel.

[0092] Severity rating It is a weighted evaluation index that integrates multiple factors. In one embodiment of the present invention, the formula for calculating the severity score is:

[0093] ,

[0094] in: The severity score ranges from 0 to 100, with higher scores indicating more severe defects. , , These are weighting coefficients, corresponding to area factor, type factor, and location factor, respectively, satisfying... The preferred value is , , ; For area scoring functions, For type scoring functions, This is the location scoring function.

[0095] Area scoring function Piecewise linear mapping is used:

[0096] ,

[0097] in: This is the threshold for the area of ​​minute defects; The threshold for medium defect area; is the threshold area of serious defects; defects with an area less than are scored from 0 to 20 points, to are scored from 20 to 50 points, to are scored from 50 to 100 points, and defects exceeding are scored 100 points (full marks).

[0098] Type scoring function is a discrete mapping, and fixed scores are assigned according to defect types: missing printing defect is 100 points, overprinting defect is 90 points, overprinting offset defect is 85 points, color deviation defect is 70 points, ink splash defect is 60 points, pattern blur defect is 50 points, pinhole and bubble defect is 40 points. This scoring system reflects the differences in the severity of the impacts of different defect types on product quality.

[0099] Position scoring function scores according to the position of the defect on the nameplate. In an embodiment of the present invention, the nameplate surface is divided into three position levels: the core area, the secondary area, and the edge area. The core area is the area where the text and key patterns in the center of the nameplate are located, the secondary area is the area of decorative patterns and borders, and the edge area is the blank area around the nameplate. The position scores are: 100 points for the core area, 60 points for the secondary area, and 30 points for the edge area.

[0100] After calculating the comprehensive severity score, it is compared with the preset product quality threshold to determine whether the product is qualified. In an embodiment of the present invention, two levels of quality thresholds are set: the minor defect threshold points and the serious defect threshold points. When the severity scores of all defects are lower than , it is determined that the product is qualified; when there are defect scores between and , it is determined that the product is a repairable product; when there are defect scores exceeding , it is determined that the product is a defective product.

[0101] The output of the test result data includes: product qualification status flag, total number of defects, type label, area, position coordinates, severity score of each defect, and defect distribution statistical information of the overall test. For the nameplate determined to be a defective product, the system outputs a defective product rejection signal to trigger the rejection mechanism on the production line to remove the defective product from the conveyor belt.

[0102] In one embodiment of the present invention, the detection result data is also used for closed-loop feedback optimization. The system statistically analyzes the defect distribution characteristics over a period of time. If it finds that the frequency of a specific type of defect is abnormally high, it automatically adjusts the preprocessing parameters in step S1 to improve image quality, or sends process adjustment suggestions to the production management system. For example, if color deviation defects occur frequently, the system can automatically enhance the contrast correction intensity to improve the detection sensitivity of color differences; if pattern blur defects increase, it may indicate camera focus drift or light source aging, and the system will issue an equipment maintenance reminder.

[0103] like Figure 2 As shown in the figure, this embodiment of the invention also provides a nameplate printing defect detection system based on image segmentation. This system is used to implement the detection method described in the above-mentioned method embodiments. The system adopts a modular architecture design, and the functional modules are connected and transmit data through data interfaces to form a complete detection pipeline.

[0104] Image acquisition module 1 is deployed at the inspection station on the production line and is responsible for real-time acquisition of images of the nameplate surface. In one embodiment of the present invention, the image acquisition module includes an industrial camera, a light source system, and a trigger control unit. The industrial camera is a CMOS camera with a resolution of 2048 pixels by 1536 pixels, a frame rate of not less than 30 frames per second, and a fixed-focus lens. The focal length is selected according to the working distance and the size of the nameplate, preferably 16mm or 25mm. The light source system uses a ring LED light source to provide highly uniform diffused illumination, with a light source power of 20W to 50W and a color temperature of 6000K. The trigger control unit receives position signals from the production line and triggers the camera to acquire images when the nameplate reaches the inspection position, ensuring consistency in acquisition timing.

[0105] Image preprocessing module 2 is connected to image acquisition module 1 via a high-speed data interface and is responsible for performing distortion correction, brightness uniformization, and size normalization on the acquired raw images. In one embodiment of the present invention, image preprocessing module 2 is implemented using an embedded GPU-accelerated computing platform, utilizing parallel computing capabilities to accelerate the image processing process. Image preprocessing module 2 stores camera calibration parameters and preprocessing algorithm parameters, processes the image according to the algorithm described in step S1 of the method embodiment, and outputs standardized preprocessed image data.

[0106] The semantic segmentation module 3 is connected to the image preprocessing module 2 and is responsible for performing inference computations for the deep learning semantic segmentation network. In one embodiment of the present invention, the semantic segmentation module 3 is deployed on an edge computing device or industrial server equipped with a GPU, and the semantic segmentation network model is optimized by inference engines such as TensorRT or ONNX Runtime to improve operating efficiency. The module receives a preprocessed image as input and outputs a pixel-level segmentation mask. For real-time detection requirements, the inference time is optimized to within 50ms per image.

[0107] The template registration module 4 is connected to the semantic segmentation module 3 and is responsible for managing the standard template images and calculating registration differences. In one embodiment of the present invention, the template registration module 4 includes two sub-units: a template database and a registration calculation engine. The template database stores standard template images of various model nameplate products and their pre-calculated segmentation masks, supporting fast retrieval by product model. The registration calculation engine implements the deformation field estimation algorithm and difference map calculation described in step S3 of the embodiment, and outputs the registration difference map and deformation field statistics.

[0108] The defect identification module 5 is connected to the template registration module 4 and is responsible for extracting defect regions from the registration difference map and classifying them by type. In one embodiment of the present invention, the defect identification module 5 implements the threshold segmentation, connected component analysis, and multi-type defect determination algorithm described in step S4 of the method embodiment, and outputs a list of defect regions and their type labels. The module has built-in judgment rules and feature parameters for seven types of nameplate printing defects.

[0109] The quantitative assessment module 6 is connected to the defect identification module 5 and is responsible for the quantitative assessment of defects and the final quality judgment. In one embodiment of the present invention, the quantitative assessment module 6 implements the area calculation, severity scoring, and pass / fail judgment algorithms described in step S5 of the method embodiment. The module outputs detection result data, including pass / fail status, defect details, and statistical information, and outputs a rejection signal to the production line control system when a defective product is detected.

[0110] In one embodiment of the present invention, the system further includes a human-machine interaction module and a data storage module as auxiliary functional units. The human-machine interaction module provides a visual interface displaying real-time detection images, segmentation results, defect annotations, and statistical reports, supporting operators in monitoring the detection process and adjusting parameters. The human-machine interaction module uses a touchscreen display, preferably with a screen size of 21.5 inches or 27 inches and a resolution of no less than 1920 pixels by 1080 pixels. The interface design follows industrial automation human-machine interface specifications, providing clear status indicators and convenient parameter adjustment access. The data storage module is responsible for storing and managing the detection result data. It uses a relational database to store structured detection result data and a file system to store raw images and intermediate processing results. It supports historical data queries, quality trend analysis, and detection report generation. The storage capacity of the data storage module is determined based on the daily detection volume and data retention period, typically configured with more than 1TB of storage space, supporting automatic data backup and archiving functions.

[0111] The detection system provided in this invention achieves high-precision automatic detection of nameplate printing defects through the coordinated operation of various functional modules. The overall system detection cycle is less than 500ms, meeting the real-time detection requirements of the production line; the defect detection rate reaches over 99%, and the false detection rate is less than 1%, significantly better than manual visual inspection; the system supports continuous operation 24 / 7, effectively reducing the labor costs of quality inspection.

[0112] The embodiments of the present invention are not limited to the specific embodiments described above. Those skilled in the art can make various equivalent changes or substitutions based on the technical solutions of the present invention, and all such changes or substitutions should be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in nameplate printing based on image segmentation, characterized in that, The method includes: The image acquisition and preprocessing steps are as follows: An industrial camera is deployed at the workstation on the production line to acquire images of the nameplate surface under constant light source conditions. The image of the nameplate surface is then subjected to distortion correction and brightness uniformization processing in sequence to obtain a preprocessed image. The distortion correction processing includes compensating for radial and tangential distortion based on camera calibration parameters, and the brightness uniformization processing includes adaptive brightness compensation based on the brightness statistics of local areas of the image. The semantic segmentation step is as follows: The preprocessed image is input into the semantic segmentation network. The encoder of the semantic segmentation network extracts multi-scale features from the preprocessed image to generate a multi-scale feature pyramid. The decoder of the semantic segmentation network performs cross-layer feature fusion and upsampling reconstruction on the multi-scale feature pyramid to output a pixel-level segmentation mask. The pixel-level segmentation mask contains classification and labeling information for the nameplate pattern area, text area, and color block area. The template registration comparison steps are as follows: Obtain the standard template image corresponding to the current nameplate; establish the spatial correspondence between the preprocessed image and the standard template image based on the pixel-level segmentation mask; calculate the deformation field vector to characterize the local displacement of the preprocessed image relative to the standard template image; after spatial transformation and alignment of the preprocessed image according to the deformation field vector, calculate the pixel difference between the aligned preprocessed image and the standard template image to generate a registration difference map. The defect identification and classification steps are as follows: threshold segmentation and connected component analysis are performed on the registration difference map to extract the defect regions with significant differences. Features are extracted from each defect region and matched with the preset defect type template to determine the defect type label. Perform quantitative assessment and judgment steps: calculate the defect area and location coordinates for each identified defect area, calculate the severity score based on the defect area, defect type and defect location, compare the severity score with the product quality standard for judgment, and output the test result data and defective product rejection signal.

2. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The distortion correction process includes: performing inverse mapping correction of radial and tangential distortion on each pixel coordinate of the nameplate surface image based on the pre-calibrated camera intrinsic parameter matrix and distortion coefficients, wherein the radial distortion coefficient ranges from -0.3 to 0.3 and the tangential distortion coefficient ranges from -0.1 to 0.

1.

3. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The brightness uniformity processing includes: dividing the preprocessed image into multiple local regions, calculating the average brightness value of each local region, calculating the brightness compensation amount based on the difference between the global average brightness and the local average brightness, and performing brightness compensation on each local region to achieve brightness uniformity of the entire image. The size of the local region is from 32 pixels by 32 pixels to 128 pixels by 128 pixels.

4. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The encoder includes a cascaded structure of multiple convolutional layers and pooling layers. Each convolutional layer uses a 3x3 convolutional kernel for feature extraction, and the pooling layer uses 2x2 max pooling for feature downsampling. The encoder has 4 to 6 layers, and the number of feature channels increases from 64 to 512 layer by layer.

5. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The generation of the multi-scale feature pyramid includes: saving feature maps at each downsampling level of the encoder to form a feature pyramid from shallow to deep layers. The shallow feature maps retain spatial detail information, while the deep feature maps contain semantic abstract information. The number of layers in the feature pyramid corresponds to the number of downsampling times of the encoder.

6. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The cross-layer feature fusion includes: at each upsampling level of the decoder, concatenating or element-wise adding the upsampling features of the current level with the encoder features of the corresponding level in the multi-scale feature pyramid, and then integrating the features through a convolutional layer. The cross-layer feature fusion uses a skip connection method to realize the information transfer between the encoder and the decoder.

7. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The calculation of the deformation field vector includes: dividing the preprocessed image and the standard template image into multiple local windows, calculating the displacement offset between the preprocessed image and the standard template image for each local window, and solving for the deformation field vector that minimizes the registration error through an iterative optimization algorithm. The size of the local window is from 16 pixels by 16 pixels to 64 pixels by 64 pixels, and the convergence threshold of the iterative optimization is 0.01 pixels.

8. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The identification of the overprinting offset includes: judging the overprinting offset based on the statistical characteristics of the deformation field vector. When the deformation field vector shows an overall translation trend in a specific printing area and the translation amount exceeds a set threshold, it is determined that there is an overprinting offset defect in the area. The overprinting offset judgment threshold is 0.1mm to 0.5mm.

9. The method for detecting nameplate printing defects based on image segmentation according to claim 1, characterized in that, The defect types include missing print, multiple prints, misregistration, ink splatter, color deviation, pattern blurring, and pinholes / bubbles. The severity score is calculated by weighted summation based on defect area, defect type weight, and defect location weight. The larger the defect area, the higher the score. Defects located in critical areas of the nameplate have a higher score weight than those in edge areas. The severity score ranges from 0 to 100. When the severity score exceeds the set quality threshold, the product is judged as defective.

10. A nameplate printing defect detection system based on image segmentation, characterized in that, The system for implementing the method according to any one of claims 1 to 9, the system comprising: The image acquisition module is deployed at the production line workstation and is equipped with an industrial camera and a constant light source to acquire images of the nameplate surface. The image preprocessing module, connected to the image acquisition module, is used to perform distortion correction and brightness uniformization processing on the image of the nameplate surface and output the preprocessed image. The semantic segmentation module is connected to the image preprocessing module. It configures the semantic segmentation network to perform multi-scale feature extraction and cross-layer feature fusion on the preprocessed image and outputs a pixel-level segmentation mask. The template registration module, connected to the semantic segmentation module, is used to spatially align the preprocessed image with the standard template image based on a pixel-level segmentation mask, and to calculate the deformation field vector and registration difference map. The defect identification module, connected to the template registration module, is used to extract defect regions and identify defect types from the registration difference map, and output defect region and defect type labels. The quantitative assessment module, connected to the defect identification module, is used to calculate the defect area, location coordinates, and severity score. It determines whether a product is qualified based on the product quality standards and outputs test result data and defective product rejection signals.

Images